Handling the Digg Effect with Wordpress Caching
Sunday, September 17, 2006
Last week, I was very proud to have my Subversion Quick Reference guide hit the Digg front page, which was a first for me! I’ve been on the del.icio.us frontpage a number of times, but nothing else compares to being dugg.
Even though my article hit the frontpage during a non-peak time, I still received over 3,000 hits within a fairly short period of time. Sure, I didn’t get the huge digg effect, but I was still very surprised that my little wordpress blog hosted on dreamhost handled the digg effect with no problems whatsoever.
I have since come to the conclusion that the reason my blog held up to the traffic because I very aggresively use the Wordpress caching module. Here’s the list of things I’ve done that helped me keep my blog afloat.
1) Install the Wordpress Caching module if it’s not already installed.
If you haven’t already got the Wordpress Caching module installed, you will want to get it and install it now. Note: Dreamhost installs it by default, so all you have to do is turn it on if you are a dreamhost customer.
If you don’t have the module, you can follow the instructions from the WP-Cache 2.0 page to install it. Once you’ve installed the plugin, you’ll want to click the activate button.
2) Configure the Cache Timeout to infinite
Unless you have content that requires constant updates, there’s really no good reason that you can’t change the caching time to infinite, because the caching module is smart enough to remove the pre-cached page if you update a post, or it will remove the cached version of the index page
Also, the WP-Cache module provides a syntax for executing dynamic code on static pages. You can include a function into the cached pages that will still execute dynamically, or you can include a php file that will always execute. The syntax works something like this:
<!–mclude ads.php–>
<?php include_once(ABSPATH . ‘ads.php’); ?>
<!–/mclude–>
With that background information, we can move on to configuring the timeout setting. This setting determines how long the pages stay cached before Wordpress automatically reloads them. The issue with this from my standpoint is that during peak loads, if a user decides to click on another link on your site, Wordpress will potentially reload that link if the timeout period has expired. I hate when I load up a page, click on another link on their site, and it either takes forever to load, or fails entirely.
In the administration panel, browse over to Options, and then to WP-Cache. I typically just type in a whole bunch of 9s into the expiration field, cause it’s simpler than remembering some number.
Click the Change Expiration button, and we’re good to go. We now have cached our blog to the point where it’s nearly static…. except it isn’t.
3) Delete and reload the cache when you change things.
Clearly we don’t want to revert back to a completely static blog, we’ve grown up since the days of Geocities accounts. Do you remember which Geocities server you were on? I think I was on the siliconvalley one, but I can’t remember.
There are a few options for how we can reload the cache once something is actually updated. First, if we are making changes to the structure of the page…. adding new links to the sidebar, moving elements around… then we will want to reload the entire cache. There’s an easy way to do so in the WP-Cache options panel… just click the Delete Cache button.
Alternatively, there’s another way to do this in a more automated fashion. I prefer to set a certain time for reloading the cache. It can be nightly, hourly, or whatever time period you would like to run it.
WP-Cache stores it’s files in the directory blogdirectory/wp-content/cache/ if we take a peek in there, we should see something like this:
wp-cache-685c5b56dc5181ffc5f2fe4753f9f25d.html wp-cache-e14d8b7d4c4a23bcdf808c8a5e41ec9e.html
wp-cache-685c5b56dc5181ffc5f2fe4753f9f25d.meta wp-cache-e14d8b7d4c4a23bcdf808c8a5e41ec9e.meta
wp-cache-692dd0426c4eda24b4d68f367fb100fb.html wp-cache-e4ca2bec97c1638bc9ccd6cffea0722d.html
wp-cache-692dd0426c4eda24b4d68f367fb100fb.meta wp-cache-e4ca2bec97c1638bc9ccd6cffea0722d.meta
wp-cache-6a318f2b596c174d8d45833c408d443b.html wp-cache-e9a3a0001759927ce99febae0e7df1ec.html
All we need to do to delete the cache, is make a cron that runs rm wp-cache* in that directory. I’ll leave that exercise up to you.
But now we have another problem…. we’ve deleted the cache, which means the entire website isn’t cached anymore. Here’s where a handy php script and the Google Sitemaps Plugin comes into play! You are using Google Sitemaps aren’t you? If not, you really should be. I’m going to assume that you’ve installed it, and we’ll proceed.
I’ve created a little PHP script that is Dreamhost friendly. Dreamhost blocks URL access through the file methods, so you can’t just use something like file_get_contents(url). Thankfully they do provide a workaround on their wiki, which I incorporated into this script:
<? $xmldata = get_url_contents(”http://www.mysitename.com/sitemap.xml“); $xml = simplexml_load_string($xmldata); $cnt = count($xml->url); for($i = 0;$i < $cnt;$i++){ $tmp = get_url_contents($xml->url[$i]->loc); } echo(”Completed!“); function get_url_contents($url){ $ch = curl_init(); $timeout = 5; // set to zero for no timeout curl_setopt ($ch, CURLOPT_URL,$url); curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, $timeout); $str = curl_exec($ch); curl_close($ch); return $str; } ?>
This script will pull down your Google Sitemap, which contains a list of all the pages in your site. It will then make a request to each page in your site, at which point Wordpress will automatically cache each and every page in your site. If we check the WP-Cache panel, we should see… sure enough, we’ve got 35 cached pages now.
Go on, check out the speed difference in your website. If you haven’t been caching, you’ll probably be amazed.
Latest Posts
September 18th, 2006 at 11:37 am
Where do we put the PHP script you wrote?
September 18th, 2006 at 11:50 am
Hmm… could you add some sort of server load check into your cache clearing/reloading scripts so it only reloads if the server has been reasonably quiet for the past x number of minutes?
That would prevent a reload right in the middle of a digg-effect. Not much chance of that occurring, but you never know.
September 18th, 2006 at 11:53 am
Don’t know about wordpress, but Drupal definitely survives the digg effect even without caching
September 18th, 2006 at 12:14 pm
Manejando el efecto Digg / Barrapunto / Meneame…
Explicacion de los pasos a seguir para, al menos, disminuir el efecto que puede provocar en tu blog de wordpress aparecer en la portada de una web importante….
September 18th, 2006 at 12:46 pm
With WP-cache, there is tendency that the latest post (or comments) don’t get updated instantly over the site. Maybe I am wrong, but I believe this is good for site that has high traffic.
September 18th, 2006 at 1:08 pm
Forgive my ignorance, but that little PHP script throws an error about an “undefined function: simplexml_load_string()”
I don’t see anywhere in your article about where to put this script or if it is to be included in a file, what file that is.
Did I completely miss something? Can you explain where this script should go exactly
September 18th, 2006 at 1:34 pm
A week back my site got listed on the Digg.com homepage and I did not even know what was happening. I tried to SSH into my server but it was overrun. I am running Wordpress, but what I did to fix it was to limit the Apache web server to only 10 forked children. Before it seems it was spawning more children than the server could handle in an attempt to keep up with requests.
Hopefully next time the server gets pounded it will be alright.
September 18th, 2006 at 1:41 pm
This is the first I’ve heard of the Digg effect, but I had a similar surge from Reddit and Blogger (blogspot) seemed to have handled it fine. Is there a Digg effect issue with blogspot.com?
-Stevie D. (http://lazycomic.blogspot.com)
September 18th, 2006 at 2:09 pm
Koby,
Dreamhost has php4 and php5, you’ll want to run it with the php5 version, which is located at:
/usr/local/php5/bin/php
You can do a php -v to find the version.
September 18th, 2006 at 2:17 pm
Ok, nevermind, I figured out it’s a PHP5 thing which my server doesn’t support.
September 18th, 2006 at 7:32 pm
I’ve gone through the exact same thing as you but I concluded that in addition to wp-cache, it is also a good idea to use the ‘Coralize this’ plugin which redirects all or some of your page and site contents to the cache servers. Very handy.
Also, doesn’t wp-cache cache comments too? SO if you set the reset time to infinity, how will people know there have been comments?
September 18th, 2006 at 9:41 pm
The caching plugin is smart enough to reset the cache once you’ve approved comments. You can set a couple different options on the comments…. I’ve found that if I don’t moderate them that I’ll end up with a ton of spam.
September 19th, 2006 at 10:58 am
I have done everything, but for some reason after clearing the cache and reloading via your script, every page I go to for the first time shows up as blank.
September 19th, 2006 at 11:40 am
Chris,
There’s a problem with dreamhost and the caching module that I should have mentioned… I’m copying this off the wiki:
If you have “blank pages” in Wordpress with wp-cache turned on after you upgrage to PHP 5.1.2 - there is simple fix to solve the problem:-
1. Open wp-cache-phase2.php file* in your favourite text editor
2. Find out wp_cache_ob_end function
3. then inside that function find out line with: ob_end_clean(); (it should be line 219 or about)
4. and finally replace that line with: ob_end_flush();
October 25th, 2006 at 7:25 pm
Hello Johnny,
I have further optimized WP-Cache to handle better the “Digg Effect” by adding support of serving highly pre - compressed pages reducing both serving time and bandwidth required per visitor.
The required modifications are described at my blog page - Modifying WP-Cache 2.0 to generate and cache gzipped output once and serve it multiple times
October 30th, 2006 at 10:05 am
Is this working with WP 2.0.5? I’m just trying to get wp-cache working for the first time, but it’s only caching my index and archive pages, not individual entries. Any suggestions? Thanks!
November 5th, 2006 at 6:01 pm
I’ve just written a post that explains how to make wordpress pages faster and how to make better use of caching to protect against being ‘Dugg’ - worth having a look.
http://www.connectedinternet.co.uk/2006/11/05/1028/
November 15th, 2006 at 6:29 am
handling the digg effect with wordpress caching…
Looking for a good WordPress plugin that will deliver a sitemap to Google? Check out Google Sitemap Generator by arnebrachhold.de. The plugin installs smoothly and has some cool priority options. You can prioritize your posts based on comments or just …
February 14th, 2007 at 3:40 pm
Thanks! Now that I have that installed I just need to get dugg :)
March 13th, 2007 at 3:41 am
www.e-fanyi.net翻译公�
www.e-fanyi.net/index02.htm北京翻译公�
March 23rd, 2007 at 5:18 pm
Thanks for the script! What does it mean if all the cache files are made but only the feeds show up in the cache admin panel?
March 30th, 2007 at 7:50 am
Thanks for the great script suggestion - am going to try this on a few of my installs. Hat tip to SEOmoz for the shout out for this neat plugin. :)
May 23rd, 2007 at 10:07 am
Wp-cache plugin has been a life saver for the sciencetext.com site this week which got a fairly big Stumble over a post about how to use your GMail account as a virtual hard drive. Nothing clever, just the basics, but amazing how many people hit it once it got picked up on SU. Tens of thousands…
db
May 24th, 2007 at 10:06 pm
æ¦æž—
August 7th, 2007 at 8:32 am
Thanks for the tip on surviving the Digg/reddit effect. I just implemented your suggestions to help keep the FastWallpapers.com blog working after one of its posts got dugg/reddited.
Thanks again,
Gabriel
August 8th, 2007 at 7:53 pm
thanks! i’m installing wordpress for our site www.txtsignal.com at blog.txtsignal.com (not live just yet) and i’m sure this will help us.
August 24th, 2007 at 11:56 am
all 9s is not working How many sec max i can cache.. I have installed in our site www.thepicky.com please guide.
September 13th, 2007 at 4:06 pm
I had this happen to me, 2,000 Diggs and nearly 20,000 hits in one day and Wordpress is shut down at the moment since it was conflicting with my other sites. Do you know of a solution? I can not reactivate the subdomain, otherwise the hits start pouring in and the server crashes.
October 12th, 2007 at 5:39 am
Thanx a lot! This is very useful.
October 18th, 2007 at 6:23 am
Great, indepth and insightful review! A+ I enjoyed reading it very much and I learned a lot.
Thanks again and keep up the good work.
November 6th, 2007 at 1:07 am
“Handling the Digg Effect with Wordpress Caching” - Good work. Cogratulations
December 10th, 2007 at 12:57 pm
Check my free tutorial on how to get your site listed on Digg’s front page:
http://www.moneymakerhot.com/Webmasters/Get_on_front_page_of_Digg.html
December 27th, 2007 at 4:21 am
You should also set-up front end and back end balancing. This will help a lot in long run.
February 28th, 2008 at 4:47 am
Thanks for the heads up. It really pays to be updated with the latest news on technology and keep abreast with the innovations.
March 28th, 2008 at 11:27 pm
Natural herbal health care medicines, Articles, informations and daily updated health concerns issues and their solutions for better health and better life. www.naturalherbalz.com
April 4th, 2008 at 4:49 pm
Awesome - appears to work beautifully
of course, I’ll have to be dugg or slashdotted to know for sure :-)
April 8th, 2008 at 12:06 pm
Penis enlargement products, American Most Trustworthy Natural herbal productz store for penis enlargement,Which increase your size 4-6 inches, Double the size Double the fun.(Yes No Pain ALL Gain) And more productslike weight loss, male enhancement, skin care, acne solutions, breast enlargement, hairLoss for men and women , multi Vitamin for men and women, optimum Diabetics, eazol Pain Relief , stretch mark prevention, revitol skin brightener, revitol hair removal cream, revitol anti-aging solution.
http://www.naturalherbalproductz.com
April 11th, 2008 at 9:57 am
I am glad to post my views and points in this blog, but I must say that webmaster of this blog has done a very great job to make his blog more informative and more discussable but unfortunately everything is same here that more than 80% in this and other blogs post their comments for making spam!!!, so i will really all this spam links to google band tool, because webmaster makes blogs for making discuss and for sloving each other problems. thanks
http://www.naturalherbalproduct.com
April 12th, 2008 at 3:18 pm
[…][…]Penis enlargement products, American Most Trustworthy Natural herbal productz store for penis enlargement,Which increase your size 4-6 inches, Double the size Double the fun.(Yes No Pain ALL Gain) And more productslike weight loss, male enhancement, skin care, acne solutions, breast enlargement, hairLoss for men and women , multi Vitamin for men and women, optimum Diabetics, eazol Pain Relief , stretch mark prevention, revitol skin brightener, revitol hair removal cream, revitol anti-aging solution.
http://www.naturalherbalproductz.com
[…][…]
April 12th, 2008 at 11:21 pm
Penis enlargement products, American Most Trustworthy Natural herbal productz store for penis enlargement,Which increase your size 4-6 inches, Double the size Double the fun.(Yes No Pain ALL Gain) And more productslike weight loss, male enhancement, skin care, acne solutions, breast enlargement, hairLoss for men and women , multi Vitamin for men and women, optimum Diabetics, eazol Pain Relief , stretch mark prevention, revitol skin brightener, revitol hair removal cream, revitol anti-aging solution.This blog is very informative and thanks for the updates and the blog colour and desiging is very beautiful after seeing this u must be a creative man
http://www.naturalherbalproductz.com
April 20th, 2008 at 4:17 am
Thanks for explaining the use of wp-cache plugin….I am using it as you have said and it has shown some good improvement
April 21st, 2008 at 7:11 am
thanks also for the explaination..my WP blog also shows improvement
April 22nd, 2008 at 2:26 am
Penis enlargement products at guaranteed cheap price!
penis enlargement pills, penis enlargement patches, penis enlargement devices, penis enlargement exercises, penis enlargement gel, and penis enlargement treatments with FDA approroved guaranteed. www.gordoniihoodia.net
April 23rd, 2008 at 5:39 am
wow, this comment is very nice !!
April 23rd, 2008 at 6:27 am
I have to laugh at how many people didn’t know what to do with the little piece of code you wrote about.
So sad…
April 24th, 2008 at 5:57 am
it seems that this code needs a repair somehow..
April 26th, 2008 at 2:27 pm
MAXWILLY is the fastest way to increase your size perminantly.
Whether, you have 1 inch or 8 inches… MAXWILLY can help.
Since its invention in 1994 Maxwilly has sold over
1 million devices to men in over 100 countries
“MAXWILLY” is Is totally safe, extremly easy to use, and is
suitable to fit all penis lengths from 1 inch to 18 inches.
“MaxWilly” was developed by, medical doctor Jorn Ege Siana,
who specializes in penis lengthening. After extensive research
he came up with the first prototype of MAXWILLY device in 1994.
Since that time the product has undergone many different clinical
studies and has won the trust of thousands of Doctors worldwide.
Clinics worldwide use our product as a safe alternative to penis
surgery. You can stop dreaming of having a larger penis and finally
achieve the results, you have only dreamed of.
April 27th, 2008 at 10:30 am
American top penis enlargement vigrx plus’s online store with very CHEAP PRICES, Health store for men and women Approved by doctors and adult magzine and FDA approroved guaranteed.Including skin products,hairloss , weight control, diet supplement and above all please (Becareful by all others Scams Fraud sites).Good luck. I am offering you money back guarentee and free coupons on all products.Goodluck
http://www.penisenlargementincs.com
April 27th, 2008 at 10:45 am
Dont trust the salesman,always trust the doctor.Only one and only men’s and women’s natural health prdoucts,eurpore and american’s largest store Recommended by FDA approved Doctors.They are offering cheap prices and free coupons make sure u get it before coupons expires.
http://www.penisenlargementincs.com
May 2nd, 2008 at 6:06 am
lorr..so many sales man here arr ???
May 2nd, 2008 at 10:49 pm
Men who need penis enlargement solutions like penis enlargement pills penis enlargement oil and women needing breast enlargement pills can get an easy guide
http://www.skincarefairy.com
May 2nd, 2008 at 10:51 pm
MoreNiche affiliate program, the worlds best affiliate network. Start your own home based business and make a success out of niche affiliate marketing and visit our affiliate marketing forum for advice and tips.
http://www.moreniche.tk
May 4th, 2008 at 12:15 am
FDA approroved guaranteed,free coupons on all products.Goodluck.American top penis enlargement vigrx plus’s,Men’s health and women’s health products online store with very CHEAP PRICES Goodluck
http://www.penisenlargementincs.com
May 4th, 2008 at 1:23 am
hahaha…. thats true….. so many sales men here
May 4th, 2008 at 3:10 pm
Digg Effect :p, i wish my site reach front page
May 5th, 2008 at 2:46 am
Nobody can stop me by posting these comments.
http://www.businesswebhostings.net
May 7th, 2008 at 11:45 am
Penis enlargement treatments, solutions and natural herbal medicines to enlarge your penis naturally like 2-3 inches gain in penis size without any hesitation, worries and side effects for making your life more pleasureable and excited. www.penissizeenlargement.net
May 8th, 2008 at 10:52 am
Natural Gain Plus Natural Male Enhancement for increasing your sexual drive and pleasure.
May 10th, 2008 at 11:22 am
Generic Cialis is used to treat male impotence also referred to as erectile dysfunction (ED).
Generic Cialis has the same active ingredient as brand name Cialis, and is equivalent in effect,
strength, and dosage. This bestselling pill offers the longest performance period on the market.
When taken 15 minutes before intercourse it can last up to 36 hours. The success rate of oral ED drugs
is very high, above 90%; however different people require different dosages to attain optimum results.
only available at http://www.penisenlargementv.com/
May 12th, 2008 at 10:32 am
The Natural Gain Plus program is the most powerful natural
male enhancement program available anywhere in the world.
With years of success behind our product and being one of
the only companies that uses 100% natural ingredients we
foresee you having great success
May 15th, 2008 at 1:12 am
Great blog, great post, my blog are dofollow too.
http://www.internationalremovals-ltd.com/en/logistics-company.html