Don’t let the image resizer crash your server
Posted under: This is the story of many page requests in a short amount of time. Over 500 requests from the same IP in under 10 seconds. They all pinged the TimThumb script (a thumbnail generator, very popular among WordPress themes and plugins) asking it for resized, uncached images. The fact that this was done in one go saturated 16GB of RAM on the server and stopped other processes running, thus crashing the server.
The client was unhappy with their site being offline for the past day and a half (they were in fact losing business while being offline) and in reply the host expressed yet more serious concerns:
Naturally as you can imagine, we have to consider other customers on this platform, several of whom have complained about these crashes. Like yourself, downtime costs us money. We either:
a) need to be absolutely sure that you yourselves, armed with the information above, know exactly what the issue is and can rectify it.
b) cut our losses, offer you a refund pro-rata and ask you to move your hosting account away from ourselves.The memory usage of the script does imply however that you will have the same issue anywhere else. No shared provider will accept a single site making use of 16GB of memory.
So we either needed to guarantee that our website wouldn’t be a trouble maker in the future, or pack our stuff and move to our mother’s.
There could have been multiple reasons for the hundreds of requests. Someone may have been scrapping the site (an online store) for product images. Someone may have been pinging the PHP image resizer and asking it for fresh images just for fun. One of our own Ajax scripts may have been doing that accidentally under certain circumstances. Hell, one of our Ajax scripts may have become self-aware and was trying to take over the hosting company for all I knew. I needed a solution to prevent any of these.
“500 requests in 10 seconds” I thought… “No single IP would need to make more than a dozen requests in any given 5 seconds.”
And there’s the answer. The scripts shouldn’t honor more than 10 requests in under 5 seconds. You may be tempted to simplify that to 2 requests per second, but don’t. Sometimes you go from one page to another in under one second, or you open 5 new tabs in a couple of seconds and there’s nothing illegitimate about that. In fact, let’s change the numbers to say that “no more than 40 requests in 20 seconds” are acceptable from the same IP.
Throttling the requests on a per IP basis
No more than 40 requests in 20 seconds are acceptable from the same IP is materialized in the PHP script I will paste below. The code would need to be saved as “limit.php” and included using:
include_once('limit.php');
…at the top of any scripts that are prone suck a lot of memory. If you’re using WordPress and are paranoid enough (as I was at the time) you can include it at the top of your theme’s functions.php file. But even more importantly, if your theme is using timthumb.php or another image resizer you should also include it at the top of that file.
TimThumb is very susceptible to suck a lot of memory. It does set a “MEMORY_LIMIT”, but that only applies each time the script runs. That means processing a very large image will be prevented by default, but running the TimThumb script 500 times on regular-sized images within a few seconds will not be prevented.
If you have the timthumb.php file somewhere on your server, anyone could write a simple JavaScript that generates different URLs and continuously requests new images from your TimThumb script until the server goes the way of the Titanic.
A simple PHP throttling script
These are the contents of “limit.php”:
The code, explained
I’ll go through every piece of it below, but you’ll probably need to know how PHP sessions work to completely understand everything.
The limit_requests() function takes two parameters: the number of allowed requests ($nr with a default of 40) and a number of seconds ($t with a default of 20) — the time interval in which those requests can be made.
We will use a session to store a couple of variables:
- $_SESSION['hits'] — the number of hits (requests) made in the last interval
- $_SESSION['tzero'] — a starting point for the time interval which we’ll use to check how much time has passed (by comparing it with the current time)
We will only start a session if one isn’t currently active:
if (!session_id()) {
start_session_based_on_ip();
}
Instead of PHP’s default “session_start()” we use a custom start_session_based_on_ip() function. If we only used session_start() the script would have worked exactly the same except a malicious user could have bypassed it by turning off their cookies. So instead we’re giving our session an id based on the visitor’s IP address.
if( !isset($_SESSION['tzero']) ) {
$_SESSION['tzero']=time();
}
If $_SESSION['tzero'] isn’t set, it means it’s the first time this user is running the script and we’ll set it to the current time.
$since_interval_start = time() - $_SESSION['tzero'];
How much time (in seconds) has passed since $_SESSION['tzero']. We’ll need this below.
if( $since_interval_start > $t ) {
$_SESSION['tzero'] = time();
$_SESSION['hits'] = 1;
} else {
$_SESSION['hits']++;
}
If $since_interval_start is larger than our time interval it means our interval is up. We reset ['tzero'] to the current timestamp (thus we start a new interval) and the hits to 1.
If not (the else bit) we increment the hits.
if( $_SESSION['hits'] > $nr ) {
die('<h1>Too many requests!</h1> You will be able to make a new request in '.($t-$since_interval_start).' seconds.');
}
This is the most important part. If the number of hits exceeded the $nr limit we kill the running script, effectively preventing any future requests until the time interval runs out (at which point a new interval will start and the hits will be reset — see the previous paragraph).
Check out a demo of the script here. If you Refresh the page a few times, you’ll see the hit tracker increase.
Where else to use it
The best place to include this script would be CPU intensive PHP programs to prevent them from being repeatedly accessed from the same machine. I’m thinking PHP scripts that generate images, generate PDF files, scripts that work with audio or video files, scripts that compress files into archives.
Password protected area in WordPress
If you ever need to set up a simple password protected section under WordPress, these snippets and plugins might come in handy when you’re hacking through your theme.
I believe this would be the logical approach: you create a main password protected page for your private section and then you create a bunch of child pages under that page. This way you’ll be using native WordPress functionality (page organization and page visibility) for most of your private section.
But you may find that a couple of patches will be in order.
WordPress Plugins
FT Password Protect Children Pages
One thing you will not want will be to have to password protect each subpage individually. You’ll just want to set a password for the main parent page and have all its child pages inherit the password protection. This plugin does exactly that.
Page Template Extended
Similarly, if you decide to use a specific page template for the protected section, it would be ideal to just set the page template to the main parent page and not have to worry about setting it for each of its child pages. This plugin makes subpages inherit their parents template.
Logout Password Protected Posts
You’ll also need a log out link that WordPress doesn’t provide by default. After installing this plugin you’ll be able to use <?php do_action(‘posts_logout_link’); ?> anywhere in your WordPress theme to produce the log out link.
Listing the subpages
Use this code in your theme to display the main parent page as a heading and the child pages below it.
If you’ve created a page template for your protected pages, you can use the code above in it to give your users a navigation between the subpages.
A couple of snippets for functions.php
Change the default password protected text
The default text on password protected posts or pages is This post is password protected. To view it please enter your password below. Should you wish to change this text, add the following code to your theme’s functions.php file:
Change the part where it says “This is my custom text…” in the code above.
Remember: never edit WordPress core files. Either use a plugin or use the functions.php file and filter your changes in from there.
Remove the word “Private:” from the titles
By default the titles of the password protected pages are preceded by the word “Private:” (followed by a colon). To prevent this from happening, add the following filter to your theme’s functions.php file:
Password un-masking:
Show them in clear text
Windows has a nice option labeled Show characters
right below password input fields:
“Show characters” checkbox under a password field in Windows
Useit.com recommends it: It’s time to show most passwords in clear text as users type them.
So without further ado, here’s what it looks like on the web: unmasking the password field This is the HTML code from the example:
PHP Screencast: Hidden Captcha
As I was saying in a past–not so documented–article, this is how the Hidden Captcha concept works:
Does the user have JavaScript enabled?
If yes, they’re okay — let’em comment, no annoying captcha required.
No? We’ve got a suspect. Read them their rights and serve them the ultimate “are you human?” test.
I made this 5 minute screencast to earn fame and fortune on Nettuts, but I’m also posting it here along with some textual comments. Figured I’d make it easier for you to copy/paste the whole 2 lines of JavaScript.
Here’s what you start with, the source code from this tutorial.
(Note: be sure to have the Arial font file called arial.ttf in the fonts folder–copy it from your System in there because their archive does not come with it).
This is the JavaScript/PHP I’m pasting in:
<script type="text/javascript">// <![CDATA[
document.getElementsByName('code')[0].value = '<?php echo $string; ?>';
document.getElementById('captcha').style.display = 'none';
// ]]></script>
Both lines of JavaScript work on elements from this chunk of HTML:
<div id="captcha">
<img src="captcha.php" alt="" /><input type="text" name="code" /> Are you human?
</div>
The first line of JavaScript sets the correct value for the code text field.
And the second line of JavaScript sets display:none to the captcha div, thus hiding it from anyone with JavaScript enabled.
Hidden Captcha instead of Akismet?
Using captcha without displaying it
How I use captcha without making my users complete the barely readable word
Capthca sucks. For more information on how much captcha can suck see John Willis’ post Top 10 Worst Captchas.
But at the same time it can be really annoying for webmasters to have their forms unprotected with all the spam bots running free out there.
What I wanted was to have the commenting feature protected against spam bots without having the innocent human users ruining their eyes on captcha like images, or complete any mathematical equation or any other additional question fields.
One very important difference between a spam bot and a human using a web browser is that the first can’t run JavaScript code. However, this isn’t a perfect criteria of selection, because there are humans browsing the web using browsers without JavaScript support (Opera Mini for mobile devices for example).
My ideea (and as I did some Google searches, I found out other people had similar ideas) was the followig algorithm:
Does the user have JavaScript enabled?
If yes, he’s ok. Let him comment.
No? He’s a suspect. Read him his rights and give him the ultimate “are you human?” test.
To do this I left the captcha system enabled and in place and wrote 2 extra lines of JavaScript that:
//complete the text field with the correct word from the image:
$('secretword').value='nospam';
//hide the div containing the captcha image and the text field:
$('captcha').style.display='none';