May 132011
 

During my exploration of optimzing website using memcached, i’ve found this tutorial. It’s quite interesting using key value application (memcached) for serving website’s static content, Such as image, jss or js file. As we know, Reading/writing Hardrive is slower than accessing memory. although compared with SSD or 7,200 RPM hard drives. Even when an application is loaded, any files that are opened for use in that application are loaded into memory. As well as how web servers work.

First time following the tutorial, it isn’t work. image files not loaded into memcached, no warn, now error. after slightly modifying the script, image files can be stored into the memcached perfectly.

Here’s the php script after modification, i saved it as mem.php

<?php

function rscandir($base='', &$data=array()) {
        $array = array_diff(scandir($base), array('.', '..'));

        foreach($array as $value) {
                if (is_dir($base."/".$value)) {
                        $data = rscandir($base."/".$value,$data);

                }
                elseif (is_file($base."/".$value)) {
                $rest = substr($value, -4);
                        if ((!strcmp($rest,'.jpg')) || (!strcmp($rest,'.png')) || (!strcmp($rest,'.gif'))) {
                                $data[] = $base."/".$value;
                        }
                }

        }
return $data;

}

$mylist=rscandir("/path/to/public_html");

$srch = array('/path/to/public_html');
$newval = array('');

$memcache_obj = memcache_connect("127.0.0.1", 11211);

while (list($key, $val) = each($mylist)) {
        $url=str_replace($srch,$newval,$val);
        echo "$key => $val -> ".filesize($val)."\n";
        $value = file_get_contents($val);
        memcache_add($memcache_obj, $url, $value, false, 0);
}

?>


We just need run the php script onetime

$ php mem.php
0 => /path/to/public_html/img/3glive.png -> 884
1 => /path/to/public_html/img/Little-Idas-Flowers.jpg -> 263271
2 => /path/to/public_html/img/Making-the-legs.jpg -> 15139
3 => /path/to/public_html/img/Marsh-Kings-Daughter.jpg -> 27216
4 => /path/to/public_html/img/benchmark.png -> 2856
5 => /path/to/public_html/img/dsc00277.jpg -> 83911
6 => /path/to/public_html/img/kancil.jpg -> 44945
7 => /path/to/public_html/img/kav.png -> 89663
8 => /path/to/public_html/img/nginx-logo.png -> 15382
9 => /path/to/public_html/img/nginx-memcached-php-1.png -> 9413
10 => /path/to/public_html/img/nginx-memcached-php-2.png -> 10073
11 => /path/to/public_html/img/nightingale.png -> 310143

I’m using fastcgi backend, in my case i’m using this nginx configuration

        server {
                listen       192.168.200.18:80;
                server_name  www.example.com;
                access_log  /var/log/nginx/nginx-test-access.log  main;

                location ~* \.(jpg|png|gif)$ {
                        access_log   off;
                        expires      max;
                        add_header   Last-Modified "Thu, 26 Mar 2000 17:35:45 GMT";
                        set $memcached_key $uri;
                        memcached_pass     127.0.0.1:11211;
                        error_page         404 = @cache_miss;
                }

                location @cache_miss {
                        fastcgi_pass 127.0.0.1:9000;
                        fastcgi_index index.php;
                        fastcgi_param SCRIPT_FILENAME /path/to/public_html$fastcgi_script_name;
                        include /etc/nginx/fastcgi_params;
                }

        }

this is example header when accessing image using curl

$ curl -I http://192.168.200.18/img/Making-the-legs.jpg
HTTP/1.1 200 OK
Server: nginx/1.0.2
Date: Fri, 13 May 2011 02:55:36 GMT
Content-Type: image/jpeg
Content-Length: 15139
Connection: keep-alive
Expires: Thu, 31 Dec 2037 23:55:55 GMT
Cache-Control: max-age=315360000
Last-Modified: Thu, 26 Mar 2000 17:35:45 GMT

If we noticed Last-Modified Header, that indicated files were serving from memcached. we set add_header Last-Modified in nginx configuration, remember? 🙂

add_header   Last-Modified "Thu, 26 Mar 2000 17:35:45 GMT";

ok, that’s it for now. I hope this guide can help others to implement the tutorial from original site.

  3 Responses to “Nginx Memcached Module, Caching Webiste Image Using Memcached”

Comments (3)
  1. Thanks for your article. It’s very fantastic.

  2. Hi,

    I have an medium forum with ~100 users online, and i want to cache all static images, even external images like from imagestack imgur and other file hosting image.

    what solution i can have ?

  3. hi, for internal static images run on local server, basically you used the same way that i’ve done in tutorial, using external images, you have no control over them, clients directly load images from external web server.

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

(required)

(required)

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.