bitstorm.org

weblog of a webdeveloper

  1. Introduction to the Google Closure Compiler

    Last year, on November 5th, Google released three JavaScript code libraries that are used in Google's impressive webapps like GMail and Google Maps. One part of the release is the Google Closure Library at which we'll take a closer look in this article.

    The Google Closure Library can do two things. First, it can optimize and minify your JavaScript code and second, it can do code checking.

    The big advantage of minifying your JavaScript code is that there's less to download so the page using it will appear faster. And since comments are removed, it allows you to write good documented code wihout being afraid it will make your code too big.

    Using the online Closure Compiler

    We start with some very simple JavaScript code:

    function showTotal(subtotal, shipping) {
       var total = calculateTotal(subtotal, shipping);
       showAmount(total);
    }

    function calculateTotal(total, shipping) {
       return total+shipping;
    }

    function showAmount(total) {
       document.getElementById("amount").innerHTML = "$ "+total;
    }

    showTotal(120, 30);

    We can use the online Closure Compiler to quickly see what the compiler does.

    The Closure Compiler has three levels of optimization. In the first level, it only removes white space like spaces, returns and tabs. This is the result:

    function showTotal(subtotal,shipping){var total=calculateTotal(subtotal,shipping);showAmount(total)}function calculateTotal(total,shipping){return total+shipping}function showAmount(total){document.getElementById("amount").innerHTML="$ "+total}showTotal(120,30);

    The second level is Simple optimization, in which local variable and function names are renamed to a shorter one.

    function showTotal(a,b){var c=calculateTotal(a,b);showAmount(c)}function calculateTotal(a,b){return a+b}
    function showAmount(a){document.getElementById("amount").innerHTML="$ "+a}showTotal(120,30);

    The third, Advanced level, it the most aggressive. The result is as follows:

    document.getElementById("amount").innerHTML="$ 150";

    That's impressive, isn't it?

    The Advanced level needs some extra work when your code is using external JavaScript code or when other code is calling your minified code.

    Take for example the following script, a very simple jQuery plugin:

    jQuery.fn.data = function(name) {
       var first = $(this).eq(0);
       if (first) {
          return first.attr('data-'+name);
       }
    }

    Compiled with Advanced level optimization, you get this:

    jQuery.c.data=function(b){var a=$(this).b(0);if(a)return a.a("data-"+b)};

    As you can see, the jQuery functions are shortened too. Ofcourse, this will not work. To fix this, we first have to make the Closure Compiler run on our local machine.

    Running the Closure Compiler locally

    Since the Closure Compiler is written in Java, we'll first make sure the Java Runtime Environment is installed. Then we can download the compiler-latest.zip and unzip it. To get an easy start, we put all files (data.js, compiler.jar and the batch/shell script below) in one directory.

    java -jar compiler.jar --js data.js --compilation_level ADVANCED_OPTIMIZATIONS --warning_level QUIET > data-min.js

    Running the batch/shell script without any changes will produce data-min.js with the same non functional code as with the online version above.

    Using externs

    To get functional code from the advanced level optimization, we have to supply the file containing the JavaScript that is called, in this case the jQuery source code. After copying jquery.min.js to the same directory, we can perform advanced level optimization. We only have to let the compiler know about the external JavaScript file with the --externs option:

    java -jar compiler.jar --js data.js --compilation_level ADVANCED_OPTIMIZATIONS --warning_level QUIET --externs jquery.min.js > data-min.js

    Now he output is what we want:

    jQuery.fn.data=function(b){var a=$(this).eq(0);if(a)return a.attr("data-"+b)};

    With the --externs option, you have to add all files your code is calling.

    External references

    The --externs option only works for JavaScript files your code is calling. If your code is called from another JavaScript file or even from a HTML-document or you just want to provide an API, things are a bit harder.

    Assume you have this function and you want to minimize it:

    function getData(element, name) {
        return element.getAttribute('data-'+name);
    }

    If you optimize this with advanced level optimization, you get an empty file. The compiler "sees" that the function is never called and thinks the function can be removed.

    There are couple of ways to solve this.

    1. The way Google suggests: attach the function to the global window object like this:

    function getData(element, name) {
        return element.getAttribute('data-'+name);
    }
    window['getData'] = getData;

    Minified, it becomes:

    function a(b,c){return b.getAttribute("data-"+c)}window.getData=a;

    Personally, I think adding every function to the window object is very ugly. First, you pollute your own code just to get it compressed and second, you are adding code which doesn't help the minification.

    2. A better way is to use a namespace. This is a better practice anyway and prevents name collisions. Now you only have to add the namespace to the global window object. This is also what jQuery does. In JavaScript, there are many ways to create a namespace.

    If jQuery would only contain the function above, it would look like this (simplified):

    (function() {
        var jQuery = {};
        jQuery.getData = function(element, name) {
            return element.getAttribute('data-'+name);
        }
        window['jQuery'] = jQuery;
    })();

    The resulting code is:

    (function(){var a={};a.getData=function(b,c){return b.getAttribute("data-"+c)};window.jQuery=a})();

    3. The last way is to not provide any API at all. You could just merge all you JavaScript code into one single minified file. It's easy and minifies very well. The Closure Compiler accepts multiple files with multiple --js parameters:

    java -jar compiler.jar --js data.js --js mylib.js --js jquery-min.js --compilation_level ADVANCED_OPTIMIZATIONS --warning_level QUIET > all-min.js

    Annotations

    The Google Closure Compiler can do more than only minifying. It can also do type checking. You thought type checking is not possible in JavaScript? Well, now it is. With the Closure Compiler you can annotate the types of variables and signatures of functions in comments. These comments look just like the comments used in JSDoc, so you can generate documentation with it, too.

    With annotations you effectively turn JavaScript into a strongly typed language and prevent a common source of errors from happening.

    The function we used earlier can now be written as:

    /**
    * @param {!Element} element
    * @param {string} name
    * @return {string}
    */
    function getData(element, name) {
        return element.getAttribute('data-'+name);
    }

    The two arguments have the types Element and string and the return type is also string. The exclamation mark in front of Element means the Element object can't have the null value. So what happens if you call the annotated function with the wrong parameters? Let's call getData like this:

    getData('body', 'size');

    The error output you get after running the Closure Compiler is:

    C:\Users\Edwin\Desktop\Closure Compiler Tests\cc-bat\typing.js:10: WARNING - actual parameter 1 of getData does not match formal parameter
    found : string
    required: Element
    0 error(s), 1 warning(s), 100,0% typed

    A long, nicely documented list of all possible annotations can be found in the Annotating JavaScript documentation. The Closure Compiler has type information for the most common JavaScript functions, including, for example, the W3C DOM bindings. In the externs part of the source code, you'll find the whole list.

    There are no build in extern files for common JavaScript libraries, so you have to search for one or create one yourself. There is a contrib directory in the Closure Compiler source code where you can find a couple of contributed extern files. One of them being a extern file for jQuery.

    Using externs only work when the warning level is set to VERBOSE. So an example batch/script file using typed externs is:

    java -jar compiler.jar --js data.js --externs jquery.externs.js --compilation_level ADVANCED_OPTIMIZATIONS --warning_level VERBOSE > data-min.js

    Even is you're not going to use the type checking feature, you should at least know about the @license or @preserve annotation. They both do the same thing: preserve the comment. This is very handy for copyright and license information.

    Windows users might be interested in a small script I wrote to minify a JavaScript file by just dragging it to this script. It already includes the Closure Compiler. Download compile-wsf.zip.

    Do you want more information? You can read more about the Google Closure Compiler on Google Code.

    Comments

  2. Report of the Dutch PHP Conference 2010

    For the third time, I visited the Dutch PHP Conference. Just as before, it was held in the RAI Convention Centre in Amsterdam. And as before, this two-day conference took place on Friday and Saturday. I always wonder why it's Friday and Saturday. Is it that if your employer or school doesn't let you go on Friday, you can at least go on Saturday? Or if your employer does let you go, you can trade in your work day, but you don't have to trade in your free Saturday? Being a free lancer, it's not really an issue for me, so I went both days.

    The conference has three tracks, so planning which presentations to see is a task on itself. There was also an unconference-track. Because during the whole conference there was always something interesting in the regular tracks and because you never know what the quality of the unconference-track will be, I didn't visit any unconference-track.

    Friday

    The conference started with a keynote from Kevlin Henney, editor of 97 Things Every Programmer Should Know. This book contains little pieces of wisdom written by different programmers. Kevlin took a couple of them and explained them. For example how to make estimates, how to have code reviews, why you shouldn't change code on the production server, why you should test, etcetera. Very informative, especially if you haven't already learned it the hard way.

    The first talk in a track was Designing for Reusability by PHP-contributor and XDebug-developer Derick Rethans. He talked about how to prevent hard dependencies, why singletons are bad for testing, the problems of ActveRecord and why traits will be a solution. Very intersting talk.

    The second talk was The Cake Is A Lie by Sebastian Bergmann. It was not a talk about how bad CakePHP is, but more a opinion about how bad generated code is. I couldn't really relate to his problems. I've worked on a large project in which the ORM-code was generated. In this project, we didn't change the generated code but instead we inherited from it and it was nice to work with. Sebastian did not present any alternatives.

    I was a bit dissapointed by The Art of Scalability, a talk from Lorenzo Alberton, and I think I was not the only one. He didn't talk about scalability from a technical point of view, but from a management point of view.

    In Under PHP's hood, Johannes Schlüter talked about the inner workings of PHP. He explained how the Zend engine and PHP Core are seperate things. He explained the difference between include and require and the -_once-variants. He also explained how references work and why you should keep having references to a minimum. He also showed how it can even hurt performance.

    Although I've never used GIT, I thought visiting Advanced Git from David Soria Parra might be interesting anyway. He adviced to make a branch for every task you do. And only rebase local branches, never pulled or pushed branches. I don't know what pulled or pushed branches are, but when I'm start using GIT, I know this tip :-) He said GIT doesn't work well on Windows because FAT doesn't support symlinks and execute-bits. Huh? Which Windows-user still uses FAT, except from portable storage? A strange assertion, I think.

    The talk Crash, Burn, Recover! from from Cal Evans showed Flex using Flash Builder. He build a recipe search app from scratch. Well, at some point it didn't work and he had to revert to a prepared app. He showed the power of Flash Builder and how you can build a program in one hour. The result can be deployed as a Flash swf-file or a AIR application. It was nice to see how Flex works.

    I didn't go to the DPC Conference Social that evening (with free beer!). Maybe next year.

    Saturday

    The second day started off with Security-Centered Design: Exploring the Impact of Human Behavior by Chris Shiflett. He showed, nicely illustrated, how humans are sensitive to certain kind of security attacks. He explained Ambient signifiers: how you can subconsciously direct your users. For example, in Tokyo they introduced different kinds of background music in the subway, depending on where you are. It turned out that as a result, many more people didn't forget to leave the train on time, resulting in a more profitable subway system (and, of course, happier customers). He showed the movie Person Swap and the Colour changing card trick, which shows how easy it is to deceive people. He also explained the Twitter Clickjacking Hack. Although it didn't really had much to do with PHP, it was a very good presentation.

    The next talk I visited was Async webservices with php and node.js by Sebastian Schürmann. He showed how asynchronous calls can improve performance and how it is done in PHP, compared to Node.js. He showed an example in PHP with Curl and an example in Node.js. The Node.js-example was much cleaner and easier. Other asynchronous API's in PHP are the Streams API and MySQLnd. He did a call to the developers of PHP to make asynchronous programming in PHP as easy as in Node.js.

    In the talk APC & Memcache the High Performance Duo, Ilia Alshanetsky explained how APC works and how you can use it as a persistent key-value store. He also compared it to Memcache and explained what the differences are. He also explained the advantages and disadvantages of igbinary: it's much faster and smaller, but not interchangeable with other languages. For me this was the most interesting and informative talk of this conference.

    After the lunch Ian Barber had a talk about local search engines: In Search Of... Integrating Site Search. He showed the pros and cons about the most popular onces: sphinx, swish-e, lucene, solr and xapian. He also showed the importance of a good stemmer.

    The last talk I visited was Plant Pyrus in your system - A guide to a plugin system by Helgi Þormar Þorbjörnsson. He explained how hard it is to develop your own plugin system which has to deal with dependencies. He worked on Pyrus, which you can integrate in your own software and should also become part of PEAR.

    The conference ended with a panel discussion about the future of PHP.

    Just like the years before, it was a very interesting conference with something for everybody. I'm quite sure I'll be there again next year.

    Comments

  3. Animate anything in jQuery

    I've been using jQuery for three years now in several projects and I love it. One of the many reasons I like jQuery is its extensibility. It's very easy to write your own plugin, your own CSS pseudo selector-like filter or your own easing function.

    What's less known is that you can also customize the animate function. Every jQuery-user knows you can animate all number-based CSS-properties. (Did you know you can also animate scrollTop and scrollLeft?) In this article I explain how to customize the animate function.

    Customizing the animation-function is not so hard and gives you a couple of features for free: think about the fx-queue, easing, callbacks and setting the duration.

    The first thing I wanted to animate was fading one color to another. It turned out John Resig himself already wrote a plugin to does just that: Color Animations.

    Unfortunately, it has a bug which makes the plugin unreliable. To initialize the animation, the plugin checks for fx.state being zero. I turned out that with consecutive calls, the animation can start without fx.state being zero and as a result, the animation not performing.

    My next quest was to rewrite the plugin to my liking. I fixed the bug. I added the missing borderColor, so to change the border color, it's not longer necessary to specify all four borders seperately. I also tried to keep the plugin very small.

    Take a look at the result: the Color animation jQuery-plugin.

    So how can you make your own custom animation? Just like the other jQuery-customizations, it's actually quite simple. Take for example the code to animate the background color:

    $.fx.step.backgroundColor = function(fx) {
        if (!fx.init) {
            fx.begin = parseColor($(fx.elem).css('backgroundColor'));
            fx.end = parseColor(fx.end);
            fx.init = true;
        }

        fx.elem.style.backgroundColor = calculateColor(fx.begin, fx.end, fx.pos);
    }

    Give $.fx.step a new property and assign it a function which will handle the animation. This function has one argument, say fx, which belongs to that one animation, so you can use it anyway you like. It has some properties already set. The most important are fx.elem, fx.end and fx.pos. In fx.elem you'll find the element on which the animation is applied, and in fx.end you'll find the end-value you want to animate to. fx.pos is a float, going form zero to one during the animation.

    In the example above, some initialization is done at the first call of the function in an animation. Here, the begin and end values are calculated. The rest of the functions assigns the calculated value to the CSS-property of the element. The calculation looks like this:

    begin + pos * (end - begin)

    If the animation is called with the end-value being a string, then fx.end will be that string. So when calling a.animate({color: "#ccc"}), fx.end will be "#ccc". However, if the string starts with a number then fx.end becomes that number. So when calling a.animate({myProp: "10px 20px"}), fx.end will be 10 and you'll lose the rest of the string. To get the whole string you should look at fx.options.curAnim.myProp instead of fx.end. In this last example, myProp is a self defined property.

    I thought a cool thing to animate is the new shadow CSS-property. A box shadow property consists of several parts: the color, whether it is inset, the x- and y-offset, the blur-radius and the spread-radius. All of these had to be taken into account when writing the plugin.

    The result is the Shadow animation jQuery-plugin.

    Unfortunately, this plugin does not work in Opera and Internet Explorer. Opera has a strange bug in which you can't request the current shadow-value. Internet Explorer is miles behind and doesn't support CSS-shadows. I expect the plugin will work in Internet Explorer 9.

    These two animation plugins are the first on my list of jQuery-plugins. It's my intention more will follow.

    Comments

  4. Faster websites with gzip compression

    One of the best ways to improve the user experience of a website is to make sure it loads fast. In a test done by Mozilla, it also improves conversion. Research by Google also shows advantages.

    One of the easiest ways to improve download speed is using gzip. Since many webbased files like html, css and javascript are text based, compressing them with gzip will result in more than 50% size reduction.

    Older browsers had some problems with gzip, so site owners where a bit reluctant to use gzip. Currently, all modern browers (even IE with the right service pack applied) have no problem with using gzip.

    Apache has a module called mod_deflate, which is easy to enable. For example:

    AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript text/css

    Mod_deflate will compress the content on the fly for every request. On a high-performance site, that's not really what you want. There are better ways.

    Compressing static content

    First you have to compress all text files. Typically, there are files ending with .html, .css, .js and .xml. When using Linux, make sure you have gzip installed. Don't just gzip the files you want to compress, because the original file will be gone. Better is to do this:

    gzip -nc myfile.css >  myfile.css.gz

    Disadvantage is, you can't do this with a bunch of files. I wrote a little script to fix this:

    echo -n $@ | xargs -d ' ' -I '{}' sh -c "gzip -n -c {} > {}.gz"

    Put it in a file, for example /usr/local/bin/zipit, make it executable: chmod +x /usr/local/bin/zipit and now you can use it to compress single files, but also files with wildcards. For example:

    zipit *.js *.css

    If you had lib.js and style.css, now you will also have the compressed versions: lib.js.gz and style.css.gz.

    Warning: you have to be careful here: uploading a new .js- or .css-file won't work! You have to run zipit again, otherwise the .gz-file with the old content might be picked up.

    In Windows, you can gzip your files in a batchfile. You can use the free 7-Zip to to that. For example:

    "C:\Program Files\7-Zip\7z.exe" a -tgzip style.css.gz style.css

    Let Apache serve compressed content

    There are several ways to let Apache serve the .gz-files. I prefer using mod_rewrite. I wrote the following configuration file to serve the gzipped files:

    <FilesMatch "\.html\.gz$">
      ForceType text/html
      Header set Content-Encoding: gzip
    </FilesMatch>

    <FilesMatch "\.js\.gz$">
      ForceType text/javascript
      Header set Content-Encoding: gzip
    </FilesMatch>

    <FilesMatch "\.css\.gz$">
      ForceType text/css
      Header set Content-Encoding: gzip
    </FilesMatch>

    RewriteEngine On
    ReWriteCond %{HTTP:accept-encoding} gzip
    ReWriteCond %{REQUEST_FILENAME} !\.gz$
    RewriteCond %{DOCUMENT_ROOT}%{REQUEST_FILENAME}.gz -f
    RewriteRule ^(.+\.js|.+\.css|.+\.html)$ $1.gz [L]

    It's quite low-level and therefore very clear. The rewrite rules checks the browser supports gzip, then it makes sure it's not already getting the .gz-file. As a last check it checks for the existance of the .gz-file. If that's the case, the url is rewritten to have .gz at the end. The FilesMatch-directive makes sure the content-type is correct and adds "Content-Encoding: gzip" to the http-header, so the browser knows it's gzipped content.

    Gzip and PHP

    Most server-side languages also support gzipped output. PHP has several ways to do that.

    If you're using ob_start to buffer the output, you can easily enable gzip-compression by using ob_gzhandler:

    ob_start("ob_gzhandler");

    A better way might be to enable gzip on all PHP-scripts. Just enable zlib.output_compression in your php.ini-file:

    zlib.output_compression On

    PHP has quite a lot of gzip-related functions. You can easily zip or unzip strings and work with gzipped files as they where normal files.

    Check gzip is working

    How can you be sure your content is compressed? Just open the Net-tab within Firebug and reload the page. Open the headers of the file you want to inspect. It will look similar to this:

    Date                Mon, 10 May 2010 14:19:06 GMT
    Server              Apache/2.2.4
    Vary                accept-encoding
    Last-Modified       Thu, 29 Apr 2010 09:23:59 GMT
    Etag                "9043e7-856-afc129c0"
    Accept-Ranges       bytes
    Content-Length      2134
    Cache-Control       max-age=172800 proxy-revalidate
    Content-Encoding    gzip
    Content-Type        text/html

    When you see "Content-Encoding gzip" as in the example above, you'll know it is compressed. Or use YSlow or Page Speed and look at the A's you'll get!

    Comments

  5. Introduction

    I'm a technology-enthiousiast. When I see a new technology, I want to try it, use it, master it. Unfortunately, a day only has 24 hours, from which a large part is gone when asleep. So I had to make some choices. When I saw the Internet for the first time, I was hooked. Mind you, that was in a terminal, browsers didn't exsist at that time. At least I could focus on a manageable set of technologies.

    But not for long. Java, Flash, JavaScript, CSS, and recently stuff like Node.js and WebGL to name a few, makes it impossible to grasp it all. Now I'm settling a bit with HTML(5), CSS, JavaScript, PHP and MySQL. Pretty much mainstream, but as a freelancer, you can't go too wild.

    In my work or with my own little projects, I sometimes stumble on some interesting uses of technology. Or I have an opinion about something. Or I think some technology is underexposed. I want to write them down, for use for everybody. And maybe, sometimes, as a reference for myself. So I started this blog. Let me know what you think about it. And when you see a spelling error, you can also let me know, English is not my first language.

    A bit about me: I almost lived my whole life in Amsterdam. A couple of years ago I moved to Hilversum, a village 20 minutes from Amsterdam. I live there with my wife Jolie and two daughters Leora and Rosalie.

    Comments

<>