Monday, December 12, 2011

Adaptative web design: loading resources dinamically

Adaptative web design (also known as responsive web design) is a tecnique that adapts web pages to the devices (desktops, tablets and phones).

This tecnique uses many differents technologies. In this blog post I don't want to explain every detail of this tecnique but it could be useful reading an in-depth tutorial:
http://webdesignerwall.com/tutorials/responsive-design-with-css3-media-queries

The remaining Issue
Using media queries to adapts a web page to mobile has a huge drawback.  We load to a mobile phone every bit of the web page and then we hide text and shrink full size images and video. The results is great, but the speed ? Is it possible to avoid downloading all this unused stuff ?

I think I found an elegant solution with these scripts https://github.com/sithmel/jQuery-decomment .
They are very simple (only 31 LOC) and are designed to work together. They are doWhenVisible and decomment.

An example:
Our Web page contains a very big and complex slideshow but in the mobile version of our web site we'll hide it using css (is very big and heavy to load).

HTML

<div class="bigslideshow">
<!--
    <img src="bigimage1.jpg" />
    <img src="bigimage2.jpg" />
    <img src="bigimage3.jpg" />
-->
</div>

CSS

@media screen and (max-width: 650px) {
    .bigslideshow{
       display: none;
    }
}

Javascript

$.doWhenVisible('.bigslideshow', function (){
    this.decomment(); // remove the comments
    this.jcarousel(); // initialize the slideshow
});

Very easy ! doWhenVisible execute a callback (only once) when a DOM element becomes visible (it checks every time the page is resized).
Decomment obviously removes comments. Removing comments is just like adding dinamically DOM nodes. The browser reacts downloading the resources and rendering the whole.

We can go even further using a script loader to load our js only if effectively used (we use require.js here):

$.doWhenVisible('.bigslideshow', function (){
    require(["jquery.jcarousel.js"], function() {
    //load jcarousel
        this.decomment(); // remove the comments
        this.jcarousel(); // initialize the slideshow
    });
});
Right now I'm very proud of these little scripts!

Tuesday, October 11, 2011

How to write a good Javascript widget


Writing a Javascript widget is almost easy but I learnt at my expense that it may be tricky. So I wrote some simple rule that help to do the job well.

1 - keep in mind: progressive ehnancement
I already wrote about it, just a recap: write html semantically correct (your mark up MUST have sense even WITHOUT Javascript)
2 - do not use inline styling (use CSS)
When your script are about to create your widget, find the outermost element and give to it a specifical class. Then do not attach style directly to the DOM elements, instead use CSS.
In this way you can customize the style of your widget without touching a single line of Javascript.
3 - avoid FOUC
Usually you modify the DOM after it is loaded. That way you may see the page while change. This is called FOUC (flash of unstyled content). You can avoid this hiding the DOM nodes you are about to change (using CSS) and show after (using Javascript).
Just a suggestion: you can use a noscript tag to show what you have hidden before in case Javascript is not enabled.
4 - use a library
A library like jQuery helps a lot with the DOM manipulation, It helps to forget the differences between the browsers. You can find a thorough guide to make a plugin here.
5 - do not calculate size and position during the widget creation
You can't assume what part of your page will be displayed and when. So you can't calculate position and size of elements at creation time. An example: I used this spinner widget on my application (http://btburnett.com/spinner/example/example.html). The widget calculates the input dimension and adds a lot of on-line styling for the little arrows. But in my application display the widget inside an hidden tab and all the size calculation returns wrong results.

Javascript script injection

Javascript script injection is a tecnique that has two main uses. You can use to load and execute a script in asynchronous way and to build a sort of ajax call without using the xmlhttprequest object.
How the browser work
Before showing how this tecnique works, let me explain how the javascript is loaded and executed in the browser.
The main concept is that there is multiple threads for downloading the resources (a limited number for every subdomain) but only one thread of execution renders the page and execute the javascript code.
The first thing a browser does when loading a page is starts to parse the page and create the DOM tree. When it meet an inline script this is executed right away (even if the DOM tree is not completely loaded). When the parser met an external resource, it stops and download them (mainly stylesheets, and external javascripts. Images are loaded asynchronously).
It's usually a good idea to put your script inside a function executed when the DOM is fully loaded, for example the onload event or jquery's ready event.
The external resources are executed as soon as they are loaded, in the order they appear in the page. Since the execution of code and page rendering use the same thread, during the scripts execution and  loading the page stops to render and become unresponsive.
Download scripts asynchronously
Javascript injections permits to load a script asynchronously and execute just after loaded. This technique use DOM manipulation to inject a script node in the DOM tree:
var script = document.createElement('script');
script.setAttribute('src', 'script.js');    // load the script
document.getElementsByTagName('head')[0].appendChild(script);
This technique has some limitation: if you use it to download more than one script the order execution is not guaranteed. Some browser execute the code in order, others execute the code as the download is finished.
There are many Javascript libraries trying to generalize this approach: give a look to YEPNOPE, LABJS, REQUIRE.JS (they use other techniques as well).
JSONP
Another useful way to use this technique is to inject a call to a URL that respond with a JSONP.
var script = document.createElement('script');
script.setAttribute('src', url + '?callback=handler');
// load the script
document.getElementsByTagName('head')[0].appendChild(script); 
function handler(data){
//do stuff with data
}
A JSONP is a JSON wrapped inside a function call. For example:
handler({...various data...});
The script tag execute the code (using our handler). This is a clever hack but it has some drawbacks:

  • the way the callback is defined pollutes the namespace (It must be a window attribute)
  • the data can be sended using GET method and urlencoded (a limited amount of data)
  • downloading a file this way triggers many "busy indicators": the various way the browser has to say "I'm not finished yet". Read something of Steve Souders to learn more ...

A more "appropriate" tecnique to make cross site ajax request is CORS (but is not yet widely used).
Not only Javascript
I read an interesting post of Stoyan Stefanov who investigates the injection of stylesheets .
To say the truth the browsers load dinamically every type of external resources ....  this is how the browser works ... and it's amazing :-)

Monday, June 27, 2011

jquery.spacetree: an example of Progressive Enhancement

I have recently developed a Jquery plugin: Jquery.spacetree. It uses the Javascript Infovis Toolkit to display a spacetree based on an html subtree.

Let me show an example. This is a nested list:
<div id="tree">
  <div id="root">Root content</div>
  <ul>
    <li>
      <div id="child1">first child</div>
      <ul>
        <li>
          <div id="subchild1">subchild element 1</div>
        </li>
        <li>
          <div id="subchild2">subchild element 2</div>
        </li>
      </ul>
    </li>
    <li>
      <div id="child2" class="selected">child element 2</div>
    </li>
    <li>
      <div id="child3">child element 3</div>
    </li>
    <li>
      <div id="child4">child element 4</div>
    </li>
  </ul>
</div>
 
It's a perfectly valid html. This is the result:
Root content
  • first child
    • subchild element 1
    • subchild element 2
  • child element 2
  • child element 3
  • child element 4
The plugin is called in the usual "ready" event:
$(document).ready(function (){
    $('#tree').spacetree('#spacetree');
});
Et voilĂ :






So, what's the point ? The point is that, even without Javascript, the semantic of the list is well defined. The spacetree plugin, in this case, adds only a better visualization.

Sunday, June 12, 2011

The art of progressive enhancement

For some developer javascript is just a toy for adding fancy effects to a web page. This approach brings to poor developing practices. Inline javascript is one of those. But I don't want to point out on this now.


In the last few years the javascript community develop some simple design principle that It's better to keep in mind.

Separation of concerns
One of the most important design principle is separate the semantic (html), style (css) and behaviour.
The consequencies of this statement are to keep our html free of logic and style and enforce the semantic inside the html.


and consequences
Another important principle is called "progressive enhancement" and is a direct consequences of the previous. This could be translated in "keep the basic functionality in html and use Javascript to add usability".
The main reason to follow this principle is to keep untouched the semantic of the document.
This is better both for accessibility and search engine optimization.


An example of a right use of progressive enhancement is fancybox. This jquery plugin implements a lightbox effect when a user clicks on an image.


You can link an external image like this one:


<a id="single_image" href="image_big.jpg"><img src="image_small.jpg" alt=""/></a>

or you can use an internal link:


<a id="single_image" href="#image_big"><img src="image_small.jpg" alt=""/></a>
...
<div id="image_big"><img src="image_big.jpg" /></div>

The semantic is clear and simple and the page works even without javascript (without the effect obioviously).


The next example shows the wrong approach.

This is jquerytools overlay markup (taken from an example on the jquerytools website)


<img src="thumbs/barcelona-pavilion.jpg" rel="#mies1"/>

<div class="simple_overlay" id="mies1">

 <!-- large image -->
 <img src="photos/barcelona-pavilion-large.jpg" />

 <!-- image details -->
 <div class="details">
  <h3>The Barcelona Pavilion</h3>

  <h4>Barcelona, Spain</h4>

  <p>The content ...</p>
 </div>

</div>

In this example the img element is connected to a div through a rel attribute (rel="#mies1"). But the image tag purpose is not to link a resource. And this is not what the rel attributes is intended for.
The rel attribute (http://www.w3.org/TR/html4/struct/links.html) is used to specify the relations between two resources and not the resource itself.
This markup semantically doesn't make sense and without Javascript is even useless.


Progressive enhancement tricks

As a matter of fact the browser renders the page while is loading the DOM tree. The scripts see the DOM node right after the loading.
When you use javascript to enhance a web page you often add markup and inline styles to elements.
Thus, following this approach, you could see the page while changing.

For example in the previous examples you can see the overlay content before the javascript manage to hide it.


You can avoid this hiding the overlay container in the css (using display:none).


For other effect you can hide like the previous example and then shows only after adding the style and markup (for example with .show() method of jquery).
Another nice effect is to remove the element opacity in the styleshhet and, at the end of the script, animate the opacity with a fade-in effect.


This approaches have a drawback: if someone is not using javascript (and search engines) the area will be empty. This issues can be easily fixed adding a css rule in a noscript tag (display: block !important).

Tuesday, June 7, 2011

Javascript Unit testing: how to

Javascript, from its humble origin, it was perceived as a toy but now has gained the status of "serious language". Every "serious language" needs "serious tools" to test and document the code.
In the last months I have done some interesting experiences unit testing Javascript that is worth sharing.
I have used Qunit and MockUpHTTPServer (to fake ajax requests).

Setting Up Qunit
It's very easy:
First of all It's better to wrap all the tests stuff in a directory (for example "tests"). Place this folder on the same level of the library you want to test. Then copy inside the "tests" directory qunit.js and mock.js.
The second one is optional: It is useful to test AJAX call.

Inside the test directory you need a simple html (test.html is a good name). The html is mostly boiler plate code:
<!DOCTYPE html>
<html>
<head>
 <title>QUnit Test Suite</title>
 <link rel="stylesheet" href="qunit.css" type="text/css" media="screen">
 <script type="text/javascript" src="qunit.js"></script>
 <script type="text/javascript" src="mock.js"></script> <!-- optional: only for ajax testing -->

 <!-- support -->
 <script type="text/javascript" src="jquery-1.4.4.js"></script>
 <!-- Your project file goes here -->
 <script type="text/javascript" src="../mylib.js"></script>
 <!-- Your tests file goes here -->
 <script type="text/javascript" src="test.js"></script>
</head>
<body>
 <h1 id="qunit-header">QUnit Test Suite</h1>
 <h2 id="qunit-banner"></h2>
 <div id="qunit-testrunner-toolbar"></div>
 <h2 id="qunit-userAgent"></h2>
 <ol id="qunit-tests"></ol>

    <div>
    <!-- Your can place here some markup to be tested -->
    </div>
</body>
</html>

This html contains the dependencies (in my case jquery) and the code to test (../mylib.js).
Obviously you need to copy your dependencies here (you can also use a CDN).
Now all the pieces are on the right place. You can launch the tests opening the page test.html. If you use chrome/chromium the page works without problems from your file system.
In Firefox you have to access to the page through a web server. The simplest way is to use the simplewebserver class embedded in the standard python library.

cd mydir
python -m SimpleHTTPServer

and then open a browser on

http://localhost:8000/tests/test.html

Until now we haven't write a single test (to tell the truth we haven't write a single line of mylib.js either). Let's develop some tests and simple functions in an TDD fashion.

Writing simple tests
Writing tests is simple. Just open test.js and write:

test('Example', function (){
    var result = mylib.add(3,5);
    ok(result == 8,'3 + 5 is equal to 8');
});

If you run the tests they will fail because there isn't a function named mylib.add. Let's create it in mylib.js:

var mylib = {
    add:function (x,y){}
};

Now the test runs but fail. Finally finish to implement the function:

var mylib = {
    add : function (x, y){
        return x + y;
    }
};

Now the test pass: Hurrah !

If you want, you can wrap a group of tests into a module. You have to place this row between the groups of tests.

module('Module A');
test('Test 1', ...
test('Test 2', ...
test('Test 3', ...
module('Module B');
...

In the previous example the function "ok" was an assertion. When an assertion fails the test show it colored in red. It means that something goes wrong !
The last argument of every assertions is a string that describe what the test is for.
Qunit gives to us various kind of assertions:

  • ok(condition, string) - pass the test if the condition is true
  • equals(result1,result2, string) - pass if result1 and result2 are equals.
We can rewrite our example using "equals":

test('Example', function (){
    var result = mylib.add(3,5);
    equals(result,8,'3 + 5 is equal to 8');
});

If you compare two mutable objects with "equals" you must pay attention: if the objects contain the same content but are actually two different object they are not considered equals.
For example this test will fail:

    var a = {};
    var b = {};
    equals(a,b,'Two different object are not equals so this test fail');

while this test succeed:

    var a = {};
    var b = a;
    equals(a,b,'a and b refers to the same object');

If we are interested in the contents we can use the "same" assertion.

same(result1, result2, string)

This assertion check recursively for equality so this assertion succeed:

same( {a: 1}, {a: 1} , 'passes, objects have the same content');

Another difference between "equals" and "same" is the use of the equality operator: "equals" use the '==' operator while "same" use the '===' operator.
The previous coerce the type of operands. So 0 == false evaluate as true but 0 === false evaluate as false.
Now we can test various functions but ... how can we test interactions between our scripts and the DOM ?

Testing DOM elements
Testing DOM is straighforward. Let's add some mark up to the test.html. For example:
<div id="container" />

Then we write the test in test.js

jQuery(document).ready(function (){
    test('Hello world', function (){
        mylib.hello('world');
        var text = jQuery('#container').text();
        equals(text,'Hello world','The text is Hello world');
    });
});

If you want to test the DOM you have to be sure the dom is entirely loaded so you must wrap your test in the ready event.
Ok now the code (mylib.js):

var mylib = {
    add : function (x, y){
        return x + y;
    },
    hello: function (text){
        jQuery('#container').text('Hello ' + text);
    }
};

The test should pass.

Test asynchronous events
Javascript in your browser run inside an event loop. Scripts are most frequently registered as callback that run when some event trigger.
You can often predict when an event trigger. For example:

mylib.js
var mylib = {
    ...
    click_container: function (){
        jQuery('#container').click(function (){
            jQuery(this).text('already clicked');
        });
    }
}

test.js
jQuery(document).ready(function (){
    test('Click the container', function (){
        jQuery('#container').click(); // trigger the event
        var text = jQuery('#container').text(); // read the text on the dom
        equals(text,'already clicked','Change the container's text');
    });
});

There are cases  you can't predict when an event will trigger. Every time the event is triggered by the browser. For example: functions scheduled with setInterval and setTimeout and AJAX callbacks.
(Pay attention !!!! this case applies also for animations: they are usually done with setInterval or using requestAnimationFrame event).
Qunit has a special testcase called asyncTest. "asynctest" schedule the test to run when the function "start" is invoked:

    asyncTest('Hide with an animation', function (){
        mylib.hello_hide();
        setTimeout(function (){
            ok(jQuery('#container').is(':invisible'),'Hello world is hidden');
            start();
        },400);
    });

    asyncTest('Show with an animation', function (){
        mylib.hello_show();
        setTimeout(function (){
            ok(jQuery('#container').is(':visible'),'Hello world is visible');
            start();
        },400);
    });

The test function call our functions. Then it waits until the animation ends (we are using setTimeout). At the end we call "start" so Qunit can go ahead with the tests.

Let's implement the hello_hide function:

var mylib = {
    ...
    hello_hide: function(){
        jQuery('#container').fadeOut();
    },
    hello_show: function(){
        jQuery('#container').fadeIn();
    }
};

Testing ajax
Testing AJAX is the most trickiest part. Not only the callback is called asynchronously but it interact with the server. The number one rule of unit testing is avoid interaction between the code we are testing and the the rest of the program.
We can solve this issue using a mockup object and substitute it to the XMLHttpRequest object used for AJAX requests. This is a job for MockHttpRequest (https://github.com/philikon/MockHttpRequest).
On top of our test.js we put this snippet:

//setting up the mock ups
var request = new MockHttpRequest();
var server = new MockHttpServer();
server.handle = function (request) {
    request.setResponseHeader("Content-Type", "text/html");
    request.receive(200, "hello world from ajax!");
};
server.start();

This piece of code create a fake XMLHTTPRequest object (request). The "server" object handle ajax call and the method start substitute the original XMLHTTPRequest object with the mock (there is also a method stop to restore the original object).
Let's write the test:

...
    // test using timeout
    asyncTest('Example ajax', function (){
        mylib.example_ajax('/hello');
        setTimeout(function (){
            equals(jQuery('#container').text(),'hello world from ajax!','contains hello world from ajax');
            start();
        },200);
    });
...

This is the implementation:

var mylib = {
    ...

    example_ajax: function(url){
        jQuery('#container').load(url);
    }
};


Useful tricks
If the function you are testing has callbacks it can be used instead setTimeout. For example let's test the jQuery.ajax function:

    asyncTest('Example ajax with callback', function (){
        jQuery.ajax({
            url:'/hello',
            success:function (data, textStatus, XMLHttpRequest){
                equals(data,"hello world from ajax!",'success ajax');
                start();
            }
        });
    });

It's possible to configure your mock up handler to simulate various kind of response (put a glance over the docs for more informations):

//setting up the mock ups
var request = new MockHttpRequest();
var server = new MockHttpServer();
server.handle = function (request) {
    request.setResponseHeader("Content-Type", "text/html");
    switch (request.urlParts.file){
        case 'hello':
            request.receive(200, "hello world from ajax!");
            break;
        case 'broken':
            request.receive(404, "Nothing here");
            break;
        default:
            request.receive(200, "Default");
    }
};
server.start();

Epilogue
Just a couple of rows to explain why unit testing and why is better to write the tests and the code at the same time.
- you can't trust in untested code
- you can refactor your code and launch the test to be sure you haven't broke the code
- code simple to test is usually simple to read and more modular than untested one