HTML5 framework with metacircular code

Most of us probably is used to think that we can program only with code of the given language. You may guess that I’m going to say that there are other options. The alternative I’d like to talk about is not graphical programming with blocks, Excel or GUI configurations. I rather have on mind the tool that allows to feed Virtual Machine with instructions that in first place are produced from source code. In case of MetaES those instructions are AST. MetaES expects to see AST compatible with Mozilla Parser API. Such AST can be trivially created from pure code running esprima.parse(code), but this is not the only way.

Prepare the ground

Do you know AngularJS and its syntax, like:

<div ng-repeat="item in items">
    {{item}}
</div>

?
Probably the answer is ‘yes’. Authors of AngularJS created their own language (kind of JavaScript subset), grammar and parser. Probably it aims to be compatible with WebComponents specification of DOM Expressions. Let’s invent something similar, but this time based on ES6 syntax without adding any special syntax for filters or iteration. How about:

// html part
<div id="app">
    <h1>
        {{"hello "+ name + "!"}}
    </h1>
    <div script="for(let i of array)">
        {{"element: " + i}}
    </div>
</div>

// js part
run(document.querySelector("#app"), {
    array: [1, "test", 3, "Hello", "world!"],
    name: "Visitor"
});

and run would be a function of the signature:

function run(element, env);

and the result we’d like to see is the:

<div id="app">
   <h1>hello Visitor!</h1>
   <div script="for(let i of array)">element: 1</div>
   <div script="for(let i of array)"></div>
   <div script="for(let i of array)">element: test</div>
   <div script="for(let i of array)"></div>
   <div script="for(let i of array)">element: 3</div>
   <div script="for(let i of array)"></div>
   <div script="for(let i of array)">element: Hello</div>
   <div script="for(let i of array)"></div>
   <div script="for(let i of array)">element: world!</div>
   <div script="for(let i of array)"></div>
</div>

I feel that it looks very readable and clear. Ok, great. Everything is settled, we’re ready for implementation.

The implementation

Let’s use top-bottom approach while designing and analysing this code.

function run(element, env){
    // (...)
    var $compiled = compile(element);
    $compiled.node.innerHTML = '';
    metaes.evaluate($compiled.source, env, {interceptor: interceptor});
}

The raw DOM should be at first preprocessed somehow, let’s use compile name which nicely describes that process. Then we remove contents of the DOM element (.innerHTML= '';). The last part is running MetaES using computed source in $compiled.source using special interceptor function. The compile function has the following implementation:

function compile(node) {
    function addSources($node) {
        // (...)
        return $node;
    }
    function compileInner(node) {
        return {
            type: 'script' in node.attributes ? 'block' : 'plain',
            node: node,
            childNodes: toArray(node.childNodes).map(function (node) {
                if (node.nodeType === Node.TEXT_NODE && node.textContent.indexOf('{{') >= 0) {
                    return {
                        type: 'expression',
                        node: node
                    };
                } else if (node.nodeType === Node.ELEMENT_NODE) {
                    return compileInner(node);
                }
            }).filter(identity)
        };
    }
    return addSources(compileInner(node));
}

First part is compileInner. This is a helper function used for recurrent calls. What is does, is checking if the node is TEXT_NODE, like {{x}} or ELEMENT_NODE, like <div script="...">. Depeneding on that it goes to do the same on the node’s children or it doesn’t. The final result is tree structure with subsequent references to descantant DOM nodes with their type information:

  • block – for nodes with @script attribute,
  • plain for nodes without @script
  • expression for text nodes.

There are also helper functions, just pasting them here for clarity:

function toArray(arrayLike) {
    return [].slice.call(arrayLike);
}
function identity(x) {
    return x;
}
function clone($node) {
    return $node.node.clone();
}

The second and the last part of compile is addSources:

function addSources($node) {
    function inner($node, startIndex) {
        var buildedSource;
        if ($node.type === 'expression') {
            $node.source = buildedSource = $node.node.textContent.match(/{{(.+?)}}/g).map(function (result) {
                return result.match(/{{(.+)}}/)[1] + ';';
            }).join('');
        } else if ($node.type === 'block' || $node.type === 'plain') {
            buildedSource = $node.type === 'block' ? $node.node.attributes.script.value + '{\n' : '';
            $node.childNodes.forEach(function (child) {
                buildedSource += inner(child, startIndex + buildedSource.length) + '\n';
            });
            if ($node.type === 'block') {
                buildedSource += '}';
            }
            $node.source = buildedSource;
        }
        $node.range = [
            startIndex,
            startIndex + $node.source.length - 1
        ];
        return buildedSource;
    }
    inner($node, 0);
    return $node;
}

Not going much into details, it transforms:

<div id="app">
    <h1>
        {{"hello "+ name + "!"}}
    </h1>
    <div script="for(let i of array)">
        {{"element: " + i}}
    </div>
</div>

into JavaScript code

"hello "+ name + "!";

for(let i of array){
"element: " + i;
}

This is the code that is executed in MetaES and observed using interceptor to make DOM manipulation. addSources is very fragile right now and not widely tested, so it is probably buggy and poor in terms of supporting the complexity of the DOM. About to interceptor, let’s take look on it:

function interceptor(e, value, env) {
    if (supportedTokens[e.type]) {
        supportedTokens[e.type](e, value, env);
    }
}

Coudn’t be simpler? 🙂 What about supportedTokens?

var supportedTokens = {
    Program: function (e, value, env) {
        e.body.forEach(function (e, index) {
            e.$nodeData = {
                node: $compiled.childNodes[index],
                parent: $compiled.node
            };
        });
    },
    ExpressionStatement: function (e, value, env) {
        if (e.$nodeData && value) {
            var node = e.$nodeData.node.node.cloneNode();
            node.textContent = value;
            e.$nodeData.parent.appendChild(node);
        }
    },
    ForOfStatement: function (e, value, env) {
        e.body.$onEnter = function () {
            var node = e.$nodeData.node.node.cloneNode();
            node.innerHTML = value;
            e.$nodeData.parent.appendChild(node);
            return {
                parent: node,
                childNodes: e.$nodeData.node.childNodes,
                $nodeData: e.$nodeData
            };
        };
    },
    BlockStatement: function (e, value, env) {
        if (e.$onEnter) {
            e.$nodeData = e.$onEnter();
            e.body.forEach(function (eInner, index) {
                eInner.$nodeData = {
                    parent: e.$nodeData.parent,
                    node: e.$nodeData.childNodes[index]
                };
            });
        }
    }
};

Right now there are four AST nodes supported: Program, ExpressionStatement, ForOfStatement and BlockStatement. $nodeData is a custom property which indicates that interceptor should do something special with this node once visited.

Program binds $nodeData to each of its child tokens, ExpressionStatement simply takes value from MetaES and writes it into DOM node, ForOfStatement binds function to its body node that will be called every time the body of ForOfStatement is visited. This function creates new DOM node for each block enter and binds new $nodeData values to its children.
The naming conventions probably could be better, but this is a quck’n’dirty example to show what is possible to achieve with relatively low cost of development.

Summary

Looking at the final result it came to my mind, that it would be possible to create simplest version of such library even without MetaES, but using just ES5, esprima and some kind of code instrumentation. But step debugger that has the references to DOM attributes and DOM textContents, handling exceptions with preserving the valuable stack or handling SyntaxErrors or ReferenceErrors would become to be tricky using only code instrumentation. That’s why having full interpreter gives the whole power into programmers hands.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: