Sergi Mansilla

Virtual list in vanilla JavaScript

2013-05-12T00:00:00+02:00

Let's say you need to show a scrolling list with millions of rows, or even a reasonably big list with visually complex rows, each one composed by several DOM elements or CSS3 effects.

If you try to render this the naive way, for example by appending rows into a DOM container with the CSS overflow property set to scroll (or auto), you will most likely run into performance issues. This is because all the items in the list are cached, thus your memory consumption will certainly go up, and since all the DOM nodes composing the rows of the list are appended in the document, the browser is trying to properly render them all making your CPU pretty busy as you scroll. If these rows change any style say, on hover, you will trigger a reflow and the experience will be even more sluggish.

Many websites and apps have run into this issue before and solved it using virtual renderers, which trick the viewer into thinking that she is looking at a massive list, when in reality only the items that are currently in the viewport are being loaded and rendered. A popular application that does that is the ACE editor, which can load a massively big source code file while keeping scrolling snappy using this technique.

Anyway, I needed a virtually rendered list and I couldn't find readily available components in simple JavaScript that I could use (and that I liked), so I created my own: virtual-list. Virtual list is a short and simple piece of code that doesn't depend on anything else, so you can drop it into any website and use it right away.

For example, here you have a 1 million row list:

Each of these rows is generated on demand, not wasting any memory, and the list never has to manage more than ~30 rows at a time, no matter how long the list is in reality.

Please read more about how to use it in the project's README and report any bugs or improvements you may find if you happen to start using it in your projects.

Implement cursor-swiping in half an afternoon

2013-02-21T00:00:00+01:00

If you haven't seen it yet, I suggest that you take a look at this concept video that made the rounds on the internet some months ago:

I would pay to have this in my phone. Only, money wouldn’t help since I own an iPhone, and a developer has simply no way to access an internal component like the phone keyboard.

Well, it only took a small part of an afternoon to implement a proof of concept of cursor-swiping in Firefox OS. And even if this swipe-selection implementation is only a prototype, it’s perfectly functional and I've already gotten used to its convenience. See a rough video I made here:

You can check out the code for it in Comoyo's Gaia branch on GitHub.

I can't even express how amazing it is to hack the OS of the phone you use and adapt it to your needs in such an easy way and with such a fast development cycle. It makes me excited to think about what will happen once all these JavaScript developers discover that they can hack their phone and contribute to it using their favorite language.

Disclaimer: This implementation is in the Gaia fork that I use and I have no idea of whether it will make it into the actual OS eventually, since there are obvious UX implications that might be uncomfortable for some.

The promise of Firefox OS

2013-02-09T00:00:00+01:00

"But how is it going to beat Android or iOS?”

That’s the reaction many people have when I tell them that I am working on Firefox OS, the new mobile operating system from Mozilla. It is a logical reaction. After all, we live in times where every major software company and its mother is releasing a mobile platform, struggling to lure developers into their new proprietary environment, APIs, libraries, etc. And indeed, many of these companies barely make it or don’t make it at all.

But Firefox OS will not be directly battling against other mobile platforms. Its main objective is to change the way the world develops mobile apps, and even in the unlikely event that Firefox OS itself disappears in the process, if web-apps become mainstream, it will have succeeded.

The fact that any website is a potential app can’t be underestimated. By tapping into extremely popular and flexible technologies such as HTML5, CSS3 and JavaScript, Firefox OS instantly promoted millions of web and JavaScript developers into app developers. All they have to do is download a free simulator addon (and not even that is strictly necessary if your app is not going to use phone APIs). Developers already know the browser environment and the tools, and there’s no need to learn any new language or framework.

I hear some of you, already. Just when you were over with that mess that it is to manipulate the DOM and that sneaky JavaScript language. Just when you learned to love the highly architected Android classes and managers or iOS’s beautiful method naming, why would we be back to that mayhem that is writing web applications? Didn’t we agree that HTML was not, after all, good enough for making real and performing apps?

Well, that might have been true some time ago, but we live in a brave new world now. Several approaches exist for developers to develop solid and real-world web applications, in the form of high-quality frameworks. At Telenor/Comoyo, where I work, we are leaning towards using AngularJS for building our apps, but there are multiple well-thought, reliable frameworks that build on years of existing knowledge of modern application development. And if your problem is with JavaScript as a language, you can already use a myriad of languages that reliably compile to it. Do you come from a Java background? You’ll probably like Dart, from Google. More of a functional kind of developer? Try ClojureScript, which is an impressive, well-maintained and well-performing implementation of Clojure on top of JavaScript. Coming from Ruby? You’ll be almost at home with CoffeeScript. You get the picture.*

While some other mobile vendors such as Blackberry also provide ways to develop apps for their systems using HTML5, Mozilla is going one step further by pushing hard to standardize the WebAPI through the W3C, guaranteeing that your app will work in any device that follows the WebAPI standard.

In my humble opinion, this puts some sense into the madness that developing for mobile devices has become, in which the developer has to know several languages, frameworks and APIs, not to mention pay a developer fee in order to build apps in some cases. It really feels like a step from the open web philosophy right back into the 90s, ridden with vendor lock-ins but without the good music.

Mozilla has a good track record of looking after the web, and it is a trusted company by its users. In the past they played a big role in freeing us from browser monopolies and pushing for better web standards, a movement that browsers like Chrome hooked onto, contributing to a better, faster and more accessible web for everybody. We should strive to do the same with our mobile environments. Fewer walled gardens, more standards and openness.

That is the promise of Firefox OS.

* Although hey, it wouldn’t hurt to learn a bit of JS to know what’s going on under the hood, after all, it is a powerful language that will only get better with the upcoming ES6.

Moving on

2013-01-29T00:00:00+01:00

A little bit more than two years ago I was sitting on a meeting room, listening to Ruben Daniels and Rik Arends, the two founders of the company I was considering to join, Cloud9 IDE. These were two of the smartest, most inspiring people I’ve met, and they were working hard on a ground-breaking concept: a full-fledged development environment that run in the browser, and in the cloud. At the time, that sounded wildly adventurous, to say the least. But the challenges were so many, the concept so revolutionary and the people so inspiring that I joined the company two weeks after that conversation.

What followed were two exhilarating years in which we got funding, surpassed almost every technical limitation that got on our way, got a huge user base and learned (many times the hard way) about software development, startup world and companionship.

Today Cloud9 IDE is by far the most advanced online IDE in the world, and what many told us that would not be possible (the DOM will never be fast enough for an editor, you won’t be able to implement real-time code analysis/refactor, developers won’t move their code to the cloud, etc.) is now a reality.

I am extremely proud to have been part of this great adventure, which has definitely made me a better developer and human being. I have no doubt that Cloud9 will continue improving and taking the development wold by storm, because they have the most talented, visionary and experienced team I’ve worked with behind.

But the time has come for me to move on. I have officially accepted a position at Telenor to work on core Firefox OS development, Mozilla’s new mobile operating system based on the open web. It is a very exciting project, and extremely different in technological challenges and userbase demographics compared to Cloud9.

Whatever the future may bring, Cloud9 will always have a special place in my heart, and I will keep using and evangelizing it on my day-to-day work.

Nodejsconf in Italy

2012-11-12T00:00:00+01:00

Last weekend I was at the Node.js conference in Brescia, Italy. Like every year the organizers managed to stage a great conference with really good talks. Even if I was there only for a lightning visit (less than 24 hours between leaving Amsterdam and being back), I had tons of fun and the talks I saw were really top-notch.

As a side note, I’m continually impressed by how much hacker spirit there is in Italy. There was lots of Arduino, BeagleBone boards and the real excitement of building stuff in Node.js that nobody has built before. Check out the talks from Simone Chiaretta or Matteo Collina to see an example of what I mean, when you have some time.

I was a speaker in the conference too, and my talk was about dependency handling and modularization in our codebase. Perhaps not as exciting as ROVs controlled by Node.js but this is one of the important parts of the architecture of Cloud9 IDE, where the ever-growing amount of code and functionality has to be extendable while keeping the dependency graph complexity low.

During my talk I recorded the slides with voice over, and you can see the video below. Unfortunately, the image and the sound of it are in a very low quality, but I think it is enough to get a rough impression of the talk.

Hopefully there will be videos of all the conference talks soon!

My talk at DynCon 2011

2011-04-04T00:00:00+02:00

The video with my talk about Cloud9 IDE at DynCon 2011 is already out! It’s quite long, but definitely worth it if you are interested in the future of the IDE, since I had the chance to answer many questions about it during the talk. About half of the video contains demo and questions from the audience, as it was quite interactive.

Also, I want to thank Peter Svensson for creating such a wildly interesting conference and being a really great host and organizer. It is a very, very hard job that he pulled off amazingly well.

This talk was especially interesting to me because although the audience included mainly professional developers, most of them weren’t using JavaScript as their main language (as is often the case in our talks). It was fun to show them the power of JavaScript. And as always when you add people with different backgrounds to a discussion, ideas and good questions quickly started popping up. It was great. We’re also working hard at getting more languages incorporated, so hopefully we can have more of these types of discussions in the future, hang in there!

The best part was to see how programmers from all kinds of backgrounds were equally interested in having the advantages of a development platform like Cloud9 on the Cloud. It is incredibly motivating to get approval and excitement from the very audience that we are targeting with Cloud9.

As a side note, it is amazing how much our own perception of past events diverges from the real thing. I thought the talk was quite fluent, but I can (painfully) see that I could have used more practice (LOTS of “ums”, especially at the beginning!). Ah well…it was my first big public talk, so I guess this is to be expected :)

Still, it’s a talk I’m proud of about a product that I’m proud of. Check it out if you want to see a glimpse of the future of development!!

Update: Find the slides for the talk here

Dynamically generating MIDI in JavaScript

2010-11-08T00:00:00+01:00

Last weekend I open-sourced a small side project consisting of a JavaScript library that generates MIDI in a simple manner. As an example, the code to generate a MIDI file that plays 3 notes (C, E and G) would look like the following:

// We pass some notes to |MidiWriter.createNote| to create the MIDI
// events that define the notes that will be in the final MIDI stream. If
// no other parameters are specified to |createNote|, a NoteOff event
// will be inserted automatically, instead of letting the note ring forever.

// Disregard the |push.apply|, it is used here simply to flatten the
// resulting array, since |createNote| returns an array of events.

var noteEvents = [];
["C4", "E4", "G4"].forEach(function(note) {
    Array.prototype.push.apply(noteEvents, MidiEvent.createNote(note));
});

// Create a track that contains the events to play the notes above
var track = new MidiTrack({ events: noteEvents });

// Creates an object that contains the final MIDI track in base64 and some
// useful methods.
var song  = MidiWriter({ tracks: [track] });

// Alert the base64 representation of the MIDI file
alert(song.b64);

// Play the song
song.play();

// Play/save the song (depending of MIDI plugins in the browser). It opens
// a new window and loads the generated MIDI file with the proper MIME type
song.save();

The library is work in progress, but it already outputs standard MIDI files (SMF) encoded in base64, and can play them in any browser that has a plugin that can reproduce MIDI installed. It is usually Quicktime in most machines. And now, please allow me to rant a little bit…

h2. The surprisingly bad state of MIDI in the browser

While developing this small library I was shocked at how inconsistent the state of MIDI reproducibility is in the browser. I might have been very naive, but I expected that a MIDI file would just sound right away inside a browser. Boy was I wrong.

Disclaimer: Any time I talk about reproducing MIDI, I am trying to do so using a data URI encoded in base64. Although all the browsers mentioned should be compatible with it, there are some bugs related to how data URI is supported when using it along with external plugins.

The new and shiny HTML5 <audio> element doesn’t support MIDI in any browser, so I had to revert to the good ol’ <embed> element, which supports it ONLY if the user has a proper plugin installed. All the browsers are using the Quicktime 7 plugin (I was very surprised to see that the mighty VLC can’t reproduce MIDI out of the box).

In some cases, even with the plugin installed, things didn’t work. That is the case for Webkit in Mac, which can’t reproduce the MIDI file generated by the library, while Webkit on Windows and Firefox 4b6 (Mac/Win) have no problems with it. I suspect this is due to bad interaction of the plugins with data URIs.

Of course you can always try to force a file download, but that’s not the point. It would be very nice if something as basic as cross-browser MIDI reproducibility would exist, and it is surprising that it is not possible.

But its inconsistent behavior isn’t actually the worse part. The worse part is that the user has to manually install a plug-in if she wants to be able to reproduce MIDI inside the browser. Of course iTunes and Media Player can play MIDI, but that’s not the experience I am aiming for. MIDI is still a strong standard nowadays (apparently the new iOS 4.2 is betting hard for it), specially in the world of professional musicians. For some reason this format has been ditched from HTML5 audio, or was never considered for it, leaving MIDI reproduction in the browser in the hands of mediocre plugins and careless browser vendors.

Try it!

Go and try it! Find bugs, suggest improvements to the API or features you would like to see there. And if you find solutions or have ideas on how to improve MIDI reproducibility in the browser, please don’t hesitate to tell me!

You can find the library here. Remember that it is not complete and there is heavy work in progress going on, so expect that new versions will break compatibility (certainly) and keep updating it because I will improve it during the following weeks.

Writing a JavaScript interpreter for DBN using PEG.js and canvas (Part II)

2010-08-20T00:00:00+02:00

In this article I build an interpreter for the DBN language from the parsed AST generated by the grammar we defined in the previous article in PEG.js. If you haven’t read the first part, I strongly reccomend to do so, otherwise this will make little sense to you.

It should be quite easy to follow by just looking at the code and its comments. Still, I will be explaining how the interpreter works as I put the code examples in.

Choices and assumptions

Due to little time and given the lack of any reference of the language, I made some assumptions and I left out some advanced features of the language. I hope I can include them soon into the project.

Scope

Since I don’t know how scope works in DBN, I assumed Lexical scoping, given that it is the most common among modern programming languages (JavaScript has lexical scoping as well, for example).

The interpreter

The interpreter takes care of evaluating the arguments for each statement and then execute the statement with its parsed arguments and the given scope, if any:

var Interpreter = function(canvas) {
    this.canvas = canvas
    this.ctx = canvas.getContext('2d')
    this.vars = {}
}

Interpreter.prototype = {
    evalType: function (e, scope) {
        return Interpreter.typeTable[e.type].call(this, e, scope)
    },
    evalExpression: function(ast, scope) {
        _.each(ast, function (c) {
            // Parse the arguments real value
            var args = _.map(c.args, function(a) {
                return this.evalType(a, scope)
            }, this)
            Interpreter.expTable[c.name].call(this, {
                args: args,
                block: c.block,
                scope: scope
            })
        }, this)
    },
    reset: function() {
        var ctx = this.ctx
        // Resets all cnavas state and properties
        this.canvas.width = this.canvas.width
        this.r = { vars: {}, cmds: {} }
    }
}

The global variables will reside in the vars property, and the most important methods here are obviously evalType and evalExpression.

h3. Evaluating types

evalType receives the type object and its scope and calls the proper static method in Interpreter.typeTable with the same parameters. Interpreter.typeTable holds the functions that interpret the different types in DBN:

// In DBN, everything is an integer or a set of integers, and the developer has
// the possibility to put the values directly in the code or store them in
// variables.

Interpreter.typeTable = {
    "command": function(e, scope) {
        var values = _.map(e.args, function(a) {
                return this.evalType(a, scope)
            }, this)
        return Interpreter.cmdTable[e.name].apply(this, values)
    },
    "string": function(e, scope) {
        var vars = this.r.vars
        var v = e.value

        if (scope && scope.hasOwnProperty(v)) return scope[v]
        else if (vars.hasOwnProperty(v))      return vars[v]

        return v
    },
    "integer": function(e, scope) { return e.value },
    "point": function(e, scope) {
        return {
            type: "point",
            x: this.evalType(e.x, scope),
            y: this.evalType(e.y, scope)
        }
    }
}

The types are the following:

Command

A user-defined function in DBN jargon. The user can define custom commands with parameters that execute a block of code when called.

String

Used mainly for variable names. Given a string from a generated AST, I check whether the variable is defined in its local scope (in case there is one) or in the global scope. In case it is in neither of them, I assume that it is the name of a parameter and doesn’t have an associated value.

Integer

The basic type. Its value is returned right away.

Point

A point refers to a coordinate in the canvas. It contains x and y values that can be integers, variables or arithmetic expressions. Returns an object with the evaluated x and y values.

Evaluating expressions

Expressions are DBN functions. In this interpreter, everything that computes is considered a function, including the arithmetic operators.

evalExpression takes an AST (or a portion of one, as it is done for code blocks) and loops through every statement, resolving the types of its arguments using evalType and calling the proper static method in Interpreter.expTable with them. Interpreter.expTable is a dictionary that stores the supported DBN expressions’ equivalents in JavaScript (same as with types).

Inside the Interpreter.expTable we can find the following kinds of expressions:

Arithmetic expressions

"*": function(a, b) { return parseInt(a * b) },
"/": function(a, b) { return parseInt(a / b) },
"+": function(a, b) { return parseInt(a + b) },
"-": function(a, b) { return parseInt(a - b) }

Arithmetic expressions are always used inside other expressions parameters, never by themselves. The functions are called always with 2 arguments that have been already resolved by evalType, so they are integers when they get to the expression.

Drawing expressions

/*
 * paper will fill up the canvas with the specified color (the only argument
 * of the expression)
 */
paper: function(p) {
    var ctx = this.ctx
    var w = ctx.canvas.clientWidth
    var h = ctx.canvas.clientHeight

    ctx.fillStyle = Interpreter.gray2rgb(p.args[0])
    ctx.fillRect(0, 0, w, h)
},
pen: function(p) {
    this.ctx.strokeStyle = Interpreter.gray2rgb(p.args[0])
},
line: function(p) {
    var a = p.args
    var ctx = this.ctx
    var height = ctx.canvas.clientHeight

    ctx.moveTo(a[0], (height-a[1]))
    ctx.lineTo(a[2], (height-a[3]))
    ctx.closePath()
    ctx.stroke()
}

The main difference between the DBN ‘canvas’ and HTML5 canvas is that the Y coordinate is inverted; in DBN the Y coordinate starts at the bottom whereas in HTML5 canvas it starts at the top. The other important difference is that DBN’s pixels color can only be a gray value from 0 (black) to 100 (white), so we have to use the very simple gray2rgb method to translate them to canvas rgb syntax:

/*
 * gray2rgb transforms a 0 to 100 gray value into a rgb
 * string compatible with canvas
 */
Interpreter.gray2rgb = function(gray) {
    var value = Math.ceil(255 - (gray * 2.55))
    return "rgb(" + value + "," + value  +"," + value + ")"
}

With these two changes in place it is quite straightforward to translate drawing functions using the basic HTML5 canvas functions and passing the arguments properly.

Block expressions

A block expression is a statement that contains a block of nested expressions. DBN’s repeat and command statements are the ones implemented here that belong to this kind. The peculiarity of block expressions is that they have a local scope:

repeat: function(p) {
    var args = p.args,
        id = args[0], f = args[1], c = args[2],
        // Cloning context object because otherwise we will change
        // properties of the parent context, messing up everything
        scopeObj = _.clone(p.scope) || {}

    _.each(_.range(f, c), function(n) {
        scopeObj[id] = n // Update the 'counter' variable to reflect the current loop number
        this.evalExpression(p.block, scopeObj)
    }, this)
},
command: function(p) {
    var args = _.rest(p.args)

    Interpreter.expTable[p.args[0]] = function(_p) {
        // Cloning context object because otherwise we will change
        // properties of the parent context, messing up everything
        var scopeObj = _.clone(p.scope) || {}

        // Associate every argument name with the given argument
        // value of the command call
        _.each(args, function(p, i) { scopeObj[p] = _p.args[i] })
        this.evalExpression(p.block, scopeObj)
    }
}

The local scope here is implemented in a naïve (and a bit inefficient) way. Basically and if it exists, the parent scope of the block expression is cloned and then the local variables of the block are copied on it. After that, the block of nested expressions is executed with the cloned context.

repeat is a ‘for’ loop that gets as arguments the ‘counter’ variable, the lower bound of the range and the higher bound of the range. From that it creates a loop that assigns the counter variable to the local scope and executes the block with this scope.

command builds a function from the given name and arguments and the block expression that forms the body of the function. In that case I create a function and store it in Interpreter.expTable so it can be called anywhere in the code. I do the same trick with the scope as in repeat, with the difference that in this case I asociate the name of the command arguments with the value passed to the call of the function.

Demo

That’s it, you can try a quick demo here. In the demo page you can find some examples and you can play around to better understand how DBN works and what its powers and limitations are.

Conclusion

As you can see this is a very simple implmentation, but it is enough to run most of the DBN programs around. Some things like arrays, timers and mouse position and events were left out and will be implemented at some point when I have time, but the actual ‘meat’ of the intepreter won’t change much anyway.

You can find the complete code for this interpreter here and the complete repository here. If I missed something or yo have some suggestions of how to better implement DBN drop me a line, or even better: fork it!

Writing a JavaScript interpreter for DBN using PEG.js and canvas (Part I)

2010-08-03T00:00:00+02:00

In this first part of the article, I will define a grammar for DBN and generate a parser for it that outputs an AST, so I can interpret the syntax tree it later on with JavaScript and draw it into an HTML5 Canvas.

John Maeda created the DBN language as a tool to teach programming to non-developers. It was quite an influential language; in fact it was the precursor of the popular processing language, developed by Maeda’s students Casey Reas and Ben Fry, taking many ideas from DBN.

Unfortunately, DBN hasn’t been open-sourced and the documentation of it is almost nonexisting, let alone any official specification of the language. There is only one book written about it (although a blurry copy exists in google books, censored enough to miss very important parts), and the DBN software amounts to a Java applet in the official website and a Java program that you can run on your own computer.

All this lack of proper information was a challenge enough to get me interested on building a JavaScript interpreter for it (this, and my deep hatred for Java applets) and a good excuse to play with some cool JavaScript technologies.

h2. Crash course in PEG.js grammar definitions

PEG.js is a parser generator for JavaScript. Given a proper grammar definition of the language, it will generate a parser for it that you can use wherever.

If you want to quickly test the grammar output as you go, use PEG.js online version, it is immensely useful on the early stages of grammar definition.

Also, keep in mind that what I explain here is a 10.000 foot overview of PEG.js and it is by no means a tutorial; if you want to learn real language grammar definition with PEG.js you must read the real documentation for it, and have some idea about parsers, lexers, tokens and their friends.

1. Defining the basics

Before we start, one thing worth oting is that PEG.js doesn’t have a separate stage for “lexing”. The lexer is integrated seamlessly in the generated parser, so we don’t have to worry about it.

Ok, so the first thing to do is to define the building bocks for the language. Note that the parser doesn’t take anything for granted, so we will have to define how our language handles whitespace as well.

variable
 = v:([a-zA-Z_][a-zA-Z0-9_]*) { // The first character must not be a number
     return {
       type: "string",
       value: v.join("")
     }
   }

integer
  = digits:[0-9]+ {
      return {
        type: "integer",
        value: parseInt(digits.join(""), 10)
      }
    }

point // Matches expressions of the form [value1 value2]
= "[" left:value ws right:value "]" {
    return {
        type: "point",
        x:left,
        y:right
      }
    }

special // Matches expressions of the form <command value1 value2 ...>
= "<" _ left:variable args:(ws value)+ ">" {
    return {
      type: "special",
      args: [left, args.map(function(a) { return a[1] })]
    }
  }

additive
  = left:muldiv _ sign:[+-] _ right:additive {
      return {
        type: "command",
        name:sign,
        args:[left,right]
      }
    }
  / muldiv

muldiv
  = left:primary _ sign:[*/] _ right:muldiv {
      return {
        type: "command",
        name: sign,
        args:[left, right]
      }
    }
  / primary

primary
 = (variable / integer / special)
 / "(" _ additive:additive _ ")" { return additive }

value
= variable / integer / additive / point / special

comment // Matches single-line comments
 = "//" (!lb .)*

ws
 = [ \t]

_ // Matches any number of whitespace/comments in a row
 = (ws / comment)*

As you can see, the language is very clear and takes inspiration from regular expressions. The above peg grammar defines strings, integers (DBN has no decimals), points in a 2D canvas, and basic arithmetic (the latter adapted from the arithmetic grammar given at the PEG.js site as example).

A rule is composed of several parts: its name, the matching expression and the optional parser action, which is enclosed between brackets and transforms the output of the rule.

For example, for the point rule:

point
 = "[" left:value ws right:value "]" // <-- This is the matching expression
   // The following code is the parser action
   {
     return {
       type: "point",
       x:left,
       y:right
     }
   }

This rule checks for a “[” character followed by a value rule (which is defined separately), followed by a whitespace rule (also defined separately), followed by another value rule and a closing “]”.

In most non-trivial rules we include a parser action, which is the JavaScript code inside the brackets. The parser action can access the matched values of the rule and modify them using JavaScript before the output is returned. Here I return an object that contains the two values inside a point, which were labeled left and right in the matching expression so they can be accessed from the parser action. Without a parser action the output would be a JavaScript array with all the matches in the matching expression, so with the input [4 2] we would get the output ["[", "a", " ", "b", "]"].

The point rule references two other rules, value and ws, which look like this:

value // matches any of the standard values in DBN
= variable / integer / additive / point / special

ws // Matches space and tab characters
= [ \t]

value will try to sequentially match any of the expressions separated by “/”, and it will return the first sucessful match. The whitespace ws rule uses a RegExp construct to match any single space or tab character.

2. DBN Commands

Now we will define the “meat” of our grammar. Fortunately enough, DBN is a language with a rather simple syntax. A basic program could look like this:

paper 30

repeat A 0 300
{
    pen 50
    line 0 55 100 55
    line 0 20 100 20

    pen 100
    line (A*2) 0 (A*2) 20
    line (A/3)  55 (A/3) 100
    paper 30
}

This program runs the instructions inside the repeat loop, which is equivalent to a classical for loop. It loops from 0 to 300 while assigning the current iteration value to the variable A and drawing different lines on the canvas.

As we see above, a basic command in DBN takes the following form:

command_name value_1 value_2 value_n

And a rule command to parse it:

command
 = _ cmd:[A-Za-z0-9?]+ args:((ws+ value)+)? _ lb+ {
     return {
       type: "command",
       name: cmd.join("").toLowerCase(),
       args: args ? args.map(function(a) { return a[1] }) : null
     }
   }

Let’s break the matching expresion down:

_: Match 0 or more whitespace characters.
cmd:[A-Za-z0-9?]+: Match an alphanumeric word that might include the ‘?’ char, and label it as “cmd”.
args:((ws+ value)+)?: Match 0 or more DBN value expressions preceded by a whitespace and label them as “args”.
lb+: Match at least one linebreak.

In case an expression command is matched, we will pass the cmd and the args results to the parsing expression, who will return an object with some particular properties that manipulate the values in the way we want. In that case we just do some basic filtering to the arrays resulting from the matches.

An important thing to note down is that any match gotten using the operators [] * or + will return an array with as many coincidences as there are in the matching expression. That means that we will have to take care of filtering these arrays so we obtain the results we want. A clear example is the cmd variable, which returns an array with all the characters of the command name matched, so in the parsing expression we have to join it to form a real string.

3. DBN Blocks

Our grammar can already process DBN commands, but this is pretty limited functionality. Any interesting program will at least have loops and block commands, and we still can’t parse these.

In order to process block commands we will create the new rules block and block_command:

block
 = "{" lb* e:(block_command / command)+ lb* _ "}" { return e }

block_command
 = e:command _ b:block lb* {
     e.block = b;
     return e;
}

In block we match any group of commands or nested block_commands that are inside braces, taking into account the possible whitespaces that we might find.

In block_command the matching expression is very straightforward, but there is an interesting bit in the parsing expression, where we attach b to c as a property, and we return c after that, so the result is a JavaScript object with the properties type: “command”, array of parameters args and that block property we just added, which contains the commands inside the block.

Conclusion

And we’re finished! Our grammar can now parse DBN programs and generate its JSON AST. Parsing the basic program that I wrote above as an example generates this tree. You can find the complete grammar file here.

As you can see, using PEG.js is a very clean and concise way to define language grammars. And best of all, we can use JavaScript for doing it which is pretty awesome. Moreover, the generated parser for it is pure JavaScript and thus independent from the PEG.js library itself. In fact, using the online version of it, you can just download the generated parser without even having to download the library itself.

I am sure very soon we will hear of apps that use PEG.js in very creative ways, surely involving node (as it seems to be the trend now) or some other cool technologies. JavaScript is definitely ready for prime time.

In the second part of this article I will create the interpreter that turns the AST into JavaScript code that paints directly into canvas. Stay tuned!

GitHub in Catalan

2010-07-28T00:00:00+02:00

This is awesome. From now on, Catalan users of GitHub can use the website in their native language. Thanks to the Github team initiative and the brilliant approach of crowd-sourcing the translations (along with some very convenient tooling to do so), I could translate almost all the strings of the website in a record time two weeks ago.

What I found out today is that the website is already making use of it and available to everyone! Just point your browser to http://github.com/blog?locale=ca and the page will be offered to you in Catalan already. The translation is currently 87% complete, so there is still some work to be done, which I expect to finish between today and tomorrow.

Anyway, if Catalan is your native language and the lack of it in Github was stopping you from using it, now you have the chance!

GitHub en Català

Des d’ara mateix, els usuaris Catalans de Github poden utilitzar la web en el seu propi llenguatge. Gràcies a la iniciativa de l’equip GitHub i a la manera brillant d’utilitzar crowd-sourcing per a traduïr la web a multitud de llenguatges (i a unes eines de traducció molt útils), he pogut traduïr quasi totes les cadenes de la web fa dues setmanes en un temps récord.

El que no m’esperava i he descobert avui és que GitHub ja està disponible online en Català! Només cal que introduïu la següent direcció al vostre navegador: http://github.com/blog?locale=ca i la pàgina us preguntarà si voleu establir Català com a llenguatge del lloc per defecte. La traducció està 87% complerta, així que encara hi ha una mica de feina per fer, que espero finalitzar entre avui i demà.

Així doncs, si la teva llengua és el Català i el fet que GitHub estigués només en anglès t’impedia utilitzar-lo, ara ja no tens excuses!