Code Pristine

JavaScript and application architecture by Dan Stocker

Building an Enterprise Framework

I’m maintaining a family of libraries on GitHub. They started out as two data manipulation libs, jOrder and Flock, one for tables, the other addressing tree-structured data. Then I went into creating a browser-based desktop video tagging app, Wwidd, which, beyond making use of these libraries has helped me establish a series of architectural practices.

Both the libs and practices were put to test when I joined Production Minds, to create the front end for its flagship product, PMP, an excercise that ignited a doubly reflective process. I used my tools in the development of PMP, learned from their effects, advantages, disadvantages, and how we were using them. Then I made modifications to the libraries or spawned entirely new ones, and introduced the changes back to the application.

At this point you’re perfectly justified to say that “anyone could do that”, so first let’s get the obvious question out of the way.

Why not Angular?

The PMP front end codebase, after 20 months of development stretches across about 80.000 lines, excluding libraries. The line count is not meant to be a testament to how large, but rather to how small it is despite being an immensly complex, stateful app with long class inheritance chains and deep data structures (all of which is necessary). It is truly what you call an enterprise application.

I am convinced that relying on Backbone, Knockout, or Angular alone wouldn’t have made it any easier, or better. I even have numbers to back up this claim. It may be only a single aspect of a framework out of many, but we already do OOP faster and better than Ext, Ember, or MooTools, thanks to Troop, the first library distilled right from our experiences, and shaped by them ever since.

But we mustn’t stop at OOP. Data manipulation is also a huge part of the picture. As massive amounts of Mongo-documents are pouring in from the servers, widgets and models must be notified, and local search indexes kept updated. I contend that Backbone models could not tackle the granularity of the data we’re dealing with here, (unless through excessive code repetition or the use of an external tool) nor would the “just works” data binding of Knockout or Angular satisfy our needs. So we built Flock.

Then, there’s the UI, we’re on the front end, after all. There’s a debate on whether business logic belongs in the markup (templates) or vice versa (markup generation from code). We went with the latter after careful examination and tests, and the emergence of Facebook’s recent React library seems to justify us. We’ve built a widget framework based on our existing tools to abstract away user interactions and markup generation.

Framework in the making

Slowly all the building blocks of a full-fledged application framework are coming together. OOP with Troop, assertions with Dessert, collections and behaviors with Sntls, and content-independent eventing with Evan are already available as single downloads (see GitHub wikis), or as NPM packages.

There are four big items left on the to-do list, however. First, we need to integrate the two original libraries, jOrder and Flock into the family (both in progress), so the whole field is leveled in terms of interoperability and testing. Currently the most painful problem about writing unit tests for model classes in PMP is that Flock is not Troop-based. (In case you were wondering, mocking Troop class methods is a blast.) Third item is releasing the widget framework, and fourth, icing the cake with MV* layers that define an overall architecture. A model for this last one is already there.

Convergence

Tools that weren’t created for building something specific rarely get successful. Popular front end frameworks no doubt excel at what they’ve been designed for, but when it comes to the enterprise domain, engineers still find themselves puzzled as to how to proceed. In order to get to a framework for enterprise applications, one has to build enterprise applications, in a perpetual sequence of reflection and iteration.

This particular framework is the byproduct of a sequence exactly like that: the day to day work we’re getting done converges on it. By using the framework, a lot of efforts may be saved from being wasted.

I invite everyone to participate in making this vision a reality by using / experimenting with the libs that are already released (Dessert, Troop, Sntls, Evan), and providing feedback on them. Contributions are also warmly welcome. The code is clean, thoroughly tested, well-documented, getting started is easy (git clone, npm install, and grunt test), and there are lots of tasks already lined up.

You can reach me at dan at kwaia dot com.

Front-end Lessions Learned: Show some class!

This is a re-post of my original article on the Production Minds dev blog.

I remember in the wake of 2010, when I really started getting into front end development, I was very determined to resist any attempt to treat JavaScript as it were some other programming language. Having been involved with C++ and C# for years, I found myself enthralled by the capacity of dynamic languages. However, I was facing pressure from my employer at the time to investigate so-called class libraries with a goal of adding much needed structure to our font end codebase.

Fueled by my new-found enthusiasm, I argued relentlessly that relying on a library to deal with OOP is cowardice, a departure from the true nature of JavaScript, and that instead we need to embrace features like prototypal inheritance, closures, lambda functions, and continuation passing style.

Now, three years have passed, and I submit that this still counts as the best argument against classes. Three years ago I wasn’t very knowledgable about JavaScript, and as I didn’t have all the information I needed, my assessment of OOP libraries was rather inaccurate. In short, I was wrong.

Here’s why.

Most debates about programming style or even paradigms are subjective. Sometimes the same issue however, becomes quite palpable when applied to a different language. Douglas Crockford brings up the placement of block delimiters as an example in one of his speeches. In C, it’s basically a matter of taste whether you put the opening curly brace on the right or the left. In JavaScript, it must be on the right, otherwise you risk getting unexpected behavior.

The same applies to classes. In compiled object-oriented languages classes might have an impact on efficiency and maintainability, but they’re essentially irrelevant with regards to performance. Not in JavaScript. In JavaScript, when the interpreter encounters a function definition for example, it creates a new Function object in memory, along with its scope chain, and user-defined properties. If it hits the same function definition twice, two objects will be created, taking altogether twice as long and consuming twice as much memory until they’re garbage collected. The widely used revealing pattern creates a new function for each method of every object it churns out. For an application that is both complex and stateful, this is a major bottleneck. Even with classes, since one way of mimicking them is exactly through the revealing pattern. So not only does it matter whether we use classes at all, but also how we emulate them. And there are a few ways.

But here’s where the specialty of JavaScript comes into play. The only way available at creating an abundance of objects in a memory-conservative and performant manner is to have a static class structure, so methods need not be re-created for each instance, but only once during the entire lifetime of the application. Working this argument backwards, in order to achieve low memory footprint and object creation cost you have to have a static structure, which in turn would behave exactly like classes.

MooTools, Ext.js, and many other libraries and frameworks have recognized this. The Production Minds Platform front end is based on Troop, an OOP library we built ourselves upon these very principles, and extended with additional features. Following our preliminary tests performed on a set of simple classes (chart below), it turned out that instantiation is in fact much faster when you switch to a static class-structure. The more complex your code, the higher the gain.

Static vs. dynamic instantiation

So when it comes to classes and JavaScript, it’s best to leave emotional attachments and loosely grounded convictions behind, and instead look at the sobering figures. It is a clear choice indeed.

Front-end Lessions Learned: Why Scale?

This is a re-post of my original article on the Production Minds dev blog.

There’s an increasing amount of talk among JavaScript engineers on the importance of scaling. More tweets, blog posts, talks, and conferences have addressed this issue during the past twelve months than ever before, and the trend seems to continue.

But what is scaling, and why do we need it?

Historically, scaling concerns the back-end, where it reflects a system’s capacity of accomodating additional resources as stress figures like traffic or storage consumption grow. In contrast, the front-end runs on the user’s device - an environment unlikely to suffer anything like the stress imposed on the servers, so scaling in this context cannot be connected to usage. But, as more of the business logic shift to the front-end, (take Google Docs, Facebook, or PMP for instance) they grow big, and the nature of JavaScript specifically poses an obstacle in keeping up a healthy balance between complexity and everything else.

Scaling, on the front-end equals to keeping maintainability and responsiveness figures high while enriching the application with features. In other words, if you(r users) feel like your app is slow, fixing one issue opens three others, and starting over seems like the most attractive option, it probably won’t scale well. Avoiding this scenario, especially beyond a certain size is not straightforward, as JavaScript, unike C# or Java - doesn’t come with tools that imply a sound architecture for our applications. JavaScript leaves the engineer with a great deal of freedom, which is good, but that freedom also means that often the engineer finds herself on uncharted waters when it comes to finding a suitable way to manage a large codebase.

In order to manage code, first one has to make it managable. Of course, the basic principles that govern this process are not new: design patterns, polymorphism, namespaces, lazy initialization, static analysis, unit testing, and many more have all been around for decades. The question is how we apply these constructs to our JavaScript codebase, because often this is what makes or breaks the whole application. Fortunatelly, here the multi-paradigm nature of the language pays off, as we may bend the architecture to our advantage in ways that would be unimaginable in classical object-oriented languages.

The art of crafting websites and developing applications merged in the past decade, and as we see browsers run JavaScript ever more efficiently, it’s just common sense to separate front and back completely. Leaving aside extra gains like running web applications offline (hint: Chrome apps), this new approach relieves server load while simultaneously offering a richer, faster experience to users. All this comes at the cost of all new technical challenges, and those taking on them - like PMP - get a shot at shaping the future of application development.

In this series’ following posts I will be looking into the specifics of scaling and how they relate to the design of PMP.

blog comments powered by Disqus