How I built a real-time collaborative app with MongoDB

This is an account of how Review19 was built in just a couple of weeks and how MongoDB was an excellent choice for this genre of browser based, real-time collaborative apps.

First things first — what is Review19? Review19 is the next generation, real-time project collaboration tool for web, creative and software teams. Think of it as a minimalist, real-time Pivotal Tracker alternative. It’s like Trello, but caters to a focused target audience.

Give it a shot, try out Review19. If you’re stuck behind a proxy that doesn’t like web sockets, here’s the failover link.

The Technology Stack

Obviously an app can’t be built with a data store alone.

In addition to a MongoDB based storage engine, the APIs and backend for Review19 are built on the Node.js platform and make use of popular frameworks including Express and Socket.io.

The frontend client app — which is under heavy refactoring as we speak — leverages advances in browser capability with respect to JavaScript execution and CSS renderings. The frameworks here include jQuery, jQuery UI and Sugar among others.

Why am I mentioning all of this? The overall technology stack needs to be known to highlight a huge advantage:

JSON. Everywhere!

I’ve been a professional programmer for ten years now and I’ve had the fortune (misfortune?) of working with a diverse set of technologies.

I happen to have dealt with:

  • the typical enterprise Java stack – Spring, Hibernate etc.
  • recent advances in Java technologies including Scala, LiftWeb, Play! etc.
  • built apps in Python using Django, Tornado and Google App Engine
  • built apps in PHP and its set of frameworks
  • built apps in Adobe Flex working with different types backend APIs

In each of these technologies, a developer has to deal with the mundane task of transforming his model between the client, server and data store. Better development teams would automate this process with the use of ORMs or helper frameworks. However this would still need the inclusion of these dependencies and the configuration of these frameworks.

While developing Review19 with the chosen technologies, things worked out of the box. It was JSON on the server, JSON on the client and, thanks to MongoDB, JSON on the data store. The productivity benefit was huge, especially for an independent developer like me, or small teams with tough deadlines.

The advantage of this cannot be underplayed. It strikes you right away when you build your first proof-of-concept and it stays with you throughout your development cycle.

Flexibility is not just a buzz-word

About 10-12 months ago, I was working on a project for an SF Bay startup that wanted to built an analytics / BI suite running on top of customer databases.

It wasn’t a very complex project, but just a large and poorly managed one. I remember being asked repeatedly about how the database was being versioned. The answer “it was being versioned in the code” wasn’t good enough for the ‘expert’ and his evil overlords.

Hmm.. generating and storing multiple versions of the data model as an .sql file in your source code repository. Sounds familiar? Of course it does. And it sounds just as painful as it sounds familiar. Even if this process is somehow painstakingly automated and devs don’t have to deal with it any more, you still often end up with multiple version of SQL files never being used.

Whatever your application be, the entities / models are bound to evolve with time. Things will change, sometimes so much that they won’t share even the slightest resemblance with the original designs.

With MongoDB, however, the data store adapts seamlessly to your app. There isn’t the slightest need to run scripts, create tables, alter columns etc. on the data store prior to each deployment. This is how it’s done for Review19 and I’d assume for a majority of projects using MongoDB.

This will bring a small morale boost and a large productivity boost for your dev and devops teams.

Node.js + MongoDB = MongoJS

MongoDB is one of the most mature NoSQL options available today and that maturity is proven by the rich set of APIs MongoDB provides that allow you to perform basic and advanced operations on your data.

Furthermore the MongoDB Querying APIs are ridiculously simple, yet powerful and easy to master. In fact its so easy to learn that you wouldn’t miss the very same SQL that you spent years to master.

A majority of MongoDB evangelists and proponents I’ve interacted with swear by this capability. After spending sometime on the MongDB Interactive Shell, I could see what they meant.

I needed to access the same capabilities in my Node.js backend app. Node.js has got a well maintained MongoDB driver and a lot of interesting abractions on top of that driver.

My choice zeroed in on MongoJS and the clinching argument was the MongoJS mission statement:

“A node.js module for mongodb, that emulates the mongodb API as much as possible.”

Your actual executable JavaScript code looks exactly the same to what you do the MongoDB console. How can this not be a #win? I like minimalism and MongoJS was as minimalist as possible.

There are a lot of other options as well for Node.js – MongoDB interaction and it would be best to cover them in detail in a future post.

Speed

Did I mention Review19 is incredibly fast? Almost in all the feedback received, the users mention among other things how fast it is and how it improved their experience.

I understand that application architecture has a role to play in this but one can’t take all the credit for the speed and performance witnessed by Review19 users. One must thank the underlying platforms it was built on which includes Node.js and MongoDB.

I’m sure you’ll find all the MongoDB speed benchmarks you want. They’re just a Google search away. What I intend to highlight here is that in a real world collaborative app which has a single instance running on standard, non-fancy hardware, Node.js and MongoDB are serving the needs brilliantly.

Remember this is a real-time collaboration app where instead of frequent smaller requests, a frontend client has a long lived connection with the backend server. This is exactly the kind of app that would have faced scaling issues few years ago and would have needed a lot of infrastructure both in terms of code and hardware.

Node.js and MongoDB make this economical for independent devs like me without the resources big corps can provide.

Conclusion

I’m extremely pleased with the technology choices made in which of course MongoDB is a critical component. I’ve completed the entire product cycle from conceptualization and design to testing, development and deployments.

Described above are ways MongoDB made this enitre process simpler, kept deployments easy and provided all the features and more needed by Review the app.

Now I look forward the the next set of challenges which include maintaining, scaling and enhancing the system. And I promise to let you know how that goes. Stay tuned to this blog! ;-)

4 thoughts on “How I built a real-time collaborative app with MongoDB

  1. Really interesting post, could you explain some details about performance, scalability & fault tolerance. Do you use replica sets?

  2. Hi, really interesting and self-reassuring, I started a project with the same stack. One question, why not use the node module mongoose ?

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>