Node.js and MongoDB – Getting started with MongoJS

It won’t be an exaggeration if one claims that in the past few months Node.js and MongoDB have literally taken the software and web industries by storm.

Not just bleeding-edge startups but even medium and large enterprises are leveraging these two technologies to deliver a better experience to their users by build more capable, performant and scalable apps.

So what is Node.js?

Node.js is a platform built on Chrome’s JavaScript runtime for easily building fast, scalable network applications. Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.

..and what is MongoDB?

MongoDB is a scalable, high-performance, open source NoSQL database.

This post will cover the basics and get you started with your Node.js + MongoDB app. Make sure you have Node.js installed and MongoDB setup on your developer machine.

Let’s verify your Node.js installation and start the MongoDB server:

$ node -v
$ mongod

Introducing MongoJS
MongoJS is a brilliant little Node.js package that lets you access MongoDB using an API that is extremely similar to MongoDB’s JavaScript shell.

Installing MongoJS
Once Node.js has been setup correctly on your machine, you can use its internal package manager NPM to install MongoJS:

$ npm install mongojs

We can now start building our JavaScript application and connect to our MongoDB server:

// app.js
var databaseUrl = "mydb"; // "username:password@example.com/mydb"
var collections = ["users", "reports"]
var db = require("mongojs").connect(databaseUrl, collections);

The databaseUrl can contain the database server host and port along with the database name to connect to. By default the host is “localhost” and the port is “27017“. If you’re using the default, which is likely on a developer environment, the databaseUrl can contain just the actual database name for our app.

The collections is a set (array) of collections our application uses. It isn’t mandatory but is preferred as it allows us to emulate a MongoDB JavaScript client like API within our Node.js app.

Here’s an example for finding documents within a collection specifically in this case to find all female users.

// app.js
db.users.find({sex: "female"}, function(err, users) {
  if( err || !users) console.log("No female users found");
  else users.forEach( function(femaleUser) {
    console.log(femaleUser);
  } );
});

Notice how our initial query is a near duplication of the corresponding query in MongoDB’s console. In addition to the query, in our Node.js source code (i.e. app.js) we pass a callback function to handle the results of the query.

Node.js implements an event based concurrency paradigm and almost everything is always a callback. This allows your Node.js app to be non-blocking and high performing.

What happens in our specific callback is self-explanatory — we check for errors and results, and if the query returns female users they are logged to the console.

Okay, how do I save a new user in my collection? Exactly how you would do it in the console, your code will look like this:

// app.js
db.users.save({email: "srirangan@gmail.com", password: "iLoveMongo", sex: "male"}, function(err, saved) {
  if( err || !saved ) console.log("User not saved");
  else console.log("User saved");
});

Here’s an example for updating a record / document:

// app.js
db.users.update({email: "srirangan@gmail.com"}, {$set: {password: "iReallyLoveMongo"}}, function(err, updated) {
  if( err || !updated ) console.log("User not updated");
  else console.log("User updated");
});

Okay, now we run this app in the console:

$ node app.js

And there we have it, an incredibly simple quick start for Node.js + MongoDB enthusiasts. Happy coding!

At what stage should scalability be addressed?

In this post I will try to be as non-partisan as possible with respect to technology, platforms, programming languages, databases etc. Do forgive me if I fail in this attempt.

Here’s the question on hand:
When should business, leadership, dev teams and other stakeholders address the scalability question.

It’s a dilema I face and others depend on my advise so I’m pooling in your wisdon.

Allow me to ellaborate the question. We’re building a product and we have two choices:

Choice 1

Go in for a technology that is extremely popular. It has been around in the mainstream for at least half a decade. It’s a good business choice because of the maturity of the environment, the availability of talent / outsourcing firms and positive feedback overall.

However there are some questions over scalability. There are many case-studies where headline grabbing companies have had to abandon this technology and go in for a complete / partial rewrite on another platform.

The argument here goes like this:

“If we use this technology and we face scalability problems few months down the line, it’s a good problem to have because it means we have done really well. So lets go ahead with this for now, even if it means a rewrite six months down the line.”

Choice 2

We can opt for another much younger technology. It has an active community but comparatively less mature environment / toolkits. In comparison to Choice 1, it is harder to find good and affordable talent.

The programming language here is extremely expressive yet heavily misunderstood.

However there are bona fide scaling credentials. It’s been a choice for organisations that are proven technology leaders. And its perhaps a safer bet in the long run on the scalability parameter.

Your call

As a programmer / hacker / geek, Choice 2 is an obvious winner. At least for me its a no-brainer.

However put yourself in a product owner / manager / CTO / decision-making role. Does your opinion change?

Welcoming all your comments, thanks!

Really simple file uploads with Node.js and Express

Few days ago I was working on a fairly typical web application and I faced the challenge of implementing a fairly typical web application feature – file uploads. It was the first time I was implementing file uploads with Node (and Express) and I did what anyone else would do – I googled it.

Unfortunately all the articles / posts out there are either outdated, too complex or plain wrong. So I did the next most obvious thing – post a question on the mailing list. As always Mr. Holowaychuk was incredibly quick to respond. His answer lead me to do what I should have done in the first place – read the docs.

The upload form

This is the most obvious part of the challenge. You’re probably familiar with this already. Anyway, for the sake of completeness of this article, here it is.

You will need a form in your browser for the file upload. I use Jade to generate my HTML and here how it looks:

form(action="...", method="post", enctype="multipart/form-data")
  input(type="file", name="displayImage")

The form.action will point to a route that handles the file upload. More below.

Accessing the uploaded file

If you’re using recents versions of Node and Express, file uploads are a piece of cake. And I’ll back this claim but before we go any further make sure you’re familiar with routes, requests and responses in Express.

Okay, now let’s justify the “piece of cake” claim. In our file upload route, the req parameter has req.files available. Here’s an example of what the req.files would contain:

{
  displayImage: {
    size: 11885,
    path: '/tmp/1574bb60b4f7e0211fd9ab48f932f3ab',
    name: 'avatar.png',
    type: 'image/png',
    lastModifiedDate: Sun, 05 Feb 2012 05:31:09 GMT,
    _writeStream: {
      path: '/tmp/1574bb60b4f7e0211fd9ab48f932f3ab',
      fd: 14,
      writable: false,
      flags: 'w',
      encoding: 'binary',
      mode: 438,
      bytesWritten: 11885,
      busy: false,
      _queue: [],
      drainable: true
    },
    length: [Getter],
    filename: [Getter],
    mime: [Getter]
  }
}

In the req.files object above, the property displayImage is the name of the file field in your HTML form and req.files will contain one property each for every valid HTML file form field.

The file object contains the type, size and name properties for your server side validations.

Saving the uploaded file

Assuming the file is valid, you use the path property for the next step. The path would typically contain a location in the tmp folder. Your application logic could either require you to access the contents of the file or simply move the uploaded file to another location.

fs.readFile(req.files.displayImage.path, function (err, data) {
  // ...
  var newPath = __dirname + "/uploads/uploadedFileName";
  fs.writeFile(newPath, data, function (err) {
    res.redirect("back");
  });
});

In the fs.readFile callback, we have the data parameter through which we can access the contents of the file. The example above is taken from an application that needed to modify the file and save it in a new location. Thus fs.writeFile is used to write data to the newPath.

If your app needs to simply move the uploaded file without modifying the contents fs.rename can be used as more simpler option.

That’s all there is to it. I’ve done file uploads in many server side languages including Python, Java, Scala and PHP and I don’t think its ever been this simple.

So much for JavaScript being labeled as an inferior server side language.