Exploring the MongoDB Document Database: A Primer for .NET Developers

Relational databases have ruled the storage medium space for a very long time. When constructing a greenfield application, a relational database is always the programmer's implicit choice. Recently, a number of new document databases have emerged, as part of the NoSQL movement, offering non-relational alternatives to relational databases. This article examines one popular new document database, MongoDB, and gives examples of how you can start putting it to work in your .NET and ASP.NET applications.

What Is a Document Database?

Unlike relational databases, document databases store information in the form of a hierarchical structure of attributes, collectively creating a single document. A document can contain nested documents as an instance of an array of documents. Document databases do not use joins to build a record set, but instead use pointers. One of the advantages of using pointers in document database is that fetching operations are extremely fast. Another advantage of using document databases is the interoperability between systems. A document that is persisted by one system can easily be accessed by a completely different system.

How Does MongoDB Store Documents?

MongoDB uses a special format when storing documents, called BSON, which stands for binary JSON. Each document is uniquely identified by an identifier that is represented by the _id attribute. The data type of _id is a 12-byte binary, which provides greater probability in producing unique identifiers. Although MongoDB supports joins, persisting objects as a nested hierarchy is considered to be a better practice. You can find more information about the BSON ObjectID specification by visiting the MongoDB Object IDs page.

Getting Started

The first step in unleashing MongoDB's power is to download the bits from the official MongoDB website. MongoDB works on all operating systems and provides extra room for storage for 64-bit machines.

The bin folder in the download contains several files, but we are interested only in mongo.exe and mongod.exe. The mongod.exe file starts the server, which is used to communicate with the MongoDB database. The mongo.exe file is an interactive command prompt, which I'll explain later in this article. To make life easier, and to be able to execute the Mongo interactive shell and the Mongo server from any directory, you should place the path to the MongoDB bin folder in the system ENVIRONMENT variable called PATH.

By default, MongoDB creates the database in the c:datadb directory. You must have this directory in place before executing the server or else the server will throw an error and fail to start. You can place your database files in a separate folder by using the -dbpath option of the mongod server, but for the sake of simplicity we'll assume that you've already created the c:datadb directory. For more help with the different options provided by the mongod server, run the following command:

mongod --help

You'll see output like that shown in Figure 1.

Figure 1: MongoDB server in action

MongoDB Interactive Console

The fastest way to get started with MongoDB is by using the MongoDB interactive console. To start the console, simply type the following at the command prompt:

Mongo

Because MongoDB uses BSON to store the documents, the console is JavaScript compatible. Figure 2 shows the addition of a new document to the Northwind database using the console.

Figure 2: MongoDB interactive console

The Mongo class is primarily responsible for interacting with the MongoDB database. The mongo.getDB method is used to fetch the database by using the supplied name. If the database does not exist, it will first be created, then it will be returned. A customer object is created with two properties, FirstName and LastName, and is inserted into the Customers collection of the Northwind database. Finally, the find() method is called on the Customers collection to retrieve and display all the documents in the collection.

MongoDB also supports several different operators that can be used to perform advanced queries. The following code shows the use of the $or operator and the $in operator.

db.Products.find({$or : [{Status:”Open”},{Status:”Closed”}]});// $in operatordb.Products.find({"Status":{$in : ["Pending","Open"]}});

There are several other operators that can be used with MongoDB database. You can find a complete liston the MongoDB Advanced Queries page.

MongoDB C# Drivers

MongoDB supports many different languages through the use of drivers. Almost all the mainstream line languages have provided the drivers to connect to the MongoDB database. In this article, we will examine the C# driver for MongoDB that was created by Sam Corder. You can find the list of all the supported drivers on the MongoDB Drivers page.

The C# driver allows us to communicate with the MongoDB database by using the C# programming language. After downloading the driver files, add a reference to the MongoDriver.dll file to your application. The code in Figure 3 creates a new MongoDB document and persists it in the MongoDB database.

static void Main(string[] args)   {       using(var mongo = new Mongo())       {       mongo.Connect();       var db = mongo.GetDatabase("Northwind");       var documents = db.GetCollection("Customers");       var document = new Document();       document["FirstName"] = "Mary";       document["LastName"] = "Kate";         documents.Insert(document);       mongo.Disconnect();              }   }

Before running this sample code, make sure that the MongoDB server is running. You can start the server by executing the mongod.exe command. First, we connect to the MongoDB server by using the Connect method. The getDB method fetches the database. If the database does not exist, it is created upon the insertion of the first document. The Customers collection is fetched from the Northwind database. If the collection does not exist, it is created upon request. A new document is created using the Document class. Attributes FirstName and LastName are attached to the document. Finally, the Insert method persists the document into the MongoDB database. Run the code above, and a new document is inserted into the MongoDB database. You should see a screen similar to Figure 4, which confirms that a new document has been inserted successfully into the Customers collection of the Northwind database.

Figure 4: Document persisted in the MongoDB database

Documents are not limited to a linear structure; they also support nesting and multiple hierarchies. In fact, the persistence of the nested structure is considered the main strength of the MongoDB database. The code in Figure 5 shows how to insert a collection of addresses with a new customer.

using (var mongo = new Mongo())       {       mongo.Connect();       var db = mongo.GetDatabase("Northwind");       var customers = db.GetCollection("Customers");       var customer = new Document();       customer["FirstName"] = "Alex";       customer["LastName"] = "Lowe";       var address1 = new Document();       address1["City"] = "Houston";       address1["State"] = "Texas";         var address2 = new Document();       address2["City"] = "San Jose";       address2["State"] = "California";       var addresses = new Document[] { address1, address2 };       customer["Addresses"] = addresses;       customers.Insert(customer);       mongo.Disconnect();       }

This code creates an array of addresses. The Addresses document is then added to the Customer document by using the Addresses attribute. This creates a parent/child relationship between the Customer and the Addresses documents. The persistence of the parent object causes the child object to persist the children automatically. Figure 6 shows the Customer object with the Addresses collection.

Figure 6: A one-to-many relationship in the MongoDB database

As Figure 6 shows, the Customer object contains an array of addresses. One special thing to note about the Address array is that none of the Address items has been assigned a unique identifier. The unique identifier is assigned only to the parent element, which in this case is the Customer object. This allows us to retrieve the object by using the Customer ID because it is acting as the aggregate root of the object tree.

Converting Objects into Documents

Small applications that consist of a couple of types of documents can benefit from the Document object model by representing their structure using attributes. But large applications need to provide a mechanism to convert the domain objects into documents. There are several ways of converting a domain object model into a document structure. The most reasonable is to use reflection to iterate through the object's properties and create a document structure that resembles the object hierarchy. The code in Figure 7 converts the Customer object into its Document representation.

static void Main(string[] args)   {       var customer = new Customer() { FirstName = "Mohammad", LastName = "Azam" };       var document = ConvertToDocument(customer);   }   private static Document ConvertToDocument(Object source)   {       var document = new Document();        var properties = source.GetType().GetProperties(System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags.Instance);       foreach (var property in properties)       {       document[property.Name] = property.GetValue(source, null);        }       return document;     }   }

The code in Figure 7 will work for very simple scenarios where the object has no nested properties. We can use the power of recursion to our advantage to create a helper method, the ToDocument extension method shown in Figure 8, which converts the object hierarchy to the corresponding document structure.

public static Document ToDocument(this object source)   {       var document = SerializeMember(source) as Document;       return document;   }private static object SerializeMember(object source)   {       // get the properties     if (!Type.GetTypeCode(source.GetType()).Equals(TypeCode.Object))       return source;       if (source is Oid)       return source;       // if the object is IEnumerable     var enumerable = source as IEnumerable;       if (enumerable != null)       {       return (from object doc in enumerable select SerializeMember(doc)).ToArray();       }       var properties = source.GetType().GetProperties(BindingFlags.Instance | BindingFlags.Public);       var document = new Document();       foreach (var property in properties)       {       var propertyValue = property.GetValue(source, null);       if (propertyValue == null) continue;       if (property.Name.Equals("_id"))           document.Add(property.Name, (((string)propertyValue).ToOid()));       else           document.Add(property.Name, SerializeMember(propertyValue));       }       return document;   }

The ToDocument extension method iterates through the object's properties and converts them into the Document object. The use of recursion also takes care of nested properties, which are converted into nested arrays and added to the Document structure.

To employ the ToDocument extension method, use the code in Figure 9. This code will convert the customer object and all its nested objects into a document structure that can later be inserted into the database by using the Insert method.

using(var mongo = new Mongo())       {       mongo.Connect();       var db = mongo.GetDatabase("Northwind");       var customers = db.GetCollection("Customers");       var customer = new Customer();       customer.FirstName = "Mary";       customer.LastName = "Kate";               customers.Insert(customer.ToDocument());       mongo.Disconnect();        }

If you're using code-generation tools to create entity classes, make sure that the entity classes don't have circular references between each other. The JavaScriptSerializer will not be able to serialize circular references and will throw a stack overflow exception. One way to deal with this is to null out the relationships or make the relationship an internal property. Unfortunately, both these techniques are the victims of bad programming practices.

MongoDB Fetching Strategies

MongoDB provides a number of fetching strategies, each of which is useful for a particular scenario. The C# driver for MongoDB closely mimics the syntax for using the MongoDB database. This section provides some of the basic fetching strategies from the point of view of the C# driver.

FindAll: The parameterless FindAll method can be used to retrieve all the documents in the collection. The FindAll method returns the ICursor object, which enables the developer to iterate over the documents. You can also use the Documents property of the ICursor, which returns the IEnumerable collection.

var documents = products.FindAll().Documents;   

FindOne: The FindOne method takes a single argument of Document type, which is used as the criterion to match against the documents that are stored in the database. Please note that all the attributes defined in the criteria document are matched against the persisted document, and null is returned even if a single attribute is not matched correctly.

var spec = new Document();spec["Name"] = "Coward Lion";var document = products.FindOne(spec);

FindAndModify: FindAndModify is a new method that was added to the later versions of the MongoDB C# driver. The FindAndModify method can be used to find and update the document by using the criteria that is defined in the document. The FindAndModify method also returns the updated document to the user.

A note about the unique identifier attribute: As mentioned before, the _id attribute of the document is used to uniquely identify the document. You can use any unique identifier to serve as an ID, but it is recommended that you use the default _id. When using domain objects in an application, you can expose a custom _id property, which will serve as the unique identifier, as shown in the following code:

public class Customer   {   public string _id { get; set; }   public string FirstName { get;set; }   public string LastName { get;set; }   public string Email { get; set; }   }

If you use the technique shown in the code example, it is recommended that you don't assign a value to the _id attribute. The _id will be managed by the MongoDB driver. When trying to fetch the document using the _id attribute, you must first convert the _id into the ObjectId format. You can accomplish this by using the ToOid extension method, as follows:

public static Oid ToOid(this string str)   {       if (str.Length == 24) return new Oid(str);       return new Oid(str.Replace(""", ""));   }

The ToOid method converts a string to the ObjectId representation, which can be easily used to fetch the documents from the MongoDB database.

Creating Indexes in MongoDB

Indexes in MongoDB serve the same purpose as they serve in the relational world, which is to make the database perform better and faster. By default, MongoDB creates a unique index on the _id attribute of the document, which can be viewed by using the code shown in Figure 10.

Figure 10: Displaying indexes on the Customers collection

Creating an index is a straightforward task that can be easily performed by using the MongoDB interactive console. Figure 11 shows how to create a unique index on the UserName property of the Customers collection.

Figure 11: Creating a unique index on the UserName property

The unique index ensures that no two documents will have the same value for the UserName attribute.

Figure 12 shows the unique index in action, which throws an exception when a duplicate UserName is inserted into the database.

Figure 12: Throwing a duplicate key exception

The C# driver provides the Insert method with the safeMode parameter, which can be set to true to validate the indexes. The safeMode parameter makes sure that all indexes are evaluated and the document is inserted only when the evaluation is successful.

MongoDB Viewers

Almost all relational databases provide a way to view the contents of the database via an easy-to-use interface. By default, MongoDB comes with the interactive shell that can be used to view the records in the database. If you're looking for something more graphical, there are several tools that might fit your needs. For information about some of the popular MongoDB Admin UI tools, visit the Admin UIs page.

Using MongoDB with ASP.NET

Speed and ease of persistence make MongoDB an excellent candidate to be used in ASP.NET applications. In this section, we will provide a common place where we can use the MongoDB database.

ViewState persistence. ASP.NET Web Forms applications manage page state by using the ViewState object. One drawback of using the ViewState object is the increase in the page size. Traditionally, this problem has been solved by moving the ViewState into different persistent medians. Unfortunately, most of the storage systems lacked sufficient retrieval speed, which affected the overall performance of the application.

MongoDB steps in as an excellent candidate for storing ViewState data. You can find more information about using MongoDB for ViewState storage at HighonCoding.com.

Data storage for monitoring systems. Monitoring systems keep track of the events that have occurred in the application. There are several monitoring systems available in ASP.NET, but the two most prominent are ELMAH and the ASP.NET built-in health monitoring system. Almost all the monitoring systems collect a lot of data in a short interval of time. Due to the speed and scalability of MongoDB database, it is the perfect candidate for storing this kind of information.

Explore New Territory

The NoSQL movement, comprising MongoDB and a number of other document databases, is gaining popularity in the database world. Large corporations such as Facebook, Twitter, and Digg have already moved to a NoSQL database. It seems likely that document databases will soon be competing head-to-head with traditional relational databases. For ASP.NET developers, then, now is a good time to explore the possibilities of MongoDB.

Mohammad Azam ([email protected]) is a principal consultant at Sogeti, a Microsoft ASP.NET MVC, and founder of the knowledge-based website www.highoncoding.com. He maintains his technical blog at www.azamsharp.com. When not programming, Azam likes to spend time with his beautiful wife, Naila Sheikh.