Big Data, Data Analytics and Data Science | BI-BIGDATA.com

I was working on one Social Media Streaming proof of concept where problem statement was which NoSQL Database is best fit for social media network. I’m big fan of MongoDB and I was planning to use MongoDB in this case but I wanted give another shot for other NoSQL databases where I chose Neo4j Graph Database.

As MongoDB is document store where we can store JSON and BSON documents that require fast writes or reads as it’s primary feature is storing and indexing schemaless documents. But retrieving data from MongoDB requires complex queries which is really challenging job and requires lot of efforts in terms of aggregation framework and Map Reduce.  MongoDB is highly available and consistent but not capable at handling relations as it doesn’t maintain referential integrity between DBRefs.

The primary advantage of Neo4j is its lightning fast ability to do very complex queries with unlimited depth and weighted connections. So you should definitely store graphs like “Users” and relationships like “Friendship” or “Subscriber” inside Neo4j. This way you can easily do queries like retrieving distinct friends of friends that like the same music albums as you.

MongoDB works much better when all of the data on a given partition can fit in memory.  What makes it scale is that many of data collections that can be easily stored in MongoDB are also easily partitioned.  This lets you spread your data across a number of nodes letting you fit data sets which are larger than the memory capacity of a single server still nearly entirely in memory. Partitioning graph data can be hard, it depends on the graph.  It is particularly challenging to find ways to keep large graphs (in general) entirely in memory when you outgrow your commodity hardware resources. Vertical scaling of your graph server is going to reach it’s threshold when you exhaust all your hardware resources.

Now you can think of using hybrid approach that may find useful, but make sure you are aware of the scalability concerns of each database. By using Neo4j to store relationships between “Users” and other properties which you will be using frequently. By keeping a store of their profile information as a document in MongoDB. In Neo4j GraphDB, using a MongoDB ObjectID to reference the user’s profile document and other documents/data that I do not use very frequently.

It is to be noted that for 90 percent plus of application use cases, it will be more than sufficient using one database or the other. Using two different databases can even have a negative impact on performance, especially when sharding database instances across multiple machines. Also to be noted is that Neo4j does not yet support auto-sharding.

To sum it up, I see comparing Neo4j with MongoDB pretty much as comparing apples to oranges (I’d be happier if someone corrected me here). You use a graph when your data is highly interconnected and needs to be navigated, else if your data is nothing but a bunch of denormalized structures, you go with a plain document store!

Here is System Properties Comparison MongoDB vs. Neo4j for your reference.

I hope this article was helpful to you.

Stay tuned.

Sandip