With Great Power comes Great Responsibility
posted on February 8, 2016 by Amit Golander
Programming on top of a SQL database is relatively easy.
Because the SQL database simplifies your work. Of course it supports Join operations, but more importantly it maintains the ACID properties for you. So you, as the application developer have some peace of mind. You know that once committed, your data is persistent and safe, even if there is a full data center power outage.
That is why we feel confident when we deposit money. We can trust that the database is “last transaction safe”.
As social media emerged and later M2M & IoT, the simple SQL model couldn’t keep up. NoSQL databases were the widely adopted alternatives because they could deliver the required performance levels, but unfortunately they were missing important features that made programming the application more difficult. One of these compromises was to put data updates at risk. Juggling with extra balls in the air became a necessary evil for Silicon Valley’s brightest programmers.
Years later, NoSQL has been adopted to other use-cases and by other programmers. These new applications handle more important data than the “number of likes” and are coded by programmers that do not always understand the hardware semantics of persistency.
Wouldn’t it be nicer if we humans could juggle fewer balls?
Even if you’re an octopus, wouldn’t you be more productive focusing on fewer challenging interdependencies? Wouldn’t you drop fewer balls in such an environment?
With great power (NoSQL) comes great (programmer) responsibility.
Wouldn’t it be better if we could just benefit from the great power, without the additional responsibilities?
We recently tried exactly that. To do that, we leveraged the convergence of memory and storage to run MongoDB in a fully durable configuration – Last Transaction Safety Mode.
Zero data loss, best user experience, and the fastest recovery time.
The white paper, available here, shows that without other fundamental changes, even with advanced Flash technologies, this was impossible. Configuring a MongoDB server not to put your data at risk would result in severe performance degradation. The good news is that you can see in that paper what a difference our “other fundamental changes” made.
Using persistent memory and Plexistor SDM, we were able to show 450% more operations per second, at a fraction of the latency. This enables for the first time “a last transaction safe” NoSQL with no performance compromise.