I was speaking with Michael Stonebraker this morning. I mentioned that lately many have been referencing comments he has made over the last couple of years. And I also mentioned that many had interpreted them as he was implying the RDBMS is “doomed”. Mike has been saying the same thing for years, but the current NoSQL movement seems to have picked up on this and highlighting one of the RDBMS's own pioneers is predicting its demise.
I asked Mike to clarify this. My interpretation of his response is as follows. I understand that he doesn’t believe the relational database itself is doomed. Instead the current general purpose implementations, or “elephants” using his words, were out of date. By moving away from a historical GP function into something more specific in focus, either in transaction processing or analytics, you can easily get 50x performance improvement over GP RDBMS. This doesn’t necessarily mean moving away from the “relational” nature, but instead changing some core design principles for how a RDBMS is implemented. It is this improvement factor that will see “new” specialist platforms overtake “old” general purpose platforms. That is gradually, over time. However Mike also mentioned the relational data model doesn’t make sense in a number of disciplines, particularly in sciences, and alternative modeling paradigms will offer benefits to this market (hence his focus on SciDB). So while relational is a valid data model, other data models are also needed.
I have a similar position to Mike, but perhaps with a few differences.
- Firstly I agree with the mantra that current GP RDBMS platforms provide only a “middle of the road” capability, and we gone too far in using a GP RDBMS for everything. However I do believe there is a long term future for the GP RDBMS. A general purpose application requirement will continued to be well suited for a general purpose platform. With a specialist only approach, a general purpose requirement may need both a specialist OLTP platform and a specialist Analytics platform to provide the same capability.
- I agree that with an extreme requirement, either analytics or transaction processing, a specialist platform is well suited. But I don’t see the choices of just MPP or memory resident RBDMS as being a broad enough set. Apps that use a db just as a persistence cache will benefit from a high performing, scalable database platform with much tighter integration with the object model. I am not sure any of the current NoSQL platforms have it quite right yet, but when these guys eventually get together with the database guys and work on these things together they may get there.
- I don’t think a 50x performance speed up on its own is enough to drive change in OLTP. I have written before how difficult it is to get into this market and how tight Oracle, Microsoft & IBM have this sewn up. But I don’t believe it is impossible, I think you just need to bring slam dunks on multiple fronts (performance just being one of them).
Anyway I feel like I am a bit of a broken record at the moment. I have been addressing the “is the RDBMS doomed” question a couple of times a day for some time. Time to focus on something else for a bit.
There will be plenty of detailed coverage on Exadata V2 so I won’t attempt to replicate that. However I do have a couple of initial thoughts which I would like to share. For those who missed it, Oracle has just announced Exadata V2 (which is their pre-built “machine”). Exadata V1 was built using HP equipment, Exadata V2 is using Sun. The main addition to Exadata V2 seems to be an extra tier in the memory hierarchy, a flash cache. Oracle is very quick to point out this is not flash disk, but it is flash memory, Sun’s FlashFire technology (flash disk or SSD’s was always going to be a transition technology, flash memory doesn’t have the physical constraints of moving parts disk so the whole “disk” concept for flash doesn’t make too much sense other than it fits easily with current architectures).
The new memory layer (Processor Cache’s -> DRAM -> Flash Cache -> Disk) coupled with Oracle’s algorithms to effectively use the Flash Cache layer brings performance benefit to the solution (+ all the other improvements 12 months of hardware innovation brings, faster CPU’s, more memory etc).
My initial thoughts are:
Kudos to Oracle. They are the first vendor to really bring a bunch of this leading edge technology together in a semi-mainstream way. Flash Cache, Inifiband interconnects, DBMS optimizations using flash hasn't really surfaced anywhere outside of startups yet.
So what happens to Exadata V1 customers using the HP solution? This is only about a year old. Some analysts are suggesting there has only been minor sales of Exadata V1 (I am not an analyst so don’t really know). So why would HP continue to support a platform where no new sales will be created, when potentially only a limited number of customers have it today? Possibly Oracle will offer attractive terms to move existing HP Exadata V1 customers to Sun Exadata V2.
It is a preconfigured solution that you by in certain size configurations. Small, half rack, full rack, multiple racks. I think Larry said that 3 racks will give you a PetaByte of storage capacity. This is fine, except they are targeting it for use with OLTP and data warehousing workloads. It seems odd that to get very high computational resources for transaction processing, you would also get massive volumes of potentially unnecessary storage capacity. It will be interesting to see if they allow the balance between processing & storage to be modified as part of configuration.
I have had some questions along the lines of “isn’t this back to the one size fits all approach?” Well yes it is, but Oracle never really moved away from this in terms of the core DBMS. It is my understanding that Oracle Exadata was still the general purpose Oracle DBMS & RAC but on a hardware platform optimized for accessing large data sets (making it a data warehousing solution). Using FlashFire, the hardware can now do high levels of random I/O (I think 1m random I/O’s was quoted) which makes the hardware platform general purpose as well.
One interesting question will be if, under Oracle, other vendors can buy the exact same hardware configuration from Sun and optimize their DBMS for Flash also? If so, it may be difficult for them to do this in a way that is price competitive. And will competitive DBMS vendors really want to help fill Oracle’s pockets further?
If we expect to see more of this hardware alignment between DBMS vendors where does that leave Microsoft? Maybe HP is already peeling the Exadata V1 logos off their racks and sticking Microsoft Madison logo’s in their place?
Oracle has put out a FAQ which partly answers some of the questions.
I haven’t blogged in over a month now. This is for a number of reasons. Firstly I have been flat out with various activities. This included a trip to VLDB in Lyon mid month. Secondly, a lot of the companies I have spoken with this month aren’t ready to speak publically so hence no blog posts resulting from these sorts of discussions.
However there has been a wiff of a change in the air in terms of focus that is interesting and worth highlighting. After years of lots of innovation around data analytics, OLTP is starting to make a comeback in terms of reclaiming some of the limelight. Much more on this between now and the end of the year, but a couple things to watch: