Micron Goes Open Source With Its Heterogeneous Memory Storage EngineApril 29, 2020 by Cabe Atwell
How does Micron's hefty memory hardware portfolio qualify them for this shift?
Micron recently announced that its HSE (Heterogeneous-Memory Storage Engine) platform designed for SSDs, storage-class memory, and other storage applications, is now available to the open-source community. The company claims this is the world's first open-source storage engine built for SSDs and storage-class memory.
Micron trumpets its HSE for offering lower latencies and enhancing software-defined platforms.
A "Heterogenous" Memory Storage Engine
Storage engines are an underlying software layer that DBMS (database management systems) use to create, read, update, and delete (CRUD) data from a database. In most cases, DBMS systems feature their own APIs (application programming interfaces) that circumvents the need to enter DBMS and interact directly with the underlying engine.
Micron says their hardware offerings helps them to vertically integrate this new open-source HSE into their broader systems knowledge. Image used courtesy of Micron
In the case of Micron’s HSE, the “heterogeneous” portion of the memory storage engine refers to its ability to simultaneously use different storage media, including SSDs, storage memory, and next-gen non-volatile technologies.
"Uniquely Positioned" to Build a Software Stack for Flash Storage
In a recent press release, Micron's VP of the storage business unit Derek Dicker stated, “As the only company developing storage class memory, flash, and DRAM technologies, Micron is uniquely positioned to build a software stack that accelerates applications running in today’s flash-based storage environments as well as storage class memory-based infrastructure of the future."
He added, "We have delivered a first-of-its-kind innovation for open-source storage developers, unlocking the full potential of high-performance storage applications.”
A Focus on SSD Endurance, Throughout, and Latency
Along with reducing latency in large-scale data sets, Micron also states their HSE solution will boost SSD endurance by a factor of seven. The company also states their technology improves the throughput of particular storage applications by up to six times and reduces latency by eleven times.
The Micron infographic notes the differences between the company’s HSE storage engine compared to traditional engines that are not optimized for Flash and storage-class memory. Image used courtesy of Micron
As mentioned earlier, HSE can exploit multiple classes of media, including QLC 3D NAND Flash and 3D XPoint technology. Micron states that with the addition of their X100 NVMe-based SSD (pictured above) paired with 4X Micron 5210 quad-level cell NAND SSDs, users can see throughput doubled and latency reduced by a factor of four.
Leaders in Hardware Enter Open-Source Software
Micron has been around for more than 40 years and touts itself as a leader in innovation, making breakthroughs with DRAM, NAND, SSDs, GDDR, NVDIMM, and the company’s 3D XPoint technology.
Micron claims their X100 is the fastest SSD in the world. Screenshot used courtesy of Micron
As a result of its hardware innovation, Micron produces its own software solutions to provide the best possible performance for those storage technologies. HSE is used to optimize the previously noted performance with throughput, lower latency, and SSD endurance.
Why Micron Chose MongoDB
Micron also created an interface for the HSE that allows it to connect to MongoDB, the most widely used NoSQL database in the world, according to the DB-Engines knowledge base. The company chose MongoDB over other offerings because it provides open API for development and can be better optimized SCM and SSDs. The integration allowed the company to compare its HSE engine to MongoDB’s storage solution and found performance increases that ranged two to six times over.
What Can the Open-Source Community Accomplish With HSE?
Micron has released its HSE as an open-source platform (for Linux) on GitHub that anyone can utilize to increase the performance of their SSD or storage memory solutions, and is eager to see what the open-source community can accomplish using the platform.
As a hardware engineer, are you particularly interested in memory software going open source? Why or why not? Share your thoughts in the comments below.