Next big event or bust?
The storage of holographic data has been discussed for decades. Once referred to as the next generation of optical storage, it promised higher densities and access speeds than current Blu-ray discs.
Over the years, many research teams have attempted to build holographic systems capable of meeting the increasing demands for data storage. However, these teams achieved little concrete results beyond the occasional prototype. Their efforts, however, were not in vain. Microsoft has breathed new life into holographic storage with Project HSD, a collaboration between Microsoft Research Cambridge and Microsoft Azure, whose goal is to bring holographic technology to cloud-scale storage.
Why holograms for storage?
Holographic storage – sometimes referred to as 3D storage – is a volumetric storage system that uses lasers to read and write data, similar to other optical storage. However, media such as CDs, DVDs, and hard drives can only store data on the media surface, limiting its capacity to two-dimensional storage. Holographic storage uses the entire volume, allowing more data to be stored in a smaller space and increased data writing and reading speeds.
Polaroid researcher Pieter J. van Heerden first proposed storing holographic data in the early 1960s, shortly after the invention of the laser. By the early 2000s, research teams from industry and academia had made significant strides in demonstrating the technology’s potential. Two major efforts were made by Aprilis, a spin-off of Polaroid, and InPhase Technologies, a spin-off of Bell Labs. Both companies have tried to commercialize holographic storage. Ultimately, however, neither has achieved commercial success. Dow Corning acquired Aprilis and InPhase ultimately filed for bankruptcy.
There were plenty of other efforts as well, but none could turn the tide on storing holographic data. Most of these attempts focused on using circular media similar to CDs or DVDs to support single write and multiple read (WORM) operations, but holographic storage competed with more technologies. established, which themselves had also progressed.
Hard drives, for example, have become faster and denser, and SSDs have become cheaper and more durable. At the same time, the reliance on cloud computing has increased, which has brought scalable storage and extended streaming capabilities.
Despite these trends, the need for innovative storage platforms continues to grow. According to Microsoft, the world will generate 125 zettabytes of data per year by 2024. Businesses and cloud service providers need to find cost-effective ways to store this data and meet their demands for performance, availability and sustainability.
Current storage technologies are insufficient to maintain this volume of data. Hard drives, for example, are limited by their mechanical nature. SSDs are still relatively expensive to implement on a large scale and do not always provide the necessary endurance.
HSD project tackles holographic storage
To meet future storage needs, Microsoft launched Project Holographic Storage Device (HSD), a collaborative research effort which revisits holographic technology, but this time with the idea of ââproviding cloud-scale storage to support hot data.
The HSD project is part of Microsoft’s Optics for the Cloud group at the Microsoft Research Lab in Cambridge, England. Another group effort is the Silica project, which is experimenting with use of crystals to provide long-term archival storage. However, Project Silica focuses only on WORM operations, like traditional approaches to storing holographic data. The HSD project allows data to be erased and rewritten and will provide faster read and write speeds.
According to Microsoft, the project’s mission is to “design high-endurance cloud storage without mechanical movement that is both efficient and profitable”. Microsoft also states that the project has already achieves 1.8 times higher density than previous volumetric holographic data storage. The team is working to further increase densities and achieve faster access rates.
To help deliver on these promises, the HSD Project uses core components like the high-resolution cameras and display screens of today’s smartphones. The project also uses machine learning and deep learning to further improve accuracy and performance. As a result, the team reduced optical distortions and manufacturing tolerance requirements. It uses software to compensate and calibrate the system at run time.
The material used for the storage medium also sets Project HSD apart from other attempts at storing holographic data. Many other projects used polymers to store permanent changes in the material, so they were limited to WORM operations.
In contrast, the HSD project stores holograms in electro-optical crystalline materials. The project stores each hologram as a spatial variation in the electron density distribution, which it can alter by exposing the medium to light of a specific wavelength. Holograms can also be erased by exposing the crystalline material to ultraviolet light.
Despite Microsoft’s departure from traditional holographic storage, the basic approach to reading and writing data is much the same. The storage process begins with dividing a laser beam into two signals. One of the bundles carries the data to the storage medium. The data-carrying beam – also known as a data, object, or signal beam – passes through a device called a special light modulator, which then passes or blocks the light at points corresponding to binary 1s and 0s. The modulated data beam then continues to the crystalline material.
The second light beam is called the reference beam. The beam does not pass through a light modulator but bounces off a mirror and is redirected to the storage medium, where it crosses the data beam to create a 3D interference pattern in the optical material.
The pattern forms a tiny hologram that represents a single page of data, which can contain hundreds of kilobytes of data. A data page occupies a small volume, or area, in the optical material. An area can contain multiple pages, and the storage medium can contain multiple areas.
A holographic storage device reads the data by diffracting the reference beam of the hologram into the storage medium. The data bundle is not required for this operation. A camera captures the diffracted image, which reconstructs the original data page. A holographic storage system can read different holograms by changing the angle of the reference beam, or it can erase the holograms with UV light, which can rewrite the data.
Uses for storing holographic data
In its use of crystalline materials, the HSD project takes advantage of the parallelism inherent in optics. It allows data to be written to and read from storage media in parallel, resulting in higher overall throughputs. The project approach also requires fewer mechanical parts, such as those found in hard drives. Instead, it limits movement to readjusting the angle of the laser beam; all other components remain fixed. In addition, holographic storage can use the entire volume of the medium, rather than just its surface, which offers higher densities than current types of optical storage.
Traditional approaches to holographic storage have focused on data archiving and supporting WORM operations. The HSD project targets hot data that supports read and write operations, which could benefit cloud providers and enterprise data centers. Hot data is typically accessed and updated less frequently than data that supports large business applications and is rarely managed in real time, although it typically requires more scalability. Performance requirements may vary depending on the workloads supported.
Holographic data storage promises a cost effective way to answer specific business questions efficiently and quickly. Its fast reading performance and ability to update data make it well suited for data warehousing, big data analysis, and applications that integrate advanced technologies such as predictive analytics or artificial intelligence. .
Organizations that need to generate regular reports, such as weekly call center statistics or monthly sales figures, could benefit from storing holographic data. Holographic storage could also support less critical operations, such as providing support staff with the basic information they need to help their customers.
How close are we to holographic storage?
Despite its promises, holographic storage has a long way to go from the research phase to the point where companies can buy commercial products. Manufacturers are expected to set up entirely new environments for building storage devices, which will require a high degree of precision to ensure proper alignment between components. Additionally, most of the research on the HSD project focused on writing and reading from a single area. The team still faces the challenge of delivering the same level of performance in multiple areas. Another concern is to ensure that accidental exposure to UV light does not erase the data.
Because holographic data storage is a technology with so many false starts, it’s no surprise that Microsoft has avoided making predictions about the commercial application of the technology. In the meantime, there are many other efforts being made to entertain the industry, ranging from storage class memory to DNA storage.