Overview
Challenge
The demand for data-driven decision making coupled with need to retain data to meet regulatory compliance requirements has resulted in a rapid increase in the amount of archival data stored by enterprises. As data generation rate far outpaces the rate of improvement in storage density of media like HDD and tape, researchers have started investigating new architectures and media types that can store such “cold”, infrequently accessed data at very low cost.
Synthetic dna
Synthetic DNA is one such storage media that has received some attention recently due to its high density and durability. DNA possesses three key properties that make it relevant for archival storage. First, it is an extremely dense threedimensional storage medium that has the theoretical ability to store 455 Exabytes in 1 gram; in contrast, a 3.5” hard disk drive can store 10 Terabytes and weighs 600 grams today. Second, DNA can last several centuries even in harsh storage environments; hard disk drives and tape have life times of five and thirty years. Third, it is very easy, quick, and cheap to perform in-vitro replication of DNA; tape and hard disk drive have bandwidth limitations that result in hours or days for copying large Exabyte-sized archives.
Proof of concept experiments and results
In this three year project (€3M funded by the EU), we will research all relevant aspects of DNA storage in a consortium of six partners across three countries (UK, France, Ireland) bringing together all necessary expertise. We will research all relevant technologies such as encoding different types of data in DNA, scalable DNA synthesis to store data, experimental techniques to manipulate the data, efficient sequencing and decoding approaches to read back the data as well as automation of all aspects. The final result will be an end-to-end prototype for storing data in DNA and for reading it back.
In initial work we have developed OligoArchive, an architecture for using DNA-based storage system as the archival tier of a relational database. We demonstrate that OligoArchive can be realized in practice by building archiving and recovery tools (pg_oligo_dump and pg_oligo_restore) for PostgreSQL that perform schema-aware encoding and decoding of relational data on DNA, and using these tools to archive a 12KB TPC-H database to DNA, perform in-vitro computation, and restore it back again.
Our initial results are summarised in the paper available here.
A factsheet/summary of the project can be found here.
People
Thomas Heinis
Imperial College
PI & Coordinator
Raja Appuswamy
Eurecom
PI
James MacDonalD
Imperial College
PI
Paul Freemont
Imperial College
PI
Pascal Barbry
Universite Nice & CNRS
PI
Marc Antonini
universite nice & CNRS
PI
Sachin Chalapti
Helixworks
PI
Nimesh Pinnamaneni
Helixworks
PI
CONSORTIUM PARTNERS
PUBLications
2021
DNA archival storage, a bottom up approach Conference
ACM Workshop on Hot Topics in storage and File Systems, 2021.
Nanopore Sequencing Simulator or DNA Data Storage Journal Article
In: Visual Communications and Image Processing (VCIP 2021), 2021.
DNA4DNA: Preserving Culturally Significant Digital Data with Synthetic DNA Journal Article
In: 17th International Conference on Digital Preservation (iPRES 2021), 2021.
Digital Preservation with Synthetic DNA Journal Article
In: 37eme Conference sur la Gestion de Donnees – Principes, Technologies et Applications (BDA 2021), 2021.
2020
Efficient Storage of Images onto DNA Using Vector Quantization Journal Article
In: 2020.
Storing Digital Data into DNA: A Comparative Study of Quaternary Code Construction Inproceedings
In: ICASSP, Barcelona, Spain, 2020.
2019
OligoArchive: Using DNA in the DBMS Storage Hierarchy Inproceedings
In: CIDR 2019, 9th Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, 2019, 2019.
Efficient Approximation of Sequence Hybridization Inproceedings
In: DNA Computing and Molecular Programming, 2019, ISBN: 978-3-030-26807-7.
Encoding Information in Primers Inproceedings
In: DNA Computing and Molecular Programming, 2019, ISBN: 978-3-030-26807-7.
A Biologically Constrained Encoding Solution for Long-term Storage of Images onto Synthetic DNA Inproceedings
In: EUSIPCO 2019, 27th European Signal Processing Conference, Coruna, Spain, 2019.
Cold Storage Data Archives: More Than Just A Bunch of Tapes Inproceedings
In: DAMON 2019, 15th International Workshop on Data Management on New Hardware, Held with ACM SIGMOD/PODS, Amsterdam, Netherlands, 2019.
2018
DNA Coding for Image Storage Using Image Compression Techniques Inproceedings
In: CORESA 2018, 20emes journées d'étude et d'échange sur la COmpression et la REprésentation des Signaux Audiovisuels, Poitiers, France, 2018.