Perfect storage medium

History / Edit / PDF / EPUB / BIB
Created: August 12, 2015 / Updated: December 18, 2015 / Status: in progress / 3 min read (~431 words)

The perfect storage medium is an important aspect of building an agent capable of storing and retrieving information as fast as possible. Thus, it is critical to define what would be the optimal ways to fetch and put data into storage as well as the current state of the art in that matter.

  • How to store an immense volume of data
    • While taking less space
    • While being easily to work with
    • While being as fast as possible
    • While still potentially allowing operations such as lookup or search

File systems and databases already implement various methods which may interest us. Thus, it is appropriate to look into existing tools to find how specific problems have been solved.

  • Databases
    • RDBMS
      • Unique identifier per schema
      • Foreign key relations
    • NoSQL
      • Graph based
      • Document based
        • UUID to uniquely identify documents and prevent collisions
  • Filesystem
    • Hierarchical data structure
  • Compression algorithms
    • Lossless
      • Compress repeating patterns using dictionaries
    • Lossy
      • Compress by removing data that has the least impact on the final result

  • Prevent duplication of data (compression)
  • Fast/Instant retrieval (lookup)
  • Fast/Instant storage (hashing)