paperlined.org
dev > concepts > disk-based_file_formats

document updated 16 years ago, on Jun 30, 2008

DIY variable-length data store

The first thought is to have a FAT-like table at the beginning of the file, that points to BLOBs scattered throughout the file, which are allocated via some malloc-like method. (possibly buddy allocation to reduce fragmentation somewhat)

However, that means you have problems if you ever need to increase the size of the FAT/TOC. So, instead, make the first word of the file point to where the FAT/TOC resides, and treat the FAT/TOC as just another block that can be malloc'd. When it needs to grow in size, malloc another block of the new size, copy the TOC over, update the first-word pointer to the new TOC, and free the old TOC.

You PROBABLY WANT TO USE a modern dbm for this instead though, since implementing the above is essentially re-implementing a dbm.