Hello, I have an idea of the chunking algorithm. It should work better than Rabin fingerprint and is highly configurable. The algorithm builds a content-dependent Merkle Tree and can be used for deduplication and compression. I’ve implemented it in my content-addressable storage (CAS), and it works well and efficiently. Is there any interest in implementing it for IPFS? Where should I start the discussion? Or should I?
Is there any place where we can learn how it works?
1 Like
I have this article about CDT: Content-Dependent Hash Tree. In our early discussion, we highlighted… | by Sergey Shandar | Medium Also, I’ve implemented my own open-source CAS using one of the implementation of the algorithm: GitHub - datablockset/blockset: The BLOCKSET application is a command line program that can store and retrieve data blocks using a content-dependent tree (CDT) hash function as a universal address of the blocks.