I have an idea for a thesis on how to optimize the big data architecture using blockchain / IPFS, and analyze the data using smart contracts, is it possible to do?
Netflix is using IPFS for pushing software update: New improvements to IPFS Bitswap for faster container image distribution | IPFS Blog & News
Some guy tried and succeeded a long time ago: https://www.cse.unsw.edu.au/~hpaik/thesis/showcases/16s2/scott_brisbane.pdf
https://s3-ap-southeast-2.amazonaws.com/scott-brisbane-thesis/decentralising-big-data-processing.pdf
Hope it helps.
1 Like
thank you for the reply and the thesis document that you have included is very helpful for me.
I see the thesis that they are working on combining ipfs and hdfs, where ipfs as storage and hdfs in this context is map reduce as processing.
what I want to do is how ipfs optimizes hdfs, ipfs as data storage as well as data processing with the help of smart contracts. I expect faster runtime performance, and higher availability
2 Likes
Hi! I’m David Aronchick and I’m co-director of Research Development at Protocol Labs.
We’ve been thinking about this problem as well - I’d love to hear your thoughts if we’re going in the right direction! GitHub - filecoin-project/bacalhau