How exactly Raft works in IPFS-Cluster?

Hi There,

If I understand it right, Bitswap is a main protocol for exchanging data in IPFS. And Raft is an consensus algorithm for managing a replicated log, that allows peers to agree on it’s state. So Raft itself has nothing to do with data transferring.

Is it possible scenario that Raft changed its log but the data is not physically distributed to all or even to one node in a Cluster?

Where can I read in more details how ipfs-cluster services interact with Raft log? Or maybe someone can explain it.

No.

You mean if it’s possible that the Cluster state (as represented to peers by Raft) changed -by adding a new pin to it-, but the pinned content did not arrive to the peers on which it’s pinned? Yes it’s possible. Cluster only asks IPFS peers to pin. Finding and fetching this content is IPFS’s task.

Understand that while, internally, Raft is a log of entries, the external view of this is just a map from [cid]->PinInformation
that is the same in all cluster peers. The Raft state (or shared state is what ipfs-cluster-ctl pin ls shows).

If you didn’t see it, there is some information here: https://cluster.ipfs.io/documentation/internals/

1 Like

Thanks, Hector!

What about an alternative consensus algorithm(“Collaborative pinsets prototype”) which is planned to be released by the end of Q1 2019 and replace Raft, how will it be different from Raft?

We plan to use ipfs-backed CRDT structures broadcasted via Pubsub. So it will not be consensus but it will do the same thing without the current problems to declare and update the peerset.