Chunk distribution diagnostics

[plexus]

For testing purposes, I would like to log the peer from which each chunk is fetched, when a pin is executed.
Is there any way to achieve this?

[plexus]

I could do
"ipfs ls <file_hash>"
and then
"ipfs dht findprovs <chunk_hash>"
for all the chunks, to get a list of providers for each chunk of the file.

But is this list the priority order IPFS uses to get the chunks?

[leerspace]

@plexus

For testing purposes, I would like to log the peer from which each chunk is fetched, when a pin is executed.
Is there any way to achieve this?

I think you can do this using the debug output as described in the linked thread.

Identify chunks' owners

I’m curious. What’s the use case for this? I’m not sure what you tried previously with the debug output, but I think that’s probably the way to go. You can do this by parsing the debug output from ipfs daemon and then looking up the addresses for the peer that sent the block. For illustration, here’s an example where I’m interested in who is sending me a particular block after I request it. $ ./ipfs --debug daemon 2>&1 | grep QmQHQHv9Ao16AasNS21JfW8j1QkPjBVCq6Vj7RwMmwUzsr 07:29:53.899 DEBUG …

[plexus]

@leerspace thanks for the info.
I’m afraid i cannot use "--debug" in my usecase.

Is there any chance, when a chunk is needed, IPFS alters the list returned by "ipfs dht findprovs <chunk_hash>" by considering other network metrics?

If we assume that there will be no peer downtimes, is it safe to say that each chunk will be fetched from the first peer of the above list?

EDIT:
The method i mentioned above is not working as expected, since "ls <file_hash>" fetches the file and adds local peer as a provider.

[leerspace]

@plexus

Is there any chance, when a chunk is needed, IPFS alters the list returned by “ipfs dht findprovs <chunk_hash>” by considering other network metrics?

If we assume that there will be no peer downtimes, is it safe to say that each chunk will be fetched from the first peer of the above list?

I don’t for sure, but I’d be really surprised if it does. Regardless of which peer the block is requested from first (even if you ignore your peer ID showing up in the list), there’s no guarantee they’ll be the first to send it; even if we assume that all nodes are friendly, the first node might be too busy to respond in a timely fashion due to processing other requests.

@plexus

EDIT:
The method i mentioned above is not working as expected, since “ls <file_hash>” fetches the file and adds local peer as a provider.

This doesn’t solve your issue, but you can prevent this behavior by changing the reprovider strategy in your node’s config.

ipfs/kubo/blob/master/docs/config.md#reprovider

[plexus]

When using "dag get <file_hash>" instead of "ls <file_hash>" the local peer is added to the hash providers, but the contents of the hash are not downloaded!

So i guess i can use this (followed by "dht findprovs <chunk_hash>") to know, if not the actual sender of a chunk, how many peers is each chunk stored in, without downloading the entire file to local peer.

Can someone of the code gurus confirm that?

EDIT:
@leerspace i saw your answer after i wrote this, i will dig into it.
I am (slowly) approaching what i want to achieve: Having the knowledge of where the chunks of a specific hash are stored, but without altering the provider list of any hash involved.