Hi,
Is it possible to query ipfs pinned cids by date/time?
For example return all new cids pinned from 01/JAN/2024 up to now?
Is it possible to also query unpinned ones in the above date range?
(Specifically BadgerDs which does not use flat files)
Not in ipfs.
In cluster, pins have a timestamp entry attached which should correspond to the time that they were added (iirc).
Querying unpinned ones is not possible easily, unless you go into manually parsing the CRDT chain and extracting data from there. But there would be no timestamp attached to those events, and thus time would have to be inferred from other pin events and causality relations to them.
In cluster can we query the pinned ones based on time or we should fetch all data and then filter?
Fetch all and filter.
the problem with /pins (boht pin ls and allocations) in cluster is that for some reason the created date is updated frequently for the same pin without anything changed! I am not sure why the date keeps being updated.I also tried the ipfsclute rapi in go and for some reason the statusAll does not return the timestamp although all other fields are there! Am I missing something?
I have repinning enabled but nothing should be happening that would cause a repinning (like going offline and online). how can I check the events log?
What field are you using? The timestamp field in /allocations and the Created field in /pins should be equivalent and correspond to the moment the pin was submitted to the consensus layer.
I checked both fields and it seems they keep getting updated for some reason every 15min or so and get a new timestamp without any new activity in the cluster
for now I am parsing the blocks folder directly to find out the pinned items filtered by time instead of relying on the timestamp
I checked the code and I’m inclined to think that if the timestamp gets updated it means that the pin is getting re-pinned. Do you have automatic repinning enabled (disable_repinning = false) ? Has the pin reached pinned state, is it pinning or does it report errors? Do the dates correspond to the time you are calling the command? Do they update every X minutes but stay stable in between?
yes, I enabled repining. but shouldn’t repinning only happen when something changes in the cluster, like number of nodes goes below min pin, or something?
Since in my cluster, there is no pinning error and the replication is more htan min replication set for the cluster
Is it repinning though? Do you have a node going offline from time to time and causing general re-allocations? Do all peers connect fine to the rest (any peer could trigger repinnings). The logs should show if things are getting repinned.
Can you disable and see if the date stays stable?