You have swarm control problem.
ipfs can be seen as a huge p2p disk, putting all together any CID anyone could put in here.
So if you bootstrap to the public default nodes, then your swarm peers become anyone, you are thrown in a too big datastore for your node to focus on your data.
I found a way to take control over my swarm using scuttlebutt relationship. Then you can have a quick access to your reduced ipfs file store.
I am now using https://gchange.fr identities as ipfs node identity, so each node “possess” a Libre Money wallet…