I guess most people using the X86 versions, so it might be that there is a undetected bug in the 64 bit arm version that’s cause it not to function well.
FWIW, I’ve been running a swarm of ~100-200 nodes on arm64, with maybe one x86_64 node. Mostly Raspberry Pi 4B, and it works well most of the time, the only problem I experience is `ipfs pin add` often hangs with a wantlist of ~7 CIDs - #5 by hector, and I’m told that may be fixed in 0.33.0-rc1 but I’ve not yet had a chance to try it.
So I don’t think your system being a Raspberry Pi 5 should cause you any problems. As the other posters have mentioned, I would be suspicious of your network setup - firewalls, NAT, etc.