In order to connect to the peer you should only need the swarm address (not the API or Gateway address) to be open. By default on my nodes it’s using port 4001, but you can check your config file (or use ipfs config Addresses.Swarm) for which port the daemon is configured to listen on.
There’s more discussion on this here: How should I configure my firewall?
It’s possible there might be something else you need to do if you’re using a more complicated network setup than simply opening the swam port
I’d recommend updating IPFS to 0.4.13 as it has many enhancements and fixes not present in earlier versions. 0.4.14 is also pretty close to being released. 0.4.4 was released more than a year ago in Oct 2016.