So I just found out that my company uses one port for all our other countries central admin. I have tried to change the port but I am getting access denied and I am a farm user and I used the setup ID as well.

Tried to use https://technet.microsoft.com/en-us/library/Ff607832.aspx

No luck, any help would be much appreciated.

Just re-run PSconfig to remove Central Admin and then run it again to re-install it on a different port number.  I've Run PSconfig to re-establish corrupted Central Admin sites a number of times at client locations.

Thank you for the quick reply. This might be a stupid question but will that break the sites I have running ?

No.  It has no effect on anything other than Central Admin.