Hi,
According to your post, I want to double ensure some points, please help correct answers for following question:
1. Whether you migrate Exchange 2010 to Exchange 2013 as well as public folder or not?
2. User in Exchange 2010 or in Exchange 2013 experience this issue?
3. Which Outlook client and VPN client are you used?
4. How about access public folder via OWA by external?
If this public folder exist in Exchange 2010, please refer to below link to configure legacy public folders where user mailboxes are on Exchange 2013 servers:
https://technet.microsoft.com/en-us/library/dn690134%28v=exchg.150%29.aspx
Besides, try other Outlook and VPN client version for testing, or upgrade to latest version.
Try run Outlook as Administrator to test. If the problem persists, run Test E-mail AutoConfiguration and check the URLs, it might be caused by autodiscover.
Thanks