Hi,
If you want to develope a farm solution for SharePoint, we need install Visual Studio in a SharePoint server.
If you only want to develope a Client-Side project for SharePoint, we can install visual studio in our local PC and use some Client Object Model to access SharePoint.
About multiple developers for SharePoint project, here is a discussion for your reference:
http://sharepoint.stackexchange.com/questions/83/working-in-a-team-of-multiple-sharepoint-developers
The following articles about build a SharePoint development environment for your reference:
Start: Set up the development environment for SharePoint 2013
https://msdn.microsoft.com/en-us/library/office/ee554869.aspx?f=255&MSPPError=-2147217396
Setting up a SharePoint 2013 Development Environment 101
http://blogs.technet.com/b/wbaer/archive/2012/10/10/setting-up-a-sharepoint-2013-development-environment-101.aspx
Set up an on-premises development environment for apps for SharePoint
https://msdn.microsoft.com/en-us/library/office/fp179923.aspx
Best Regards