Hi ManServ,
It is very likely that upgrading from v3 to v4 will be a supported upgrade path. If not, any incompatibilities between release versions will be mentioned in the blog post that announces the new version.
This aspect of deploying EMET was mentioned for EMET v3 and v3.5 Tech Preview:
http://blogs.technet.com/b/srd/archive/2012/05/15/introducing-emet-v3.aspx
http://blogs.technet.com/b/srd/archive/2012/07/24/emet-3-5-tech-preview-leverages-security-mitigations-from-the-bluehat-prize.aspx
In the first link above it was mentioned that upgrading from v2.1 to v3.0 was supported and would work as expected with no need to uninstall.
However since v3.5 was a Tech Preview upgrading from v3.0 to v3.5 Tech Preview required an uninstall (mentioned in the 2nd link above) but would still use any applications rules that were in place for v3.0.
From reading past User Guides the deletion of the following registry keys is usually only necessary when upgrading from a beta version of EMET to the released version e.g. section 7.4 General Questions on Page 32 of the EMET v3 Users Guide.
Upgrading to v2.1 from EMET v1.0.2 was also mentioned in section 6.1: EMET 1.0.2 Questions on Page 20 of the EMET v2.1 Users Guide.
Presumably this provides a clean installation of the release version and removes any registry keys that may have been used for debugging purposes. In addition, default registry settings may change from a beta to a release version.
As you are aware, due to the new features within EMET v4.0 Beta, there are more registry values than the older v3:
EMET v4.0 Beta:

Direct Link to Image:
http://i742.photobucket.com/albums/xx69/Jimboc/Microsoft/EMET_4_Beta_Reg_Settings.png
EMET v3:

Direct Link to Image:
http://i742.photobucket.com/albums/xx69/Jimboc/Microsoft/EMET_3_Registry_Settings.png
The above screenshots show the following registry key:
HKLM\Software\Microsoft\EMET
I hope this helps. Thank you.