Is there a specific configuration needed to allow a web page on the free App Service tier communicate with a database defined in web.config? I looked up the IP Address of the website, http://wikistats.azurewebsites.net/, and added that to the firewall list on the database server. Obviously, a web farm may be behind this DNS name which would cause it not to work. It's my first attempt with Azure; I must be missing something.

You should also allow azure service to connect to the database. That's a setting in the database firewall settings.

See here for more info: https://azure.microsoft.com/en-us/documentation/articles/sql-database-configure-firewall-settings/