All the clients in the network are recieving errors:

Forefront TMG cannot handle your request because the DNS quota was exceeded

unable to RDP to TMG server which holds the public DnS.

We needed to reboot the server.

How to solve this please?

I would focus on the DNS server itself.

Any events in the event log related to DNS (including the service)?

Has it happened before?

What does TMG logs say, excessive amount of requests at the same time?

After rebootin the server it went well.

I have opened another thread:

https://social.technet.microsoft.com/Forums/en-US/4aa6bf00-681a-4d21-9f29-90f3171524da/forefront-tmg-disconnected-a-nontcp-connection-from-domain-controller?forum=Forefrontedgegeneral

Where is a default location of forefront Flood Mitigation logs?

with best regards

I kind of suspected that the threads where related.

TMG firewall log is the place to look in.

Event logs will also have the same information as the alerts, use the alerts to correlate time and date to the logs.