what is memory leak is
Generally you won't see a memory leak in SQL Server. There have been a few rare bugs that have cropped up over the years, like the varbinary over linked server leak. If you run CLR, it's possible you may have a leak in the .NET code. It all depends and it's
not necessarily ever a leak (unless it's JAVA) :) haha!
In some leak cases you will see ERROR 703 messages in the sql error log when a process is leaking.
Do you have a specific scenario you're trying to troubleshoot? What version and build are you running? Have you installed the lasted SP + CU?
For processes outside of SQL Server I'd use perfmon and review the PROCESS\PRIVATE BYTES counter to see if it continually grows, which would be an indicator of a leak but may also indicate the process is just being used more.
I hope you found this helpful! If you did, please vote it as helpful on the left. If it answered your question, please mark it as the answer below. :)