Currently experiencing what appears to be a memory leak on a Win2k3 x64 machine. We initial noticed the issue when the server becoming unresponsive and needed to be hard reset. Upon investigation of the event logs we saw the event id 2019 was logged. Currently we are monitoring the consumed non-pool memory via poolmon and rebooting as necessary. Here is the server environment Win2k3 x64 Dell PE 2900 w/Broadcom NIC's 12GB RAM Backup Exec 12.5 TrendMicro Office Scan Server 8 I have run memtriage over a 2 1/2 period with several samples. The results can be found below. The following troubleshooting steps have been taken to attempt to resolve. -Update Broadcom NIC to latest driver as provided from Dell. -Cycled all related services to Backup Exec and TrendMicro Office Scan Server with poolmon open to see if consumed pool memory changes. No change registered. Any thoughts as to what is causing the Non Paged memory leak? Memtriage Analysis ================================ System ======================================== Name Inc-Trend Object Change Start End Percent Rate/hour System Always AvailableKByte -173724 10313780 10140056 1 -99264 System Always CommittedKByte 165712 1823896 1989608 9 94686 System Always NPagedPoolKByte 9216 108000 117216 8 5265 System Always PagedPoolKByte 7964 205928 213892 3 4550 ================================ Per Process ======================================== Name Inc-Trend Object Change Start End Percent Rate/hour HealthService.ex Sometime CommitKByte 532 39392 39924 1 303 DMServer.exe Sometime Handles 27 456 483 5 15 ================================ Kernel Pool Usage ======================================== Name Inc-Trend Object Change Start End Percent Rate/hour File Always AllocFreeDiff 331 45037 45368 0 189 File Always Bytes 93552 12265728 12359280 0 53454 ---- DRIVER : [<unknown> - File objects] IoNm Always AllocFreeDiff 138 38762 38900 0 78 IoNm Always Bytes 36496 10071536 10108032 0 20853 ---- DRIVER : [nt!io - Io parsing names] Job Sometime AllocFreeDiff 4 4 8 100 2 Job Sometime Bytes 2560 2496 5056 102 1462 ---- DRIVER : Unknown Driver LSnh Sometime AllocFreeDiff 16 28 44 57 9 LSnh Sometime Bytes 1024 1792 2816 57 585 ---- DRIVER : [<unknown> - nonpaged block header] LStr Sometime AllocFreeDiff 14 27 41 51 7 LStr Sometime Bytes 3808 7712 11520 49 2175 ---- DRIVER : [<unknown> - transaction] MmCa Sometime AllocFreeDiff 241 37651 37892 0 137 MmCa Sometime Bytes 42544 6596096 6638640 0 24309 ---- DRIVER : [nt!mm - Mm control areas for mapped files] MmSm Sometime AllocFreeDiff 245 36700 36945 0 139 MmSm Sometime Bytes 23520 3523200 3546720 0 13439 ---- DRIVER : [nt!mm - segments used to map data files] MmSt Sometime AllocFreeDiff 312 39417 39729 0 178 NtFS Sometime AllocFreeDiff 24 951 975 2 13 NtFS Sometime Bytes 5136 277616 282752 1 2934 ---- DRIVER : [ntfs.sys - SecurSup.c] Obtb Always AllocFreeDiff 31 360 391 8 17 Obtb Always Bytes 123008 1077760 1200768 11 70285 ---- DRIVER : [nt!ob - object tables via EX handle.c] Proc Always AllocFreeDiff 6382 10036 16418 63 3646 Proc Always Bytes 6739392 10598016 17337408 63 3850823 ---- DRIVER : [<unknown> - Process objects] ReEv Sometime AllocFreeDiff 236 1054 1290 22 134 ReEv Sometime Bytes 11328 50592 61920 22 6472 ---- DRIVER : [<unknown> - Resource Event] ReSe Sometime AllocFreeDiff 53 356 409 14 30 ReSe Sometime Bytes 2544 17088 19632 14 1453 ---- DRIVER : [<unknown> - Resource Semaphore] ReTa Sometime AllocFreeDiff 58 787 845 7 33 ReTa Sometime Bytes 4832 65136 69968 7 2760 ---- DRIVER : [<unknown> - Resource Extended Table] SePa Always AllocFreeDiff 122 394 516 30 69 SePa Always Bytes 22704 71392 94096 31 12972 ---- DRIVER : [nt!se - Process audit image names and captured polity s SeTd Always AllocFreeDiff 6385 10224 16609 62 3648 SeTd Always Bytes 817280 1308672 2125952 62 466985 ---- DRIVER : [nt!se - Security Token dynamic part] Sema Sometime AllocFreeDiff 144 3081 3225 4 82 Sema Sometime Bytes 13824 298048 311872 4 7898 ---- DRIVER : [<unknown> - Semaphore objects] Toke Always AllocFreeDiff 6385 10224 16609 62 3648 Toke Always Bytes 5058224 8174464 13232688 61 2890220 ---- DRIVER : [nt!se - Token objects] Usqm Sometime AllocFreeDiff 4 17 21 23 2 Usqm Sometime Bytes 448 1904 2352 23 255 ---- DRIVER : [win32k!InitQEntryLookaside - QMSG] Poolmon Output (Truncated) Memory:12577420K Avail: 9942344K PageFlts:76231725 InRam Krnl: 4212K P:220164K Commit:2188924K Limit:14136700K Peak:2267836K Pool N:121312K P:223456K Tag Type Allocs Frees Diff Bytes Per Alloc MmSt Paged 118837 78153 40684 24149904 593 Proc Nonp 19389 1 19388 20473728 1056 Toke Paged 1385263 1365675 19588 15606272 796 File Nonp 3978932 3932600 46332 12625328 272 IoNm Paged 5178704 5139127 39577 10284288 259 Ntfr Nonp 90211 14589 75622 9681552 128 BCM0 Nonp 14 0 14 8663632 618830 CcVa Nonp 1 0 1 7872512 7872512 MmCm Nonp 7175 7104 71 6821152 96072 WFC Nonp 5430149 5423324 6825 6804592 997 MmCa Nonp 651567 613077 38490 6742000 175 CM35 Paged 123 0 123 5152768 41892 UlHT Paged 1 0 1 4198400 4198400 MmSm Paged 53188 15704 37484 3598464 96 NtFs Nonp 291881 253445 38436 3078528 80 TCPt Nonp 9115 9082 33 2867536 86895 LSwi Nonp 1 0 1 2703360 2703360 SeTd Nonp 1385263 1365675 19588 2507264 128 Ntfn Nonp 125263 86818 38445 2463808 64 NtfF Paged 4121 2695 1426 1848096 1296 Mm Nonp 154 139 15 1820768 121384 Irp Nonp 902431 900332 2099 1806672 860 tmte Nonp 470 0 470 1797280 3824 TSdd Paged 1184 1151 33 1695264 51371 NDpp Nonp 412 0 412 1616128 3922 Thre Nonp 57769 56349 1420 1567680 1104 CM25 Paged 1252 1158 94 1540096 16384 UlCO Nonp 1664 379 1285 1356960 1056 Obtb Paged 60647 60234 413 1282944 3106 Gh15 Paged 53346 53100 246 1193552 4851 FSim Paged 14650 6759 7891 1136304 144 Mdl Nonp 33477 29248 4229 1119008 264

Hi, According to the output, we can see that the proc and file tags consume the most space in nonpaged pool. I suspect that the issue is caused by a handle leak. The following blog may help you analyze the issue: http://blogs.technet.com/b/askperf/archive/2008/05/09/troubleshooting-server-hangs-part-three.aspx This posting is provided "AS IS" with no warranties, and confers no rights. Please remember to click Mark as Answer on the post that helps you, and to click Unmark as Answer if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

Joson, Thank you for the response. I have reviewed the number of open handles via Process Explorer, and nothing looks to be out of the ordinary. Screenshot below.

Hi, Yes, the handle looks normal. In this case, we need to capture a dump file when the issue occurs and analyze it to identify the cause of the issue. Unfortunately, we cannot help you analyze dump file in Forum. To trouble shoot this problem efficiently, I suggest that you speak directly with a Microsoft Support Professional so that the crash dump can be analyzed. For a complete list of Microsoft Product Support Services phone numbers and information about support costs, please go to the following address on the World Wide Web: http://support.microsoft.com/directory/overview.asp This posting is provided "AS IS" with no warranties, and confers no rights. Please remember to click Mark as Answer on the post that helps you, and to click Unmark as Answer if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

Thanks Joson, That was my though initially, but I thought I would give the forums a shot first.

Just thought that I would update this thread in case any one else stumbles across this issue. The issue was caused by a driver with the file name of “aksdf.sys”. This is a driver that supports the Alladin software security key. The server no longer required the driver to be installed so we planned on removing the files. We performed the removal based on the information covered in the following article. http://www.pulsonix.com/faq.aspx?KB090025 We downloaded the commandline driver installation/uninstallation files from here, extract the files, and then ran the following command. Hinstall –remove After successful removal we reviewed the poolmon output and saw that the proc memory tag now had an increasing amount of frees occurring. The amount of memory also associated with the tag no longer increased, and in fact started to slowly decrease.