Event 2019: The server was unable to allocate from the system nonpaged pool because the pool was empty
Hi Guys,
I've been trying to solve this issue for a customer for a long time now, the problem is it's VERY intermittent so may go a month or more before issues. The server becomes more or less unresponsive (can still RDP to it, but it won't provide user services
like DNS/AD etc.) and starts logging the 2019 event below -
Event Type: Error
Event Source: Srv
Event Category: None
Event ID: 2019
Date: 24/09/2010
Time: 10:44:03
User: N/A
Computer: NTSERVER01
Description:
The server was unable to allocate from the system nonpaged pool because the pool was empty.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Server is Windows SBS 2003 on a HP DL380 with all updates applied. Sophos Antivirus, not much else installed. Something that may be important is we intalled a seperate server with Exchange 2007 into the Exchange Org, migrated mailboxes etc. then
disabled the Exchange services on this SBS server.
Issue happened again this morning so I logged on and saw the top "NP Pool" consumer was dns.exe at 700k, so I restarted the DNS Server service and the server went back to running normally. Note once the DNS service restarted it is now consuming
5000k of NP Pool. I doubt dns.exe is really the problem here, I suspect by just restarting it I freed enough NP Pool for things to start running again. So the server is still in this state, running fine for users but probably on the verge of going
down again.
From the task manager, dns.exe is consuming 5000K of NP Pool, the next is lsass.exe at 182k then the rest are less then 100k. I also note inetinfo.exe has 510000 handles, which seems excessive? The next biggest is lsass.exe with 19000 handles.
I also see that inetinfo.exe has 4000k of Paged Pool while the next is dns.exe with 800k then various other services with 300k or less.
Below is the poolmon from the server currently. Note this is after I restarted dns.exe so the server is functioning at this point.
Tag Type Allocs Frees Diff Bytes Per Alloc
Thre Nonp 9500893 ( 3) 9228029 ( 5) 272864 170267136 ( -1248) 624
Time Nonp 498966 ( 0) 244249 ( 0) 254717 46867976 ( 0) 184
Irp Nonp 10454050 ( 173) 10440924 ( 84) 13126 5819448 ( 59096) 443
Mdl Nonp 13346386 ( 8) 13314772 ( 21) 31614 4050840 ( -1664) 128
LSwi Nonp 1 ( 0) 0 ( 0) 1 2576384 ( 0) 2576384
File Nonp 243870397 (1034) 243856192 ( 915) 14205 2175432 ( 18088) 153
VoSm Nonp 28535 ( 0) 28501 ( 0) 34 2064672 ( 0) 60725
MmCm Nonp 29 ( 0) 3 ( 0) 26 1712672 ( 0) 65872
TCPt Nonp 6120280 ( 0) 6120246 ( 0) 34 1458768 ( 0) 42904
TPLA Nonp 256 ( 0) 0 ( 0) 256 1048576 ( 0) 4096
TCPA Nonp 7874940 ( 4) 7872333 ( 5) 2607 959376 ( -368) 368
AfdE Nonp 52108726 ( 6) 52105751 ( 7) 2975 833000 ( -280) 280
Pool Nonp 6 ( 0) 3 ( 0) 3 610304 ( 0) 203434
Devi Nonp 96553 ( 0) 96097 ( 0) 456 558352 ( 0) 1224
UlCO Nonp 3392 ( 0) 2559 ( 0) 833 506464 ( 0) 608
CcSc Nonp 11365692 ( 15) 11364308 ( 15) 1384 442880 ( 0) 320
Even Nonp 167545334 ( 53) 167536359 ( 99) 8975 435616 ( -2624) 48
LSwr Nonp 128 ( 0) 0 ( 0) 128 416768 ( 0) 3256
LSrf Nonp 2140496 ( 121) 2138919 ( 0) 1577 391096 ( 30008) 248
Ntfr Nonp 988808 ( 0) 983752 ( 0) 5056 324552 ( 0) 64
Vad Nonp 75198269 ( 21) 75191670 ( 69) 6599 316752 ( -2304) 48
Mm Nonp 20964 ( 0) 20945 ( 0) 19 292736 ( 0) 15407
September 24th, 2010 6:42am
Hi Guys,
I've been trying to solve this issue for a customer for a long time now, the problem is it's VERY intermittent so may go a month or more before issues. The server becomes more or less unresponsive (can still RDP to it, but it won't provide user services
like DNS/AD etc.) and starts logging the 2019 event below -
[cut]
take a look at this possible solutions
http://www.eventid.net/display.asp?eventid=2019&eventno=661&source=Srv&phase=1
HTHEdoardo Benussi - Microsoft MVP
Management Infrastructure - Systems Administration
https://mvp.support.microsoft.com/Profile/Benussi
Windows Server Italian Forum Moderator
edo[at]mvps[dot]org
Free Windows Admin Tool Kit Click here and download it now
September 24th, 2010 7:23am
The server has just started logging these events again, so restartnig DNS earlier only got it woring for a few hours, so wasnt' the problem imho. I'll read through those EventID posts but to be honest there are so many there it's a bit difficult to
use reliably.
Any comment on the 500,000 handles that IIS has? Is this normal? After running iisreset inetinfo is only using 923 handles. As Exchange is no longer on this server IIS should be doing very little or nothing.
This is poolmon output now while the server is logging these errors -
Memory: 3669472K Avail: 1398824K PageFlts: 278 InRam Krnl: 2724K P:71916K
Commit:2126424K Limit:5871164K Peak:2225484K Pool N:258396K P:72672K
System pool information
Tag Type Allocs Frees Diff Bytes Per Alloc
Thre Nonp 9521773 ( 3) 9247970 ( 7) 273803 170853072 ( -2496) 624
Time Nonp 499919 ( 0) 244310 ( 0) 255609 47032104 ( 0) 184
Irp Nonp 10550084 ( 0) 10536978 ( 5) 13106 5778304 ( -800) 440
Mdl Nonp 13408624 ( 20) 13376884 ( 32) 31740 4066888 ( -1536) 128
LSwi Nonp 1 ( 0) 0 ( 0) 1 2576384 ( 0) 2576384
File Nonp 246524579 (116090) 246510779 (116091) 13800 2113024 ( -136) 1
VoSm Nonp 28551 ( 0) 28517 ( 0) 34 2064672 ( 0) 60725
MmCm Nonp 29 ( 0) 3 ( 0) 26 1712672 ( 0) 65872
TCPt Nonp 6140943 ( 0) 6140909 ( 0) 34 1458768 ( 0) 42904
TPLA Nonp 256 ( 0) 0 ( 0) 256 1048576 ( 0) 4096
TCPA Nonp 7891931 ( 13) 7889339 ( 14) 2592 953856 ( -368) 368
AfdE Nonp 53760483 (115700) 53757547 (115701) 2936 822080 ( -280) 2
Pool Nonp 6 ( 0) 3 ( 0) 3 610304 ( 0) 203434
Devi Nonp 96866 ( 0) 96414 ( 0) 452 553304 ( 0) 1224
UlCO Nonp 3392 ( 0) 2559 ( 0) 833 506464 ( 0) 608
CcSc Nonp 11453044 ( 30) 11451688 ( 28) 1356 433920 ( 640) 320
Even Nonp 167938722 ( 107) 167929796 ( 114) 8926 433712 ( -336) 48
LSwr Nonp 128 ( 0) 0 ( 0) 128 416768 ( 0) 3256
LSrf Nonp 2158481 ( 0) 2157095 ( 14) 1386 343728 ( -3472) 248
Ntfr Nonp 993702 ( 0) 988527 ( 0) 5175 332168 ( 0) 64
Vad Nonp 75442295 ( 68) 75435772 ( 68) 6523 313104 ( 0) 48
Mm Nonp 21117 ( 0) 21098 ( 0) 19 292736 ( 0) 15407
September 24th, 2010 10:08am
The server has just started logging these events again, so restartnig DNS earlier only got it woring for a few hours, so wasnt' the problem imho. I'll read through those EventID posts but to be honest there are so many there it's a bit difficult to
use reliably.
Any comment on the 500,000 handles that IIS has? Is this normal? After running iisreset inetinfo is only using 923 handles. As Exchange is no longer on this server IIS should be doing very little or nothing.
[cut]
if you don't need IIS, you can even uninstall them.
have you tried some solution in link that i have posted ?
Edoardo Benussi - Microsoft® MVP
Management Infrastructure - Systems Administration
https://mvp.support.microsoft.com/Profile/Benussi
Windows Server Italian Forum Moderator
edo[at]mvps[dot]org
Free Windows Admin Tool Kit Click here and download it now
September 24th, 2010 10:19am
Hi,
Have you tried the troubleshooting suggestions “Edoardo Benussi” provided and what is the result?
If it does not work, I would like to suggest you update all the hardware drivers and temporarily disable antivirus program to test the issue.
You may also test the issue in Clean Boot. For the detailed steps of how to perform a Clean Boot environment, please refer to the following steps:
1.
Click
Start, type msconfig in the Start Search box, and then press
ENTER.
2.
If you are prompted for an administrator password or for a confirmation, type the password or click Continue.
3.
On the
General tab, click Selective startup.
4.
Under
Selective Startup, click to clear the Load startup items check box.
5.
Click the
Services tab, click to select the Hide All Microsoft services check box, and then click
Disable all.
6.
Click
OK, and then click Restart.
What is the result in Clean Boot?
If the issue persists, would you please send me the Process Explorer log for analyzing. For your convenience, I have created
a workspace for you. You can upload the information files to the following link. (Please choose "Send Files to Microsoft")
Workspace URL:
https://sftasia.one.microsoft.com/choosetransfer.aspx?key=73a93724-109d-47c0-bb89-1d07620242ca
Password: D#T8AEk^Wi
Note: Due to differences in text formatting with various email clients, the workspace link above may appear to be broken.
Please be sure to include all text between '(' and ')' when typing or copying the workspace link into your browser. Meanwhile, please note that files uploaded for more than 72 hours will be deleted automatically. Please ensure to notify me timely after you
have uploaded the files. Thank you for your understanding.
1.
To view the NP Pool usage in
Task Manager, click View, Select Columns and
Non-Paged Pool.
2.
Please download the Process Explorer at
http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx.
3.
Extract the zip file and double click the
procexp.exe file.
4.
Please highlight the Process Name which takes the high NP Pool resource, such as DNS.exe in Process Explorer.
5.
From the
View menu, please point to Lower Pane view and select
DLLs.
6.
From the
File menu, please select Save to save the dll list to a txt file such as IEDll.txt.
7.
Please upload it to the link I provided above.
Regards,Please remember to click Mark as Answer on the post that helps you, and to click Unmark as Answer if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.
September 27th, 2010 6:12am
Hi guys,
Sorry for not replying, I forget email notifications from this site are sporadic :).
IIS is just building up handles over time, I logged in this morning to find it at 200k again. So I've disabled the IIS Admin service as suggested above which I hope will solve the problem.
Thanks.
Free Windows Admin Tool Kit Click here and download it now
October 4th, 2010 10:20am