Server crashes constantly
Hi,
The issue described here is taking place on a Dell PE R610 running W2K8 R2 DataCenter. The machine crashes within minutes of using it. The dump file is below. Please advise.
----- 64 bit Kernel Mini Dump Analysis
DUMP_HEADER64:
MajorVersion 0000000f
MinorVersion 00001db1
DirectoryTableBase 00000000`00187000
PfnDataBase fffff800`01b14228
PsLoadedModuleList fffff800`01aa8e90
PsActiveProcessHead fffff800`01a8ab90
MachineImageType 00008664
NumberProcessors 00000010
BugCheckCode 000000d1
BugCheckParameter1 00000000`00000000
BugCheckParameter2 00000000`00000002
BugCheckParameter3 00000000`00000008
BugCheckParameter4 00000000`00000000
KdDebuggerDataBlock fffff800`01a540a0
MiniDumpFields 00000cff
TRIAGE_DUMP64:
ServicePackBuild 00000100
SizeOfDump 00040000
ValidOffset 0003fffc
ContextOffset 00000348
ExceptionOffset 00000f00
MmOffset 00002080
UnloadedDriversOffset 000020d0
PrcbOffset 00002298
ProcessOffset 00006f98
ThreadOffset 00007468
CallStackOffset 00007900
SizeOfCallStack 00000e08
DriverListOffset 00008a48
DriverCount 00000096
StringPoolOffset 0000dea8
StringPoolSize 00003300
BrokenDriverOffset 00000000
TriageOptions ffffffff
TopOfStack fffff880`0201afa8
BStoreOffset 00000000
SizeOfBStore 00000000
LimitOfBStore 00000000`00000000
DebuggerDataOffset 00008708
DebuggerDataSize 00000340
DataBlocksOffset 000111a8
DataBlocksCount 0000000c
Windows XP Kernel Version 7601 (Service Pack 1) MP (16 procs) Free AMD x86-64
Built by: 7601.17514.amd64fre.win7sp1_rtm.101119-1850
Kernel base = 0xfffff800`01863000 PsLoadedModuleList = 0xfffff800`01aa8e90
Debug session time: Fri Mar 18 13:09:11 2011
System Uptime: 0 days 0:06:33
start end module name
fffff800`01863000 fffff800`01e4d000 nt Checksum: 0055CE0C Timestamp: Sat Nov 20 02:30:02 2010 (4CE7951A)
Unloaded modules:
fffff880`06c53000 fffff880`06c6e000 USBSTOR.SYS Timestamp: unavailable (00000000)
fffff880`06c38000 fffff880`06c53000 USBSTOR.SYS Timestamp: unavailable (00000000)
fffff880`06c1d000 fffff880`06c38000 USBSTOR.SYS Timestamp: unavailable (00000000)
fffff880`01bcd000 fffff880`01bdb000 crashdmp.sys Timestamp: unavailable (00000000)
fffff880`01bdb000 fffff880`01be5000 dump_storpor Timestamp: unavailable (00000000)
fffff880`01be5000 fffff880`01bf1000 dump_megasas Timestamp: unavailable (00000000)
fffff880`0182a000 fffff880`01838000 ClusDisk.sys Timestamp: unavailable (00000000)
fffff880`00cc0000 fffff880`00cdb000 sacdrv.sys Timestamp: unavailable (00000000)
Finished dump check
Thanks,
Steve
March 18th, 2011 4:04pm
Bug Check Code 0xD1: The DRIVER_IRQL_NOT_LESS_OR_EQUAL bug check has a value of 0x000000D1. This indicates that a kernel-mode driver attempted to access pageable memory at a process
IRQL that was too high.
For more information, refer to this Microsoft article:
http://msdn.microsoft.com/en-us/library/ff560244(VS.85).aspx
As I am unable to get much information from the output of the debugger you used, please use Microsoft Skydrive to upload dump files (%systemroot%\minidumps like c:\windows\minidumps) and
post a link here. Once done, I will determine the faulty driver using my debugger.
You can also try to update all possible drivers and check if this solve your problem.
This posting is provided "AS IS" with no warranties or guarantees , and confers no rights.
Microsoft
Student Partner
Microsoft Certified Professional
Microsoft Certified Systems Administrator: Security
Microsoft Certified Systems Engineer: Security
Microsoft Certified Technology Specialist: Windows Server 2008 Active Directory, Configuration
Microsoft
Certified Technology Specialist: Windows Server 2008 Network Infrastructure, Configuration
Free Windows Admin Tool Kit Click here and download it now
March 18th, 2011 4:27pm
Thanks! Here is the link.
http://cid-d5f0bdd6544ab788.office.live.com/self.aspx/.Public/031811-13400-01.dmp
Steve
March 18th, 2011 5:22pm
11: kd> !analyze -v
*******************************************************************************
*
*
* Bugcheck Analysis
*
*
*
*******************************************************************************
DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 0000000000000000, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000008, value 0 = read operation, 1 = write operation
Arg4: 0000000000000000, address which referenced memory
Debugging Details:
------------------
READ_ADDRESS: 0000000000000000
CURRENT_IRQL: 2
FAULTING_IP:
+475952f0272db4c
00000000`00000000 ?? ???
PROCESS_NAME: System
CUSTOMER_CRASH_COUNT: 1
DEFAULT_BUCKET_ID: DRIVER_FAULT_SERVER_MINIDUMP
BUGCHECK_STR: 0xD1
TRAP_FRAME: fffff880024990f0 -- (.trap 0xfffff880024990f0)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=fffff880024993b8 rbx=0000000000000000 rcx=fffffa8040892318
rdx=00000000c0000240 rsi=0000000000000000 rdi=0000000000000000
rip=0000000000000000 rsp=fffff88002499288 rbp=0000000000000000
r8=0000000000000000 r9=0000000000000000 r10=fffffa801922bd00
r11=0000000000000002 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0 nv up ei ng nz na po nc
00000000`00000000 ?? ???
Resetting default scope
LAST_CONTROL_TRANSFER: from fffff8000188ebe9 to fffff8000188f640
FAILED_INSTRUCTION_ADDRESS:
+475952f0272db4c
00000000`00000000 ?? ???
STACK_TEXT:
fffff880`02499288 fffff880`01873e80 : fffffa80`19feb580 00000000`000007ff fffffa80`40dfd102 00000000`00000000 : 0x0
fffff880`02499290 fffff880`01873dbf : 00000000`0000000b 00000000`00000001 00000000`00000001 fffff880`04693946 : tcpip!TcpCompleteClientReceiveRequest+0x20
fffff880`02499340 fffff880`01873ed9 : 00000000`00000000 00000000`c0000240 fffffa80`40dfd180 fffffa80`40dfd180 : tcpip!TcpFlushTcbDelivery+0x11f
fffff880`02499430 fffff880`018a1cd9 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : tcpip!TcpFlushRequestReceive+0x29
fffff880`02499460 fffff880`0195e6c6 : fffffa80`411f8e20 fffffa80`1a0d24b8 fffffa80`1a0d2401 fffffa80`40c19ad0 : tcpip!TcpShutdownTcb+0x539
fffff880`024995a0 fffff880`0195e929 : fffffa80`40935600 fffffa80`40935600 00000000`00000000 00000000`00000000 : tcpip!TcpActivateDelegatedTcb+0x3c6
fffff880`02499630 fffff880`0195ec67 : 00000000`00000000 fffffa80`40e3e080 00000000`00000002 fffff880`018fe8ed : tcpip!TcpUploadComplete+0x79
fffff880`02499670 fffff880`0194ece7 : 00000000`00000000 fffffa80`19ebf280 00000000`00000000 fffff800`018aa6bb : tcpip!TcpNlClientTerminateOffloadComplete+0x27
fffff880`024996a0 fffff880`0194f1f0 : fffffa80`19bc7220 00000000`00000000 00000000`00000002 fffff880`00eee717 : tcpip!IppTerminatePathOffloadComplete+0x97
fffff880`024996f0 fffff880`019034ff : fffffa80`40d75890 fffffa80`19ebf280 fffffa80`40d75890 fffffa80`19ebf280 : tcpip!IpFlcTerminateNeighborOffloadComplete+0x2e0
fffff880`02499750 fffff880`0165ce66 : fffffa80`00000000 00000000`00000001 fffff880`024997d0 fffffa80`19ebf280 : tcpip!FlTerminateOffloadComplete+0x9f
fffff880`02499780 fffffa80`00000000 : 00000000`00000001 fffff880`024997d0 fffffa80`19ebf280 fffffa80`19879510 : bxnd60a+0xce66
fffff880`02499788 00000000`00000001 : fffff880`024997d0 fffffa80`19ebf280 fffffa80`19879510 fffff880`0165dc9b : 0xfffffa80`00000000
fffff880`02499790 fffff880`024997d0 : fffffa80`19ebf280 fffffa80`19879510 fffff880`0165dc9b fffffa80`39c11010 : 0x1
fffff880`02499798 fffffa80`19ebf280 : fffffa80`19879510 fffff880`0165dc9b fffffa80`39c11010 fffffa80`1a0d2420 : 0xfffff880`024997d0
fffff880`024997a0 fffffa80`19879510 : fffff880`0165dc9b fffffa80`39c11010 fffffa80`1a0d2420 fffffa80`4122d0b0 : 0xfffffa80`19ebf280
fffff880`024997a8 fffff880`0165dc9b : fffffa80`39c11010 fffffa80`1a0d2420 fffffa80`4122d0b0 fffffa80`19879510 : 0xfffffa80`19879510
fffff880`024997b0 fffffa80`39c11010 : fffffa80`1a0d2420 fffffa80`4122d0b0 fffffa80`19879510 00000000`00000000 : bxnd60a+0xdc9b
fffff880`024997b8 fffffa80`1a0d2420 : fffffa80`4122d0b0 fffffa80`19879510 00000000`00000000 00000000`00000000 : 0xfffffa80`39c11010
fffff880`024997c0 fffffa80`4122d0b0 : fffffa80`19879510 00000000`00000000 00000000`00000000 fffffa80`00000000 : 0xfffffa80`1a0d2420
fffff880`024997c8 fffffa80`19879510 : 00000000`00000000 00000000`00000000 fffffa80`00000000 fffff880`00f18601 : 0xfffffa80`4122d0b0
fffff880`024997d0 00000000`00000000 : 00000000`00000000 fffffa80`00000000 fffff880`00f18601 fffffa80`4122d0b0 : 0xfffffa80`19879510
STACK_COMMAND: .trap 0xfffff880024990f0 ; kb
FOLLOWUP_IP:
bxnd60a+ce66
fffff880`0165ce66 ?? ???
SYMBOL_STACK_INDEX: b
SYMBOL_NAME: bxnd60a+ce66
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: bxnd60a
IMAGE_NAME: bxnd60a.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 4ac5bc49
FAILURE_BUCKET_ID: X64_0xD1_CODE_AV_NULL_IP_bxnd60a+ce66
BUCKET_ID: X64_0xD1_CODE_AV_NULL_IP_bxnd60a+ce66
Followup: MachineOwner
---------
11: kd> lmvm bxnd60a
start end module name
fffff880`01650000 fffff880`01666000 bxnd60a T (no symbols)
Loaded symbol image file: bxnd60a.sys
Image path: \SystemRoot\system32\DRIVERS\bxnd60a.sys
Image name: bxnd60a.sys
Timestamp: Fri Oct 02 10:39:37 2009 (4AC5BC49)
CheckSum: 0002326B
ImageSize: 00016000
Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
-----------------------------------------------------------------------------------------------------------------
Your crashes are caused by bxnd60a.sys driver. This driver is a Broadcom Corporation driver. So, I suppose it is the NIC card driver of the server.
Please update it because the last update was may Friday 2 October 2009. It this does not solve your problem, contact Broadcom Technical Support.
This posting is provided "AS IS" with no warranties or guarantees , and confers no rights.
Microsoft Student Partner
Microsoft Certified Professional
Microsoft Certified Systems Administrator: Security
Microsoft Certified Systems Engineer: Security
Microsoft Certified Technology Specialist: Windows Server 2008 Active Directory, Configuration
Microsoft
Certified Technology Specialist: Windows Server 2008 Network Infrastructure, Configuration
Free Windows Admin Tool Kit Click here and download it now
March 18th, 2011 8:04pm
Thanks a lot. I have updated the drivers. Both servers seem to be working better. I will continue to monitor them. What debugging tool are you using for this? Mine was obviously less informative than yours.
I greatly appreciate your help.
Steve
March 18th, 2011 8:50pm
I used Windbg.
For more information, refer to this Microsoft article:
http://support.microsoft.com/kb/315263
This posting is provided "AS IS" with no warranties or guarantees , and confers no rights.
Microsoft Student Partner
Microsoft Certified Professional
Microsoft Certified Systems Administrator: Security
Microsoft Certified Systems Engineer: Security
Microsoft Certified Technology Specialist: Windows Server 2008 Active Directory, Configuration
Microsoft Certified Technology Specialist: Windows Server 2008 Network Infrastructure, Configuration
Free Windows Admin Tool Kit Click here and download it now
March 20th, 2011 12:13am


