This morning, one of my development machines died unexpectedly from a BSOD. I haven't been able to find any information from Google, so I'm posting my experience here for others to find.
The culprit seems to be ipnat.sys, which is interesting because even though I have a complex network setup, Windows doesn't handle the NATing. I'm running XP SP3 on this laptop, with ipnat.sys 5.1.2600.5512, modified Monday, April 14, 2008, 12:27:16 AM. The OCA failed because I had the minidump open in WinDbg at the time (oops).
Detailed network setup at the time of crash (10 adapters total):
- 1394 (FireWire) port - disabled
- Builtin wireless - disabled
- Dialup connection via physical modem - enabled but phone line not plugged in
- USB wireless - active, with Internet access
- NIC - active, local access only
- PCMCIA cell tower wireless ("dialup") - enabled but not inserted
- VMWare VMNet1 for virtual machines - enabled but not active (no virtual machines were running)
- VMWare VMNet8 for virtual machines - same
- VPN virtual adapter for OpenVPN connections - disabled; no longer used
- VPN virtual adapter #2 for OpenVPN - same
VMWare and OpenVPN both handle their own NAT, and adapter #4 is *not* shared over the network on #5, so I don't know why Windows was doing something with NAT.
Anyway, here's the WinDbg analysis:
Microsoft (R) Windows Debugger Version 6.9.0003.113 X86
Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [C:\Documents and Settings\Stephen Cleary\Local Settings\Temp\WERe5f9.dir00\Mini080208-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available
Symbol search path is: SRV*c:\symbols*http://msdl.microsoft.com/download/symbols;C:\Work\RAC\RAMDisk\Project\debug;symsrv*symsrv.dll*c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is:
Windows XP Kernel Version 2600 (Service Pack 3) UP Free x86 compatible
Product: WinNt
Built by: 2600.xpsp.080413-2111
Kernel base = 0x804d7000 PsLoadedModuleList = 0x80553fc0
Debug session time: Sat Aug 2 07:35:32.341 2008 (GMT-4)
System Uptime: 0 days 20:28:21.531
Loading Kernel Symbols
...............................................................................................................................................................
Loading User Symbols
Loading unloaded module list
...............
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
Use !analyze -v to get detailed debugging information.
BugCheck D1, {3a003e, 2, 1, ed878886}
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: nt!_KPRCB ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: nt!_KPRCB ***
*** ***
*************************************************************************
Probably caused by : ipnat.sys ( ipnat!NatDereferenceAddressPoolEntry+ce )
Followup: MachineOwner
---------
kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high. This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: 003a003e, memory referenced
Arg2: 00000002, IRQL
Arg3: 00000001, value 0 = read operation, 1 = write operation
Arg4: ed878886, address which referenced memory
Debugging Details:
------------------
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: mssmbios!_SMBIOS_DATA_OBJECT ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: nt!_KPRCB ***
*** ***
*************************************************************************
*************************************************************************
*** ***
*** ***
*** Your debugger is not using the correct symbols ***
*** ***
*** In order for this command to work properly, your symbol path ***
*** must point to .pdb files that have full type information. ***
*** ***
*** Certain .pdb files (such as the public OS symbols) do not ***
*** contain the required information. Contact the group that ***
*** provided you with these symbols if you need this command to ***
*** work. ***
*** ***
*** Type referenced: nt!_KPRCB ***
*** ***
*************************************************************************
WRITE_ADDRESS: 003a003e
CURRENT_IRQL: 2
FAULTING_IP:
ipnat!NatDereferenceAddressPoolEntry+ce
ed878886 894804 mov dword ptr [eax+4],ecx
CUSTOMER_CRASH_COUNT: 1
DEFAULT_BUCKET_ID: DRIVER_FAULT
BUGCHECK_STR: 0xD1
TRAP_FRAME: f78b6dac -- (.trap 0xfffffffff78b6dac)
ErrCode = 00000002
eax=003a003a ebx=8a4eef58 ecx=e75b94c8 edx=00000000 esi=8a4260e8 edi=8a4260e8
eip=ed878886 esp=f78b6e20 ebp=f78b6e30 iopl=0 nv up ei pl zr na pe nc
cs=0008 ss=0010 ds=0023 es=0023 fs=0030 gs=0000 efl=00010246
ipnat!NatDereferenceAddressPoolEntry+0xce:
ed878886 894804 mov dword ptr [eax+4],ecx ds:0023:003a003e=????????
Resetting default scope
LAST_CONTROL_TRANSFER: from ed878886 to 80540683
STACK_TEXT:
f78b6dac ed878886 badb0d00 00000000 ed52b35c nt!KiTrap0E+0x233
f78b6e30 ed87d965 8a4eef58 8a4260e8 8a4eefb8 ipnat!NatDereferenceAddressPoolEntry+0xce
f78b6e44 ed87f601 8a4eef58 fdfd0128 ed87f266 ipnat!NatDeleteTicket+0x2f
f78b6e88 804ffd88 ed887da0 00000000 df47447e ipnat!NatTimerRoutine+0x39b
f78b6fa4 804ffe9f 99893074 000000ab ffdff000 nt!KiTimerListExpire+0x122
f78b6fd0 80541b8d 80552da0 00000000 0047f961 nt!KiTimerExpiration+0xaf
f78b6ff4 8054185a ba43e9c8 00000000 00000000 nt!KiRetireDpcList+0x46
f78b6ff8 ba43e9c8 00000000 00000000 00000000 nt!KiDispatchInterrupt+0x2a
WARNING: Frame IP not in any known module. Following frames may be wrong.
f78b6ffc 00000000 00000000 00000000 00000000 0xba43e9c8
STACK_COMMAND: kb
FOLLOWUP_IP:
ipnat!NatDereferenceAddressPoolEntry+ce
ed878886 894804 mov dword ptr [eax+4],ecx
SYMBOL_STACK_INDEX: 1
SYMBOL_NAME: ipnat!NatDereferenceAddressPoolEntry+ce
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: ipnat
IMAGE_NAME: ipnat.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 48025786
FAILURE_BUCKET_ID: 0xD1_W_ipnat!NatDereferenceAddressPoolEntry+ce
BUCKET_ID: 0xD1_W_ipnat!NatDereferenceAddressPoolEntry+ce
Followup: MachineOwner
---------
Posts
