Thanks Pat. I'll try it out and let you know the results

Regards
Ian

[quoted text, click to view]

Run IISState in crash mode against w3wp.exe and post the log. It could be
any number of things (a bug in MS stuff, a bug in an app, etc.).

iisstate -p <pid of w3wp.exe> -sc <enter>


Pat


[quoted text, click to view]

Our production server has recently started experiencing AppPool crashes.
These seem to occur sporadically. sometimes three times a day, sometimes not
for a couple of days.
The error manifests itself to the client as "Service Unavailable".
In the system event log we see the following:
A process serving application pool 'DefaultAppPool' suffered a fatal
communication error with the World Wide Web Publishing Service
With error number: 8007006d

At the same time (but not always) we see this error in the Application Log
"Faulting application w3wp.exe, version 6.0.3790.0, faulting module
kernel32.dll, version 5.2.3790.0, fault address 0x000249d3"

The application does usually start working again after anywhere between 2
and 15 minutes (although no worker process restarts or recycles appear in
our perf logs).

This problems seems to have started occuring following the latest MS patches
being applied to our server. It may just be a coincidence though.

We are running Windows Server 2003 Standard. We are about to apply SP1 in an
attempt to solve this problem, but I wanted to find out if anyone else has
had this problem and what the solution was. I have seen several similarish
posts but nothing concrete as a solution.

I have seen this article
(http://support.microsoft.com/Default.aspx?id=885654) but it doesn't quite
match our situation as we are not running as a domain controller. We haven't
yet tried running registry monitor as I wanted to find out if error
8007006d always equates to a registry permission problem?? before installing
3rd party freeware onto our production server.

Any help is much appreciated

Regards
Ian

Your plight motivated me to jot some thoughts down... you're already going
down the right path; the blog entry is just to pull everything together into
perspective and give the "why" and "howto".

http://blogs.msdn.com/david.wang/archive/2005/08/29/HOWTO_Understand_and_Diagnose_an_AppPool_Crash.aspx

--
//David
IIS
http://blogs.msdn.com/David.Wang
This posting is provided "AS IS" with no warranties, and confers no rights.
//
[quoted text, click to view]
Thanks Pat. I'll try it out and let you know the results

Regards
Ian

[quoted text, click to view]

Thanks for that great blog entry David. Very useful


[quoted text, click to view]

Okay we tried Win 2K3 Sp1. That didn't help.
I tried running IIS state but it seems to pick up a soft crash everytime I
run it when in fact the problems we have been having only happen once or
twice a day. Unless there is some ongoing issue that it is recovering from
most of the time...

Also when IISstate detects this soft crash it kills the IIS process when it
finishes. Is this normal?

Anyway here is my IIS State log file, if any one can make sense of it that
would be much appreciated
******
Opened log file 'D:\program files\iisstate\output\IISState-3420.log'

***********************
Starting new log output
IISState version 3.3.1

Thu Sep 01 19:02:54 2005

OS = Windows 2003 Server
Executable: w3wp.exe
PID = 3420

Note: Thread times are formatted as HH:MM:SS.ms

***********************


IIS has crashed...
Beginning Analysis
DLL (!FunctionName) that failed:




Thread ID: 11
System Thread ID: bf4
Kernel Time: 0:0:1.640
User Time: 0:0:30.343
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll -
Thread Type: Other
# ChildEBP RetAddr
WARNING: Frame IP not in any known module. Following frames may be wrong.
00 01a4eb64 02d0830f 0x2f701c5
01 01a4f79c 02dfe27c 0x2d0830f
02 01a4f808 02c014a7 0x2dfe27c
03 01a4f844 02c00f13 0x2c014a7
04 01a4f8a0 023cf792 0x2c00f13
05 01a4f8ec 023cf663 0x23cf792
06 01a4f944 020f9938 0x23cf663
07 01a4f9d4 7923c206 0x20f9938
08 01a4fa14 791cc3c8 mscorsvr!CoUninitializeEE+0xcc3
09 01a4fa5c 7923c1c3 mscorsvr!Ordinal76+0x1c3c8
0a 01a4fac4 01afa179 mscorsvr!CoUninitializeEE+0xc80
0b 01a4fba8 0218e928 0x1afa179
0c 0218e928 00000001 0x218e928
Closing open log file D:\program files\iisstate\output\IISState-3420.log
Opened log file 'D:\program files\iisstate\output\IISState-3420.log'

***********************
Starting new log output
IISState version 3.3.1

Thu Sep 01 19:02:54 2005

OS = Windows 2003 Server
Executable: w3wp.exe
PID = 3420

Note: Thread times are formatted as HH:MM:SS.ms

***********************




Thread ID: 0
System Thread ID: cd4
Kernel Time: 0:0:0.15
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23




Thread ID: 1
System Thread ID: dc4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 2
System Thread ID: c80
Kernel Time: 0:0:0.156
User Time: 0:0:0.171
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c9ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c9ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c9ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c9ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c9ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c9ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c9ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 3
System Thread ID: c14
Kernel Time: 0:0:0.265
User Time: 0:0:0.250
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00cdff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00cdff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00cdff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00cdff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00cdffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00cdffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00cdffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 4
System Thread ID: 91c
Kernel Time: 0:0:0.500
User Time: 0:0:0.406
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d1ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d1ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d1ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d1ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d1ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d1ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d1ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 5
System Thread ID: ca0
Kernel Time: 0:0:0.359
User Time: 0:0:0.390
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d5ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d5ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d5ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d5ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d5ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d5ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d5ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 6
System Thread ID: 5b0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e1fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e1fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e1ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e1ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 7
System Thread ID: 658
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 017efa84 7c822124 ntdll!KiFastSystemCallRet
01 017efa88 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 017efaf8 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 017efb0c 5a3b8147 kernel32!WaitForSingleObject+0x12
04 017effb8 77e66063 w3core!HTTP_COMPRESSION::CompressionThread+0x126
05 017effec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 8
System Thread ID: 80c
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0186ff8c 7c821364 ntdll!KiFastSystemCallRet
01 0186ff90 7c815267 ntdll!NtDelayExecution+0xc
02 0186ffb8 77e66063 ntdll!RtlpIOWorkerThread+0x3f
03 0186ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 9
System Thread ID: da8
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 0196fe50 7c821364 ntdll!KiFastSystemCallRet
01 0196fe54 77e42439 ntdll!NtDelayExecution+0xc
02 0196febc 77e424b7 kernel32!SleepEx+0x68
03 0196fecc 791bf4f9 kernel32!Sleep+0xf

I guess I'm misunderstanding how IISState using the -sc switch works.
When ever I run it it catches an exception and quits

Anyway, we just had another incident and I was able to run IISState during
it.

Here is the log...hopefully its more useful than the last one

Your thoughts are muchly appreciated

****
Opened log file 'D:\program files\iisstate\output\IISState-3420.log'

***********************
Starting new log output
IISState version 3.3.1

Fri Sep 02 18:27:05 2005

OS = Windows 2003 Server
Executable: w3wp.exe
PID = 3420

Note: Thread times are formatted as HH:MM:SS.ms

***********************


IIS has crashed...
Beginning Analysis
*** WARNING: Unable to verify checksum for
c:\windows\assembly\nativeimages1_v1.1.4322\mscorlib\1.0.5000.0__b77a5c56193
4e089_1e3d3e9e\mscorlib.dll
*** ERROR: Module load completed but symbols could not be loaded for
c:\windows\assembly\nativeimages1_v1.1.4322\mscorlib\1.0.5000.0__b77a5c56193
4e089_1e3d3e9e\mscorlib.dll
DLL (!FunctionName) that failed: mscorlib_79990000




Thread ID: 10
System Thread ID: fe0
Kernel Time: 0:0:9.875
User Time: 0:3:35.859
Thread Type: Managed Thread. Possible ASP.Net page or other .Net worker
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll -
PDB symbol for mscorsvr.dll not loaded
succeeded
Loaded Son of Strike data table version 5 from
"C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll"
Thread 10
ESP EIP
019edc38 79ad8e18 [DEFAULT] Class
System.Runtime.Remoting.Messaging.LogicalCallContext
System.Runtime.Remoting.Messaging.Message.PropagateCallContextFromMessageToT
hread(Class System.Runtime.Remoting.Messaging.IMessage)
019edc44 79ad08da [DEFAULT] Void
System.Runtime.Remoting.Contexts.Context.DoCallBackFromEE(I4,I4,I4)
019ef914 791b3208 [FRAME: NDirectMethodFrameStandaloneCleanup] [DEFAULT] I4
System.Web.UnsafeNativeMethods.EcbGetUnicodeServerVariable(I,String,I,I4)
019ef92c 020f79f2 [DEFAULT] [hasThis] String
System.Web.Hosting.ISAPIWorkerRequestInProcForIIS6.GetUnicodeServerVariable(
String,Class System.Web.Hosting.ServerVarCharBuffer)
019ef944 020f76c4 [DEFAULT] [hasThis] Void
System.Web.Hosting.ISAPIWorkerRequestInProcForIIS6.ReadRequestBasics()
019ef990 020f7615 [DEFAULT] [hasThis] Void
System.Web.Hosting.ISAPIWorkerRequest.Initialize()
019ef998 020f660f [DEFAULT] Class System.Web.Hosting.ISAPIWorkerRequest
System.Web.Hosting.ISAPIWorkerRequest.CreateWorkerRequest(I,I4)
019ef9d8 020f6457 [DEFAULT] [hasThis] I4
System.Web.Hosting.ISAPIRuntime.ProcessRequest(I,I4)
019efaa0 79217188 [FRAME: ContextTransitionFrame]
019efb80 79217188 [FRAME: ComMethodFrame]


Begin System Thread Information

# ChildEBP RetAddr
00 00000001 00000000 mscorlib_79990000+0x148e18
Closing open log file D:\program files\iisstate\output\IISState-3420.log
Opened log file 'D:\program files\iisstate\output\IISState-3420.log'

***********************
Starting new log output
IISState version 3.3.1

Fri Sep 02 18:27:10 2005

OS = Windows 2003 Server
Executable: w3wp.exe
PID = 3420

Note: Thread times are formatted as HH:MM:SS.ms

***********************




Thread ID: 0
System Thread ID: cd4
Kernel Time: 0:0:0.15
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23




Thread ID: 1
System Thread ID: dc4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 2
System Thread ID: c80
Kernel Time: 0:0:2.421
User Time: 0:0:1.640
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c9ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c9ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c9ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c9ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c9ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c9ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c9ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 3
System Thread ID: c14
Kernel Time: 0:0:2.265
User Time: 0:0:1.578
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00cdff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00cdff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00cdff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00cdff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00cdffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00cdffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00cdffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 4
System Thread ID: 91c
Kernel Time: 0:0:1.859
User Time: 0:0:1.421
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d1ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d1ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d1ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d1ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d1ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d1ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d1ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 5
System Thread ID: ca0
Kernel Time: 0:0:1.984
User Time: 0:0:1.546
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d5ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d5ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d5ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d5ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d5ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d5ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d5ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 6
System Thread ID: 5b0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e1fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e1fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e1ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e1ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 7
System Thread ID: 658

Hey, didn't I suggest in the blog entry to NOT try to install random
patches/service-packs or make configuration changes to fix a crash? :-) You
need to catch the crash and identify it first, then determine whether a
patch/service-pack, config change, etc can address the issue.

You want to wait for real crashes. Not whatever you are catching right now

When dealing with a crash, you just wants the log file and stack trace.
Everything else in the process is pretty much toast since data corruption
could have happened on the exceptional event (like a crash) and you
certainly do not want to continue working with known data corruption.

It is much the same rationale for BSOD. Windows rather BSOD when a bad thing
happens in kernel mode than attempt to continue run, possibly causing other
data corruption (especially to the network or filesystem...).

--
//David
IIS
http://blogs.msdn.com/David.Wang
This posting is provided "AS IS" with no warranties, and confers no rights.
//
[quoted text, click to view]
Okay we tried Win 2K3 Sp1. That didn't help.
I tried running IIS state but it seems to pick up a soft crash everytime I
run it when in fact the problems we have been having only happen once or
twice a day. Unless there is some ongoing issue that it is recovering from
most of the time...

Also when IISstate detects this soft crash it kills the IIS process when it
finishes. Is this normal?

Anyway here is my IIS State log file, if any one can make sense of it that
would be much appreciated
******
Opened log file 'D:\program files\iisstate\output\IISState-3420.log'

***********************
Starting new log output
IISState version 3.3.1

Thu Sep 01 19:02:54 2005

OS = Windows 2003 Server
Executable: w3wp.exe
PID = 3420

Note: Thread times are formatted as HH:MM:SS.ms

***********************


IIS has crashed...
Beginning Analysis
DLL (!FunctionName) that failed:




Thread ID: 11
System Thread ID: bf4
Kernel Time: 0:0:1.640
User Time: 0:0:30.343
*** ERROR: Symbol file could not be found. Defaulted to export symbols for
C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorsvr.dll -
Thread Type: Other
# ChildEBP RetAddr
WARNING: Frame IP not in any known module. Following frames may be wrong.
00 01a4eb64 02d0830f 0x2f701c5
01 01a4f79c 02dfe27c 0x2d0830f
02 01a4f808 02c014a7 0x2dfe27c
03 01a4f844 02c00f13 0x2c014a7
04 01a4f8a0 023cf792 0x2c00f13
05 01a4f8ec 023cf663 0x23cf792
06 01a4f944 020f9938 0x23cf663
07 01a4f9d4 7923c206 0x20f9938
08 01a4fa14 791cc3c8 mscorsvr!CoUninitializeEE+0xcc3
09 01a4fa5c 7923c1c3 mscorsvr!Ordinal76+0x1c3c8
0a 01a4fac4 01afa179 mscorsvr!CoUninitializeEE+0xc80
0b 01a4fba8 0218e928 0x1afa179
0c 0218e928 00000001 0x218e928
Closing open log file D:\program files\iisstate\output\IISState-3420.log
Opened log file 'D:\program files\iisstate\output\IISState-3420.log'

***********************
Starting new log output
IISState version 3.3.1

Thu Sep 01 19:02:54 2005

OS = Windows 2003 Server
Executable: w3wp.exe
PID = 3420

Note: Thread times are formatted as HH:MM:SS.ms

***********************




Thread ID: 0
System Thread ID: cd4
Kernel Time: 0:0:0.15
User Time: 0:0:0.31
Thread Status: Thread is in a WAIT state.
Thread Type: HTTP Compression Thread
# ChildEBP RetAddr
00 0006fc08 7c822124 ntdll!KiFastSystemCallRet
01 0006fc0c 77e6baa8 ntdll!NtWaitForSingleObject+0xc
02 0006fc7c 77e6ba12 kernel32!WaitForSingleObjectEx+0xac
03 0006fc90 5a36467a kernel32!WaitForSingleObject+0x12
04 0006fca0 5a366e63 w3dt!WP_CONTEXT::RunMainThreadLoop+0x10
05 0006fca8 5a3af41d w3dt!UlAtqStartListen+0x2d
06 0006fcb8 5a3bc259 w3core!W3_SERVER::StartListen+0xbd
07 0006ff0c 0100187c w3core!UlW3Start+0x26e
08 0006ff44 01001a23 w3wp!wmain+0x22a
09 0006ffc0 77e523cd w3wp!wmainCRTStartup+0x12b
0a 0006fff0 00000000 kernel32!BaseProcessStart+0x23




Thread ID: 1
System Thread ID: dc4
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Type: Other
# ChildEBP RetAddr
00 009eff9c 7c821364 ntdll!KiFastSystemCallRet
01 009effa0 7c81fe26 ntdll!NtDelayExecution+0xc
02 009effb8 77e66063 ntdll!RtlpTimerThread+0x47
03 009effec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 2
System Thread ID: c80
Kernel Time: 0:0:0.156
User Time: 0:0:0.171
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00c9ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00c9ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00c9ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00c9ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00c9ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00c9ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00c9ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 3
System Thread ID: c14
Kernel Time: 0:0:0.265
User Time: 0:0:0.250
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00cdff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00cdff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00cdff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00cdff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00cdffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00cdffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00cdffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 4
System Thread ID: 91c
Kernel Time: 0:0:0.500
User Time: 0:0:0.406
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d1ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d1ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d1ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d1ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d1ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d1ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d1ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 5
System Thread ID: ca0
Kernel Time: 0:0:0.359
User Time: 0:0:0.390
Thread Type: HTTP Listener
# ChildEBP RetAddr
00 00d5ff24 7c821bf4 ntdll!KiFastSystemCallRet
01 00d5ff28 77e6611a ntdll!NtRemoveIoCompletion+0xc
02 00d5ff54 5a30249e kernel32!GetQueuedCompletionStatus+0x29
03 00d5ff8c 5a3026bc W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x33
04 00d5ffa0 5a301db9 W3TP!THREAD_POOL_DATA::ThreadPoolThread+0x24
05 00d5ffb8 77e66063 W3TP!THREAD_MANAGER::ThreadManagerThread+0x39
06 00d5ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 6
System Thread ID: 5b0
Kernel Time: 0:0:0.0
User Time: 0:0:0.0
Thread Status: Thread is in a WAIT state.
Thread Type: Other
# ChildEBP RetAddr
00 00e1fcec 7c822114 ntdll!KiFastSystemCallRet
01 00e1fcf0 7c83acfd ntdll!NtWaitForMultipleObjects+0xc
02 00e1ffb8 77e66063 ntdll!RtlpWaitThread+0x161
03 00e1ffec 00000000 kernel32!BaseThreadStart+0x34




Thread ID: 7
System Thread ID: 658
Kernel Time: 0:0:0.0

I know you suggested that. Unfortunately, by that point we had already
arranged the service pack install and downtime as well as testing it out on
our test servers, so we were fairly committed.

The problem is that when ever I run IISState (e.g. IISState -p 1234 -sc -d)
it dumps out whatever I sent you and then exits.
Am I doing something wrong? The other problem is that when it does this it
takes out my w3wp process which obviously the users notice.

Your help is much appreciated

Ian

[quoted text, click to view]