all groups > iis smtp nntp > november 2007 >
You're in the

iis smtp nntp

group:

SMTP Causes 100% CPU Usage



SMTP Causes 100% CPU Usage Lori
11/5/2007 10:38:00 AM
iis smtp nntp: Last Friday I began a migration from a Win2K/Exc2003 server (DNEXOLD) to a
Win2003/Exc2003 server (DNEXNEW). Note that both servers are FULLY updated
with all MS updates & patches.

I had been using DNEXNEW for my own mailbox for the week without any issues.
On Friday I moved over all other mailboxes in our organization. Everything
was working wonderfully that eve, thru the weekend and this morning. For an
hour. At 9:35am it suddenly came to a halt when the CPU Usage spiked to 100%
and stopped processing mail.

I uninstalled McAfee GroupShield 6.0 after reading dozens of articles
blaming it for similar issues, alas, same problem. I also uninstalled McAfee
VirusScan in desperation. No help.

I gave up and moved ALL the mailboxes back to the old server (DNEXOLD) just
to get people functioning again.

I'm at a loss.....

Please understand I'm NOT an Exchange admin by any stretch so while I beg
for your input and help please bear with me if I ask for clarification on
some of your suggestions....

Re: SMTP Causes 100% CPU Usage Lori
11/5/2007 12:09:01 PM
I enabled SMTP logging & it created a log file! I put a check to log
EVERYTHING. However I moved my own mailbox over to DNEXNEW, sent a few test
messages in & out & nothing got recorded. Since 3:30pm, all that's been in
the log is:

(Note that 172.16.1.5 is the IP of DNEXOLD. 172.16.1.6 is the IP of DNEXNEW.)

#Software: Microsoft Internet Information Services 6.0
#Version: 1.0
#Date: 2007-11-05 19:30:18
#Fields: time c-ip cs-method cs-uri-stem sc-status
19:30:18 172.16.1.5 - - 0
19:30:18 172.16.1.5 EHLO - 0
19:30:18 172.16.1.5 - - 0
19:30:18 172.16.1.5 - - 0
19:30:18 172.16.1.5 - - 0
19:30:18 172.16.1.5 - - 0
19:30:18 172.16.1.5 X-LINK2STATE - 0
19:30:18 172.16.1.5 - - 0
19:30:18 172.16.1.5 MAIL - 0
19:30:18 172.16.1.5 - - 0
19:30:18 172.16.1.5 RCPT - 0
19:30:18 172.16.1.5 - - 0
19:30:18 172.16.1.5 XEXCH50 - 0
19:30:18 172.16.1.5 - - 0
19:30:18 172.16.1.5 - - 0
19:30:18 172.16.1.5 BDAT - 0
19:30:18 172.16.1.5 - - 0
19:30:18 172.16.1.5 QUIT - 0
19:30:18 172.16.1.5 - - 0
19:30:38 172.16.1.5 EHLO - 250
19:30:38 172.16.1.5 x-exps - 0
19:30:38 172.16.1.5 x-link2state - 200
19:30:52 172.16.1.5 MAIL - 250
19:31:02 172.16.1.5 RCPT - 250
19:31:02 172.16.1.5 xexch50 - 354
19:31:02 172.16.1.5 BDAT - 250
19:31:02 172.16.1.5 MAIL - 250
19:31:02 172.16.1.5 RCPT - 250
19:31:02 172.16.1.5 xexch50 - 354
19:31:02 172.16.1.5 BDAT - 250
19:31:02 172.16.1.5 QUIT - 240
#Software: Microsoft Internet Information Services 6.0
#Version: 1.0
#Date: 2007-11-05 19:39:12
#Fields: time c-ip cs-method cs-uri-stem sc-status
19:39:12 172.16.1.5 - - 0
19:39:12 172.16.1.5 EHLO - 0
19:39:12 172.16.1.5 - - 0
19:39:12 172.16.1.5 - - 0
19:39:12 172.16.1.5 - - 0
19:39:12 172.16.1.5 - - 0
19:39:12 172.16.1.5 X-LINK2STATE - 0
19:39:12 172.16.1.5 - - 0
19:39:12 172.16.1.5 MAIL - 0
19:39:12 172.16.1.5 - - 0
19:39:12 172.16.1.5 RCPT - 0
19:39:12 172.16.1.5 - - 0
19:39:12 172.16.1.5 XEXCH50 - 0
19:39:12 172.16.1.5 - - 0
19:39:12 172.16.1.5 - - 0
19:39:12 172.16.1.5 BDAT - 0
19:39:13 172.16.1.5 - - 0
19:39:13 172.16.1.5 QUIT - 0
19:39:13 172.16.1.5 - - 0
19:41:17 172.16.1.5 EHLO - 250
19:41:17 172.16.1.5 x-exps - 0
19:41:17 172.16.1.5 x-link2state - 200
19:41:37 172.16.1.5 MAIL - 250
19:41:53 172.16.1.5 RCPT - 250
19:41:53 172.16.1.5 xexch50 - 354
19:41:54 172.16.1.5 BDAT - 250
19:41:54 172.16.1.5 QUIT - 240
19:48:15 172.16.1.5 EHLO - 250
19:48:15 172.16.1.5 x-exps - 0
19:48:15 172.16.1.5 x-link2state - 200
19:48:15 172.16.1.5 MAIL - 250
19:48:15 172.16.1.5 RCPT - 250
19:48:15 172.16.1.5 xexch50 - 354
19:48:18 172.16.1.5 BDAT - 250
19:48:18 172.16.1.5 QUIT - 240
19:51:09 172.16.1.5 EHLO - 250
19:51:09 172.16.1.5 x-exps - 0
19:51:09 172.16.1.5 x-link2state - 200
19:51:09 172.16.1.5 MAIL - 250
19:51:09 172.16.1.5 RCPT - 250
19:51:09 172.16.1.5 xexch50 - 354
19:51:12 172.16.1.5 BDAT - 250
19:51:12 172.16.1.5 QUIT - 240

[quoted text, click to view]
Re: SMTP Causes 100% CPU Usage Sanford Whiteman
11/5/2007 2:21:58 PM
[quoted text, click to view]

First suggestion: look at your SMTP logs. You may have created a mail
loop.

The symptoms are constant connections resending the same e-mail (MAIL FROM,
RCPT TO).

--Sandy


------------------------------------
Sanford Whiteman, Chief Technologist
Broadleaf Systems, a division of
Cypress Integrated Systems, Inc.
Re: SMTP Causes 100% CPU Usage Sanford Whiteman
11/6/2007 12:24:02 AM
[quoted text, click to view]

I would suggest that you create test accounts on DNEXNEW instead of
using your production personal account!

Also, switch to W3C Extended Log Format. It is much more verbose.

Anyway, what you're seeing are only topology exchanges; indeed, there
is no mail loop yet. But what happens when you add more test accounts
on the new server? Do you see a geometric increase in log entries for
every message you send?

Does the CPU on DNEXNEW stay low, or increase, with no production
load?

--Sandy


------------------------------------
Sanford Whiteman, Chief Technologist
Broadleaf Systems, a division of
Cypress Integrated Systems, Inc.
Re: SMTP Causes 100% CPU Usage Lori
11/6/2007 9:17:02 AM
"I would suggest that you create test accounts on DNEXNEW instead of using
your production personal account!"
-- Yes - that dawned on me right after posting this. I moved my account
back to DNEXOLD & created a dummy on DNEXNEW to test with. Blonde moment...
;)

"Also, switch to W3C Extended Log Format. It is much more verbose."
-- It IS W3C Extended Log Format!!

An update....

I found it's not the SMTP SERVICE, but rather the Default SMTP Virtual
Server that's causing the problem. I changed the SMTP service to Manual,
booted the server. Started the SMTP service - all was aok. THEN I started
the Default SMTP Virtual Server - that's when all hell breaks loose. CPU
Usage spikes to, and stays at, 100%. I can't even STOP the SMTP service OR
Default SMTP Virtual Server after that... The only 'way out' is to reboot....

I also created a second Virtual Server & started it (while the Default was
stopped after a reboot). It starts aok & the CPU Usage does NOT spike up,
however I can't get any mail to pass thru it w/ my test account.... but
that's a side issue.... (at least I think....)

Last but not least when I started the Default SMTP Virtual Server it created
a new log for today with lots of goodies... I have NO clue what all this
means.... but maybe you can pinpoint the problem? Also, I found that it
won't ADD anything to the log even after a reboot. I noticed that yesterday
but thought it was a fluke - it's doing the same thing today. It appears
that the log will be CREATED but never altered..... That can't be
right.......!

Anyway, here's the log from today....
#Software: Microsoft Internet Information Services 6.0
#Version: 1.0
#Date: 2007-11-06 16:12:03
#Fields: date time c-ip cs-username s-sitename s-computername s-ip s-port
cs-method cs-uri-stem cs-uri-query sc-status sc-win32-status sc-bytes
cs-bytes time-taken cs-version cs-host cs(User-Agent) cs(Cookie) cs(Referer)
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - -
220+dnexchange.datanomics.com+Microsoft+ESMTP+MAIL+Service,+Version:+5.0.2195.6713+ready+at++Tue,+6+Nov+2007+11:12:03+-0500+ 0 0 124 0 15 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 EHLO - DNEX01.datanomics.com 0 0 4 0 15 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 250-dnexchange.datanomics.com+Hello+[172.16.1.6] 0 0 48 0 15 SMTP - -
- -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 334+GSSAPI+supported 0 0 20 0 469 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - -
334+oYIBLDCCASigAwoBAKELBgkqhkiC9xIBAgKigYgEgYVggYIGCSqGSIb3EgECAgIAb3MwcaADAgEFoQMCAQ+iZTBjoAMCAReiXARayvj+SHlyOimcg+mRX1jVHFGgJJYNK63NhYCc8FUEQUh6dr8ZjLTHJlYq8UywdWtgGokqo2DEAHgYYOMYYzUkyMsafD9Zk4ftkbRtopvsLOcu0PUertvlrCZqo4GIBIGFYIGCBgkqhkiG9xIBAgICAG9zMHGgAwIBBaEDAgEPomUwY6ADAgEXolwEWsr4/kh5cjopnIPpkV9Y1RxRoCSWDSutzYWAnPBVBEFIena/GYy0xyZWKvFMsHVrYBqJKqNgxAB4GGDjGGM1JMjLGnw/WZOH7ZG0baKb7CznLtD1Hq7b5awmag== 0 0 412 0 469 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 235+2.7.0+Authentication+successful. 0 0 36 0 484 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 X-LINK2STATE -
LAST+CHUNK={0000006a}+MULTI+(5)+({00000051}+DIGEST_QUERY+00dd79b02250d3429e4461ed90c9ee95+d7b7b3d489351f213a321e70d1d11931++)++ 0 0 12 0 500 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 200+LAST+CHUNK={00000029}+MULTI+(5)+({00000010}+DONE_RESPONSE++)++ 0 0
66 0 500 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 MAIL - FROM:<DNEX01-IS@datanomics.com>+SIZE=3121 0 0 4 0 515 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 250+2.1.0+DNEX01-IS@datanomics.com....Sender+OK 0 0 47 0 515 SMTP - -
- -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 RCPT - TO:<DNEXCHANGE-IS@datanomics.com>+NOTIFY=NEVER 0 0 4 0 531 SMTP - -
- -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 250+2.1.5+DNEXCHANGE-IS@datanomics.com+ 0 0 39 0 547 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 XEXCH50 - 3140+2 0 0 7 0 547 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 354+Send+binary+data 0 0 20 0 562 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 250+XEXCH50+OK 0 0 14 0 562 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 BDAT - 3121+LAST 0 0 4 0 562 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - -
250+2.6.0++<47A4AAF365366240AE2A83940FB05C61B8A3@DNEX01.datanomics.com>+Queued+mail+for+delivery 0 0 96 0 640 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 RSET - - 0 0 4 0 672 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 250+2.0.0+Resetting 0 0 19 0 672 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 MAIL - FROM:<administrator@example.org>+SIZE=5296 0 0 4 0 672 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 250+2.1.0+administrator@example.org....Sender+OK 0 0 48 0 672 SMTP - -
- -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 RCPT - TO:<webmaster@datanomics.com> 0 0 4 0 672 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 250+2.1.5+webmaster@datanomics.com+ 0 0 35 0 687 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 XEXCH50 - 1084+2 0 0 7 0 687 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 354+Send+binary+data 0 0 20 0 687 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 250+XEXCH50+OK 0 0 14 0 687 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 BDAT - 5296+LAST 0 0 4 0 687 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - -
250+2.6.0++<ABAFA5A7BFB5487594E311A19AE7AE5D@datanomics.com>+Queued+mail+for+delivery 0 0 85 0 734 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
25 RSET - - 0 0 4 0 765 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionResponse SMTPSVC1 DNEX01 -
25 - - 250+2.0.0+Resetting 0 0 19 0 765 SMTP - - - -
2007-11-06 16:12:03 172.16.1.5 OutboundConnectionCommand SMTPSVC1 DNEX01 -
Re: SMTP Causes 100% CPU Usage Lori
11/7/2007 1:58:01 PM
It WAS a third-party software, Policy Patrol, that caused the problem.

As I mentioned, I reverted all my users to the old server which had been
stripped of ALL third-party software. Since there was no end in sight to the
problem on the new server I started adding software back - one by one. Policy
Patrol was first since it manages our signatures, disclaimers & forwarding.
Around 2pm today the OLD server started exhibiting the SAME problem as the
new server. There were only 2 common elements between the 2 servers -
Exchange 2003 & Policy Patrol.

I contacted Policy Patrol and they had a fix for it which resolved the issue
immediately however they were surprised that it happened on a server with NO
other third party applications installed - not even virus protection. I know,
a no-no, but desperate times.... They believe the database got corrupted and
when I installed it on the old server and imported the database from the new
server, along with the problem.... Why it was delayed in showing itself, in
both cases, they don't know...

But thank you Sandford, for taking time to post replies I really do
appreciate it & you taught me a thing or two along the way!!

[quoted text, click to view]
Re: SMTP Causes 100% CPU Usage Sanford Whiteman
11/8/2007 1:16:52 AM
[quoted text, click to view]

It certainly is a mandate to remove all inline add-ons (event sinks)
before blaming the core SMTP service, yep. I'd assumed that McAfee was
the only such sink. My bad for not pushing you on that further. Glad
you found the problem.

--Sandy




------------------------------------
Sanford Whiteman, Chief Technologist
Broadleaf Systems, a division of
Cypress Integrated Systems, Inc.
Re: SMTP Causes 100% CPU Usage Lori
11/8/2007 5:56:07 AM
No, it was my bad for not uninstalling it b/f I reached out for help. I
don't know why I had the sense to uninstall McAfee VirusScan & GroupShield
but not Policy Patrol... I was in a panic and just not thinking clearly, I
guess.

Please know how much I appreciate your time & help. It was just comforting
to know someone out there was lending a hand.... This forum has been awesome
time & again and all the techs that respond, yourself included, have been
wonderful..... ;)

[quoted text, click to view]
AddThis Social Bookmark Button