If you have a machine with 4gb of ram, you may still see out of memory errors coming from ASP.NET. It seems impossible, since you have so much memory, yet it can happen for a simple reason...

As a safeguard against memory leaks, an ASP.NET process will consume virtual memory up to a defined limit before automatically recycling the process and releasing the memory. This limit is controlled by the memoryLimit attribute in machine.config, which indicates a percentage of physical ram. By default, memoryLimit is set to 60.

So for example, if your server has 1gb of physical ram installed, and memoryLimit is set to 60, then ASP.NET processes will recycle if they utilize over 600mb of memory (60% of 1gb).

The problem occurs if you have a server with a lot of ram, e.g. 4gb. 60% of 4gb is 2.4gb, which means that ASP.NET won't recycle until it uses over 2.4gb of ram. If it's using less, it doesn't think there's a memory problem. However, since by default a process can only address 2gb of ram, you will run into memory allocation errors as you approach 2gb.

The errors will actually occur before 2gb is reached, because as your application allocates, releases, and consumes memory, the available pool becomes more fragmented. That means that fewer contiguous blocks of memory are available, and the blocks that are available are smaller. In a simplified example, let's say there are 100 blocks of memory left, and each block is 1mb. It will look like there's 100mb of memory free. However, if your application requests a 2mb block, it will get an out of memory error, because there aren't any available blocks that are 2mb (or larger).

So, back to the server with 4 gigs of ram. ASP.NET will try to go up to 2.4gb, but 2gb is an address barrier. So there are a few solutions:

enable the /3gb switch in boot.ini. That will allow a process to address up to 3gb of ram instead of 2gb

change machine.config's memoryLimit parameter to a lower value like 40. That would cause ASP.NET to recycle at 1.6gb (40% of 4gb).

configure your app pool in IIS to recycle if it hits 1.6gb of virtual memory.

As an aside, you should also try to understand why your application is using so much memory in the first place. 2gb is a lot of data, so you may want to see if you have a memory leak.

MSDN has a great article on tuning and performance monitoring that you should check out if you want to learn more.

Normally SQL Server is installed to be case insensitive, yet you need to perform a case sensitive search. Thus when trying to do a normal string comparison you run into problems.

if 'foo' = 'FOO'
print 'strings match'
else
print 'strings are different'


Will tell you the strings match, yet they clearly aren't the same if you're talking case. There are least two ways in SQL Server. First, you can convert them to binary values:

if convert(varbinary,'foo') = convert(varbinary,'FOO')
print 'strings match'
else
print 'strings are different'

Or if you're on SQL Server 2000 you can change the collation to be case-sensitive:

if 'foo' collate Latin1_General_CS_AI = 'FOO' collate Latin1_General_CS_AI
print 'strings match'
else
print 'strings are different'

So there you go. Happy string matching!

One Late Night With Perl: A Troubleshooting Story

A few weeks ago we needed to do a search and replace on a few hundred IIS web log files. Each file was between 100MB and 2GB. We're all windows here, so I figured I needed to download a tool. With a GUI. And many buttons I can click.

I first tried numerous "Search and Replace" tools from download.com and other shareware sites. SuperSearch, WinGrep, WGrep, Advanced Search and Replace, Fast Search, etc, etc. I must have tried out 6 or 7 different apps on a small subset. They ranged from bleh to moderate to very useful for smaller files, but my best hopes were that it would take anywhere from 12 hours to 2 days to update all this data. Did I mention that it was 9pm and we needed to update reports for the following morning?

I finally found TextPipe...it looked promising, but we had to buy a copy in order to update all the files. The web site said we'd get a registered version in 15 minutes, so I popped in my CC, got my "thanks for ordering" email, and waited. And waited. After 20 minutes I got an email telling me my order would be fulfilled in 4 hours. WTF!?

So it's back to square one, and I figured to try out Perl. BTW I don't know Perl from Clipper. Downloaded ActivePerl (which is free) and started hitting the news groups. There were numerous samples, but all for Unix, Linux, etc...no Windows. Anyhow, I finally found a sample that would work:

== replace.bat ==
rem replace "foo" with "bar" in *.log
for %%n in (*.log) do perl -pi.bak -e "s/foo/bar/g" %%n
== EOF ==

And it was fast as hell! I was hoping for something that would recurse through directories, but I didn't have time to find it. So instead I just made a batch file kinda like

for %%n in (f:\w3svc1\*.log) do perl -pi.bak -e "s/foo/bar/g" %%n
for %%n in (f:\w3svc2\*.log) do perl -pi.bak -e "s/foo/bar/g" %%n
for %%n in (f:\w3svc3\*.log) do perl -pi.bak -e "s/foo/bar/g" %%n

etc. There were only a dozen directories, so no big deal. Hmm, spend 2 hours learning Perl so you can write a function to recurse 10 subdirectories, or write 10 lines in a batch file? 10:30pm, you tell me. I added a few lines to the batch file to write out timestamps when it finished each directory, kicked it off, & cabbed home. I didn't know how long it would take, but it was the best I could do for the time being.

Epilogue: I come in the next morning around 8am to hear that the reports were running. I checked the timestamps from the search & replace -- under 4 hours to update everything! Pretty amazing, and we were abe to get the reports done in time. Oh, and TextPipe, the program I paid several hundred for and expected to get in 15 minutes to save my day? It finally arrived 11 hours after I ordered it. Needless to say I got a refund immediately, and cozied up to my new friend Perl.

I was having problems at work getting iisreset to run...kept getting "class not registered" errors. Luckily, I found the answer from Google and its vast, vast field, which led me to Doug Thews' IlluminatiLand. You just need reregister IISRSTAS.EXE with

c:\windows\system32\inetsrv\iisrstas.exe /RegServer