Hi- an application is trying to construct a string builder out of the lines
between separator lines in a huge file. All lines between two neighboring
separator lines will belong to the same stringbuilder. however, sometimes it
could be long. e.g., in one instance, the stringbuffer has a length of more
than 140MB and when the operation
myStringBuilder.ToString()

is called, an OOM exception is thrown.

Is there a bullet proof way to avoid OOM exception? In case of an OOM
exception, is there still a way to recover and continue with the application?
It appears only a restart of the application is helpful.

thanks,

Simply catch the OOM, free some space (e.g. set object references to huge
arrays to null) and call GC.Collect().


"Tao" <Tao@discussions.microsoft.com> schrieb im Newsbeitrag
news:A396D58F-48DD-4D1D-BB01-F3588B68FBEB@microsoft.com...
[quoted text, click to view]

[quoted text, click to view]

Although it's not an immediatly helpfull comment to you, perhaps you
should change your design, passing 140MB strings is really not a good
idea in the long run.

If possible, have a look at Stream, and see if you can do some streaming
processing of your data instead.

--
Helge Jensen
mailto:helge.jensen@slog.dk
sip:helge.jensen@slog.dk

Using a string builder will help a little but you may have some issues.
First the StringBuilder starts out being able to hold 16 characters. It
doubles in size every time it needs to grow. You can calculate the total
amount of memory needed to create your 140 MB string using that starting
point. The other issue is that once your sting becomes greater the 85KB, the
..Net runtime creates it on the large object heap. The large object heap does
not get compacted so it may keep growing with your allocations. Thirdly,
when you call the ToString() method of the StringBuilder, it creates a copy
of the string. So in essence you have two 140MB strings temporarily in
memory.

I would suggest looking at ways to reduce the amount of data you need to
handle. I would also try to initialize the size of the StringBuilder to the
size you anticipate you'll need. This will help with the intermediate
allocation issue.

IHTH

Jon

[quoted text, click to view]

[quoted text, click to view]

Actually StringBuilder.ToString doesn't always copy the string, but that's
just a detail. It performs some magic which non-framework code isn't
allowed to and wraps its internal buffer in a string, and passes a reference
to that. Any subsequent modification of the same StringBuilder will cause
it to allocate a new buffer. Since 99.99% of the time the StringBuilder is
never reused after .ToString(), this is a nice optimization.


Here's a copy of the Rotor implementation
(http://www.123aspx.com/rotor/RotorSrc.aspx?rot=42060)


/// <include file='doc\StringBuilder.uex'
path='docs/doc[@for="StringBuilder.ToString"]/*' />
public override String ToString() {
String currentString = m_StringValue;
int currentThread = m_currentThread;
if (currentThread != 0 && currentThread !=
InternalGetCurrentThread()) {
return String.InternalCopy(currentString);
}

if ((2 * currentString.Length) < currentString.ArrayLength) {
return String.InternalCopy(currentString);
}

currentString.ClearPostNullChar();
m_currentThread = 0;
return currentString;
}

David

In my opinion this suggestion would be solving the symptom instead of the
problem. I wouldn't recommend calling GC.Collect since you should generally
allow the Garbage Collector to handle itself since it has been optimized to
do so. There are algorithms in place which handle cleaning up, compacting,
etc.

I would begin by looking at using a different solution to what you're trying
to accomplish instead of loading 140MB string into memory. While some say
"memory is cheap", any application using 140MB of strings could be
re-designed to do things more efficiently (IMHO).

You might try streaming or "chunking" the data into the application.

[quoted text, click to view]