Hi,
a c# app of mine that parses 30,000 xml files writes large
amounts of data to flat file (> 2GB) using a streamwriter
object (sw). everything works fine except that the memory
used by the app grows linearly during execution and
eventually crashes the computer. without going into too
much detail of my code, i made the following observations:

- if you comment out the sw.Write(x) statement (which is
inside the loop that parses the xml files), memory doesn't
increase. thus it must be something to do with the
streamwriter or file I/O.

- if you change x in sw.Write(x) to be a constant, memory
doesn't increase, but if x is an expression or variable,
memory does increase. Not sure what this means.

I've tried many things to solve this problem and am
completely stuck. I've tried making sw = null every so
often within the loop, making a new streamwriter and
output file (so that the file sizes never exceed 10MB),
and calling GC.Collect() to try and force the compiler to
clean up memory, but there is no effect. I've tried using
{} statements which don't do anything either. The only
thing I can think of is to re-run the entire application
multiple times and pass command line parameters to
indicate where the last one left off, but this is
obviously not an ideal solution.

I did get it to work by using the
Scripting.FileSystemObejct via COM instead of
streamwriter, but it's prohibitively slow. However this
does indicate that there are no memory leaks within the
rest of my code, and that the problem is with streamwriter
somehow.

Any thoughts? Any help at all is greatly appreciated!!!!!!!
thanks in advance

Hello Rob,

Thanks for posting in the group.

After reviewing the post, I have the following suggestions:

1) Could you please move sw.write(x) related code to a separate function?
After that, please call that function in the loop to see if the memory
still keep on growing.

2) Besides, in order to know which object keeps the object, we need to
create a dump file of this application and use WinDBG to analyze it.

For more information on it, please refer to:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/
DBGrm.asp?frame=true
D:\Program Files\Microsoft Visual Studio .NET 2003\SDK\v1.1\Tool Developers
Guide\Samples\sos\SOS.htm

They are all good materials for debugging managed application on .NET
platform. Hope that helps.

If you have any more concerns on it, please feel free to post here.

Best regards,
Yanhong Huang
Microsoft Online Partner Support

Get Secure! - www.microsoft.com/security
This posting is provided "AS IS" with no warranties, and confers no rights.

Hi Yanhong,
We have a discussino thread at this site:

http://www.developersdex.com/csharp/message.asp?p=1111&ID=%
3CO80sz1zmDHA%2E2436%40TK2MSFTNGP09%2Ephx%2Egbl%3E

I tried moving sw.write to a separate funciton and nothing
happened... any more advice is appreciated.
thanks
rob




[quoted text, click to view]