all groups > visual studio .net enterprise tools > october 2007 >
You're in the

visual studio .net enterprise tools

group:

merging .coverage files causes OutOfMemory


merging .coverage files causes OutOfMemory Dinan5
10/18/2007 1:21:04 PM
visual studio .net enterprise tools:
Hello All,

Hope i picked the right Group. If not maybe someone can move it for me?

I have an interesting situation. My team uses a propietary test
infrastructure that kicks off nightly On 8+ Servers. Each Server runs our
product in different VM enviroments (each with 3 to 7 VMs). The enviroments
simulate the different permutations that we expect to run into in the field.
So for instance we have a set of VMs that simulate a 64bit Exchange 2007
enviroment.

Anyway, I am currently collecting coverage results once a week across all of
these Virtual environments. For any given run I have upwords of 200+
coverage files. Usually these coverage files are 148k or 212k (why is that
anyway? they are always the same size. that's not important now).
Everything was working pretty well until this week when apparently I found
the limit in terms of the number of coverage files I could merge together. I
have a utility that merges all the coverage files for me and produces a
single "Results.XML". The nut of that code is:

public void Merge(string inputFile)
{
Console.WriteLine("File being merged: " + inputFile);
CoverageInfoManager.SymPath = pdbFilePath;
CoverageInfoManager.ExePath = binaryFilePath;
ci = CoverageInfoManager.CreateInfoFromFile(inputFile);
if (resultInfo == null)
resultInfo = ci;
else
resultInfo = CoverageInfoManager.JoinInfo(resultInfo, ci,
CoverageJoinType.CoverageJoinUnion);
}

public void ExportXml(string xmlPath)
{
resultData = resultInfo.BuildDataSet(null);
resultData.ExportXml(xmlPath);
}

The merge method is invoked in a loop for every .coverage file. Then the
exportXml is invoked once at that end. Again up until this week this worked
fine now what I'm seeing is the following exception when I run the code above:

Microsoft.VisualStudio.CodeCoverage.CoverageQueryException: Error when
querying
coverage data: Error when calculating line coverage --->
System.OutOfMemoryExcep
tion: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Collections.ArrayList.GetEnumerator()
at System.Data.InternalDataCollectionBase.GetEnumerator()
at System.Data.ConstraintEnumerator.GetNext()
at System.Data.DataTable.CascadeAll(DataRow row, DataRowAction action)
at System.Data.DataTable.RaiseRowChanging(DataRowChangeEventArgs args,
DataRo
w eRow, DataRowAction eAction, Boolean fireEvent)
at System.Data.DataTable.SetNewRecordWorker(DataRow row, Int32
proposedRecord
, DataRowAction action, Boolean isInMerge, Int32 position, Boolean
fireEvent, Ex
ception& deferredException)
at System.Data.DataTable.InsertRow(DataRow row, Int32 proposedID, Int32
pos,
Boolean fireEvent)
at System.Data.DataRowCollection.Add(DataRow row)
at
Microsoft.VisualStudio.CodeCoverage.CoverageInfo.FillLineData(CoverageDS c
ov_ds, DataRow method_row, RVASet* rvaset, Guid[] testcases, ModuleInstance
curr
_mod_inst)
--- End of inner exception stack trace ---
at
Microsoft.VisualStudio.CodeCoverage.CoverageInfo.FillLineData(CoverageDS c
ov_ds, DataRow method_row, RVASet* rvaset, Guid[] testcases, ModuleInstance
curr
_mod_inst)
at Microsoft.VisualStudio.CodeCoverage.CoverageInfo.BuildDataSet(Guid[]
tests
)
at CoverageMerge.CoverageMerge.Main(String[] args)

I have played around and tried to do some Garbage Collection at certain
strategic places in the code i published above. I've also chaged the code
around a bit so that i could rid myself of the relatively interum resultInfo
object before actually generating the xml from the resultData object. This
helped a bit but eventually I always run out of memory either while building
the Data Set (BuildDataSet) or when I go to Export the XML (ExportXml). I'm
running the merge code on a 2003 Enterprise Server with 8GB of memory, I am
certainly not even coming close to using all of the memory available in the
system. If I watch the memory usage of my merge tool's process I see that it
seems to die right around 850MB. So maybe a stop gap solution here would be
to allow my process to use more memory, unfortunately I can't seem to figure
out how to properly do that. But again that will only be a stop gap, because
I expect that I may double or even triple the amount of coverage data I'm
collecting by the end of this project.



While debugging I've also done two separate "smaller" merges of the
..coverage files. Of course these two merges work. but when I attempt to
merge the resulting "Results.XML" files within the Visual Studio IDE i get a
total of 0% coverage with no errors thrown by the IDE. So this is either a
bug with merging XML result files...or it simply is not possible.



Well any and all help will be very appreciated, Thanks in advance,

Justin
AddThis Social Bookmark Button