Groups | Blog | Home
all groups > visual studio .net debugging > december 2004 >

visual studio .net debugging : Mysterious Value Dissapears :(


Bilbao
12/8/2004 8:47:05 AM
Hi,

Last night (12/08/04) I modified a code (MFC & DirectX9) and after that, at
a specific loop one of the values (a pointer) gets set to NULL:
for (int iPaletteEntry = 0; iPaletteEntry < pMesh->m_dwNumPaletteEntries;
++iPaletteEntry)
{
iMatrixIndex = pAttributeTable[i].BoneId[iPaletteEntry];
if (iMatrixIndex != UINT_MAX)
{
pMatList[iPaletteEntry] = pMesh->m_pFinalBoneMatrices[iMatrixIndex];
}
}

Now the variable that gets set to NULL is "pAttributeTable" (which is a
pointer to D3DXBONECOMBINATION, and is locally declared within the function),
the mystery lies on the fact that it gets set to NULL when iPaletteEntry
reaches 27!

This is an MFC application and it is doesn't have threads (at least created
by me).

The value it references still exist and untouched. It is as if it looses
the reference of is simply set to NULL.

The situation got way more frustrating when I modified the line to read like
this:
iMatrixIndex = pMesh->m_AttributeTable[i].BoneId[iPaletteEntry];

Now "m_AttributeTable" is a member variable that belongs to the pMesh object
which persist in a global scope.
But to my biggest surprise when iPaletteEntry reaches 28 pMesh gets
completely deleted (most of the values/members inside say "CXX0030: Error:
expression cannot be evaluated"), as if it were deleted (intentionally using
the destructor), but not set to NULL, as pMesh still holds the value of the
address and is not equal to 0.
But I set a "Breakpoint" in the destructor of pMesh and it is never accessed.

Note that "i" never changes and the code breaks when i is still equals to 0
(which is it's initial value).

The version of VS.NET I am using is 2002 and the code is traditional MFC
C++, and not managed.

Surprisingly enough I copied the .h and .cpp files to another computer with
VS.NET 2003 and I still get the same problem!

This is sooo frustrating, the first thing that comes to my mind is that
there might be something hidden, because it all seems so "smart".

Please, let me know if you have any question or need more information.

Thanks a million in advance!
--
Bilbao
12/9/2004 4:35:02 AM
Oleg,

You are absolutely right, pMatList is not big enough. And what was really
happenning was that the variables that were allegedly being "deleted" or "set
to NULL" were being somehow modified by going out of pMatList bounds.

After all these years I keep falling in the same trap :(

Thanks for your suggestions, I greatly appreciate your time helping me!

Cheers,
~Bilbao

[quoted text, click to view]
Oleg Starodumov
12/9/2004 11:15:25 AM

It looks like a write past the end of an array allocated on the stack
(as a local variable).

Without seeing the code for the whole function, I would suspect
that pMatList could point to a local array, whose size is not big enough
to contain 27 (28?) elements.

While debugging, you can try to catch it in the following way:

1. When entering the function, determine the address of the variable
that is being corrupted (pAttributeTable, or pMesh) (e.g. enter "&pMesh"
in Watch window to determine the address).

2. After the value of the variable has been initialized, set a data breakpoint
at the address of the variable (use "Data" tab in New Breakpoint dialog,
and after closing the dialog make sure that the breakpoint's checkbox is checked
in Breakpoints window).

3. Step through the function and wait until the breakpoint is hit.
The execution will stop at the line that wrote to the variable,
or at the next line after it. Now you can see exactly who changes the value
of the variable.

4. Disable the breakpoint after control leaves the function, since this place
on the stack can be reused and can cause false alarms (breakpoint hits).

Or, without using a data breakpoint, simply step through the function's code
and see where the value of the variable is changed.

Also it makes sense to build the project with /RTC1 (debug build only)
and /GS compiler options (note that these options help to detect various
stack-related problems, but they also change the layout of the stack,
so if they are not yet enabled, it might be better to enable them only after
you have fixed this particular problem)

Regards,
Oleg





AddThis Social Bookmark Button