Heap Magic: Process Heap Internals, Part 2

In part 1 of this series, I described a technique that is helpful during live debugging, but not very helpful when you receive crash dumps from the field. In this second article of the series, I discuss further techniques that will help you reproduce bugs and troubleshoot behaviors related to heap memory issues.

For this discussion, I again focus on the MyClass type as well as the intentional dangling pointer I created in main() in Figure 1, where I delete the object pointed to by the anInstance variable, then subsequently dereference it.

The process I described last month is not helpful when you receive crash dumps from the field because end users typically won't be running with the debug heap and instead will most likely be running with the low fragmentation heap (LFH). You might be wondering how the debug heap got turned on in the first place. The OS loader did that for us automatically because it noticed a debugger was attached to the process. If you look at the global flags for the process from within the debugger, you'll see the following even though I didn't explicitly turn on any flags:

0:000> !gflag

Current NtGlobalFlag contents: 0x00000070

    htc - Enable heap tail checking

    hfc - Enable heap free checking

    hpc - Enable heap parameter checking

These are all enabled for you by the OS loader. It's the Enable heap free checking flag with the hfc moniker that's responsible for filling the freed memory with 0xfeeefeee.

This behavior is not very helpful when you need to reproduce a bug in the field with the debugger attached. So, thankfully, this is easy to turn off by setting the environment variable _NO_DEBUG_HEAP=1 or by launching WinDbg with the -hd option. I like the environment variable approach because it works no matter what.

After setting _NO_DEBUG_HEAP=1 and restarting within the debugger, you should see that after calling the delete operator the memory looks like that in Figure 2 (of course, the memory location is different than what I mentioned previously).

This is where the tag field really shines because although this block of memory is freed, you can still tell what the pointer used to point to. That might be helpful when you know you have a dangling pointer; this just might be the evidence you need to identify the bug.

Let's also take a look at the pointer using !heap -x to find out more about it, as Figure 3 shows. Everything looks almost identical except that you can see that I'm definitely using the LFH heap. You can also verify that after executing the call to SetField() on the freed instance that !heap -v cannot find the modification as it did before, now that the fill pattern is missing.

At this point, you're probably wondering what the pad fields are all about in the MyClass type and why I put them there. As an exercise, try removing them, then perform the previous steps again without the debug heap. You should see that the tag and potentially other fields within the dead instance are stomped. This is because the heap implementation uses the beginning of the user portion of the allocation to manage freed blocks. For example, if the heap puts the entry on a linked list of allocations of the same size, it can simply pluck an entry off that list if someone asks for an allocation of that size. It then could use the beginning of the allocation to manage the pointers of the linked list. Notice that the LFH stomped a portion of my 0xBAADF00D filler word during deallocation.

In short, the padding is an extra measure to prevent the heap from stomping the fields within the dead object, thus making it easy to inspect the object's state at the time it was freed. You should reserve two pointer-size fields for the padding. Although the LFH doesn't use that much space, the non–low-frequency heap uses the pointers to put the entry into a linked list. Several flags could turn off LFH, including any of the flags that trigger the debug heap as well as heaps created with the HEAP_NO_SERIALIZE flag.

The flags that trigger the debug heap include the following and can be set outside the debugger using the gflags.exe tool that comes with the Debugging Tools for Windows:

For example, if you want to set the heap parameter-checking global flag for the process, you could execute the following from an elevated command prompt (assuming the executable is named HeapMagic.exe):

gflags /i HeapMagic.exe +hpc

Ultimately, this adds an entry for HeapMagic.exe in the HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindows NTCurrentVersionImage File Execution Options registry key. And if you launch the process within the debugger, assuming you still have _NO_DEBUG_HEAP=1, the !gflag command within the debugger should reflect the previously mentioned setting.

Searching Heap Memory

Another very useful facet of the tag field within the MyClass instance is that it lets you easily find instances of your type on the heap simply by searching memory. For example, if you execute the app within the debugger and break on the delete operator call, you can search for all the MyClass instances within the list in the instances variable. Use the techniques previously described to find the heap that the instance pointed to by anInstance lives in (hint: !heap -x). Once you have that, you can use !heap -m to find all the segments of the heap, as Figure 4 shows.

Heaps consist of segments where each segment represents a virtual memory allocation within the process address space. Using the Base and Last Entry fields in Figure 4 to get a range for a segment, you can then use the debugger's s command to search the memory range for the tag, as follows:

s -d 00260000 L?10000 416a624f

The -d option tells the debugger that I'm searching for a DWORD. 0x416a624f is the little-endian representation of ObjA, the tag for MyClass. 0x00260000 tells the command where to start searching, and L?10000 tells it to search a 0x10000 byte range.

Once you find the pointer to the tag, you can do the math to find the offset to the beginning of the instance, thus obtaining the pointer to the instance. For example, below is a line from the output generated on my machine:

00000000`002694e0  416a624f 00000000 0000000c 00000058  ObjA........X...

Using the dt command, I can quickly see the type layout of MyClass showing the tag offset is 0x10:

0:000> dt MyClass

HeapMagic!MyClass

   +0x000 pad1             : Uint8B

   +0x008 pad2             : Uint8B

   +0x010 tag              : Uint8B

   +0x018 m_fieldA         : Int4B

   +0x01c m_fieldB         : Int4B

Now it's a simple matter to calculate that the pointer to the instance from the previous output is 0x00000000`002694d0. Dumping it out with !heap -x shows something interesting in Figure 5. Notice that the pointer in the User column is not the same as the pointer to the MyClass instance. This is because std::list adds some overhead to the allocation to manage its internal list. From the data above, you can see that the overhead is 16 bytes (2 pointers in length).

As a final bonus and as an exercise, try adding the -v option to !heap -x as shown below to show you which memory locations contain a pointer to this heap block:

!heap -x -v 00000000`002694d0

This is a nifty trick to identify who might be holding a pointer to this object instance.

Valuable Knowledge

I've demonstrated that knowing a little bit about how the process heap works can go a long way. If it makes engineering sense to do so, you can add some padding at the beginning of your type as well as a tag value to help identify your instances within the heap. This will proactively prepare your application for troubleshooting after release. When you receive dumps from customers, you have one more tool in your debugging forensics arsenal to find out what went wrong.

Look for more upcoming columns in which I'll cover process heaps—including how to use pageheap, which is arguably the nuclear weaponry of heap corruption detection tools built into the OS. Until then, happy troubleshooting!