Hi, I'm using MSXML4 and I'm having an issue with memory allocation which 
appears to be a memory leak, but isn't exactly. Let me explain.

I have this one XML file (the "IXM" file) that contains the names of 148 
other XML files (the "external" files). What is supposed to happen is that 
the app reads the IXM file, then reads each external file named in the IXM 
file, then copies the external file's data to the database.  The problem is 
that after about 36 external files, it runs out of memory. Each external file 
is generally between 104 MB and 106 MB, but some are larger. The largest is 
136 MB.  The external files are in ASCII.

I run performance monitor and monitor virtual bytes.  It seems that the 
virtual bytes are increasing all the time, and by the time of the 36th 
external file, virtual bytes tries to exceed 2 GB, and it bombs.

Based on experiments I've done, I conclude the following:

1. When MSXML is allocating memory, it is not doing so on the process heap, 
but on its own heap, so memory looks like this:

[PROCESS-HEAP][HEAP-1][HEAP-2][MSXML-HEAP-1][MSXML-HEAP-2]

2. The SAX parser in MSXML reads the entire external file into a contiguous 
region of memory before it will call the content handler.

3. If MSXML cannot load the entire external file into a contiguous region of 
memory in its own heaps, it allocates another heap:

[PROCESS-HEAP][HEAP-1][HEAP-2][MSXML-HEAP-1][MSXML-HEAP-2][MSXML-HEAP-3]

[PROCESS-HEAP][HEAP-1][HEAP-2][MSXML-HEAP-1][MSXML-HEAP-2][MSXML-HEAP-3][MSXML-HEAP-4]

[PROCESS-HEAP][HEAP-1][HEAP-2][MSXML-HEAP-1][MSXML-HEAP-2][MSXML-HEAP-3][MSXML-HEAP-4][MSXML-HEAP-5]

It does not seem to be freeing the extra heaps that it is creating.  So 
eventually, it will try to allocate a memory block that's too big for the 
existing heaps and too big to allocate a new heap.

Assuming that's the case:

1. How do I free up all the heaps that MSXML allocates, preferably between 
external files?

2. How do I get MSXML to use the app's heap instead of creating its own?

3. Is there any other strategy that I could try?

-- Thanks
-- Darren --