Home. 
.

transparent

transparent

transparent

Altova Mailing List Archives


Re: how to handle large xml files

From: "Anthony Jones" <Ant@------------.--->
To: NULL
Date: 12/2/2006 1:49:00 PM


"mindspring" <ardinnot@h...> wrote in message
news:Hp9ch.5945$tM1.5263@n......
>
> "Anthony Jones" <Ant@y...> wrote in message
> news:up2rZgZFHHA.420@T......
> >
> > "mindspring" <ardinnot@h...> wrote in message
> > news:GZ0ch.5415$sf5.2581@n......
> >> How does one parse a huge xml file?
> >> I am using asp / vba and msxml 4
> >>
> >> using declaration as
> >> Dim oXMLHTTP As MSXML2.ServerXMLHTTP40
> >>    Set oXMLHTTP = New MSXML2.ServerXMLHTTP40
> >>
> >> so you get the idea of what parser I use.
> >>
> >> What do you think?
> >> this works for small files, but the huge ones crash my system.
> >>
> >
> > Define huge?
> >
> > Your only other realistic option is to use WinHTTP in asynchonous mode
and
> > push the bytes through SAX in some way.  What are you did you intent to
do
> > with the data once you have parsed it into XML?
> >
> >
> >> Thanks
> >>
> >>
> >
> >
> here is the code I use:
>
> theurl = 'the url to the xml file'
>
>
> Dim oXMLHTTP As MSXML2.ServerXMLHTTP40
>    Set oXMLHTTP = New MSXML2.ServerXMLHTTP40
>
>    With oXMLHTTP
>        .setTimeouts 30000, 30000, 120000, 300000
>        .Open "POST", theurl, , "username", "password"
>        .setRequestHeader "CONTENT-TYPE", "application/x-
> www-form-urlencoded"

I've seen this a couple of times (although it could be from your previous
posts).  Why specify a Content-Type header then you're not sending any
content?  For that matter why use a post instead of a GET?

>        .send
>        strresult = .responseText
>    End With
>
>  strfile = strresult
>
>
> test.loadXML strfile

I take it you haven't got the issues with the data supply regarding
appropriate headers and encoding sorted yet.  It might help if you could use
the responseXML property since this will significantly reduce the memory
required.

>
> --------------
> I guess the above just loads the big xml file into memory first, then my
> parser will go through it.
> When I parse, it just store into an mdb table, but it just kills the pc.
I
> wish I could somehow save bits to the hard drive as I read in or
something.
> I will looking into the sax..

Yes looking into it you probably don't need the WinHTTP object at all.  The
use

rdr.putFeature  "server-http-request", true

on the SAX reader and it will use Server HTTP to fetch the uri provided in
the parseURI method.
Of course that does mean a GET not a POST.

>I do also have the problem of msaccess failing
> when the records are more than a few thousand, so I guess can use sql or
> mysql client on the pc and an ms access front end? how to do?

How to do what? There is probably a better place to ask how to use
Access/SQLServer/MySQL than in an XML group.

>
> Thanks
>
>




transparent
Print
Mail
Like It
Disclaimer
.

These Archives are provided for informational purposes only and have been generated directly from the Altova mailing list archive system and are comprised of the lists set forth on www.altova.com/list/index.html. Therefore, Altova does not warrant or guarantee the accuracy, reliability, completeness, usefulness, non-infringement of intellectual property rights, or quality of any content on the Altova Mailing List Archive(s), regardless of who originates that content. You expressly understand and agree that you bear all risks associated with using or relying on that content. Altova will not be liable or responsible in any way for any content posted including, but not limited to, any errors or omissions in content, or for any losses or damage of any kind incurred as a result of the use of or reliance on any content. This disclaimer and limitation on liability is in addition to the disclaimers and limitations contained in the Website Terms of Use and elsewhere on the site.

.
.

transparent

transparent