Just as an update, i'm giving up on the custom indexing approach. I only
(test file 10mb). Unfortunately that's still far too big, especially if my
XML is 1GB. I'll perhaps come back to this when I get time. :)
"Andrew Brook" <ykoorb@hotmail.com> wrote in message
news:u9FugvHvHHA.4908@TK2MSFTNGP03.phx.gbl...
> Thanks, i'll look into that.
>
> At the moment i'm implementing a text reader that has buffering like a
> streamreader, but still exposes the correct position in the base stream.
> Not sure where i'll head after that, but maybe use it as a basis for an
> XmlTextReader.
>
> Andrew
>
> "Angel J. Hernández M." <angeljesus14@hotmail.com> wrote in message
> news:6D67DC0F-9208-401B-A3D9-E59EA665AA06@microsoft.com...
>> Hi there,
>>
>> What about if you insert that file into an XML Column in a SQL Server
>> 2005 Table. In SQL Server 2005 you can have XML indexes
>>
>> Best regards,
>>
>>
>> --
>> Angel J. Hernández M.
>> MCP,MCAD,MCSD,MCDBA,MCT
>> Microsoft MVP
>>
http://msmvps.com/blogs/angelhernandez >> angeljesus14@hotmail.com
>>
>>
>>
>> "Andrew Brook" <ykoorb@hotmail.com> wrote in message
>> news:%23ZQ9uZztHHA.1060@TK2MSFTNGP06.phx.gbl...
>>> Hi Everyone,
>>>
>>> I have a very large XML file (~1GB). I would like to essentially
>>> pre-navigate the entire structure using an XmlReader and somehow index
>>> the positions of important elements.
>>>
>>> I suppose I'd hoped that I could access the exact file position of an
>>> xml element, store that position with a unique id in a hashtable and
>>> then be able to quickly seek back to that position at a later point to
>>> get the data out.
>>>
>>> The only way I can think of achieving this is to implement my own Stream
>>> Reader which will allow seeking as well as returning accurate position
>>> information (incorporating buffering etc).
>>>
>>> I've seen documentation about special XPath Indexed Navigators but these
>>> only work when the XML is in memory and that's definitely something I
>>> need to avoid.
>>>
>>> I don't suppose anyone has come across a problem like this, or perhaps
>>> have some ideas about solving the problem?
>>>
>>> Thanks,
>>> Andrew
>>>
>>
>
>