all groups > dotnet clr > september 2007 >
You're in the

dotnet clr

group:

starting streams at offsets


starting streams at offsets Lee Crabtree
9/24/2007 1:55:16 PM
dotnet clr:
Is there an existing means of starting a FileStream at a given offset
into a file? That is, can the origin be set somewhere other than the
beginning of the actual file?

As an example, let's suppose I'd like to disregard the first 32768 bytes
in a file. I know I could create a normal file and just seek forward,
but if I could set the origin of the stream 32768 bytes from the
beginning of the file, I wouldn't have to know that offset anywhere
other than the creation of the stream.

If it's not possible I'm not going to be doomed to failure, but it's
something that would be handy.

Re: starting streams at offsets Doug Semler
9/24/2007 9:02:07 PM
[quoted text, click to view]

Why? Creating the stream *always* sets the offset to 0. You seek
independently of the creation.

Stream s = (new whateverstream)
s.Seek(offsetdesired, SeekOrigin.begin);



--
Doug Semler, MCPD
a.a. #705, BAAWA. EAC Guardian of the Horn of the IPU (pbuhh).
The answer is 42; DNRC o-
Gur Hfrarg unf orpbzr fb shyy bs penc gurfr qnlf, abbar rira
erpbtavmrf fvzcyr guvatf yvxr ebg13 nalzber. Fnq, vfa'g vg?
Re: starting streams at offsets Jon Skeet [C# MVP]
9/25/2007 12:00:00 AM
[quoted text, click to view]

One point about the above code: I personally find the Position property
clearer than the Seek method. It tends to be overlooked for historical
reasons, but:

Stream s = (new whateverstream);
s.Position = offsetDesired;

Just a bit simpler, IMO.

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
Re: starting streams at offsets Lee Crabtree
9/25/2007 10:06:41 AM
Here's a more concrete example of what I'm currently tasked with doing.

Multiple images have been put into one file, with a "master" header that
gives the offsets into the file to the beginning of each image. Now,
I COULD use a single stream and seek to the beginning of each file when
I need to, but it seems to me that it would make MUCH more sense to use
one stream for each image (everything is reading only, so I don't have
to worry about writing) with each stream's origin being the offset into
the file of the associated image.

If that's not possible, I'm basically forced to hold multiple offsets
and jump a single stream around the file, and I can just see that being
all kinds of error-prone.

Lee Crabtree

[quoted text, click to view]
Re: starting streams at offsets Günter Prossliner
9/25/2007 5:48:56 PM
Hello Lee!

[quoted text, click to view]

Of cause it is possible to do so. You may implement a special
System.IO.Stream which represents another stream starting from offset X.

e.g (untested).:

class OffsetStream : Stream {

Stream _s;
long _o, _l;

public OffsetStream(Stream stream, long offset, long length){
_s = stream;
_o = offset;
_l = length;

if(!_s.CanSeek)
throw new ArgumentException("not seekable...");
}

public override bool CanRead{
return _s.CanRead;
}

// do so with "CanWrite", "CanSeek", ...

public override long Length {
get{return _l;}
}

public override void SetLength(long value){
// would affect other parts of the stream
throw new NotSupportedException();
}

// seeking

public override long Seek(long offset, SeekOrgin orgin){
long p;
switch(orgin){
case SeekOrgin.Begin: p = _o+offset; break;
case SeekOrgin.Current: p = Position + offset;
case SeekOrgin.End: p = _o + _l - offset; break;
}

_s.Seek(SeekOrgin.Begin, p);
return p - _o;
}

public override long Position {
get{ return _s.Position - _o; }
set{Seek(value, SeekOrgin.Begin);}
}

// implement the other abstract members
}


This class may not be used to split up an Stream and to concurrenlty use the
parts individually (even in single-threaded environments). Because all
"OffsetStream" instances manipulate the Postion within the supplied Stream,
they would affect each other.

You have to know what operations are happening with your "OffsetStream" to
know if this is a problem or not. Image.FromStream() is a "Black-Box". It is
not documented when and how the Stream is used. So I would NOT recommend to
use the "OffsetStream" in this scenario.

You may implement the "OffsetStream" in a way that it loads all Data into
memory when it gets constructed. This would actually increase the
Memory-Footprint, but each Stream can operate individually.

e.g.

class OffsetStream : MemoryStream {

public OffsetStream(Stream stream, long offset, long length)
: base(GetData(stream, offset, length), false) {}


static byte[] GetData(Stream stream, long offset, long length){

byte[] buffer = new byte[(int)length];
stream.Read(buffer, (int)offset, buffer.Length);
return buffer;

// you should check the return-value of "stream.Read" to ensure that
// the data is read completly.

// you can also use a "BinaryReader" (which checks the Return-Value
by itself):
BinaryReader reader = new BinaryReader(stream);

// allthough this lock does NOT garanty that this code is
thread-safe in any case,
// it garanties that it is threadsafe when you create multiply
OffsetStream instances
// based on the the same Stream-Object.
lock(stream){
stream.Position = offset;
return reader.ReadBytes((int)length);
}
}

}




GP

Re: starting streams at offsets Doug Semler
9/25/2007 8:15:14 PM
[quoted text, click to view]


Hehe, can you tell i come from the Unix world? I forgot about Position,
because I am so used to random seeks from current position <g>

--
Doug Semler, MCPD
a.a. #705, BAAWA. EAC Guardian of the Horn of the IPU (pbuhh).
The answer is 42; DNRC o-
Gur Hfrarg unf orpbzr fb shyy bs penc gurfr qnlf, abbar rira
erpbtavmrf fvzcyr guvatf yvxr ebg13 nalzber. Fnq, vfa'g vg?
Re: starting streams at offsets Jon Skeet [C# MVP]
9/26/2007 7:29:01 PM
[quoted text, click to view]

You're not alone - I see calls to Seek() far more often than calls to
Position. The Position property is one of those "discovery" things -
once you've seen it, you tend to use it, but a lot of people haven't
seen it :)

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
Re: starting streams at offsets Doug Semler
9/26/2007 7:40:12 PM
[quoted text, click to view]


Not sure if it's applicable to this group but I was curious about it. Is
there any performance hit by using the "Position" property versus
"Seek(offset, SeekOffset.Current)?

Reason I ask: Example, the FileStream class Position property resets the
seek from the beginning of the file regardless of the current
position...just wondering if the reset back to the beginning of the file
hurts performance any versus a seek from the current offset...

(BTW I know what you mean about "discovery" things....just found out about
the ?? operator the other day ... thanks for mentioning it!!! lol)

--
Doug Semler, MCPD
a.a. #705, BAAWA. EAC Guardian of the Horn of the IPU (pbuhh).
The answer is 42; DNRC o-
Gur Hfrarg unf orpbzr fb shyy bs penc gurfr qnlf, abbar rira
erpbtavmrf fvzcyr guvatf yvxr ebg13 nalzber. Fnq, vfa'g vg?
Re: starting streams at offsets Jon Skeet [C# MVP]
9/27/2007 5:35:13 PM
[quoted text, click to view]

I don't know - I would expect there to be any though.

[quoted text, click to view]

It doesn't reset back to the beginning of the file - it resets
*relative* to the beginning of the file, i.e. an "absolute" position.
Assuming the OS maintains some sort of integer to say where the stream
currently is, it could easily optimise an absolute call to a relative
one if that gives a performance improvement in a particular filesystem.

I guess if you were doing a lot of this it and had performance issues
would be worth benchmarking, but I wouldn't expect to see a lot of
difference.

[quoted text, click to view]

:)

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
Re: starting streams at offsets Doug Semler
9/28/2007 6:56:13 PM
[quoted text, click to view]


That's what I wasn't sure about. Some (older) OSs that I had worked with in
the past (notably earlier versions of HP-UX and I think SunOS 4 IIRC) would
basically invalidate the I/O cache for the file if you did a seek from begin
over current because it made different assumptions based on Begin, Current,
End...whereas a forward look/backward look off a current offset may or may
not have have invalidated the cache (depending on the size of the seek,
etc). But that may have been more of a hardware architecture layout than
anything else)

in other words, if your current fp was located at say offset 32,127,181

seek(32,100,101, BEGIN) would often cause a cache flush whereas
seek(-27,180, CURRENT) more often than not did not...

(but, like I said, Windows wasn't the OS I grew up on, and I/O is not the
bottlenecks in image processing which is my current balliwick. :)

--
Doug Semler, MCPD
a.a. #705, BAAWA. EAC Guardian of the Horn of the IPU (pbuhh).
The answer is 42; DNRC o-
Gur Hfrarg unf orpbzr fb shyy bs penc gurfr qnlf, abbar rira
erpbtavmrf fvzcyr guvatf yvxr ebg13 nalzber. Fnq, vfa'g vg?
AddThis Social Bookmark Button