| Groups | Blog | Home |
dotnet performance : byte[] to byte*... What is the fastest way?
"ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote: > The subject says it all... I want to use a byte[] and use it as byte* so I > can increment the pointer to iterate through it. I really hate to be pedantic, but I'm willing to bet that the difference in how you iterate through your array makes little to no difference in the overall performance of your code. People frequently are guilting of over-optimizing things that are already "Fast Enough". Unless you've verified this section is slow via a Profiler, you're better off not getting fancy with optimizations. -- Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP http://www.coversant.com/blogs/cmullins
On Aug 23, 10:05 pm, "ThunderMusic" [quoted text, click to view] <NoSpAmdanlatathotmaildot...@NoSpAm.com> wrote: > on a performant checksum algorithm. It will be for an error checking > mecanism for tcp and udp communication on a closed network environment. So > it doesn't need to be human-modification resistant, it's just to prevent > modification due to the noise on the line (if it can happen)... It can't happen. Now, how's that for optimization?
I understand well the "This code it called alot, therefore it needs to be
optimized." frame of mind. I build big, high-performance, network socket servers for a living. I know this space really well, and have done lots in both TCP & UDP land at scales that are pretty signfigant. Put a profiler on this, and you'll likley find that this chunk of code won't even show up. Certainly the difference between the two (pointer vs iterator) is unlikley to show up at all. -- Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP http://www.coversant.com/blogs/cmullins [quoted text, click to view] "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:eZjuTUc5HHA.4964@TK2MSFTNGP06.phx.gbl... > hi, > I know it may be overoptimizing, but this part of the code will be called > many many many times per seconds, so I'm must make sure it's as optimized > as it can be. It's for a checksum mecanism, so each time a message must be > sent, it is called to compute the checksum to append to the message and > will be computed again when the client receives it so it can verify the > validity of the message. > > Thanks for the comment tought... > > ThunderMusic > > "Chris Mullins [MVP]" <cmullins@yahoo.com> wrote in message > news:uyDEaDc5HHA.484@TK2MSFTNGP06.phx.gbl... >> "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote: >> >>> The subject says it all... I want to use a byte[] and use it as byte* so >>> I can increment the pointer to iterate through it. >> >> I really hate to be pedantic, but I'm willing to bet that the difference >> in how you iterate through your array makes little to no difference in >> the overall performance of your code. >> >> People frequently are guilting of over-optimizing things that are already >> "Fast Enough". Unless you've verified this section is slow via a >> Profiler, you're better off not getting fancy with optimizations. >> >> -- >> Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP >> http://www.coversant.com/blogs/cmullins >> > >
Hi,
The subject says it all... I want to use a byte[] and use it as byte* so I can increment the pointer to iterate through it. What is the fastest way of doing so in C#? Thanks ThunderMusic
ThunderMusic,
Use unsafe code: byte[] bytes = ...; unsafe { fixed (byte* p = bytes) { // Work with pointer here. } } As a matter of fact, that's the only way to do it, as you need to pin down the location of the array to prevent the reference from moving around. Is there a reason you need the pointer, or are you just looking for a faster way to iterate through the array? -- - Nicholas Paldino [.NET/C# MVP] - mvp@spam.guard.caspershouse.com [quoted text, click to view] "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:%23KkK5zb5HHA.5316@TK2MSFTNGP04.phx.gbl... > Hi, > The subject says it all... I want to use a byte[] and use it as byte* so I > can increment the pointer to iterate through it. > > What is the fastest way of doing so in C#? > > Thanks > > ThunderMusic >
Hi,
[quoted text, click to view] "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:%23KkK5zb5HHA.5316@TK2MSFTNGP04.phx.gbl... > Hi, > The subject says it all... I want to use a byte[] and use it as byte* so I > can increment the pointer to iterate through it. > > What is the fastest way of doing so in C#? Out of curiosity, why you want to do that? What is wrong with using a indexer like for( int i=0; i< buffer.Lengh; i++ buffer[i] .....
hi,
thanks for the really quick answer... Your solution works, but I can't do p++ because it's fixed. So I must use another pointer like byte* p2 = p; so now I can do p2++;... Actually, I'm looking for a faster way to compute a checksum on a byte array... For now, I'm using the Adler-32 algorithm, but I'm open to advises on a performant checksum algorithm. It will be for an error checking mecanism for tcp and udp communication on a closed network environment. So it doesn't need to be human-modification resistant, it's just to prevent modification due to the noise on the line (if it can happen)... Thanks ThunderMusic "Nicholas Paldino [.NET/C# MVP]" <mvp@spam.guard.caspershouse.com> wrote in message news:uezij7b5HHA.5360@TK2MSFTNGP03.phx.gbl... [quoted text, click to view] > ThunderMusic, > > Use unsafe code: > > byte[] bytes = ...; > > unsafe > { > fixed (byte* p = bytes) > { > // Work with pointer here. > } > } > > As a matter of fact, that's the only way to do it, as you need to pin > down the location of the array to prevent the reference from moving > around. > > Is there a reason you need the pointer, or are you just looking for a > faster way to iterate through the array? > > > -- > - Nicholas Paldino [.NET/C# MVP] > - mvp@spam.guard.caspershouse.com > > "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message > news:%23KkK5zb5HHA.5316@TK2MSFTNGP04.phx.gbl... >> Hi, >> The subject says it all... I want to use a byte[] and use it as byte* so >> I can increment the pointer to iterate through it. >> >> What is the fastest way of doing so in C#? >> >> Thanks >> >> ThunderMusic >> > >
On Thu, 23 Aug 2007 15:36:18 -0400, "ThunderMusic"
[quoted text, click to view] <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote: >Hi, >The subject says it all... I want to use a byte[] and use it as byte* so I >can increment the pointer to iterate through it. > >What is the fastest way of doing so in C#? > I have recently started with C#, converting C++ applications that must communicate with external equipment using protocols with rigidly fixed byte sequences. Initially I was terribly frustrated abandoning my precious pointer manipulation and access to byte arrays. But I developed a bunch of utilities to create managed code byte[] arrays from pieces and extract appropriate values from them and can now avoid all unmanaged code. Look into Encoding.Convert for converting between ASCII byte sequences and Unicode, MemoryStream.Write and ToArray along with BinaryWriter.Write for creating byte[] data and the BitConverter set of methods to extract values from byte[]. I keep telling myself that staying entirely within managed code is worth all that effort, but I am still early in the process. Some old timers here can probably fill you in on much more detail, including the virtues of doing so.
ThunderMusic,
If you are looking to tune this algoritm, I would say that the iteration through the bytes is NOT the way to do it. There are probably a bunch of other areas that can be improved upon. -- - Nicholas Paldino [.NET/C# MVP] - mvp@spam.guard.caspershouse.com [quoted text, click to view] "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:ukULcEc5HHA.2312@TK2MSFTNGP06.phx.gbl... > hi, > thanks for the really quick answer... Your solution works, but I can't > do p++ because it's fixed. So I must use another pointer like byte* p2 = > p; so now I can do p2++;... > > Actually, I'm looking for a faster way to compute a checksum on a byte > array... For now, I'm using the Adler-32 algorithm, but I'm open to > advises on a performant checksum algorithm. It will be for an error > checking mecanism for tcp and udp communication on a closed network > environment. So it doesn't need to be human-modification resistant, it's > just to prevent modification due to the noise on the line (if it can > happen)... > > Thanks > > ThunderMusic > > "Nicholas Paldino [.NET/C# MVP]" <mvp@spam.guard.caspershouse.com> wrote > in message news:uezij7b5HHA.5360@TK2MSFTNGP03.phx.gbl... >> ThunderMusic, >> >> Use unsafe code: >> >> byte[] bytes = ...; >> >> unsafe >> { >> fixed (byte* p = bytes) >> { >> // Work with pointer here. >> } >> } >> >> As a matter of fact, that's the only way to do it, as you need to pin >> down the location of the array to prevent the reference from moving >> around. >> >> Is there a reason you need the pointer, or are you just looking for a >> faster way to iterate through the array? >> >> >> -- >> - Nicholas Paldino [.NET/C# MVP] >> - mvp@spam.guard.caspershouse.com >> >> "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message >> news:%23KkK5zb5HHA.5316@TK2MSFTNGP04.phx.gbl... >>> Hi, >>> The subject says it all... I want to use a byte[] and use it as byte* so >>> I can increment the pointer to iterate through it. >>> >>> What is the fastest way of doing so in C#? >>> >>> Thanks >>> >>> ThunderMusic >>> >> >> > >
hi,
I know it may be overoptimizing, but this part of the code will be called many many many times per seconds, so I'm must make sure it's as optimized as it can be. It's for a checksum mecanism, so each time a message must be sent, it is called to compute the checksum to append to the message and will be computed again when the client receives it so it can verify the validity of the message. Thanks for the comment tought... ThunderMusic [quoted text, click to view] "Chris Mullins [MVP]" <cmullins@yahoo.com> wrote in message news:uyDEaDc5HHA.484@TK2MSFTNGP06.phx.gbl... > "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote: > >> The subject says it all... I want to use a byte[] and use it as byte* so >> I can increment the pointer to iterate through it. > > I really hate to be pedantic, but I'm willing to bet that the difference > in how you iterate through your array makes little to no difference in the > overall performance of your code. > > People frequently are guilting of over-optimizing things that are already > "Fast Enough". Unless you've verified this section is slow via a Profiler, > you're better off not getting fancy with optimizations. > > -- > Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP > http://www.coversant.com/blogs/cmullins >
hi,
So, here is the code to get my checksum. If you see something that can be optimized, let me know. thanks ThunderMusic [Code] private const UInt16 MOD_ADLER = 65521; private unsafe uint GetChecksum(byte[] databytes) { UInt32 a = 1, b = 0; fixed (byte* tmpdata = databytes) { byte* data = tmpdata; int len = databytes.Length; /* Length in bytes */ while (len > 0) { int tlen = len > 5550 ? 5550 : len; len -= tlen; do { a += *(data++); b += a; } while ((--tlen) > 0); a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); } /* It can be shown that a <= 0x1013a here, so a single subtract will do. */ if (a >= MOD_ADLER) a -= MOD_ADLER; /* It can be shown that b can reach 0xffef1 here. */ b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); if (b >= MOD_ADLER) b -= MOD_ADLER; } return ((b << 16) | a); } [/Code] "Nicholas Paldino [.NET/C# MVP]" <mvp@spam.guard.caspershouse.com> wrote in message news:ewAkXGc5HHA.4712@TK2MSFTNGP04.phx.gbl... [quoted text, click to view] > ThunderMusic, > > If you are looking to tune this algoritm, I would say that the > iteration through the bytes is NOT the way to do it. There are probably a > bunch of other areas that can be improved upon. > > > -- > - Nicholas Paldino [.NET/C# MVP] > - mvp@spam.guard.caspershouse.com > > "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message > news:ukULcEc5HHA.2312@TK2MSFTNGP06.phx.gbl... >> hi, >> thanks for the really quick answer... Your solution works, but I can't >> do p++ because it's fixed. So I must use another pointer like byte* p2 = >> p; so now I can do p2++;... >> >> Actually, I'm looking for a faster way to compute a checksum on a byte >> array... For now, I'm using the Adler-32 algorithm, but I'm open to >> advises on a performant checksum algorithm. It will be for an error >> checking mecanism for tcp and udp communication on a closed network >> environment. So it doesn't need to be human-modification resistant, it's >> just to prevent modification due to the noise on the line (if it can >> happen)... >> >> Thanks >> >> ThunderMusic >> >> "Nicholas Paldino [.NET/C# MVP]" <mvp@spam.guard.caspershouse.com> wrote >> in message news:uezij7b5HHA.5360@TK2MSFTNGP03.phx.gbl... >>> ThunderMusic, >>> >>> Use unsafe code: >>> >>> byte[] bytes = ...; >>> >>> unsafe >>> { >>> fixed (byte* p = bytes) >>> { >>> // Work with pointer here. >>> } >>> } >>> >>> As a matter of fact, that's the only way to do it, as you need to pin >>> down the location of the array to prevent the reference from moving >>> around. >>> >>> Is there a reason you need the pointer, or are you just looking for a >>> faster way to iterate through the array? >>> >>> >>> -- >>> - Nicholas Paldino [.NET/C# MVP] >>> - mvp@spam.guard.caspershouse.com >>> >>> "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message >>> news:%23KkK5zb5HHA.5316@TK2MSFTNGP04.phx.gbl... >>>> Hi, >>>> The subject says it all... I want to use a byte[] and use it as byte* >>>> so I can increment the pointer to iterate through it. >>>> >>>> What is the fastest way of doing so in C#? >>>> >>>> Thanks >>>> >>>> ThunderMusic >>>> >>> >>> >> >> > >
Hi,
[quoted text, click to view] "ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:eZjuTUc5HHA.4964@TK2MSFTNGP06.phx.gbl... > hi, > I know it may be overoptimizing, but this part of the code will be called > many many many times per seconds, so I'm must make sure it's as optimized > as it can be. It's for a checksum mecanism, so each time a message must be > sent, it is called to compute the checksum to append to the message and > will be computed again when the client receives it so it can verify the > validity of the message. > > Thanks for the comment tought... Why don't you do something, try using the "vanila" for statement versus the pointer arithmetic you want to use, I will be surprised if you find any noticeable difference.
[quoted text, click to view]
>The subject says it all... I want to use a byte[] and use it as byte* so I >can increment the pointer to iterate through it. > >What is the fastest way of doing so in C#? The fixed statement? Mattias -- Mattias Sjögren [C# MVP] mattias @ mvps.org http://www.msjogren.net/dotnet/ | http://www.dotnetinterop.com
[quoted text, click to view]
"ThunderMusic" <NoSpAmdanlatathotmaildotcom@NoSpAm.com> wrote in message news:OhgmWYc5HHA.2108@TK2MSFTNGP02.phx.gbl... > hi, > So, here is the code to get my checksum. If you see something that can be > optimized, let me know. > > thanks > > ThunderMusic > > [Code] > > private const UInt16 MOD_ADLER = 65521; > private unsafe uint GetChecksum(byte[] databytes) > { > UInt32 a = 1, b = 0; > fixed (byte* tmpdata = databytes) > { > byte* data = tmpdata; > int len = databytes.Length; /* Length in bytes */ > while (len > 0) > { > int tlen = len > 5550 ? 5550 : len; > len -= tlen; > do > { > a += *(data++); > b += a; > } while ((--tlen) > 0); > a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); > b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); > } > /* It can be shown that a <= 0x1013a here, so a single subtract > will do. */ > if (a >= MOD_ADLER) > a -= MOD_ADLER; > /* It can be shown that b can reach 0xffef1 here. */ > b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); > if (b >= MOD_ADLER) > b -= MOD_ADLER; > } > return ((b << 16) | a); > } > > [/Code] > You can try to do some partial loop unrolling, this is what a C++ compiler does and what the C# compiler/JIT unfortunately fails to do. UInt32 a = 1, b = 0; fixed (byte* tmpdata = databytes) { byte* data = tmpdata; int len = databytes.Length; /* Length in bytes */ while(len > 0) { int tlen = len > 5550 ? 5550 : len; len -= tlen; for(int c = 0; c < tlen / 4; c++) { a += *(data++); b += a; a += *(data++); b += a; a += *(data++); b += a; a += *(data++); b += a; } a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); for(int c = 0; c < tlen % 4; c++) { a += *(data++); b += a; } a = (a & 0xffff) + (a >> 16) * (65536 - MOD_ADLER); b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); } if (a >= MOD_ADLER) a -= MOD_ADLER; /* It can be shown that b can reach 0xffef1 here. */ b = (b & 0xffff) + (b >> 16) * (65536 - MOD_ADLER); if (b >= MOD_ADLER) b -= MOD_ADLER; } return ((b << 16) | a); Willy.
Don't see what you're looking for? Try a search.
|
June 2003
July 2003
August 2003
September 2003
October 2003
November 2003
December 2003
January 2004
February 2004
March 2004
April 2004
May 2004
June 2004
July 2004
August 2004
September 2004
October 2004
November 2004
December 2004
January 2005
February 2005
March 2005
April 2005
May 2005
June 2005
July 2005
August 2005
September 2005
October 2005
November 2005
December 2005
January 2006
February 2006
March 2006
April 2006
May 2006
June 2006
July 2006
August 2006
September 2006
October 2006
November 2006
December 2006
January 2007
February 2007
March 2007
April 2007
May 2007
June 2007
July 2007
August 2007
September 2007
October 2007
November 2007
December 2007
January 2008
February 2008
March 2008
April 2008
May 2008
June 2008
| home · blog · groups | Questions? Comments? Contact the dn |