Bob Altman wrote:
> I am processing a very large (20+ GB) file. As part of my processing
> I need to compute a CRC or hash value for the file. To improve the
> efficiency of my algorithm I'm looking for a way to do this without
> having to read the file twice (once for my processing and again to
> compute the hash).
> My processing involves reading the file one 64K byte chunk at a time
> into a byte array. As I'm reading the file I'd like to generate a
> running CRC or hash code that I can update each time I read another
> bunch of bytes from the file. Is there a way to do this without
> writing my own CRC32 implementation?

wrote in message news:eBkNHshcGHA.1264@TK2MSFTNGP05.phx.gbl...
> Bob Altman wrote:
>> I am processing a very large (20+ GB) file. As part of my processing
>> I need to compute a CRC or hash value for the file. To improve the
>> efficiency of my algorithm I'm looking for a way to do this without
>> having to read the file twice (once for my processing and again to
>> compute the hash).
>> My processing involves reading the file one 64K byte chunk at a time
>> into a byte array. As I'm reading the file I'd like to generate a
>> running CRC or hash code that I can update each time I read another
>> bunch of bytes from the file. Is there a way to do this without
>> writing my own CRC32 implementation?
>
> Yes.
>
> While the .NET framework doesn't include a CRC32 implementation, there are
> many articles/libraries available that do provide just one you're looking
> for.
>
> Here's one:
>
> http://www.codeproject.com/csharp/crc32_dotnet.asp?df=100&forumid=4026&exp=0&select=913712
>
> that exposes CRC32 as a class derived from
> System.Security.Cryptography.HashAlgorithm. You can easily hash an
> artibrarily large object in blocks as you read it in and process it.
>
> If you don't like that one, do a few network searches - there are
> literally dozens of implementations out there.
>
> -cd
>
>

Bob Altman wrote:
> Thanks for the instant answer! I guess my question should have been
> phrased a little more generally: Is there a way to use any of the
> existing
> hash algorithms to compute a hash for an arbitrarily large file in blocks
> as I read and process it?

Bob Altman wrote:
> Thanks for the instant answer! I guess my question should have been phrased
> a little more generally: Is there a way to use any of the existing hash
> algorithms to compute a hash for an arbitrarily large file in blocks as I
> read and process it?
>
> "Carl Daniel [VC++ MVP]" <cpdaniel_remove_this_and_nospam@mvps.org.nospam>
> wrote in message news:eBkNHshcGHA.1264@TK2MSFTNGP05.phx.gbl...
>> Bob Altman wrote:
>>> I am processing a very large (20+ GB) file. As part of my processing
>>> I need to compute a CRC or hash value for the file. To improve the
>>> efficiency of my algorithm I'm looking for a way to do this without
>>> having to read the file twice (once for my processing and again to
>>> compute the hash).
>>> My processing involves reading the file one 64K byte chunk at a time
>>> into a byte array. As I'm reading the file I'd like to generate a
>>> running CRC or hash code that I can update each time I read another
>>> bunch of bytes from the file. Is there a way to do this without
>>> writing my own CRC32 implementation?
>> Yes.
>>
>> While the .NET framework doesn't include a CRC32 implementation, there are
>> many articles/libraries available that do provide just one you're looking
>> for.
>>
>> Here's one:
>>
>> http://www.codeproject.com/csharp/crc32_dotnet.asp?df=100&forumid=4026&exp=0&select=913712
>>
>> that exposes CRC32 as a class derived from
>> System.Security.Cryptography.HashAlgorithm. You can easily hash an
>> artibrarily large object in blocks as you read it in and process it.
>>
>> If you don't like that one, do a few network searches - there are
>> literally dozens of implementations out there.
>>
>> -cd
>>
>>
>