I need to build an infrastructure that moves large files around the
Internet. The scenario is many clients all submitting information to a
single server. The data in question is confidential, so encryption and
signatures are important. Customers currently write data to magnetic tape
and have that tape delivered via courrier.
The files in question will range from 10 megs to about 50 gigs, and we may
see (on a very rare occasion) as many as 10,000 files in a single day. More
normal load will be 1000 or fewer files per day. Bandwith isn't an issue,
and there are lots of machines available in a web farm.
As each client is likley to want to develop their own client side
implementation (so they can seamlessly integrate into their existing
business processes), it's important that the protocols used be as standard
as possible. Everyone's first impression was FTP until they realized it
doesn't meet the security constraints.
I've been looking at MTOM (via WSE 3.0) and trying to decide if it's going
to do the trick. So far, it doesn't look as if MTOM can deal well with large
files, and everything I've seen really requires a chunking approach. While I
can live with requiring chunking, I would rather have a solution that didn't
require this.
I am, I must confess, disappointed that MTOM doesn't seem to cope at all
with large files. From what I can see in WSE, the entire MTOM image is
stored in memory until the whole thing is received, then is written out to
disk. I need something that is a bit more of a streaming approach. Am I
doing something wrong? I'm open to suggestions...
--
Chris Mullins