Hello,

While trying to get multiple files from an IIS FTP site using the standard
FTP command line program included with Windows XP, the transfer seems to
freeze. When I look at the FTP server log files, I see the following:

#Software: Microsoft Internet Information Services 5.0
#Version: 1.0
#Date: 2004-10-11 00:15:50
#Fields: time cs-method cs-uri-stem sc-status
....
23:43:18 [13]sent
/ftpsnapshots/DC_FdmJournal_FdmFtp/20041011161536/IncidentCategory1_267.sch
226
23:43:18 [13]sent
/ftpsnapshots/DC_FdmJournal_FdmFtp/20041011161536/IncidentCategory2_218.sch
226
23:43:18 [13]sent
/ftpsnapshots/DC_FdmJournal_FdmFtp/20041011161536/IncidentCategory3_141.sch
425
23:43:18 [13]sent
/ftpsnapshots/DC_FdmJournal_FdmFtp/20041011161536/IncidentCategory3_141.sch
426

As you can see, the first x number of files transfer fine, but at some point
I always receive a 425 error and then a 426 error.

From the command line, I'm simply using a set of commands like the
following:
ftp machine.mydomain.com
....
cd nameOfDir
prompt
bin
mget *.sch

What's interesting is that once the FTP client freezes, if I click the
cmd.exe window close box before the FTP server times out, the session acts
like the mget command has been restarted and attempts to download all the
*.sch files again.

Another interesting fact is that the command line FTP client included with
Cygwin does not hang at any point. Unfortunately, a third party application
(Microsoft SQL Server) is attempting to retrieve files over FTP and I
believe it is running into the same problem as I do when using the FTP
client included in windows.

The only info I've been able to dig up on the web is the following short
descriptions of these error types:
FTP STATUS Error code from IIS:
.. 425 Cannot open data connection.
.. 426 Connection closed; transfer aborted.
10048 (sc-win32-status) - net msghelp says: Only one usage of each socket
address (protocol/network address/port) is normally permitted.

Can anyone give me more detail on why this error is occurring or possible
workarounds to look into?

Thanks,
- Eli

[quoted text, click to view]

This is a very important ellipsis. How many files are you trying to
transfer?

[quoted text, click to view]

What's "x"? I would guess that each of these files is pretty quick to
transfer, too.

[quoted text, click to view]

Here's a little explanation of some of the deep areas of TCP.

TCP sockets each have two ends - local and remote - and each end has an
address and a port. TCP sockets must be unique - that is, there must be no
two sockets that share the same combination of local address, local port,
remote address and remote port. To help with this, most connections are
made from a pool of "ephemeral" port numbers, with the port number for one
connection being one or two greater than the port number for the last
connection. The ephemeral port range in Windows' TCP is 1024-5000.

FTP has two modes - the command-line FTP client provided with Windows uses
only one mode, "active". In that mode, data transfers are made from the
server's local IP address, and port 20, to an ephemeral port supplied by the
client (it uses a PORT command to indicate this information). This means
that each PORT command must supply an IP address and port combination that
is not currently in use for an FTP data connection.

If it were that simple, there'd probably be no problem. Unfortunately for
your setup, "in use" doesn't get restricted to simply "data being
transferred". Each TCP socket, to prevent bounced and repeated packets from
causing disconnections of later sockets, goes into a dormant "TIME_WAIT"
state for four minutes after it is closed. During that time, noone can
create a socket with the same local and remote IP address and port
combination.

If everything is working correctly, the situation you describe would most
likely occur when downloading a couple of thousand files within four
minutes. There are other aspects that might cause this to be exascerbated -
for instance, an ephemeral port will not be created at a port number that
already has something listening at it. Also, if someone has applied the
changes described in http://support.microsoft.com/?id=812873, you may have a
limited range of unreserved ephemeral ports.

When you get an error like the one you describe, open up a command window at
the servermachine, and run the "netstat" command, and you should see a whole
slew of entries with "TIME_WAIT". [Of course, you'll have to do that inside
of four minutes, or they'll all disappear]

The solution, if you've managed to read this far, is to increase the range
of ephemeral ports alloted by the client (this may be difficult if the
client is behind a restrictive firewall or NAT), or to use a client that
works in PASV mode.

Alun.
~~~~

Depending on when I do it, there are either one or two connections listed as
FTP when I do a netstat -ao which are in the CLOSE_WAIT state.

There is one port associated with the FTP process ID that is in the
listening state, and another with output like this:

Proto Local Address Foreign Address State PID
UDP verbal:4407 *:* 2320

- Eli

[quoted text, click to view]

Alun, thanks for the detailed reply.

I'm trying to download 253 files. Depending on the client machine and
network that I try to transfer from, 4 to over 50 files are transferred
successfully before the error.

You stated that the windows command line FTP client uses active mode to
transfer files, and perhaps this is the root of the problem. I'm trying to
transfer a SQL Server snapshot (just a set of files) using SQL Server MSDE,
and it uses FTP to transfer files. It appears to be using active FTP mode,
but I suppose asking about using passive mode is a question for the SQL
Server group.

When I do a netstat -ao on the server, I do see a number of ports in the
TIME_WAIT state, but only a dozen or so (see below).

You mention that one of the solutions would be to have the client "increase
the range of ephemeral ports alloted by the client (this may be difficult if
the client is behind a restrictive firewall or NAT)". How would one go
about doing this?

Thanks again,
Eli

netstat -io output:
Active Connections

Proto Local Address Foreign Address State PID
TCP RAINIER:ftp RAINIER.subnet.mycompany.com:0 LISTENING
1424
TCP RAINIER:http RAINIER.subnet.mycompany.com:0 LISTENING
1784
TCP RAINIER:epmap RAINIER.subnet.mycompany.com:0 LISTENING
756
TCP RAINIER:microsoft-ds RAINIER.subnet.mycompany.com:0 LISTENING
4
TCP RAINIER:1025 RAINIER.subnet.mycompany.com:0 LISTENING
952
TCP RAINIER:1029 RAINIER.subnet.mycompany.com:0 LISTENING
576
TCP RAINIER:1030 RAINIER.subnet.mycompany.com:0 LISTENING
1104
TCP RAINIER:1031 RAINIER.subnet.mycompany.com:0 LISTENING
1424
TCP RAINIER:ms-sql-s RAINIER.subnet.mycompany.com:0 LISTENING
1444
TCP RAINIER:pptp RAINIER.subnet.mycompany.com:0 LISTENING
4
TCP RAINIER:3001 RAINIER.subnet.mycompany.com:0 LISTENING
1572
TCP RAINIER:3389 RAINIER.subnet.mycompany.com:0 LISTENING
812
TCP RAINIER:5800 RAINIER.subnet.mycompany.com:0 LISTENING
1652
TCP RAINIER:5900 RAINIER.subnet.mycompany.com:0 LISTENING
1652
TCP RAINIER:8181 RAINIER.subnet.mycompany.com:0 LISTENING
1784
TCP RAINIER:8282 RAINIER.subnet.mycompany.com:0 LISTENING
1424
TCP RAINIER:3004 RAINIER.subnet.mycompany.com:0 LISTENING
1072
TCP RAINIER:ftp-data 67.42.144.125:4343 TIME_WAIT 0
TCP RAINIER:ftp-data 67.42.144.125:4345 TIME_WAIT 0
TCP RAINIER:ftp-data 67.42.144.125:4346 TIME_WAIT 0
TCP RAINIER:ftp-data 67.42.144.125:4347 TIME_WAIT 0
TCP RAINIER:ftp-data 67.42.144.125:4360 TIME_WAIT 0
TCP RAINIER:ftp-data 67.42.144.125:4364 TIME_WAIT 0
TCP RAINIER:ftp-data 67.42.144.125:4365 TIME_WAIT 0
TCP RAINIER:ftp-data 67.42.144.125:4366 TIME_WAIT 0
TCP RAINIER:ftp-data 67.42.144.125:4367 TIME_WAIT 0
TCP RAINIER:ftp-data 67.42.144.125:4371 TIME_WAIT 0
TCP RAINIER:ftp-data 67.42.144.125:4373 FIN_WAIT_1 1424
TCP RAINIER:ftp 67.42.144.125:49372 ESTABLISHED 1424
TCP RAINIER:netbios-ssn RAINIER.subnet.mycompany.com:0 LISTENING
4
TCP RAINIER:ms-sql-s 67.42.144.125:15286 ESTABLISHED 1444
TCP RAINIER:ms-sql-s rdsl149.other.place.net:32006 ESTABLISHED
1444
TCP RAINIER:ms-sql-s rdsl149.other.place.net:32007 ESTABLISHED
1444
TCP RAINIER:ms-sql-s rdsl149.other.place.net:34515 ESTABLISHED
1444
TCP RAINIER:ms-sql-s rdsl149.other.place.net:36108 ESTABLISHED
1444
TCP RAINIER:3389 67.42.144.125:49309 ESTABLISHED 812
TCP RAINIER:3389 68-184-88-115.cpel0.ga.charter.com:15529
ESTABLISHED 812
UDP RAINIER:microsoft-ds *:* 4
UDP RAINIER:isakmp *:* 576
UDP RAINIER:1026 *:* 884
UDP RAINIER:1027 *:* 884
UDP RAINIER:1028 *:* 884
UDP RAINIER:ms-sql-m *:* 1444
UDP RAINIER:l2tp *:* 4
UDP RAINIER:3456 *:* 1424
UDP RAINIER:4500 *:* 576
UDP RAINIER:ntp *:* 952
UDP RAINIER:3009 *:* 952
UDP RAINIER:3010 *:* 952
UDP RAINIER:3456 *:* 1424
UDP RAINIER:ntp *:* 952
UDP RAINIER:netbios-ns *:* 4
UDP RAINIER:netbios-dgm *:* 4

[quoted text, click to view]