Unable to download deep files and directories recursively with mget -R command...

Hello.

I just installed and set up vsftpd v2.0.1 (apt-get) on my Debian v3.1
(Kernel v2.6.8) box. I seem to have problems using ncftp client (yes, I
am old school text user) in downloading a folder/directory with many
deep subfolders/subdirectories. I use mget -R foldername to download
everything in that folder as requested. It worked fine with wu-ftpd
server I had before, but needed to dump it.

What am I missing? I tried ncftp from both a PowerBook G4 (Mac OS X
10.2.8) and localhost of the Debian box. Both only got the files and
folders/directories in the root.

Thank you in advance.
--
"I look at an ant and I see myself: a native South African, endowed by nature with a
strength much greater than my size so I might cope with the weight of a racism that crushes
my spirit." --Miriam Makeba
/\___/\
/ /\ /\ \ Ant @ The Ant Farm: http://antfarm.ma.cx
| |o o| | Ant's Quality Foraged Links: http://aqfl.net
\ _ / Please remove ANT if replying by e-mail.
( )

In comp.os.linux.networking (E-Mail Removed) wrote:
> Hello.

> I just installed and set up vsftpd v2.0.1 (apt-get) on my Debian v3.1
> (Kernel v2.6.8) box. I seem to have problems using ncftp client (yes, I
> am old school text user) in downloading a folder/directory with many
> deep subfolders/subdirectories. I use mget -R foldername to download
> everything in that folder as requested. It worked fine with wu-ftpd
> server I had before, but needed to dump it.

> What am I missing? I tried ncftp from both a PowerBook G4 (Mac OS X
> 10.2.8) and localhost of the Debian box. Both only got the files and
> folders/directories in the root.

> Thank you in advance.

I discovered debug and enabled it in ncftp:

ncftp /home/ant > mget -R test
> mget -R test

Cmd: MLST test
500: Unknown command.
Cmd: PWD
257: "/home/ant"
Cmd: CWD test
250: Directory successfully changed.
Cmd: CWD /home/ant
250: Directory successfully changed.
Cmd: MDTM test.tar
550: Could not get file modification time.
Cmd: TYPE I
200: Switching to Binary mode.
Cmd: PASV
227: Entering Passive Mode (127,0,0,1,233,118)
Cmd: RETR test.tar
550: Failed to open file.
Cmd: PWD
257: "/home/ant"
Cmd: CWD test
250: Directory successfully changed.
Cmd: TYPE A
200: Switching to ASCII mode.
Cmd: PASV
227: Entering Passive Mode (127,0,0,1,121,228)
Cmd: LIST -Ra
150: Here comes the directory listing.
226: Directory send OK.
Cmd: CWD /home/ant
250: Directory successfully changed.
Cmd: MDTM test/LastTVshows.txt
213: 20050108073325
Cmd: PASV
227: Entering Passive Mode (127,0,0,1,95,41)
Cmd: RETR test/LastTVshows.txt
150: Opening BINARY mode data connection for test/LastTVshows.txt (108 bytes).
test/LastTVshows.txt: ETA: 0:00 108.00/108.00 B 1.00 MB/s 226: File send OK.
test/LastTVshows.txt: 108.00 B 460.56 kB/s
Cmd: MDTM test/SmallSquirrel.gif
213: 20030628041208
Cmd: TYPE I
200: Switching to Binary mode.
Cmd: PASV
227: Entering Passive Mode (127,0,0,1,65,9)
Cmd: RETR test/Squirrel.gif
150: Opening BINARY mode data connection for test/Squirrel.gif (1844 bytes).
test/SmallSquirrel.gif: ETA: 0:00 1.80/ 1.80 kB 20.69 MB/s 226: File send OK.
test/SmallSquirrel.gif: 1.80 kB 8.41 MB/s
Cmd: MDTM test/VisiBoneHTML.gif
213: 20020225090427
Cmd: PASV
227: Entering Passive Mode (127,0,0,1,223,30)
Cmd: RETR test/VisiBoneHTML.gif
150: Opening BINARY mode data connection for test/VisiBoneHTML.gif (622402 bytes).
test/VisiBoneHTML.gif: ETA: 0:00 32.00/607.81 kB 93.28 MB/s 226: File send OK.
test/VisiBoneHTML.gif: 607.81 kB 95.40 MB/s
Cmd: MDTM test/ant.gif
213: 20050101163408
Cmd: PASV
227: Entering Passive Mode (127,0,0,1,147,133)
Cmd: RETR test/foo.gif
150: Opening BINARY mode data connection for test/foo.gif (585 bytes).
test/foo.gif: ETA: 0:00 585.00/585.00 B 3.49 MB/s 226: File send OK.
test/foo.gif: 585.00 B 1.21 MB/s
Cmd: MDTM test/readme.txt
213: 20050126061143
Cmd: TYPE A
200: Switching to ASCII mode.
Cmd: PASV
227: Entering Passive Mode (127,0,0,1,239,48)
Cmd: RETR test/readme.txt
150: Opening BINARY mode data connection for test/computers.txt (3641 bytes).
test/readme.txt: ETA: 0:00 3.56/ 3.56 kB 27.56 MB/s 226: File send OK.
test/readme.txt: 3.56 kB 15.16 MB/s
Cmd: MDTM test/movies.txt
213: 20050202045100
Cmd: PASV
227: Entering Passive Mode (127,0,0,1,188,231)
Cmd: RETR test/movies.txt
150: Opening BINARY mode data connection for test/movies.txt (951 bytes).
test/movies.txt: ETA: 0:00 951.00/951.00 B 10.19 MB/s 226: File send OK.
test/movies.txt: 951.00 B 4.63 MB/s
Cmd: MDTM test/signature.txt
213: 20050218092832
Cmd: PASV
227: Entering Passive Mode (127,0,0,1,117,211)
Cmd: RETR test/signature.txt
150: Opening BINARY mode data connection for test/signature.txt (402 bytes).
test/signature.txt: ETA: 0:00 402.00/402.00 B 4.56 MB/s 226: File send OK.
test/signature.txt: 402.00 B 2.05 MB/s

Any ideas why I am getting error for test.tar?
--
"I look at an ant and I see myself: a native South African, endowed by nature with a
strength much greater than my size so I might cope with the weight of a racism that crushes
my spirit." --Miriam Makeba
/\___/\
/ /\ /\ \ Ant @ The Ant Farm: http://antfarm.ma.cx
| |o o| | Ant's Quality Foraged Links: http://aqfl.net
\ _ / Please remove ANT if replying by e-mail.
( )

In comp.os.linux.setup (E-Mail Removed) wrote:
> ncftp /home/ant > mget -R test
>
> Cmd: MDTM test.tar
> 550: Could not get file modification time.
> Cmd: TYPE I
> 200: Switching to Binary mode.
> Cmd: PASV
> 227: Entering Passive Mode (127,0,0,1,233,118)
> Cmd: RETR test.tar
> 550: Failed to open file.

> Any ideas why I am getting error for test.tar?

Maybe you have no read permissions for the file.

Yours,
Laurenz Albe

(E-Mail Removed) wrote:

> In comp.os.linux.networking (E-Mail Removed) wrote:
>> Hello.
>
>> I just installed and set up vsftpd v2.0.1 (apt-get) on my Debian v3.1
>> (Kernel v2.6.8) box. I seem to have problems using ncftp client (yes, I
>> am old school text user) in downloading a folder/directory with many
>> deep subfolders/subdirectories. I use mget -R foldername to download
>> everything in that folder as requested. It worked fine with wu-ftpd
>> server I had before, but needed to dump it.
>
>> What am I missing? I tried ncftp from both a PowerBook G4 (Mac OS X
>> 10.2.8) and localhost of the Debian box. Both only got the files and
>> folders/directories in the root.
>
>> Thank you in advance.
>
> I discovered debug and enabled it in ncftp:
>
> ncftp /home/ant > mget -R test
>> mget -R test
>
> Cmd: MLST test
> 500: Unknown command.
> Cmd: PWD
> 257: "/home/ant"
> Cmd: CWD test
> 250: Directory successfully changed.
> Cmd: CWD /home/ant
> 250: Directory successfully changed.
> Cmd: MDTM test.tar
> 550: Could not get file modification time.
> Cmd: TYPE I
> 200: Switching to Binary mode.
> Cmd: PASV
> 227: Entering Passive Mode (127,0,0,1,233,118)
> Cmd: RETR test.tar
> 550: Failed to open file.
> Cmd: PWD
> 257: "/home/ant"
> Cmd: CWD test
> 250: Directory successfully changed.
> Cmd: TYPE A
> 200: Switching to ASCII mode.
> Cmd: PASV
> 227: Entering Passive Mode (127,0,0,1,121,228)
> Cmd: LIST -Ra
> 150: Here comes the directory listing.
> 226: Directory send OK.
> Cmd: CWD /home/ant
> 250: Directory successfully changed.
> Cmd: MDTM test/LastTVshows.txt
> 213: 20050108073325
> Cmd: PASV
> 227: Entering Passive Mode (127,0,0,1,95,41)
> Cmd: RETR test/LastTVshows.txt
> 150: Opening BINARY mode data connection for test/LastTVshows.txt (108
> bytes).
> test/LastTVshows.txt: ETA: 0:00 108.00/108.00 B 1.00
> MB/s 226: File send OK.
> test/LastTVshows.txt: 108.00 B 460.56
> kB/s Cmd: MDTM test/SmallSquirrel.gif
> 213: 20030628041208
> Cmd: TYPE I
> 200: Switching to Binary mode.
> Cmd: PASV
> 227: Entering Passive Mode (127,0,0,1,65,9)
> Cmd: RETR test/Squirrel.gif
> 150: Opening BINARY mode data connection for test/Squirrel.gif (1844
> bytes).
> test/SmallSquirrel.gif: ETA: 0:00 1.80/ 1.80 kB 20.69
> MB/s 226: File send OK.
> test/SmallSquirrel.gif: 1.80 kB 8.41
> MB/s Cmd: MDTM test/VisiBoneHTML.gif
> 213: 20020225090427
> Cmd: PASV
> 227: Entering Passive Mode (127,0,0,1,223,30)
> Cmd: RETR test/VisiBoneHTML.gif
> 150: Opening BINARY mode data connection for test/VisiBoneHTML.gif (622402
> bytes).
> test/VisiBoneHTML.gif: ETA: 0:00 32.00/607.81 kB 93.28
> MB/s 226: File send OK.
> test/VisiBoneHTML.gif: 607.81 kB 95.40
> MB/s Cmd: MDTM test/ant.gif
> 213: 20050101163408
> Cmd: PASV
> 227: Entering Passive Mode (127,0,0,1,147,133)
> Cmd: RETR test/foo.gif
> 150: Opening BINARY mode data connection for test/foo.gif (585 bytes).
> test/foo.gif: ETA: 0:00 585.00/585.00 B 3.49
> MB/s 226: File send OK.
> test/foo.gif: 585.00 B 1.21
> MB/s Cmd: MDTM test/readme.txt
> 213: 20050126061143
> Cmd: TYPE A
> 200: Switching to ASCII mode.
> Cmd: PASV
> 227: Entering Passive Mode (127,0,0,1,239,48)
> Cmd: RETR test/readme.txt
> 150: Opening BINARY mode data connection for test/computers.txt (3641
> bytes).
> test/readme.txt: ETA: 0:00 3.56/ 3.56 kB 27.56 MB/s
> 226: File send OK.
> test/readme.txt: 3.56 kB 15.16 MB/s
> Cmd: MDTM test/movies.txt
> 213: 20050202045100
> Cmd: PASV
> 227: Entering Passive Mode (127,0,0,1,188,231)
> Cmd: RETR test/movies.txt
> 150: Opening BINARY mode data connection for test/movies.txt (951 bytes).
> test/movies.txt: ETA: 0:00 951.00/951.00 B 10.19
> MB/s 226: File send OK.
> test/movies.txt: 951.00 B 4.63
> MB/s Cmd: MDTM test/signature.txt
> 213: 20050218092832
> Cmd: PASV
> 227: Entering Passive Mode (127,0,0,1,117,211)
> Cmd: RETR test/signature.txt
> 150: Opening BINARY mode data connection for test/signature.txt (402
> bytes).
> test/signature.txt: ETA: 0:00 402.00/402.00 B 4.56
> MB/s 226: File send OK.
> test/signature.txt: 402.00 B 2.05
> MB/s
>
> Any ideas why I am getting error for test.tar?

Is it downloading the contents of test.tar? There is something about this
in man ncftp. Personally, I don't seem to be able to get any recursive
command to go deeper than one directory. The man page doesn't specifically
say that you can go deeper; it just says that you can retrieve the
conttents of an entire directory with the -R switch.

I still like ncftp the best, and I am a KDE user.

Doug.
--
ICQ Number 178748389. Registered Linux User No. 277548.
Love can heal the world, and forgiveness is the catalyst to make it happen.
- John Gray, author of Men are from Mars, women are from Venus.

On Fri, 18 Feb 2005 04:05:04 -0600, ANTant thoughtfully wrote:

> Hello.
>
> I just installed and set up vsftpd v2.0.1 (apt-get) on my Debian v3.1
> (Kernel v2.6.8) box. I seem to have problems using ncftp client (yes, I am
> old school text user) in downloading a folder/directory with many deep
> subfolders/subdirectories. I use mget -R foldername to download everything
> in that folder as requested. It worked fine with wu-ftpd server I had
> before, but needed to dump it.
>
> What am I missing? I tried ncftp from both a PowerBook G4 (Mac OS X
> 10.2.8) and localhost of the Debian box. Both only got the files and
> folders/directories in the root.
>
> Thank you in advance.

I suggest switching to wget instead of mget.

Man wget.

> > ncftp /home/ant > mget -R test
> >
> > Cmd: MDTM test.tar
> > 550: Could not get file modification time.
> > Cmd: TYPE I
> > 200: Switching to Binary mode.
> > Cmd: PASV
> > 227: Entering Passive Mode (127,0,0,1,233,118)
> > Cmd: RETR test.tar
> > 550: Failed to open file.

> > Any ideas why I am getting error for test.tar?

> Maybe you have no read permissions for the file.

I think test.tar is from ncftp or something for download compression.
wu-ftpd did this, but I don't know why vsftpd is having this problem?

I do not have this .tar file anywhere so it is a temporary file during
download.
--
"I look at an ant and I see myself: a native South African, endowed by nature with a
strength much greater than my size so I might cope with the weight of a racism that crushes
my spirit." --Miriam Makeba
/\___/\
/ /\ /\ \ Ant @ The Ant Farm: http://antfarm.ma.cx
| |o o| | Ant's Quality Foraged Links: http://aqfl.net
\ _ / Please remove ANT if replying by e-mail.
( )

> >> I just installed and set up vsftpd v2.0.1 (apt-get) on my Debian v3.1
> >> (Kernel v2.6.8) box. I seem to have problems using ncftp client (yes, I
> >> am old school text user) in downloading a folder/directory with many
> >> deep subfolders/subdirectories. I use mget -R foldername to download
> >> everything in that folder as requested. It worked fine with wu-ftpd
> >> server I had before, but needed to dump it.
> >
> >> What am I missing? I tried ncftp from both a PowerBook G4 (Mac OS X
> >> 10.2.8) and localhost of the Debian box. Both only got the files and
> >> folders/directories in the root.
> >
> >> Thank you in advance.
> >
> > I discovered debug and enabled it in ncftp:
> >
> > ncftp /home/ant > mget -R test
> >> mget -R test
> >
> > Cmd: MLST test
> > 500: Unknown command.
> > Cmd: PWD
> > 257: "/home/ant"
> > Cmd: CWD test
> > 250: Directory successfully changed.
> > Cmd: CWD /home/ant
> > 250: Directory successfully changed.
> > Cmd: MDTM test.tar
> > 550: Could not get file modification time.
> > Cmd: TYPE I
> > 200: Switching to Binary mode.
> > Cmd: PASV
> > 227: Entering Passive Mode (127,0,0,1,233,118)
> > Cmd: RETR test.tar
> > 550: Failed to open file.
> > Cmd: PWD
> > 257: "/home/ant"
> > Cmd: CWD test
> > 250: Directory successfully changed.
> > Cmd: TYPE A
> > 200: Switching to ASCII mode.
> > Cmd: PASV
> > 227: Entering Passive Mode (127,0,0,1,121,228)
> > Cmd: LIST -Ra
> > 150: Here comes the directory listing.
> > 226: Directory send OK.
> > Cmd: CWD /home/ant
> > 250: Directory successfully changed.
> > Cmd: MDTM test/LastTVshows.txt
> > 213: 20050108073325
> > Cmd: PASV
> > 227: Entering Passive Mode (127,0,0,1,95,41)
> > Cmd: RETR test/LastTVshows.txt
> > 150: Opening BINARY mode data connection for test/LastTVshows.txt (108
> > bytes).
> > test/LastTVshows.txt: ETA: 0:00 108.00/108.00 B 1.00
> > MB/s 226: File send OK.
> > test/LastTVshows.txt: 108.00 B 460.56
> > kB/s Cmd: MDTM test/SmallSquirrel.gif
> > 213: 20030628041208
> > Cmd: TYPE I
> > 200: Switching to Binary mode.
> > Cmd: PASV
> > 227: Entering Passive Mode (127,0,0,1,65,9)
> > Cmd: RETR test/Squirrel.gif
> > 150: Opening BINARY mode data connection for test/Squirrel.gif (1844
> > bytes).
> > test/SmallSquirrel.gif: ETA: 0:00 1.80/ 1.80 kB 20.69
> > MB/s 226: File send OK.
> > test/SmallSquirrel.gif: 1.80 kB 8.41
> > MB/s Cmd: MDTM test/VisiBoneHTML.gif
> > 213: 20020225090427
> > Cmd: PASV
> > 227: Entering Passive Mode (127,0,0,1,223,30)
> > Cmd: RETR test/VisiBoneHTML.gif
> > 150: Opening BINARY mode data connection for test/VisiBoneHTML.gif (622402
> > bytes).
> > test/VisiBoneHTML.gif: ETA: 0:00 32.00/607.81 kB 93.28
> > MB/s 226: File send OK.
> > test/VisiBoneHTML.gif: 607.81 kB 95.40
> > MB/s Cmd: MDTM test/ant.gif
> > 213: 20050101163408
> > Cmd: PASV
> > 227: Entering Passive Mode (127,0,0,1,147,133)
> > Cmd: RETR test/foo.gif
> > 150: Opening BINARY mode data connection for test/foo.gif (585 bytes).
> > test/foo.gif: ETA: 0:00 585.00/585.00 B 3.49
> > MB/s 226: File send OK.
> > test/foo.gif: 585.00 B 1.21
> > MB/s Cmd: MDTM test/readme.txt
> > 213: 20050126061143
> > Cmd: TYPE A
> > 200: Switching to ASCII mode.
> > Cmd: PASV
> > 227: Entering Passive Mode (127,0,0,1,239,48)
> > Cmd: RETR test/readme.txt
> > 150: Opening BINARY mode data connection for test/computers.txt (3641
> > bytes).
> > test/readme.txt: ETA: 0:00 3.56/ 3.56 kB 27.56 MB/s
> > 226: File send OK.
> > test/readme.txt: 3.56 kB 15.16 MB/s
> > Cmd: MDTM test/movies.txt
> > 213: 20050202045100
> > Cmd: PASV
> > 227: Entering Passive Mode (127,0,0,1,188,231)
> > Cmd: RETR test/movies.txt
> > 150: Opening BINARY mode data connection for test/movies.txt (951 bytes).
> > test/movies.txt: ETA: 0:00 951.00/951.00 B 10.19
> > MB/s 226: File send OK.
> > test/movies.txt: 951.00 B 4.63
> > MB/s Cmd: MDTM test/signature.txt
> > 213: 20050218092832
> > Cmd: PASV
> > 227: Entering Passive Mode (127,0,0,1,117,211)
> > Cmd: RETR test/signature.txt
> > 150: Opening BINARY mode data connection for test/signature.txt (402
> > bytes).
> > test/signature.txt: ETA: 0:00 402.00/402.00 B 4.56
> > MB/s 226: File send OK.
> > test/signature.txt: 402.00 B 2.05
> > MB/s
> >
> > Any ideas why I am getting error for test.tar?

> Is it downloading the contents of test.tar? There is something about this
> in man ncftp. Personally, I don't seem to be able to get any recursive
> command to go deeper than one directory. The man page doesn't specifically
> say that you can go deeper; it just says that you can retrieve the
> conttents of an entire directory with the -R switch.

It only downloaded the files and main subdirectories in the root
directory like shown in the debug log. You know what's weird, I had NO
problems with wu-ftpd. I am thinking it is on vsftpd side. I cannot
figure out where the option to tweak this. I do NOT see anything.


> I still like ncftp the best, and I am a KDE user.

I agree. Text mode ftp client.
--
"I look at an ant and I see myself: a native South African, endowed by nature with a
strength much greater than my size so I might cope with the weight of a racism that crushes
my spirit." --Miriam Makeba
/\___/\
/ /\ /\ \ Ant @ The Ant Farm: http://antfarm.ma.cx
| |o o| | Ant's Quality Foraged Links: http://aqfl.net
\ _ / Please remove ANT if replying by e-mail.
( )

> > Hello.
> >
> > I just installed and set up vsftpd v2.0.1 (apt-get) on my Debian v3.1
> > (Kernel v2.6.8) box. I seem to have problems using ncftp client (yes, I am
> > old school text user) in downloading a folder/directory with many deep
> > subfolders/subdirectories. I use mget -R foldername to download everything
> > in that folder as requested. It worked fine with wu-ftpd server I had
> > before, but needed to dump it.
> >
> > What am I missing? I tried ncftp from both a PowerBook G4 (Mac OS X
> > 10.2.8) and localhost of the Debian box. Both only got the files and
> > folders/directories in the root.
> >
> > Thank you in advance.

> I suggest switching to wget instead of mget.

> Man wget.

mget is a command in ncftp client, not a command in shell. I do not
want to use wget command.
--
"I look at an ant and I see myself: a native South African, endowed by nature with a
strength much greater than my size so I might cope with the weight of a racism that crushes
my spirit." --Miriam Makeba
/\___/\
/ /\ /\ \ Ant @ The Ant Farm: http://antfarm.ma.cx
| |o o| | Ant's Quality Foraged Links: http://aqfl.net
\ _ / Please remove ANT if replying by e-mail.
( )

Finally, I fixed it and boy do I feel like an idiot. I don't know how in the
world I miss this line in /etc/vsftpd.conf file:

# You may activate the "-R" option to the builtin ls. This is disabled by
# default to avoid remote users being able to cause excessive I/O on large
# sites. However, some broken FTP clients such as "ncftp" and "mirror" assume
# the presence of the "-R" option, so there is a strong case for enabling it.
#ls_recurse_enable=YES

I removed # from #ls_recurse_enable=YES line, restarted vsftpd, and retried.
It works now (on localhost). I will try remotely later. I can't believe I
pondered on this for days. [sighs]

Thanks to everyone who tried to help.


> Hello.

> I just installed and set up vsftpd v2.0.1 (apt-get) on my Debian v3.1
> (Kernel v2.6.8) box. I seem to have problems using ncftp client (yes, I
> am old school text user) in downloading a folder/directory with many
> deep subfolders/subdirectories. I use mget -R foldername to download
> everything in that folder as requested. It worked fine with wu-ftpd
> server I had before, but needed to dump it.

> What am I missing? I tried ncftp from both a PowerBook G4 (Mac OS X
> 10.2.8) and localhost of the Debian box. Both only got the files and
> folders/directories in the root.

> Thank you in advance.
--
"Size isn't everything. The whale is endangered, while the ant continues to do just fine." --Bill Vaughan
/\___/\
/ /\ /\ \ Ant @ The Ant Farm: http://antfarm.ma.cx
| |o o| | Ant's Quality Foraged Links: http://aqfl.net
\ _ / Please remove ANT if replying by e-mail.
( )

In comp.os.linux.setup (E-Mail Removed) wrote:
>>> ncftp /home/ant > mget -R test
>>>
>>> Cmd: MDTM test.tar
>>> 550: Could not get file modification time.
>>> Cmd: TYPE I
>>> 200: Switching to Binary mode.
>>> Cmd: PASV
>>> 227: Entering Passive Mode (127,0,0,1,233,118)
>>> Cmd: RETR test.tar
>>> 550: Failed to open file.
>>> Any ideas why I am getting error for test.tar?
>
>> Maybe you have no read permissions for the file.
>
> I think test.tar is from ncftp or something for download compression.
> wu-ftpd did this, but I don't know why vsftpd is having this problem?
>
> I do not have this .tar file anywhere so it is a temporary file during
> download.

I do not understand this, but it seems unimportant now as you said you have
solved your problem.

The fact remains that there is a test.tar on the server that you cannot
read, but it seems that you are blissfully unaware of it and it does not
concern you :^)

Cheers,
Laurenz Albe