Does sending a binary file over TCP need special treatment?

Dear all,

I have a simple multi-threaded TCP client/server application. In it, the
client can send virutally all text-based (ASCII) file correctly and easily.
However, when I try to send a binary file (e.g., a video file, a tar file or
even PDF), the client side almost always get "Broken Pipe" error. The
client is supposed to send blocks of data obtained from a global queue
structure. The queue is never empty.

Since FTP requires the user to specify whether a file is ASCII or binary, I
think suspecting that binary file transfers may need special treatment?

The codes I used at the client side is very simple and nothing special. I
just post it here in case it helps.

Thanks very much in advance.

Regards,

Patrick

while ((glob_qptr->front) != NULL)

{

//The client is supposed to send blocks of data_struct from a global
queue structure.

headofq = (struct data_struct *)malloc(sizeof(struct data_struct));


pthread_mutex_lock(&glob_mx);



headofq = glob_qptr->front;


headofq->thread_number = s_args->thread_number; //Assign the thread
number to each outgoing piece of data.


gettimeofday(&now, NULL); //Obtain the current time at the sender.


headofq->t_timestamp = now.tv_sec*1000 + now.tv_usec/1000;


if (headofq->next != NULL)

{

glob_qptr->front = headofq->next;

}


glob_qptr->queue_len--;


pthread_mutex_unlock(&glob_mx);


n = write(sockfd, headofq, sizeof(struct data_struct));


fprintf(debug_file, "Data sent from thread %ld \n", pthread_self());

if (headofq->next != NULL)

{

glob_qptr->front = headofq->next;

}


headofq = NULL;


free(headofq);

}

On Sat, 8 Apr 2006 16:23:16 +0800, Patrick Lam
<yahoo_mail@nospam_tom.com> wrote:
> Dear all,
>
> I have a simple multi-threaded TCP client/server application. In it, the
> client can send virutally all text-based (ASCII) file correctly and easily.
> However, when I try to send a binary file (e.g., a video file, a tar file or
> even PDF), the client side almost always get "Broken Pipe" error. The
> client is supposed to send blocks of data obtained from a global queue
> structure. The queue is never empty.
>
> Since FTP requires the user to specify whether a file is ASCII or binary, I
> think suspecting that binary file transfers may need special treatment?
>
In FTP, it is ascii files that need special treatment, namely conversion
of end-of-line characters used by different operating systems.


--
Sorry, no fortune this time.

Patrick Lam wrote:
> Since FTP requires the user to specify whether a file is ASCII or binary, I
> think suspecting that binary file transfers may need special treatment?

No.

TCP does not care whether the bytes you send came from an ascii file, a
binary file, or even if the bytes came from any type of file at all.
Bytes is bytes.

The FTP binary/ascii situation is actually backwards from the way you
may be guessing it works.

FTP provides special treatment only when in ascii mode. In binary mode
it leaves things alone.

The special handling in ascii mode is for cross-platform compatibility.
During ascii mode ftp transfer the files are transformed into a
universal format so that encoding peculiarities are not encountered
when sending files between platforms.

/chris

Patrick Lam wrote:
> Dear all,
>
> I have a simple multi-threaded TCP client/server application. In it, the
> client can send virutally all text-based (ASCII) file correctly and easily.
> However, when I try to send a binary file (e.g., a video file, a tar file or
> even PDF), the client side almost always get "Broken Pipe" error. The
> client is supposed to send blocks of data obtained from a global queue
> structure. The queue is never empty.
>
> Since FTP requires the user to specify whether a file is ASCII or binary, I
> think suspecting that binary file transfers may need special treatment?
>
> The codes I used at the client side is very simple and nothing special. I
> just post it here in case it helps.
>
> Thanks very much in advance.
>
> Regards,
>
> Patrick
>

The answer is no.

FTP only needs to know it's a binary file because its default is
to assume it is sending text files, and this invokes some messing
around with line endings (e.g. converting \n to \r\n), which
should not be done with binary files of course. True TCP does not
care about the contents of the byte stream.

It may be that the binary files are bigger and are therefore
uncovering some other problem in the code. One popular mistake is
to assume that read(char[]) will always fill the buffer, whereas
in fact it returns an int giving the number of bytes actually
read. Or to assume that what you send in a single write(char[])
will arrive in a single read(char[]). This is not the case -
written chunks can be split into parts, and successive chunks
written can be merged into single reads. No structure over the
order of the bytes is preserved by TCP.

HTH