receiving mjpegs over tcp and multicasting decoded rgb images slowsnetwork down?

hello dear network experts,

i am currently having trouble implementing some small application that
receives an mjpeg stream via tcp/ip and parallely multicasts the decoded
jpegs as rgb images via udp into the local network.

the application starts a dedicated reception thread for the mjpeg stream
and consequently receives everything coming. once a full jpeg frame is
detected, it is decoded and put into a queue that is secured via mutexes
and semaphores. i am able to receive and process 30fps constantly (not
more, because the sending camera is not able to send any faster). the
jpegs are each ~90kb large.

parallely, my main application thread waits for any incoming images from
the camera (which by then is already decoded into a plain rgb image with
size 640x480x3 channels = 921600 bytes), splits it into small fragments
and sends all fragments (including a small preceding descriptor) over an
udp multicast socket.

it seems this slows down the tcp/ip reception.

i am absolutely sure to use a 1 gigabit lan. all cables, connections and
devices are checked and verified. the tcp stream of the camera image,
basically comes in as ~4000 byte packages. the udp datagrams i send are
50000 bytes large in maximum. i also tested smaller datagrams (down to
100 bytes) but it always comes down to a framrate drop down to ~23 frames
per second.

hence, all i am sending over my network is receiving jpegs (appox 90000
bytes big each) via tcp/ip and sending the decoded jpegs (as plain rgbs
with 921600 bytes each) via udp multicast) and the i already encounter
limits on my ethernet.

don't get me wrong, i also thought, that this might be due to cpu
processing. i have a dualcore, the whole program only uses 16% cpu from
either one, not both. and the main slow down is measurable during the recv
()-call of the tcp-/ip connection, not the processing of the data! the
more i send over udp multicast, the longer the recv()-call takes, and if
i send everything i receive, i drop down to 23 fps.

i believe i handle the multicast sending wrong or am missing something in
the tcp connection implementation. could you please give me a hint? i
reall don't have a clue where to keep looking!

best
mikey79

On Jan 15, 11:08*am, mikey79 <mike...@gmx.net> wrote:

> parallely, my main application thread waits for any incoming images from
> the camera (which by then is already decoded into a plain rgb image with
> size 640x480x3 channels = 921600 bytes), splits it into small fragments
> and sends all fragments (including a small preceding descriptor) over an
> udp multicast socket.
>
> it seems this slows down the tcp/ip reception.

It sounds like you're not pacing your transmitted datagrams.

> i am absolutely sure to use a 1 gigabit lan. all cables, connections and
> devices are checked and verified. the tcp stream of the camera image,
> basically comes in as ~4000 byte packages. the udp datagrams i send are
> 50000 bytes large in maximum. i also tested smaller datagrams (down to
> 100 bytes) but it always comes down to a framrate drop down to ~23 frames
> per second.

1KB datagrams will probably work best. But you *must* pace them. You
can just send 50 of them in a tight loop.

DS

> It sounds like you're not pacing your transmitted datagrams.

I googled a lot today and when searching for 'udp' and 'pacing' there
come up a lot of messages and answers from you. Thank you very much for
taking your time for my problem as well; unfortunately I don't understand
the pacing scheme completely.

Today, I changed my app a bit, in that I tested two different variants:

1.) I created a buffer of size 640*480*3 (one exemplary image) and sent
it via a udp-socket and sendto(). No more threads, just plain sending in
a while-loop until forever.

2.) My main app starts one thread that receives the mjpeg stream from the
ip camera. This camera sends with 30 frames per second (jpegs, so small
messages). I do not use these images, I just let this thread run
parallely to the main app that simply sends out the same dummy buffer
(640*380*3 bytes) via a udp socket as in example 1.

Variant 1 results in ~105 MB/sec, whereas variant 2 slows down to ~55MB/
sec.

I understand that the constant udp-sending-stream blocks the ethernet bus
and I need to pace it for making the bus free for reception, correct?

But what I don't understand is, how this influences my previous app? I
receive the ip-cam-stream with 30fps and once I received a complete
image, I quickly send this out via udp. I use a simple loop for sending,
as for example:

for (unsigned int i = 0; i < numSplitDatagrams; i++)
sendto(socket, image_fragment[i], sizeof(image_fragment[i], ...);


The whole send-process happens in almost no time (at least the
accumulated time sendto() takes). How can this influence the parallel
reception via tcp, so I get an overall drop down to 25 fps and even less?
I don't send constantly, only once I have a complete image.

Where do I need to set the pace then? Between the single sendto()-calls
in the loop when sending the single image or after the loop once an image
is sent?

On Jan 16, 3:07*pm, mikey79 <mike...@gmx.net> wrote:

> But what I don't understand is, how this influences my previous app? I
> receive the ip-cam-stream with 30fps and once I received a complete
> image, I quickly send this out via udp. I use a simple loop for sending,
> as for example:

You don't "quickly send this out via udp". You quickly *queue* it for
sending via udp. The 'sendto' function doesn't actually send the
datagram and when it returns, the datagram may or may not already have
been sent.

> for (unsigned int i = 0; i < numSplitDatagrams; i++)
> * *sendto(socket, image_fragment[i], sizeof(image_fragment[i], ...);
>
> The whole send-process happens in almost no time (at least the
> accumulated time sendto() takes).

Right, because it just puts the datagram on a queue.

> How can this influence the parallel
> reception via tcp, so I get an overall drop down to 25 fps and even less?
> I don't send constantly, only once I have a complete image.

TCP includes send pacing. UDP does not. If you want to send using UDP,
transmit pacing is your responsibility. The stack will happily let you
saturate or even oversaturate the links. This will massively harm TCP
performance by causing packet loss.

> Where do I need to set the pace then? Between the single sendto()-calls
> in the loop when sending the single image or after the loop once an image
> is sent?

You need to transmit data at a reasonable speed. 921,600 bytes in a
tiny fraction of a second is *NOT* reasonable. This will *obviously*
kill the network. You need to smooth out your transmission.

You can probably transmit more than one packet at a time, but 900 at a
time is simply insane. You also do not want to send multicast packets
that exceed the smallest reasonable path MTU. That will cause huge
problems too.

DS