HTTP over both TCP and UDP

Hi,
I understand that HTTP uses TCP rather than UDP,
and most of the existing implementations of HTTP
are based on TCP , since reliability is critical for
Web pages with text.

But, TCP can be good for transferring long files,
but not for small sessions.
So, can HTTP switch-over between TCP and UDP
rather than using TCP alone ?

By having the support for both TCP & UDP, we will
be able to utilize the benefits of TCP and UDP.
Is there any opensource available for it ?

Thx in advans,
Karthik Balaguru

Maxwell Lol wrote:
> karthikbalaguru <(E-Mail Removed)> writes:
>
>> Hi,
>> I understand that HTTP uses TCP rather than UDP,
>> and most of the existing implementations of HTTP
>> are based on TCP , since reliability is critical for
>> Web pages with text.
>>
>> But, TCP can be good for transferring long files,
>> but not for small sessions.
>> So, can HTTP switch-over between TCP and UDP
>> rather than using TCP alone ?
>
> TCP is find for small sessions.
> True, sending a single byte will take about 7 packets.

No, just 2 packets (like on a Telnet session): the byte encapsulated in
a packet and an ACK in the opposite direction, assuming that both
packets actually arrive at their destinations.

Robert

>
> And UDP can do it in one packet. (And the sender never knows if it is
> even transmitted, let alone arrives)
>
> But when you go to a web page and download 50 things simultaneously,
> that differecne is small.

On Apr 25, 10:23*am, karthikbalaguru <karthikbalagur...@gmail.com>
wrote:

> But, TCP can be good for transferring long files,
> but not for small sessions.
> So, can HTTP switch-over between TCP and UDP
> rather than using TCP alone ?

I don't see why it would be good. TCP has been optimized by experts
over many, many years and is very, very good at what it does. The only
time you should prefer UDP is because TCP provides some expensive
feature that you don't need. But we need everything TCP does.
Specifically, we need the following things UDP does not provide:

Slow start, Exponential backoff, Transmit pacing, Congestion control
Retransmissions, Acknowledgements
Reordering, Duplicate packet rejection, Lost packet detection, RTT
estimation
Path MTU determination, Segmentation
Session management

So the question is, why would you think that you can implement those
things better than TCP does? In other words, every feature (pretty
much) that TCP provides that UDP doesn't is one you need. So it's
madness to use UDP.

DS

Before you even get to the technical issues (which David Schwartz
outlined, and which might be summarized as saying http was designed to
run on top of tcp), you'd need a server and browser that both used
udp. And nobody is going to create one of them until the other is in
wide use.

On Apr 26, 9:15*am, Joe Pfeiffer <pfeif...@cs.nmsu.edu> wrote:

> Before you even get to the technical issues (which David Schwartz
> outlined, and which might be summarized as saying http was designed to
> run on top of tcp), you'd need a server and browser that both used
> udp. *And nobody is going to create one of them until the other is in
> wide use.

I don't think that's true, so long as there was no net harm to the
capabilities for clients/servers that didn't support it. If there was
a standard that provided genuine advantages if supported on both ends
and had effectively no cost if it wasn't supported and someone
supported patches to Apache and Firefox to support it, I can't think
of any reason those patches would be rejected.

You would need to have a standards document explaining how it works in
sufficient detail to determine if implementations were correct or not.
Ideally, it would be submitted as an IETF draft. The general opinion
would have to be that it's, at least, not harmful. The implementation
would have to be of comparable quality to the existing code in those
projects.

The catch though is that the implementation would have to make sense.
And there's a very critical reason why it likely won't -- TCP-like
protocols don't generally interoperate well with TCP on the Internet
in general. And if you go to enough effort to make it interoperate
well, you'll wind up suffering from the fact that the TCP
implementation is tuned for the platform and tightly integrated with
the network driver architecture and your user-space implementation
can't be. For example, exactly when you send your packets will be
critical to sharing bandwidth fairly, and precision timing in user-
space is difficult to impossible.

DS

On Apr 27, 3:44*am, Maxwell Lol <nos...@com.invalid> wrote:

> Perhaps, but I can think of several disadvantages to the server.
>
> It would, for instance, be much easier to bring a server to it's knees
> if it allowed UDP-based requests.

How do you figure? A UDP request can easily be thrown away by the HTTP
server. With TCP, a connection will be established by the kernel
whether or not the server wants it to be.

> It also makes the server much more vulnerable to a denial of service attempt.

Again, how do you figure? TCP is vulnerable to all the same denial of
service attacks UDP is, plus TCP-specific attacks such as SYN floods.

DS

On Apr 27, 8:55*am, Maxwell Lol <nos...@com.invalid> wrote:

> > How do you figure?

> Well, UDP has no congestion control. Clients can increase their rate,
> and any routers that are overloaded cannot tell them to slow down.

Right, but any algorithm proposed would also have congestion control.
So it couldn't be used to overwhelm outbound bandwidth. As for inbound
bandwidth, an attacker is not going to follow the congestion control
algorithms. So there's no different there.

> It's also faster to generate requests.

Huh?

> Consider the SQL Slammer worm. See below.
>
> > A UDP request can easily be thrown away by the HTTP
> > server. With TCP, a connection will be established by the kernel
> > whether or not the server wants it to be.
>
> >> It also makes the server much more vulnerable to a denial of service attempt.
>
> > Again, how do you figure? TCP is vulnerable to all the same denial of
> > service attacks UDP is, plus TCP-specific attacks such as SYN floods.

> In a TCP SYN flood, the client must either allocate state for the
> connection, or create a fabicated TCP packet using raw sockets.

Right, and in fact the client creates fabricated packets.

> A UDP-based client doesn't need to use raw sockets. It can send out
> packets very quickly using normal sockets, and without retaining
> state. *And a complete HTTP request (that consumes significant CPU)
> could be created sing a single UDP packet.

Right, but I can attack a server with UDP packets whether or not it's
listening for them.

> Microsoft removed raw sockets from the OS. Therefore it is much easier
> to do a DDoS using UDP that TCP.

I don't see how. You can bombard the server with UDP packets and DDoS
it whether or not it is listening for those packets.

> So each and every computer can generate requests and the other
> limiting factor is the ability of the stack to queue them up and
> transmit them.

I'm not sure I follow your argument. Are you saying that they would
overload the server with a bunch of requests and it would be saturated
trying to process those requests? Or are you saying the network or the
stack would be overloaded? If the latter, that could occur whether or
not the server is listening. If the former, that is solvable in the
design of the protocol (for example, by requiring a token before the
request is processed and issuing the token over TCP).

DS

On Apr 28, 5:19*am, Maxwell Lol <nos...@com.invalid> wrote:
> David Schwartz <dav...@webmaster.com> writes:

> > On Apr 27, 8:55*am, Maxwell Lol <nos...@com.invalid> wrote:

> >> Well, UDP has no congestion control. Clients can increase their rate,
> >> and any routers that are overloaded cannot tell them to slow down.

> > Right, but any algorithm proposed would also have congestion control.

> Yes. If you can come up with something useful - let us know.

You would pretty much have to do what TCP does. Slow start,
exponential backoff, transmit pacing, and so on. That's largely why
this is pointless -- to do it, you'd have to replicate TCP.

> > So it couldn't be used to overwhelm outbound bandwidth.

> Huh? Congestion is not determined by the outbound properties.

Congestion of outbound bandwidth is. If I try to congest your outbound
bandwidth, I'll open a bunch of HTTP sessions and request a large
file. Even if I don't ACK your data packets, you'll waste a lot more
of your outbound bandwidth than I'll waste of mine getting you to do
it.

You prevent an attacker from overwhelming your outbound bandwidth by
controlling how much outbound data you send.

> It's determined by the routers in the middle.

It doesn't matter where the congestion occurs, if it's due to server-
to-client traffic, you can solve it by controlling how much data the
server sends. This is exactly the same issue with both TCP and UDP.

> Are you envisioning a special behavior on the end points?

Yes, congestion control, just like what TCP does.

> And we can't assume all clients behave properly.

Right, but that applies equally to TCP and UDP, so no difference
there.

> >As for inbound
> > bandwidth, an attacker is not going to follow the congestion control
> > algorithms. So there's no different there.
>
> >> It's also faster to generate requests.
>
> > Huh?
>
> UDP is stateless. The OS queues it, and sends it when the NIC is
> finished with the last packet.
>
> TCP has state. There is a table of the current state. There is a
> buffer for the incoming stream - like 20Kb per connection. *It's
> easier/faster to queue up a million UDP packets than a million TCP
> packets.

Right, but what does that have to do with anything? You can queue up a
million UDP packets and flood me with them whether or not I'm running
any UDP protocol.

> >> A UDP-based client doesn't need to use raw sockets. It can send out
> >> packets very quickly using normal sockets, and without retaining
> >> state. *And a complete HTTP request (that consumes significant CPU)
> >> could be created sing a single UDP packet.

> > Right, but I can attack a server with UDP packets whether or not it's
> > listening for them.

> We are talking about a UDP-based web server.

Right.

> So the attack WOULD be on a port the server is listening to
> and this means the server has to treat the request as legitimate.

Right, but so what? You think it's easier to overwhelm a web server
than the network it's on?

Perhaps you're thinking that the server would get overwhelmed by the
requests if they were TCP. Sure, because the server has to create
state and has to send tear down packets and so on. But they're UDP.
The server just has to inspect and drop them. It's quick and
efficient.

> > I don't see how. You can bombard the server with UDP packets and DDoS
> > it whether or not it is listening for those packets.

> Again - a UDP-based web server. Obviously an attacker would terget a
> UDP port the server is listening to. Which requires the server to
> process the packet.

So what? The server OS would have to process the packet even if the
server weren't listening. And the server's processing of the packet
can consist simply of dropping it. During an attack scenario, the net
cost of a UDP packet is way less than the net cost of a TCP packet.

> > I'm not sure I follow your argument. Are you saying that they would
> > overload the server with a bunch of requests and it would be saturated
> > trying to process those requests?

> Sure. It's much easier to overwhelm a server using UDP instead of TCP,
> if the web server was listening to both. Raw sockets aren't needed, so
> any computer can do it, and admin priviledges aren't needed.

But they're not even if it's not listening for UDP. Just hit it with
UDP packets anyway, say on its DNS port. The relative cost of a
listened-to UDP packet and a user-space dropped UDP packet is just a
bit of server CPU, and the server CPU typically vastly outpaces the
network connection.

> Remember, routers cannot tell UDP clients to slow down when congestion
> is detected. They just drop packets.

Actually, that's not quite true. Router can tell UDP clients to slow
down by delaying their packets. And in practice, routers don't
mutilate TCP packets. Even though ECN is available, it's not very
widely supported and there's no evidence it's a critical part of
protecting against attacks. In any event, UDP can use ECN as well as
TCP.

> If the UDP-based web server was a beefy monster with 10 different
> Internet connections, then it would be forced to treat each incoming
> packet as legit, and would try to respond to each one.

> Given an attacker can send a millions requests to download an ISO
> image with zero overhead, and the server has to process each one, it
> seems likely the server would fail first. Especially if the client
> isn't listening. (or the client targets the server to send to a third party)

If the packets were hurting the server, it can simply drop them. If it
did that to TCP, it would have to tear down the connection. The
benefits of UDP being connectionless easily make up for the slight
extra cost of having to drop the packet in user space.

Any sensible server would have a fast path to drop suspicious packets
during an attack. The protocol could specify, for example, that you
send one UDP packet and if you get no reply, you must use a TCP
request to obtain a validation token. Then each subsequent UDP request
can contain that token. A packet with an invalid or abused token can
be dropped immediately.

You're just imagining a poor implementation and then blaming the
protocol for the poor implementation.

> >If the former, that is solvable in the
> > design of the protocol (for example, by requiring a token before the
> > request is processed and issuing the token over TCP).

> If you are talking about a hybrid, where a TCP connection is used to
> get a token, and then the UDP packet uses that token, then I don't see
> how that solves anything.

A UDP request with an invalid or abused token can be immediately
dropped by the server. The overhead of setting up and tearing down a
TCP session is avoided. No kernel state need be kept. An attack on
such a protocol would be harder than a TCP SYN flood.

> * * * UDP is unreliable. Getting 52% of a URL isn't very useful.
>
> * * * An attacker can use TCP to get a token, and then generate a
> * * * million UDP requests.

Sure, and we instantly drop all but the first two because the token
has been abused. That's *perfect* since we don't have to go to the
aggravation of keeping TCP state.

> * * * The web server still has to authenticate UDP requests. And since
> * * * there is no congestion control, and UDP packets can be streamed
> * * * out without limit from the attacker's side, it's easy to swamp
> * * * the server with bogus requests. Make it a distributed DoS, and that
> * * * server's going down.

There is no congestion control on any attack. That's what makes it an
attack. The beauty of UDP is we don't have the unavoidable penalty of
session establishment and teardown.

DS

On Apr 29, 5:50*am, Maxwell Lol <nos...@com.invalid> wrote:

> My point is that while the TCP protocol can a mechanism to slow down
> congestion at the routers, UDP does not.

Right, but we're not talking about using UDP. We're talking about
using a protocol layered on top of UDP. In general, routers don't
handle TCP packets any different from UDP packets, and to the extent
they do, I know of no difference that's critical to denial of service
attack mitigation.

> So while both TCP and UDP connections can both overload servers and
> routers, a router cannot tell a UDP client to slow down.

Yes, it can. It can delay or drop the UDP packets, just as it does
with TCP packets.

> If the Distributed Denial of Service attack's purpose was to consume
> incoming bandwidth, a TCP-based attack can be addressed by the
> routers, and the server.

So can a UDP attack the same way.

> However, a UDP-based attack would not be automatically blocked by the
> routers.

Neither would a TCP-based attack. I don't know what automatic router
blocking technique you're thinking of, but I don't think it actually
exists. Service providers don't treat TCP packets any differently from
UDP packets and once you get to the server, it's too late to stop an
attack that congests inbound bandwidth.

> I still don't know what you are proposing. I understand and have used
> DCCP, which is like UDP with congestion control. But it's not reliable.

If you don't know the proposal, how can you conclude that it will be
vulnerable to denial of service attacks then?

> Now ASSUMING you solved the reliability problems with a new
> application layer on top of UDP, (and since I haven't seen any of your
> ideans on this, so I think the whole thought experiment is silly) you
> also have the problem that the client can ask a server for bandwidth
> and CPU consuming requests using UDP.

And the server can simply drop the UDP packets. That's much cheaper
than TCP that requires session establishment and tear down.

> EVen if the incoming connection is not saturated, the CPU and outgoing
> connection can easily be saturated, because the query costs nearly
> nothing. It's the response that matters.

Then, duh, you DON'T RESPOND TO THE QUERY. See how simple it is?

Whereas with TCP, it's too late. Once you've established a connection,
you have to tear it down.

> > Perhaps you're thinking that the server would get overwhelmed by the
> > requests if they were TCP. Sure, because the server has to create
> > state and has to send tear down packets and so on. But they're UDP.
> > The server just has to inspect and drop them. It's quick and
> > efficient.

> TCP has state, AND sequence numbers. That's wht the server can inspect
> and drop bogus packets.

> But UDP does not have these characteristics.

So what?

> So HOW can a server decide if this UDP request is legit or not?
> And if an attacker can generate one authorized UDP packet,
> it can generate a million more.

Simple, you put a sequence number in the UDP packet. If the sequence
number is not valid, the server drops it. Dropping a packet in user
space is cheap.

> > So what? The server OS would have to process the packet even if the
> > server weren't listening. And the server's processing of the packet
> > can consist simply of dropping it. During an attack scenario, the net
> > cost of a UDP packet is way less than the net cost of a TCP packet.

> You still haven't describe a mechanism that allows a web server to
> decide that this UDP packet is legit, but a million other nearly
> packets are fake.

You put a token in the beginning of the packet. If the token is
invalid or abused, you drop the UDP packet. It's really that simple.

> In fact, the packets could even be identical. Web servers don't reject
> duplicate requests.

We're not talking about a web server, we're talking about *THIS*
protocol. If this protocol says "drop duplicate requests", then a
proper implementation will drop duplicate requests.

You are, again, assuming a completely broken specification and then
complaining that the specification is completely broken. Try assuming
a non-idiotic specification.

> > But they're not even if it's not listening for UDP. Just hit it with
> > UDP packets anyway, say on its DNS port. The relative cost of a
> > listened-to UDP packet and a user-space dropped UDP packet is just a
> > bit of server CPU, and the server CPU typically vastly outpaces the
> > network connection.

> Sigh....
>
> Let me repeat my point.
>
> Assuming someone can create UDP-based mechanism to fetch web pages
> (and since this is nonsense, I don't know why I am continuing to
> debate this point), using UDP-based web services makes it easier to
> overload the server.

How? The network treats them the same. Once you get to the server
machine, it's a non-issue. You're not going to overwhelm the machine.
At least with UDP, you can spare the network and the server kernel the
overhead of session setup and tear down.

> Yes, UDP-based attacks exist, But your proposal requires a legitimate
> UDP-based web request. *Once you do that, all the problems I am
> talking about occur, and so far you have not explained how a web
> server will accept one UDP-based web request, and reject a million
> other requests.

Right, that's because we don't have the protocol specification yet.
You are claiming it's not possible to develop such a specification
because of inherent problems with UDP. But all of your complaints are
trivially solvable.

The solution to this one is simple: if the server detects it is under
attack, it switches to a mode in which each UDP query must contain a
valid, unabused token. Queries without one are simply dropped.
Legitimate clients are programmed to send one UDP request, and if it's
not replied to, they send a TCP request that includes a request for a
token.

Dropping a UDP packet with no valid token is no more expensive than
dropping a TCP packet. There are advantages and disadvantages on both
sides. With TCP, if the server drops it, the kernel has to tear down
the TCP session. With UDP, it doesn't. But with UDP, the drop has to
take place in user space. More or less, it's a wash.

> > Actually, that's not quite true. Router can tell UDP clients to slow
> > down by delaying their packets.

> Not true.

How do you figure?

> Clients don't know unless an application on the receiving side
> notifies them. And the clients decide to be nice and slow down.

Right. If you assume an attacker will be nice, then this will work for
both TCP and UDP. If you don't, it won't work for either TCP or UDP.
And, again, someone can flood you with UDP whether or not you are
listening for it.

> >And in practice, routers don't
> > mutilate TCP packets. Even though ECN is available, it's not very
> > widely supported and there's no evidence it's a critical part of
> > protecting against attacks. In any event, UDP can use ECN as well as
> > TCP.

> Theoretically, and if the client wants to use it.
> Evil clients would not want to use it.

For both TCP and UDP. So this is not an issue.


> >> Given an attacker can send a millions requests to download an ISO
> >> image with zero overhead, and the server has to process each one, it
> >> seems likely the server would fail first. Especially if the client
> >> isn't listening. (or the client targets the server to send to a third party)

> > If the packets were hurting the server, it can simply drop them. If it
> > did that to TCP, it would have to tear down the connection. The
> > benefits of UDP being connectionless easily make up for the slight
> > extra cost of having to drop the packet in user space.

> The benefits also make it easier to create connections on the client side..

That's a negligible difference since compromised clients will find it
plenty easy and uncompromised clients can still flood you with UDP
even if you're not listening for it. Once the packet reaches the
server, the damage is all already done.

> And since the cost to the server is much greater than the cost to the client,
> it's much easier for clients to overload a server.

Except it won't overload the server. A typical server will barely
break a sweat dropping a full pipe of bogus UDP requests. So long as
the application provides a way to easily detect bogus requests (which
is trivial as I've explained), it doesn't matter.

> > Any sensible server would have a fast path to drop suspicious packets
> > during an attack. The protocol could specify, for example, that you
> > send one UDP packet and if you get no reply, you must use a TCP
> > request to obtain a validation token. Then each subsequent UDP request
> > can contain that token. A packet with an invalid or abused token can
> > be dropped immediately.

> So an evil client can
> * *request a token using TCP
> * *Get a token
> * *Generate 10,000 UDP requests using that token.
> * *repeat
>
> Now consider 10,000 evil clients.

This is *NO* different from the following scenario:

1) An evil client issues a web request using TCP.
2) The client then floods the server with 10,000 UDP packets to a
random port.
3) Repeat for 10,000 evil clients.

You think the server CPU won't be able to drop the UDP packets?!
Seriously?

> First of all, these UDP requests have to handled by the server.
> And they have to be examined individually.

So what? A CPU is way faster than the Internet.

> An attacker can also create arbitrate UDP packets with random source
> addresses and ports.

> So we have a stream of datagrams, and the server has to examine each
> and every one. How will it determine which packets are legitimate?

It looks at the token. If it's valid and unabused, it answers the
request if it wants to. It can always drop the packet if it wants, the
client will try again using TCP if it's legitimate.

> > You're just imagining a poor implementation and then blaming the
> > protocol for the poor implementation.

> Excuse me? Perhaps you can explain how your algorithm will work?

The same way TCP does. Honestly, this is really not complicated.
Layering reliable protocols on top of TCP with special characteristics
is really not all that complicated.

You are the one arguing that no implementation is possible that
doesn't have a defect. You have the burden of proof.

> > A UDP request with an invalid or abused token can be immediately
> > dropped by the server.

> Immediately? Explain how.

Check token in table. If not issued or abused, drop packet. That
simple.

> >The overhead of setting up and tearing down a
> > TCP session is avoided.

> By creating state that allows a "valid" UDP packet to arrive.

> So you exchange TCP state with UDP state.

Exactly. The difference is, with the UDP state, you can avoid the
session overhead if you can tell an illegitimate request. This stops
an attacker from consuming as much outbound bandwidth. This will work
so well that attackers will attack you with TCP instead, because most
web servers have more inbound bandwidth to spare than outbound, and
even with TCP SYN cookies, they can consume about as much outbound
bandwidth as their flood of inbound packets.

> >No kernel state need be kept. An attack on
> > such a protocol would be harder than a TCP SYN flood.

> If one UDP packet is legit, then why not a million more identical UDP
> packets?

Because a million identical packets has no legitimate purpose. Duh.

> How can you handle this without retaining state? *Or would you rather
> let me request downloading ubuntu 9.04 a million times simulaneously?

The same way TCP handles this without retaining state. If I get, say,
a TCP RST request, I check it in my table. If it matches no TCP
session I know of, I can detect this and ignore the packet. Note that
I didn't need to retain *ANY* state about *THIS* packet. This attack
packet required no state to be silently discarded. Same way.

DS

On Apr 29, 5:50*am, Maxwell Lol <nos...@com.invalid> wrote:

> > Sure, and we instantly drop all but the first two because the token
> > has been abused. That's *perfect* since we don't have to go to the
> > aggravation of keeping TCP state.

> Ah - so you DO retain state!

You do if it doesn't hurt you to do so. If it does, then you don't
have to. The client cannot compel you to retain state, but you can do
so if it benefits you.

The difference is, an attacker cannot make you keep state with UDP. He
can with TCP, that is, if you wish to continue to accept legitimate
requests and nothing before the user-space web server application can
tell the two apart.

> So please explain how these tokens work. If I have to use TCP to get a
> one-time UDP token, that defeats the entire concept.

You don't. The server would only switch to that mode during an attack.
Under normal conditions, your initial UDP request would simply be
responded to.

> And if I can get a token that can be reused, then I can get one
> legitimate one, and make a million more valid requests as quickly as I
> can spew them out.

The server can either honor those requests or not. If it chooses not
to, it doesn't have to maintain any state to do so. Te absence of the
token would do the job. So this is no more difficult then if we
weren't listening for the requests in the first place. Not matching a
token is not significantly more complex than not matching a listening
port.

DS