slowloris | Slowloris rewrite in Python | HTTP library

TCP does not deliver messages. It delivers a stream of bytes, and it can deliver those bytes in parcels whose size need not bear any resemblance to the size of the parcels that were written by the sender. So whenever you're reading a stream of data from TCP you have to be prepared to get that data in small parcels spread over multiple read() calls on the socket.

Typically the receiver will accumulate the data from those parcels into a buffer until it has collected enough data to construct a meaningful data unit of some kind that can then be processed. Depending on the format and meaning of the data that unit of actionable data could be a certain number of bytes, or a line's worth of characters, or a (potentially multi-byte) character, or something else.

For an HTTP server the actionable unit could reasonably be a complete set of headers, which is recognisable by the empty line that marks the end of the header block. Or you might decide that the first actionable unit is a line (the HTTP request line) and the second actionable unit is the remaining set of headers. (If you decide that the actionable unit while processing headers is always a line, you have to be prepared to deal with multi-line headers. That probably involves an additional layer of buffering.)

In the usual case data from TCP on an HTTP connection will arrive in fairly large blocks, hundreds or even thousands of bytes from a single read() if the specified buffer is large enough to hold them. So it might be the case that all of the headers are collected by your first read() on a new connection. But you must not depend on that happening. You must be prepared to issue multiple read() calls to get the data you need.

Even if you do have to make multiple read() calls to get all of the headers, the time spent doing this is usually pretty brief and then you can get on with handling this request and moving on to the next one.

Whenever an HTTP server is handling a request it will have a certain number of resources devoted to that request -- it has the connection socket, data buffers, probably some structures allocated for tracking request state, maybe one or more threads devoted to this connection or request, a TLS context if this is an HTTPS request, and so on. If the request can be handled quickly then those resources can be released and recycled or applied to other requests and connections. If the request can not be handled quickly then those resources remain tied up, and if the server has to handle lots of long-lived requests then its resource demands can rise to the point where the server becomes sluggish, or becomes unable to accept new connections, or perhaps even crashes.

Resource exhaustion can happen in normal operation if the server is undersized for the load, but it's also possible for a server to be deliberately driven into exhaustion by an attacker. There are many variants of this kind of attack. The first HTTP server exhaustion attacks involved having a hostile client send very large (perhaps infinitely large) request bodies, and/or send those request bodies very slowly, and/or keep the connection open while not sending any request data at all. The server's defence against this kind of attack is to limit the size of request body it will consume, and to limit the length of time that it will wait for a request body to be completely delivered, and to limit the length of time it will wait for a certain incremental amount of request data to be received.

There's also an attack where the client opens a connection to the server but does not send any header data at all. The server's defence is to limit the time it will wait for header data to begin arriving, and also to limit the time it will wait for additional header data to arrive.

The Slowloris attack is the next step in the header-based exhaustion attack progression. Instead of sending no headers, it sends them but does it very slowly, just slowly enough to avoid having the connection dropped by the server.

And Slowloris sends a never-ending series of made-up headers, so that the server never goes into the mode where its defences against ridiculous request body attacks come into play. The attacker could even send these headers a single byte at a time if it wanted to waste more CPU at the server; I don't know if the original Slowloris does this but I wouldn't be at all surprised if some later variant does.

The usual server-side defence against Slowloris is to apply the same strategies that are used against hostile request bodies. That is, put arrival-completion-time and total-header size limits on the request headers.

The 100-Continue thing is unrelated. Your server should be prepared to send a 100-Continue interim response after processing the request headers, but only if the client explicitly specifies (by sending an Expect header that asks for 100-Continue) that it wants an interim response. In my experience this is rare, so I wouldn't worry about it too much if you're just starting to build a server. It should be straightforward to add this feature after you have the basic server up and running.

RFC 2616 is where all of HTTP 1.1 is specified. If you're writing a server then you should start by becoming very familiar with that RFC.

Technically RFC 2616 has been superseded by a series of later RFCs (7230, 7231, 7232, 7233, 7234, 7235) but the changes are small and I find it easier to read the one old document than the five new ones.