Revision 1120103 of Evolution of HTTP

<p>{{HTTPSidebar}}</p>

<p><strong>HTTP</strong> is the underlying protocol of the World Wide Web. Invented by Tim Berners-Lee in the years 1989-1991, HTTP has seen many changes, keeping most of the simplicity and further shaping its flexibility. HTTP has evolved, from an early protocol to exchange files in a semi-trusted laboratory environment, to the modern maze of the Internet, now carrying images, videos in high resolution and 3D.</p>

<h2 id="Invention_of_the_World_Wide_Web">Invention of the World Wide Web</h2>

<p>In 1989, while he was working at CERN, Tim Berners-Lee wrote a proposal to build a&nbsp;hypertext system over the Internet. Initially calling it the <em>Mesh</em>, it was later renamed to <em>World Wide Web</em> during its implementation in 1990. Built over the existing TCP and IP protocols, it consisted of 4 building blocks:</p>

<ul>
 <li>A textual format to represent hypertext documents, the <em><a href="/en-US/docs/Web/HTML">HyperText Markup Language</a></em> (HTML).</li>
 <li>A simple protocol to exchange these documents, the <em>HypertText Transfer Protocol </em>(HTTP).</li>
 <li>A client to display (and accidentally edit) these documents, the first Web browser called <em>WorldWideWeb</em>.</li>
 <li>A server to give access to the document, an early version of <em>httpd</em>.</li>
</ul>

<p>These four building blocks were completed by the end of 1990, and the first servers were already running outside of CERN by early 1991. On August 6<sup>th</sup> 1991, Tim Berners-Lee's <a href="https://groups.google.com/forum/#!msg/alt.hypertext/eCTkkOoWTAY/urNMgHnS2gYJ">post</a> on the public <em>alt.hypertext</em> newsgroup is now considered as the official start of the World Wide Web as a public project.</p>

<p>The HTTP protocol used in those early phases was very simple, later dubbed HTTP/0.9, and sometimes as the one-line protocol.</p>

<h2 id="HTTP0.9_–_The_one-line_protocol">HTTP/0.9 – The one-line protocol</h2>

<p>The initial version of HTTP had no version number; it has been later called 0.9 to differentiate it from the later versions. HTTP/0.9 is extremely simple: requests consist of a single line and start with the only possible method {{HTTPMethod("GET")}} followed by the path to the resource (not the URL as both the protocol, server, and port are unnecessary once connected to the server).</p>

<pre>
GET /mypage.html</pre>

<p>The response is extremely simple too: it only consisted of the file itself.</p>

<pre>
&lt;HTML&gt;
A very simple HTML page
&lt;/HTML&gt;</pre>

<p>Unlike subsequent evolutions, there were no HTTP headers, meaning that only HTML files could be transmitted, but no other type of documents. There were no status or error codes: in case of a problem, a specific HTML file was send back with the description of the problem contained in it, for human consumption.</p>

<h2 id="HTTP1.0_–_Building_extensibility">HTTP/1.0 – Building extensibility</h2>

<p>HTTP/0.9 was very limited and both browsers and servers quickly extended it to be more versatile:</p>

<ul>
 <li>Versioning information is now sent within each request (<code>HTTP/1.0</code> is appended to the <code>GET</code> line)</li>
 <li>A status code line is also sent at the beginning of the response, allowing the browser itself to understand the success or failure of the request and to adapt its behavior in consequence (like in updating or using its local cache in a specific way)</li>
 <li>The notion of HTTP headers has been introduced, both for the requests and the responses, allowing metadata to be transmitted and making the protocol extremely flexible and extensible.</li>
 <li>With the help of the new HTTP headers, the ability to transmit other documents than plain HTML files has been added (thanks to the {{HTTPHeader("Content-Type")}} header).</li>
</ul>

<p>A typical request was then looking like this:</p>

<pre>
GET /mypage.html HTTP/1.0
User-Agent: NCSA_Mosaic/2.0 (Windows 3.1)

200 OK
Date: Tue, 15 Nov 1994 08:12:31 GMT
Server: CERN/3.0 libwww/2.17
Content-Type: text/html
&lt;HTML&gt; 
A page with an image
  &lt;IMG SRC="/myimage.gif"&gt;
&lt;/HTML&gt;</pre>

<p>Followed by a second connection and request to fetch the image:</p>

<pre>
GET /myimage.gif HTTP/1.0
User-Agent: NCSA_Mosaic/2.0 (Windows 3.1)

200 OK
Date: Tue, 15 Nov 1994 08:12:32 GMT
Server: CERN/3.0 libwww/2.17
Content-Type: text/gif
<em>(image content)</em></pre>

<p>These novelties have not been introduced as concerted effort, but as a try-and-see approach over the 1991-1995 period: a server and a browser added one feature and it saw if it get traction. A lot of interoperability problems were common. In November 1996, in order to solve these annoyances, an informational document describing the common practices has been published, {{RFC(1945)}}. This is the definition of HTTP/1.0 and it is notable that, in the narrow sense of the term, it isn't an official standard.</p>

<h2 id="HTTP1.1_–_The_standardized_protocol">HTTP/1.1 – The standardized protocol</h2>

<p>In parallel to the somewhat chaotic use of the diverse implementations of HTTP/1.0, and since 1995, well before the publication of HTTP/1.0 document the next year, proper standardization was in progress. The first standardized version of HTTP, HTTP/1.1 was published in early 1997, only a few months after HTTP/1.0.</p>

<p>HTTP/1.1 clarified ambiguities and introduced numerous improvements:</p>

<ul>
 <li>A connection can be reused, saving the time to reopen it numerous times to display the resources embedded into the single original document retrieved.</li>
 <li>Pipelining has been added, allowing to send a second request before the answer for the first one is fully transmitted, lowering the latency of the communication.</li>
 <li>Chunked responses are now also supported.</li>
 <li>Additional cache control mechanisms have been introduced.</li>
 <li>Content negotiation, including language, encoding, or type, has been introduced, and allows a client and a server to agree on the most adequate content to exchange.</li>
 <li>Thanks to the {{HTTPHeader("Host")}} header, the ability to host different domains at the same IP address now allows server collocation.</li>
</ul>

<p>A typical flow of requests, all through one single connection is now looking like this:</p>

<pre>
GET /en-US/docs/Glossary/Simple_header HTTP/1.1
Host: developer.mozilla.org
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:50.0) Gecko/20100101 Firefox/50.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://developer.mozilla.org/en-US/docs/Glossary/Simple_header

200 OK
Connection: Keep-Alive
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Date: Wed, 20 Jul 2016 10:55:30 GMT
Etag: "547fa7e369ef56031dd3bff2ace9fc0832eb251a"
Keep-Alive: timeout=5, max=1000
Last-Modified: Tue, 19 Jul 2016 00:59:33 GMT
Server: Apache
Transfer-Encoding: chunked
Vary: Cookie, Accept-Encoding

<em>(content)</em>


GET /static/img/header-background.png HTTP/1.1
Host: developer.cdn.mozilla.net
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:50.0) Gecko/20100101 Firefox/50.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://developer.mozilla.org/en-US/docs/Glossary/Simple_header

200 OK
Age: 9578461
Cache-Control: public, max-age=315360000
Connection: keep-alive
Content-Length: 3077
Content-Type: image/png
Date: Thu, 31 Mar 2016 13:34:46 GMT
Last-Modified: Wed, 21 Oct 2015 18:27:50 GMT
Server: Apache

<em>(image content of 3077 bytes)</em></pre>

<p>HTTP/1.1 was first published as {{rfc(2068)}} in January 1997.</p>

<h2 id="More_than_15_years_of_extensions">More than 15 years of extensions</h2>

<p>Thanks to its extensibility – creating new headers or methods is easy – and even if the HTTP/1.1 protocol was refined over two revisions, {{RFC("2616")}} published in June 1999 and the series of {{RFC("7230")}}-{{RFC("7235")}} published in June 2014 in prevision of the release of HTTP/2, this protocol has been extremely stable over more than 15 years.</p>

<h3 id="Using_HTTP_for_secure_transmissions">Using HTTP for secure transmissions</h3>

<p>The largest change that happened to HTTP was done as early as end of 1994. Instead of sending HTTP over a basic TCP/IP stack, Netscape Communication created an additional encrypted transmission layer on top of it: SSL. SSL 1.0 was never released outside the companies, but SSL 2.0 and its successors SSL 3.0 and SSL 3.1 allowed for the creation of e-commerce Web sites by encrypting and guaranteeing the authenticity of the messages exchanged between the server and client. SSL was put on the standards track and eventually became TLS, with version 1.0, 1.1, and 1.2 appearing successfully to close vulnerabilities. TLS 1.3 is currently in the making.</p>

<p>During the same time, the need for an encrypted transport layer raised: the Web left the relative trustiness of a mostly academic network, to a jungle where advertisers, random individuals or criminals compete to get as much private information about people, try to impersonate them or even to replace data transmitted by altered ones. As the applications built over HTTP became more and more powerful, having access to more and more private information like address books, e-mail, or the geographic position of the user, the need to have TLS became ubiquitous even outside the e-commerce use case.</p>

<h3 id="Using_HTTP_for_complex_applications">Using HTTP for complex applications</h3>

<p>The original vision of Tim Berners-Lee for the Web&nbsp;wasn't a read-only medium. He envisioned a Web were people can add and move documents remotely, a kind of distributed file system. Around 1996, HTTP has been extended to allow authoring, and a standard called WebDAV was created. It has been further extended for specific applications like CardDAV to handle address book entries and CalDAV to deal with calendars. But all these *DAV extensions had a flaw: they had to be implemented by the servers to be used, which was quite complex. Their use on Web realms stayed confidential.</p>

<p>In 2000, a new pattern for using HTTP was designed: {{glossary("REST", "representational state transfer")}} (or REST). The actions induced by the API were no more conveyed by new HTTP methods, but only by accessing specific URIs with basic HTTP/1.1 methods. This allowed any Web application to provide an API to allow retrieval and modification of its data without having to updated the browsers or the servers: all what is needed was embedded in the files served by the Web sites through standard HTTP/1.1. The drawback of the REST model resides in the fact that each website defines its own non-standard RESTful API and has total control on it; unlike the *DAV extensions were clients and servers are interoperable. RESTful APIs became very common in the 2010s.</p>

<p>Since 2005, the set of APIs available to Web pages greatly increased and several of these APIs created extensions, mostly new specific HTTP headers, to the HTTP protocol for specific purposes:</p>

<ul>
 <li><a href="/en-US/docs/Web/API/Server-sent_events">Server-sent events</a>, where the server can push occasional messages to the browser.</li>
 <li><a href="/en-US/docs/Web/API/WebSocket_API">WebSocket</a>, a new protocol that can be set up by upgrading an existing HTTP connection.</li>
</ul>

<h3 id="Relaxing_the_security-model_of_the_Web">Relaxing the security-model of the Web</h3>

<p>HTTP is independent of the security model of the Web, the <a href="/en-US/docs/Web/Security/Same-origin_policy">same-origin policy</a>. In fact, the current Web security model has been developed after the creation of HTTP! Over the years, it has proved useful to be able to be more lenient, by allowing under certain constraints to lift some of the restriction of this policy. How much and when such restrictions are lifted is transmitted by the server to the client using a new bunch of HTTP headers. These are defined in specifications like <a href="/en-US/docs/Glossary/CORS">Cross-Origin Resource Sharing</a> (CORS) or the <a href="/en-US/docs/Web/Security/CSP">Content Security Policy</a> (CSP).</p>

<p>In addition to these large extensions, numerous other headers have been added, sometimes experimentally only. Notable headers are Do Not Track ({{HTTPHeader("DNT")}}) header to control privacy, {{HTTPHeader("X-Frame-Options")}}, or {{HTTPHeader('Upgrade-Insecure-Request')}} but many more exist.</p>

<h2 id="HTTP2_–_A_protocol_for_greater_performance">HTTP/2 – A protocol for greater performance</h2>

<p>Over the years, Web pages became much more complex, often being true applications. The amount of visual media displayed, as well as the amount and size of the scripts to add interactivity, increased a lot: many more data is transmitted over many more requests. HTTP/1.1 connections need requests sent in the right order. Also, if several parallel connections can theoretically be used (typically between 5 and 8) there was still a lot of overhead and complexity. HTTP pipelining has turned out to be a deployment nightmare, for example.</p>

<p>In the first half of the 2010s, Google demonstrated an alternative way of exchanging data between client and server, by implementing the experimental SPDY protocol. It fostered a lot of interest, both on the browser and server side. Proving its ability to increase responsiveness and solving the duplication of data transmitted problem, SPDY served as the basis of the HTTP/2 protocol.</p>

<p>The HTTP/2 protocol has a few fundamental differences from the HTTP/1.1 version:</p>

<ul>
 <li>It is a binary protocol rather than a text protocol. It cannot be read and created by hand anymore; in exchange of this drawback, new optimization techniques can be implemented.</li>
 <li>It is a multiplexed protocol, where parallel requests can be handled over the same connection, removing the order and blocking constraints of the HTTP/1.x protocol.</li>
 <li>It compresses headers as these are often very similar between consecutive requests, removing significant duplication of data transmitted and the unneeded overhead induced by it.</li>
 <li>It allows a server to populate a client cache in advance with data that will likely be needed, by a mechanism called server pushes.</li>
</ul>

<p>Officially standardized in May 2015, HTTP/2 had a great success: in July 2016, 8.7% of all Web sites<sup><a href="https://w3techs.com/technologies/details/ce-http2/all/all">[1]</a></sup> are already using it, and this represents more than 68% of all requests<sup><a href="https://www.keycdn.com/blog/http2-statistics/">[2]</a></sup>. High-traffic Web sites, that save a lot of money with the decrease of the overhead of transmissions on the wire, have adopted it extremely rapidly.</p>

<p>This rapid adoption was possible as HTTP/2 doesn't need an adaptation of the Web sites and applications: using HTTP/1.1 or HTTP/2 is transparent for them. Having an up-to-date server speaking with a recent browser is enough to enable its use: only a few limited set of actors were needed to trigger its adoption and as old browser and server versions are replaced, its usage increased naturally, without Web developers needing to act.</p>

<h2 id="Post-HTTP2_evolution">Post-HTTP/2 evolution</h2>

<p>HTTP didn't stop&nbsp;its evolution with the release of HTTP/2. Like with HTTP/1.x, its extensibility is still used to add new features. Of notable importance we can cite these new extensions of HTTP appearing in 2016:</p>

<ul>
 <li>Support of {{HTTPHeader("Alt-Svc")}} that allows to dissociate the notion of the identification and the location of a given resource, allowing for smarter {{Glossary("CDN")}} caching mechanism.</li>
 <li>The introduction of {{HTTPHeader("Client-Hints")}} allowing the browser or the client to proactively communicate information about its needs or hardware constraints to the server.</li>
 <li>The introduction of security-related prefixes in the {{HTTPHeader("Cookie")}} header to help in guaranteeing that a secure cookie has not been altered.</li>
</ul>

<p>The evolution of HTTP shows that its extensibility and simplicity allowed to create numerous unforeseen application and adaptation of the protocol. The environment in which HTTP is used today is completely different from the one envisioned in the early 1990s. HTTP's original design proved to be a masterpiece allowing the Web to evolve over a quarter of a century without the need of non-compatible revolution. By curing many flaws identified over the years while retaining the flexibility and extensibility that made HTTP success over a quarter of a century, the rise of HTTP/2 seems to hint at&nbsp;a bright future for the protocol.</p>