Authentication and Security

Every HTTP authentication exchange follows the same shape, regardless of the scheme in use. The server challenges, and the client responds.

When a client requests a protected resource without credentials, the server does not simply refuse. It tells the client how to authenticate:

The 401 Unauthorized status code means: "I know what you asked for, but I need proof of who you are before I hand it over." The WWW-Authenticate header tells the client which authentication scheme to use and provides any parameters the scheme requires.

The client gathers credentials—​usually by prompting the user—​and retries the request with an Authorization header:

If the credentials are valid, the server returns the resource normally:

If they are not, the server sends another 401 and the cycle repeats. This three-step dance—​request, challenge, authorized request—​is the foundation of all HTTP authentication.

A single server often protects different resources with different passwords. A corporate intranet might have one set of credentials for financial reports and another for the employee directory. HTTP handles this through realms.

The realm parameter in the WWW-Authenticate header names the protected area:

When the browser encounters this challenge, it displays the realm name to the user, so they know which username and password to enter. A request to a different part of the same server might trigger a different challenge:

Realms let a server partition its resources into independent protection spaces, each with its own set of authorized users. The browser remembers which credentials belong to which realm and sends them automatically on subsequent requests to the same space.

Basic authentication is the oldest and simplest HTTP authentication scheme. It is supported by virtually every client and server, and it works like this:

A complete exchange:

Base64 is an encoding, not encryption. Anyone who intercepts the Authorization header can trivially decode it and read the password in plain text. Basic authentication is only safe when the connection itself is encrypted with TLS—​that is, when the URL begins with https. Over plain HTTP, it is no more secure than shouting your password across a crowded room.

Despite this weakness, Basic authentication remains useful. It is easy to implement, universally understood, and perfectly adequate when layered on top of HTTPS. Many internal tools and APIs still rely on it.

Digest authentication was designed to fix Basic’s most glaring flaw: sending the password in the clear. Instead of transmitting the actual password, the client sends a digest--a one-way cryptographic hash that proves knowledge of the password without revealing it.

Modern APIs rarely ask users for a password on every request. Instead, the client authenticates once—​typically through a login page or an OAuth 2.0 flow—​and receives a token. This token is a string that represents the client’s identity and permissions. On subsequent requests, the client presents the token rather than a username and password.

The Bearer scheme carries these tokens:

The server validates the token—​checking its signature, its expiration, and the permissions it grants—​and either serves the resource or rejects the request.

Bearer tokens are opaque to the protocol. HTTP does not know or care what is inside them. They might be JSON Web Tokens (JWTs) containing encoded claims, or random strings that the server looks up in a database. The protocol’s only job is to carry them in the Authorization header.

Like Basic authentication, Bearer tokens must be protected by TLS. A stolen token grants the same access as a stolen password, and tokens travel in every request header. HTTPS ensures they stay confidential.

Two status codes relate to authentication and authorization, and the distinction between them matters:

A 401 is a question: "Who are you?" A 403 is a verdict: "I know who you are, and the answer is no."

Some servers return 404 Not Found instead of 403 to hide the existence of a resource from unauthorized users. If an attacker cannot tell whether a URL leads to a protected page or nothing at all, that itself is a layer of defense.

Authentication can happen at intermediaries, not just origin servers. A corporate proxy might require employees to identify themselves before any request reaches the open Internet. HTTP supports this with a parallel set of headers and a dedicated status code.

The exchange follows the same challenge/response pattern. The proxy sends a 407 with a Proxy-Authenticate header; the client retries with Proxy-Authorization. Both origin-server and proxy authentication can coexist in the same request, each using its own set of headers.

HTTP authentication proves identity, but it does not protect the conversation. Headers, bodies, and credentials all travel as readable text unless the connection is encrypted. This is where TLS—​Transport Layer Security—​enters the picture.

When a URL begins with https, the client and server perform a TLS handshake before any HTTP data is exchanged. This handshake establishes three properties:

Encryption. All data between client and server is encrypted. Eavesdroppers see only opaque bytes.

Server authentication. The server presents a certificate proving its identity. The client verifies the certificate against a trusted chain of certificate authorities. This prevents an attacker from impersonating the server.

Integrity. Every message includes a cryptographic checksum. If a single byte is altered in transit, the receiver detects the tampering and discards the message.

TLS does not replace HTTP authentication—​it complements it. HTTP authentication answers "who is the client?" TLS answers "is this really the server, and is anyone listening?" Together they provide both ends of the trust equation.

Without TLS, Basic credentials are exposed, Bearer tokens can be stolen, and even Digest authentication is vulnerable to sophisticated attacks. In modern practice, HTTPS is not optional for any authenticated endpoint. It is the foundation on which all other security mechanisms rest.

Here is an annotated exchange that ties together the concepts from this section. A client accesses a protected API endpoint:

The server challenges:

The client authenticates (perhaps through an OAuth flow) and retries:

The server validates the token and responds:

The Cache-Control: private, no-store directive is worth noting. Authenticated responses often contain data specific to one user. Caching such responses in a shared proxy would leak private data to other users. The no-store directive tells every cache along the path—​browser, proxy, CDN—​that this response must never be stored.

Authentication, authorization, and transport security each solve a different piece of the same puzzle. Authentication proves identity. Authorization determines what that identity may access. TLS ensures the entire conversation remains private. HTTP weaves all three into its stateless request/response model through a handful of headers and status codes—​no sessions, no stored state, just a protocol-level conversation that scales to billions of requests per day.