The Internet is, by default, not secure. In TCP and below, everything is transferred in plaintext; an attacker could snoop on your communications or, worse, masquerade as the person you think you're talking to. Yet today, most users don't have to worry about those kinds of attacks. Why not? The answer is the Transport Layer Security protocol, which transparently secures much of the modern Web. Strap in!

Conceptual Overview §

TLS aims to create a secure channel with three properties: confidentiality, authenticity, and integrity. These three properties go hand in hand, but each one has a distinct meaning.

Confidentiality §

It's pretty easy to see why confidentiality is important: nobody wants a third party to be able to read your communications. The solution is to encrypt your data with a cipher, an algorithm which accepts a plaintext and a key and produces a ciphertext, such that it is easy to retrieve the plaintext if you have the key, but virtually impossible without it. Ciphers that use the same key to encrypt and decrypt data are known as symmetric ciphers.

Symmetric ciphers are great because they're usually very fast and very secure, but they do have one weakness: key exchange. The two communicating parties need to figure out a way to share a secret key without the possibility of anyone else obtaining it. This creates a bit of a chicken-and-egg problem; after all, they can't establish a private means of communication without negotiating a key first, which just brings us back to square one. To solve this problem, we need to get a little more clever.

Enter the Diffie-Hellman Key Exchange. It enables a client and a server to generate the same secret value while only exchanging publicly knowable data, which sounds impossible until you take a look at how it works. Here's a diagram that explains the steps of Diffie-Hellman key exchange using colors of paint as a stand-in for cryptographic keys.

Mixing paints is a wonderful analogy that conveys the properties of DHKE in a very intuitive way, but it abstracts away the mathematical details, which you may be interested in. So here's a little cryptography primer.

Let's start with modular arithmetic. It all starts with a modulus, or some integer which we will call $p$. If we take two elements $a$ and $b$ from the set $(1, 2, 3, \dots, p - 1)$ and take the product

$$a \times b \mod p$$

the result is another integer from that set. In fact, if $p$ is prime, then the product of any two integers from the set mod $p$ must also be a member of the set. In mathematical terms, we say that the set is closed under multiplication mod $p$.

This combination of a set (which we will call $G$) and a binary operation forms a mathematical structure called a group, because it fulfills the following criteria:

• Associativity: $a \times (b \times c)$ = $(a \times b) \times c$ for all members of $G$.
• Identity element: there is an element $e$ such that for any $a$ in $G$, $a \times e = e \times a = a$. In this case, $e = 1$.
• Inverse element: for every element $a$ in $G$, there exists an element $a^{-1}$ so that $a \times a^{-1} = a^{-1} \times a = e$.

The idea of a group can be used to describe many things: everything from the permutations of a Rubik's cube to the symmetries of a crystal lattice, and much more. By representing some structure as a group, we can study its structure using the theorems and techniques of group theory.

In our case, we have the multiplicative group of integers mod $p$. It is known that if $p$ is prime then this group is cyclic, meaning that it contains some element $g$ such that any member of the group can be written as a power of $g$. In other words, by repeatedly multiplying $g$ by itself, we can step through every element of the group before arriving back at $g$. For this reason, $g$ is known as a generator of the group.

A useful property of cyclic groups is that they are always abelian, which simply means that the group operation is commutative. This is important for Diffie-Hellman!

Okay, so let's envision how we can apply this new knowledge to key exchange. Alice and Bob start by agreeing on a prime modulus $p$ and generator $g$; for TLS, these values have been standardized and are built into TLS libraries. Next, each party generates some secret value, which we will call $A$ and $B$. Alice computes $g^A$, and Bob computes $g^B$. These secondary values are exchanged.

Now, Alice can compute $(g^B)^A$, and Bob can compute $(g^A)^B$. These two must be equal, meaning that they have arrived at a shared secret! Crucially, an observer cannot do the same, because while they know the value of $g$ to the power of the secret values they do not know the secret values themselves. Furthermore, if $p$ is sufficiently large, there are no known methods to quickly determine $x$ from the value of $g^x \bmod p$. This is known as the discrete logarithm problem, and its difficulty forms the basis of many modern cryptosystems.

The method we have just described is known as finite-field Diffie-Hellman key exchange, but it is also common practice to use groups defined by elliptic curves rather than modular arithmetic. These groups have more complex structure, so the same level of security can be achieved with fewer bits. For finite-field Diffie-Hellman, you are usually dealing with primes of at least 2048 bits in length.

Authenticity §

Confidentiality is worthless if you can't ensure that the person on the other end of the line is who you actually intend to talk to. One of TLS's jobs is to help the server prove its identity. This is accomplished with the help of digital signatures.

Digital signatures are a form of public key cryptography, so the client first has to generate a keypair. This is usually done by creating a private key from random data, and then deriving a public key using a process specific to the signature scheme. The public key serves as the cryptographic identity of the client, and it can be distributed safely without compromising the private key.

To create a digital signature, the client uses the signing algorithm to convert their private key and a piece of data into a short output known as a signature. Signatures are useful since they have the following properties:

• Anyone can prove that a signature is valid using just the public key and the data.
• The signature won't validate if the data doesn't exactly match what was originally signed.
• No one can create a signature without possessing the private key.

So, suppose Bob wants to send a message to Alice, while also providing reassurance that the message was actually sent by him and not an impostor. He could sign the message, but Alice needs some way of knowing that the key he signed it with actually belongs to Bob.

TLS relies on the services of a trusted third party—we'll call her Carol—to provide authenticity. Both Alice and Bob trust Carol, and know her public key. So Bob asks Carol to sign a statement asserting that Bob's public key is <ABC>, and includes the signed statement when initiating communications with Alice. Now, Alice can verify that Bob's public key really belongs to Bob. This "signed statement" is known as a certificate.

This, in essence, is how TLS performs authentication. In reality, Carol's job is performed by a group of organizations known as certificate authorities, whose public keys are hardcoded into web browsers. The job of a CA is to respond to requests for certificates by verifying ownership of a domain (usually through methods like DNS) and then issuing a signed certificate.

You can see this at play for pretty much any website you visit. If you're on Chrome, simply click the padlock button next to the address bar in your browser, and you'll see the option to view the website's certification path. Here's what you would probably see for this page:

Notice that my website's certificate is not directly signed using a root certificate (ISRG Root X1). Instead, my CA (Let's Encrypt) has chosen to issue an intermediate keypair, R1, which it uses to sign certificates. This is done to minimize usage of the root keypair; it's a lot less of a headache to revoke an intermediate keypair than to try and get millions of users to adopt a new root certificate.

Simply receiving a valid certificate doesn't prevent man-in-the-middle attacks, though. Certificates are public knowledge; my webserver has to actually prove ownership of the corresponding private key somehow. To solve this, TLS makes the server sign the hash of all the messages sent before certificate verification. This assures the client that at no point during the handshake did an attacker mingle in their communications.

This is all well and good, but it leaves an elephant in the room: what's stopping a CA from issuing certificates for malicious purposes? For many years this was considered one of the most severe blemishes on TLS's security, but today, technologies have been developed that enable auditing of the certificates issued by a CA. We'll learn more about how this is achieved later.

It's also possible to run your own CA and issue your own certificates, though of course they will only be accepted by computers to which you have added your root certificate. This scheme is quite common in enterprise environments.

Here's a conversational illustration of all these concepts at play in a TLS handshake.

Integrity §

Even if you have authenticated with a server and established an encrypted channel, there remains a problem: an attacker who controls the network between you and your destination can still modify the ciphertexts being exchanged. Even though the attacker can't see the original message, if they know the position of some value within the ciphertext, they can simply blindly modify the messages until the desired effect is achieved.

This possibility obviously not acceptable. We want some cryptographic guarantee that the message was not tampered with in transit, which is offered by authenticated encryption schemes.

In TLS 1.3, integrity is handled by only using ciphers which include authenticated encryption with associated data (AEAD). When encrypting data using an AEAD cipher, a value known as an authentication tag is produced in addition to the ciphertext. The authentication tag allows clients to verify that the plaintext matches what was originally encrypted. In addition, the auth tag can also guarantee the validity of some associated data not included within the ciphertext, the associated data. Crucially, all of this is done without leaking any information about the encrypted data itself.

With all of that being said, let's actually examine a recorded TLS session and see how all of these cryptographic concepts are implemented.

Client Handshake Key Generation §

Before the client even begins the handshake, it first generates an x25519 keypair for use in key exchange later down the road. The private key is simply created by drawing 32 bytes of random data from an unpredictable source. Here is the client's private key, which is never sent over the network:

186b7a9daf3855fa090bf29a69391dc1ee788393f2dba58a23cab0f6b96d6355

The public key is derived by multiplying the fixed starting point (x = 9) by the private key. The elliptic-curve discrete logarithm problem prevents attackers from calculating the private key from the starting point and the public key. This is the public key, which you will see later in the exchange:

4b65e1788c1999350d0a44f50646a75c392584735d96d6d0b35b61c2f9375931

C → S: Client Hello §

TLS handshakes begin with the Client Hello message, where the client declares its capabilities to the server. Most importantly, it lists the TLS versions it understands as well as a list of supported ciphers for encryption. It also sends its public key.

Guide: Click on a section of the packet to see a description of its significance. Click the hex preview on the left to return to the top. Try enabling "show all" if you want to read all the section descriptions.

Server Handshake Key Generation §

The server uses the same process as the client to generate its keypair. It starts by generating a private key:

d8b3916b7e1ed8d6fa07c7810eef53639b77e51e0fd8e044c1c9e1186fd63c49

From this, the public key is derived.

011b5df090006e814ab8db60f6a2765cb90fe7fce73559e914796dafe6719c40

S → C: Server Hello §

The server responds to Client Hello with connection parameters such as the TLS version it has chosen to use, the session ID, and its own public key in response to the key_shares sent by the client.

Handshake Key Exchange §

At this point, both the client and the server are ready to perform the key exchange calculations necessary to secure the rest of the handshake. First, each party generates a shared secret by multiplying their private key by the other party's public key. On the client's side:

client_privkey x server_pubkey = shared_secret
186b7a9daf3855fa090bf29a69391dc1ee788393f2dba58a23cab0f6b96d6355 x
011b5df090006e814ab8db60f6a2765cb90fe7fce73559e914796dafe6719c40 =
4c75e186e47a2627bb4501955a051d516653d9570f34c660e623d26e2175b956

On the server's side:

server_privkey x client_pubkey = shared_secret
d8b3916b7e1ed8d6fa07c7810eef53639b77e51e0fd8e044c1c9e1186fd63c49 x
4b65e1788c1999350d0a44f50646a75c392584735d96d6d0b35b61c2f9375931 = 
4c75e186e47a2627bb4501955a051d516653d9570f34c660e623d26e2175b956

From here, TLS uses a process known as the key schedule to generate keys from the shared secret. There are two primary cryptographic operations involved. The first is hkdfExtract, which introduces a new "source of entropy" (e.g. the shared secret) into the current key schedule state, making all future derived keys depend on the correctness of this value. The second is deriveSecret, which uses the key schedule state to actually generate a random key of a specific length. This process is necessary because TLS generally needs to create several keys deterministically from a single shared secret, and getting it wrong can have serious security implications.

The key schedule begins with the calculation of an "early secret". This is used to include the pre-shared key in the key schedule. Since we don't have a PSK, this part is mostly irrelevant. However, the spec says that we still have to do it, just with dummy values.

earlySecret = hkdfExtract(
    Buffer.alloc(HASHLEN), // initial salt is zero 
    Buffer.alloc(HASHLEN)  // replace PSK with string of zeroes of the same length
);

Next, calculate a derived secret from the early secret.

derivedSecret = deriveSecret(earlySecret, "derived", Buffer.alloc(0));

Create the handshake secret using the shared value from key exchange and the derived secret from the previous step.

// sharedSecret = 4c 75 e1 ...
handshakeSecret = hkdfExtract(derivedSecret, sharedSecret);

For the next part, we need to pass the hash of the previously exchanged ClientHello and ServerHello messages to deriveSecret in a parameter called handshakeContext, which is created by applying a cryptographic hash function to the raw data of the ClientHello and Server Hello messages (excluding the record header).

// client side
clientHandshakeTrafficSecret = deriveSecret(handshakeSecret, "c hs traffic", handshakeContext);

// server side
serverHandshakeTrafficSecret = deriveSecret(handshakeSecret, "s hs traffic", handshakeContext);

From here, we just need to calculate two additional values that we can feed to our actual cipher. In this case, our client and server settled on AES-256, so we'll need a key and an unpredictable initialization vector.

// client credentials
clientHandshakeKey = hkdfExpandLabel(clientHandshakeTrafficSecret, "key", Buffer.alloc(0), 32);
clientHandshakeIV = hkdfExpandLabel(clientHandshakeTrafficSecret, "iv", Buffer.alloc(0), 12);

// server credentials
serverHandshakeKey = hkdfExpandLabel(serverHandshakeTrafficSecret, "key", Buffer.alloc(0), 32);
serverHandshakeIV = hkdfExpandLabel(serverHandshakeTrafficSecret, "iv", Buffer.alloc(0), 12);

The initialization vector (IV) ensures that every ciphertext is unique. For AES-GCM (and AES in general), IV reuse can have severe consequences.

The code used to explain this section is available here.

Perfect Forward Secrecy §

One important property offered by this complex key schedule is perfect forward secrecy: even if the server's stored keypair is eventually compromised, a client cannot decrypt recorded TLS traffic sent before the compromise. It's easy to see why this is the case: the public key the server sent in its key_share has nothing to do with the keypair in the server's certificate; they are ephemeral. As long as these ephemeral keys aren't saved somewhere (which should never happen under natural circumstances), the forward secrecy remains unbroken. Indeed, the TLS 1.3 RFC recommends immediately overwriting these values in memory to reduce potential for exposure. The only reason we're able to decrypt the capture is because I passed a flag to OpenSSL telling it to log some of the values used in key exchange (such as the handshake traffic secrets) to a text file that can be parsed by tools such as Wireshark.

S → C: Change Cipher Spec §

This message does nothing in TLS 1.3 and is only sent for compatibility purposes.

S → C: Encrypted Extensions §

Our first encrypted message! Here, additional TLS extensions that aren't essential for key negotation are declared, so that minimal information about the session is leaked to eavesdroppers. There aren't any additional extensions in this session, so this message is empty.

All encrypted messages are contained within application data records, which provide the necessary info to receive and decrypt the data. For all messages after this one, we will simply be showing the decrypted payload. However, for the sake of completeness, this message will be shown in its entirety.

Here's the result that we get after decryption:

You can check out the code I used to decrypt the message here.

S → C: Certificate §

The server sends its certificate to the client. Here, the certificate message is abridged since most of it consists of the actual certificate itself, which is explained below.

Only the decrypted message is shown.

The certificates themselves are serialized using a scheme called Distinguished Encoding Rules, or just DER. DER is rather complicated, so this section doesn't aim to really explain its nuances; instead, it's meant to give you an idea of what's actually in a certificate and how it's stored. There's a lot of information in the certificate that isn't really necessary for understanding the TLS cryptosystem. Pay the most attention to the parts that associate the server's keypair with its identity (its domain name), as well as the parts proving the certificate's trustworthiness (the public key, and the fields identifying the signing certificate).

Here's the first certificate, dissected:

While most of the certificate is pretty unremarkable, I recommend checking out the Certificate Transparency section—this is how CAs are audited today!

Each certificate contains some information about where to proceed next in the chain of trust. In this case, the certificate lists an authority key identifier of 142eb31...14c2c6, which matches the subject key identifier of R3. A client can then validate the signature and repeat the process until it reaches the trust root.

The three certificates are available for download: test.bithole.dev, R3, and ISRG Root X1. You can dump information about these certificates using OpenSSL from the terminal:

$ openssl x509 -in cert2.crt -inform der -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            91:2b:08:4a:cf:0c:18:a7:53:f6:d6:2e:25:a7:5f:5a
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = Internet Security Research Group, CN = ISRG Root X1
        Validity
            Not Before: Sep  4 00:00:00 2020 GMT
            Not After : Sep 15 16:00:00 2025 GMT
        Subject: C = US, O = Let's Encrypt, CN = R3
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:bb:02:15:28:cc:f6:a0:94:d3:0f:12:ec:8d:55:
                    92:c3:f8:82:f1:99:a6:7a:42:88:a7:5d:26:aa:b5:
                    2b:b9:c5:4c:b1:af:8e:6b:f9:75:c8:a3:d7:0f:47:
                    94:14:55:35:57:8c:9e:a8:a2:39:19:f5:82:3c:42:
                    a9:4e:6e:f5:3b:c3:2e:db:8d:c0:b0:5c:f3:59:38:
                    e7:ed:cf:69:f0:5a:0b:1b:be:c0:94:24:25:87:fa:
                    37:71:b3:13:e7:1c:ac:e1:9b:ef:db:e4:3b:45:52:
                    45:96:a9:c1:53:ce:34:c8:52:ee:b5:ae:ed:8f:de:
                    60:70:e2:a5:54:ab:b6:6d:0e:97:a5:40:34:6b:2b:
                    d3:bc:66:eb:66:34:7c:fa:6b:8b:8f:57:29:99:f8:
                    30:17:5d:ba:72:6f:fb:81:c5:ad:d2:86:58:3d:17:
                    c7:e7:09:bb:f1:2b:f7:86:dc:c1:da:71:5d:d4:46:
                    e3:cc:ad:25:c1:88:bc:60:67:75:66:b3:f1:18:f7:
                    a2:5c:e6:53:ff:3a:88:b6:47:a5:ff:13:18:ea:98:
                    09:77:3f:9d:53:f9:cf:01:e5:f5:a6:70:17:14:af:
                    63:a4:ff:99:b3:93:9d:dc:53:a7:06:fe:48:85:1d:
                    a1:69:ae:25:75:bb:13:cc:52:03:f5:ed:51:a1:8b:
                    db:15
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign, CRL Sign
            X509v3 Extended Key Usage:
                TLS Web Client Authentication, TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0
            X509v3 Subject Key Identifier:
                14:2E:B3:17:B7:58:56:CB:AE:50:09:40:E6:1F:AF:9D:8B:14:C2:C6
            X509v3 Authority Key Identifier:
                keyid:79:B4:59:E6:7B:B6:E5:E4:01:73:80:08:88:C8:1A:58:F6:E9:9B:6E

            Authority Information Access:
                CA Issuers - URI:http://x1.i.lencr.org/

            X509v3 CRL Distribution Points:

                Full Name:
                  URI:http://x1.c.lencr.org/

            X509v3 Certificate Policies:
                Policy: 2.23.140.1.2.1
                Policy: 1.3.6.1.4.1.44947.1.1.1

    Signature Algorithm: sha256WithRSAEncryption
         85:ca:4e:47:3e:a3:f7:85:44:85:bc:d5:67:78:b2:98:63:ad:
         75:4d:1e:96:3d:33:65:72:54:2d:81:a0:ea:c3:ed:f8:20:bf:
         5f:cc:b7:70:00:b7:6e:3b:f6:5e:94:de:e4:20:9f:a6:ef:8b:
         b2:03:e7:a2:b5:16:3c:91:ce:b4:ed:39:02:e7:7c:25:8a:47:
         e6:65:6e:3f:46:f4:d9:f0:ce:94:2b:ee:54:ce:12:bc:8c:27:
         4b:b8:c1:98:2f:a2:af:cd:71:91:4a:08:b7:c8:b8:23:7b:04:
         2d:08:f9:08:57:3e:83:d9:04:33:0a:47:21:78:09:82:27:c3:
         2a:c8:9b:b9:ce:5c:f2:64:c8:c0:be:79:c0:4f:8e:6d:44:0c:
         5e:92:bb:2e:f7:8b:10:e1:e8:1d:44:29:db:59:20:ed:63:b9:
         21:f8:12:26:94:93:57:a0:1d:65:04:c1:0a:22:ae:10:0d:43:
         97:a1:18:1f:7e:e0:e0:86:37:b5:5a:b1:bd:30:bf:87:6e:2b:
         2a:ff:21:4e:1b:05:c3:f5:18:97:f0:5e:ac:c3:a5:b8:6a:f0:
         2e:bc:3b:33:b9:ee:4b:de:cc:fc:e4:af:84:0b:86:3f:c0:55:
         43:36:f6:68:e1:36:17:6a:8e:99:d1:ff:a5:40:a7:34:b7:c0:
         d0:63:39:35:39:75:6e:f2:ba:76:c8:93:02:e9:a9:4b:6c:17:
         ce:0c:02:d9:bd:81:fb:9f:b7:68:d4:06:65:b3:82:3d:77:53:
         f8:8e:79:03:ad:0a:31:07:75:2a:43:d8:55:97:72:c4:29:0e:
         f7:c4:5d:4e:c8:ae:46:84:30:d7:f2:85:5f:18:a1:79:bb:e7:
         5e:70:8b:07:e1:86:93:c3:b9:8f:dc:61:71:25:2a:af:df:ed:
         25:50:52:68:8b:92:dc:e5:d6:b5:e3:da:7d:d0:87:6c:84:21:
         31:ae:82:f5:fb:b9:ab:c8:89:17:3d:e1:4c:e5:38:0e:f6:bd:
         2b:bd:96:81:14:eb:d5:db:3d:20:a7:7e:59:d3:e2:f8:58:f9:
         5b:b8:48:cd:fe:5c:4f:16:29:fe:1e:55:23:af:c8:11:b0:8d:
         ea:7c:93:90:17:2f:fd:ac:a2:09:47:46:3f:f0:e9:b0:b7:ff:
         28:4d:68:32:d6:67:5e:1e:69:a3:93:b8:f5:9d:8b:2f:0b:d2:
         52:43:a6:6f:32:57:65:4d:32:81:df:38:53:85:5d:7e:5d:66:
         29:ea:b8:dd:e4:95:b5:cd:b5:56:12:42:cd:c4:4e:c6:25:38:
         44:50:6d:ec:ce:00:55:18:fe:e9:49:64:d4:4e:ca:97:9c:b4:
         5b:c0:73:a8:ab:b8:47:c2

This yields pretty much the same information as what was listed in the manual breakdown, sans all the byte-level details.

S → C: Certificate Verify §

This message allows the server to prove that it controls the private keys behind the certificate by signing a hash of all the messages sent up until now. This also ensures that none of those messages were tampered with.

Only the decrypted message is shown.

contextHash = crypto.createHash("sha384").update(Buffer.concat([
    ClientHello,
    ServerHello,
    EncryptedExtensions,
    Certificate
])).digest();

data = Buffer.concat([
    Buffer.from(new Array(64).fill(0x20)), // 64 spaces
    Buffer.from("TLS 1.3, server CertificateVerify", "ascii"), // context string
    Buffer.alloc(1), // null byte
    contextHash // content to be signed
]);

console.log(crypto.verify(
    "sha256",
    data,
    {key, padding: crypto.constants.RSA_PKCS1_PSS_PADDING},
    signature
)); // -> "true"

S → C: Finished §

As the name implies, this is the last message sent by the server in the handshake. It contains the HMAC of all messages sent up to this point, created using a key derived from the server's handshake traffic secret, guaranteeing that none of the handshake has been interfered with.

Only the decrypted message is shown.

serverFinishedKey = hkdfExpandLabel(serverHandshakeTrafficSecret, "finished", Buffer.alloc(0), HASHLEN);

contextHash = crypto.createHash("sha384").update(Buffer.concat([
    ClientHello,
    ServerHello,
    EncryptedExtensions,
    Certificate,
    CertificateVerify
])).digest();

console.log(crypto.createHmac("sha384", finishedKey)
                  .update(contextHash)
                  .digest()
                  .toString("hex"));

89625ec1d535045663cf6237a38b8d220dd00fe4ea50bfe2d51573680becf1c8bbae0421cb667bff10bc546c017aaef6

C → S: Change Cipher Spec §

The client also sends a Change Cipher Spec message to the server, which is similarly ignored in TLS 1.3. The byte-level analysis is omitted since it is identical to the one sent by the server earlier in the handshake.

C → S: Finished §

Like the server, the client also sends a Finished message to complete the handshake.

Only the decrypted message is shown.

13e0061c0160d048c9ea16964b12d5340630c8e30f04c48b895f98f898113d31b2c9653af573952ad5ead6719b34fc17

S → C: New Session Ticket §

TLS 1.3 supports a feature called session resumption to reduce the number of full key exchanges a client has to do. Here's how it works: after completing a handshake, the server may send the client a value called a session ticket. Next time, the client can present the session ticket to the server as a preshared key and pick up last time the two parties left off, instead of starting from scratch. Let's examine the parts of a session ticket.

Only the decrypted message is shown.

Session tickets are single-use only, so upon initiating a new session some TLS implementations will send two tickets to give the client some added flexibility. The second session ticket is omitted since it is pretty much the same as the first one in structure.

Application Key Calculation §

From this point on, messages are encrypted by a new set of keys known as the application traffic keys. We can follow their derivation in the key schedule.

First, we generate the master secret.

derivedSecret2 = deriveSecret(handshakeSecret, "derived", Buffer.alloc(0));
masterSecret = hkdfExtract(derivedSecret2, Buffer.alloc(HASHLEN));

Next, the application secrets are derived. These values will be used by the client and server, respectively, to generate their encryption keys.

applicationContext = Buffer.concat([handshakeContext, EncryptedExtensions, Certificate, CertificateVerify, ServerFinished]);
clientApplicationTrafficSecret = deriveSecret(masterSecret, "c ap traffic", applicationContext);
serverApplicationTrafficSecret = deriveSecret(masterSecret, "s ap traffic", applicationContext);

We can now derive a set of values that we can use with our cipher (AES-256), much like we did earlier during the handshake.

clientApplicationKey = hkdfExpandLabel(clientApplicationTrafficSecret, "key", Buffer.alloc(0), 32);
clientApplicationIV = hkdfExpandLabel(clientApplicationTrafficSecret, "iv", Buffer.alloc(0), 12);

serverApplicationKey = hkdfExpandLabel(serverApplicationTrafficSecret, "key", Buffer.alloc(0), 32);
serverApplicationIV = hkdfExpandLabel(serverApplicationTrafficSecret, "iv", Buffer.alloc(0), 12);

We are finally ready to read the last part of the session.

S → C: Application Data §

The script I wrote to generate this session finishes by sending four bytes of application data to the server. Let's see what's inside.

Here's the unencrypted packet:

Now, decrypted:

And, at long last, the client and server have a secure channel which they can communicate through.

Epilogue §

This article only shows a small portion of TLS's many capabilities, which includes things like 0-RTT messages, client authentication, SNI, ALPN, and so on. Furthermore, TLS 1.3 is vastly different from TLS 1.2 in many ways—I specifically chose TLS 1.3 since it is more elegant than TLS 1.2, being designed to avoid the pitfalls of past standards. To learn more, the RFCs are your best friend. Skip ahead to the Further Reading section for some good references.

Also, much of the Internet still runs on TLS 1.2, which differs from TLS 1.3 in many ways. A full description of TLS and its many nuances is just a tad too long for a blogpost, so skip ahead to the Further Reading section for some good references.

Recording the TLS session turned out to be a little tricky. I ended up modifying OpenSSL to log the private keys used in the handshake key exchange process and rebuilding NodeJS from source. Here's the patch I applied before rebuilding:

diff --git a/deps/openssl/openssl/ssl/statem/extensions_clnt.c b/deps/openssl/openssl/ssl/statem/extensions_clnt.c
index 7b46074232..bd765f43f0 100644
--- a/deps/openssl/openssl/ssl/statem/extensions_clnt.c
+++ b/deps/openssl/openssl/ssl/statem/extensions_clnt.c
@@ -613,6 +613,11 @@ static int add_key_share(SSL *s, WPACKET *pkt, unsigned int curve_id)
         }
     }
 
+    /* print out the key >:) */
+    BIO *bp = BIO_new_fp(stdout, BIO_NOCLOSE);
+    EVP_PKEY_print_private(bp, key_share_key, 1, NULL);
+    BIO_free(bp);
+
     /* Encode the public key. */
     encodedlen = EVP_PKEY_get1_encoded_public_key(key_share_key,
                                                   &encoded_point);
diff --git a/deps/openssl/openssl/ssl/statem/extensions_srvr.c b/deps/openssl/openssl/ssl/statem/extensions_srvr.c
index 0b6e843e8a..757c49190c 100644
--- a/deps/openssl/openssl/ssl/statem/extensions_srvr.c
+++ b/deps/openssl/openssl/ssl/statem/extensions_srvr.c
@@ -1670,6 +1670,11 @@ EXT_RETURN tls_construct_stoc_key_share(SSL *s, WPACKET *pkt,
             return EXT_RETURN_FAIL;
         }
 
+	/* more devilish key printing */
+	BIO *bp = BIO_new_fp(stdout, BIO_NOCLOSE);
+	EVP_PKEY_print_private(bp, skey, 1, NULL);
+	BIO_free(bp);
+
         /* Generate encoding of server key */
         encoded_pt_len = EVP_PKEY_get1_encoded_public_key(skey, &encodedPoint);
         if (encoded_pt_len == 0) {

That was enough instrumentation to figure out all the other secrets used in the handshake. A couple weeks of banging my head on various RFCs was enough to piece together the rest of the puzzle. After that, it was a simple matter of using NodeJS to create a TLS client and server, then capturing the exchange using tcpdump. Here's the client/server code.

// run with --tls-keylog=...

const tls = require("tls");
const fs = require("fs");

const port = 8443, host = "test.bithole.dev";

const server = tls.createServer({
    cert: fs.readFileSync("fullchain.pem", "utf-8"),
    key: fs.readFileSync("privkey.pem", "utf-8")
});

server.on("secureConnection", tlsSocket => {
    console.log("connection");
    tlsSocket.on("data", data => console.log("data received:", data));
});

server.listen(port, () => console.log("Started listening"));

const secureSocket = tls.connect({host, port});
secureSocket.on("secureConnect", () => {
    console.log("client connected");
    setTimeout(() => secureSocket.write(Buffer.from([0xde,0xad,0xbe,0xef])), 1000);
});

I created a subdomain test.bithole.dev resolving to 127.0.0.1 and grabbed a certificate through Let's Encrypt to make this example a little more realistic.

You can download the full packet capture of the exchange which this page is based on here. You'll also need the keylog to decrypt the capture.

Further Reading §

• TLS 1.3 Illustrated
• IETF - RFC 8446: The Transport Layer Security (TLS) Protocol Version 1.3
• IETF - RFC 8448: Example Handshake Traces for TLS 1.3
• IETF - RFC 5869: HMAC-based Extract-and-Expand Key Derivation Function (HKDF)
• IETF - RFC 5280: Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile
• Cloudflare Fundamentals - TLS
• Certificate Transparency - How CT Works
• Let's Encrypt - A Warm Welcome to ASN.1 and DER