Base64: encoding, decoding and the traps to avoid

You paste a string that begins with iVBORw0KGgo into a decoder, it answers ‘invalid string’, and nothing tells you why. Or the other way round: you encode Café and the service at the far end fails to recover the right word. Base64 looks trivial, yet three details — the character set, the alphabet in use and the padding — account for very nearly every failure.

What Base64 does, and what it does not

Base64 cuts a stream of bytes into 3-byte slices (24 bits), re-cuts each slice into four groups of 6 bits, and swaps each group for a character drawn from an alphabet of 64 signs: A–Z, a–z, 0–9, plus + and /. When the total length is not a multiple of 3, the last slice is padded out with one or two = signs. That is all there is to it: a mechanical transformation, with no key and no secret.

The point is to push binary data through a channel that only accepts text. The common cases today:

• email attachments (MIME), the job the format was designed for;
• a file carried inside a JSON field or an XML attribute, since neither has a binary type;
• certificates and keys in PEM format, between the -----BEGIN…----- lines;
• JWT tokens and the Authorization: Basic HTTP header;
• data URIs, which embed an image directly in HTML or CSS.

What Base64 is not: it is neither encryption nor compression. Anyone can decode the string in a second. Writing a password in Base64 in a configuration file protects nothing at all. If the aim is to check that a file has not been altered, what you need is a digest rather than an encoding: a SHA-256 digest answers that question, and, crucially, it is not reversible.

The overhead: +33%, no exceptions

Four characters for every three bytes: the ratio is fixed, so the encoded string always weighs a third more than the source. That is not an average, it is arithmetic.

Add the line breaks on top: MIME traditionally wraps every 76 columns, PEM every 64, which puts another 2 to 3% on the total. That is why a 20 MB attachment — around 27 MB once encoded and split into lines — trips a server limit set at 25 MB: the limit applies to the encoded message, not to the original file.

Encoding text: it all comes down to the character set

Base64 encodes bytes, never characters. So before encoding text you have to decide how that text becomes bytes, and that is where it goes off the rails. The word Café in UTF-8 is the bytes 43 61 66 C3 A9 and gives Q2Fmw6k=. The same word in ISO-8859-1 is 43 61 66 E9 and gives Q2Fm6Q==. Two results for a single word: if sender and recipient do not agree on the encoding, the accent comes back as mojibake at the other end.

UTF-8 is the default choice everywhere today, and it is what Convertu’s Base64 encoder and decoder applies. Two traps still crop up regularly:

• in JavaScript, btoa() throws an InvalidCharacterError as soon as a character goes above code point 255; convert the text to bytes with TextEncoder before calling the function;
• in a word processor or a spreadsheet, a curly apostrophe or a non-breaking space slips into the string and throws off everything that follows. Pasting the value back as plain text sorts it out.

Standard or URL-safe: two alphabets, one decoder

The standard alphabet (RFC 4648, §4) uses + and /. Dropped into a URL, those two characters cause trouble: + is read back as a space in a query string, / splits the path, and the trailing = has to be escaped as %3D. Hence the base64url variant (§5): + becomes -, / becomes _, and the padding is usually dropped.

You come across it without realising in JWT tokens — three base64url segments separated by full stops — and in the code_challenge of OAuth PKCE. Working out which variant a string uses takes a second: a - or a _ marks URL-safe, a + or a / marks standard. Both families in the same string are a sign of corruption.

Not to be confused with percent-encoding for URLs, which answers a different question. Base64 makes binary data transportable as text; percent-encoding makes characters that are already text admissible inside a URL. The two are sometimes combined — standard Base64, then percent-encoding of the result — and that stacking explains a good many mysterious %2B sequences in server logs.

Data URIs and binary content

A data URI looks like data:image/png;base64,iVBORw0KGgo…: the file is written out in full inside the document. The benefit is real, one fewer HTTP request, but the sums turn against you fast. The image swells by 33%, it is no longer cached separately from the HTML or CSS that carries it, and it is downloaded again every time that document changes. Below roughly 2 KB — a small icon, a repeated pattern — the trade-off holds up. Above that, a separate file remains the better option; and for an icon, inline SVG is almost always lighter than the same drawing as an encoded PNG.

The other way round, a Base64 string does not necessarily hold text. The first few characters are often enough to guess what you are holding:

• iVBORw0KGgo — a PNG image
• /9j/ — a JPEG image
• JVBERi0 — a PDF
• UEsDB — a ZIP archive, so also a DOCX, an XLSX or an ODT
• R0lGOD — a GIF

Pasting that sort of string into a decoder that can only display text produces a mush of characters. A decent decoder spots the binary and offers a download rather than an unreadable dump.

When decoding fails: the diagnosis

Decoder error messages are terse. Here are the real causes, in decreasing order of frequency.

The length does not add up

A valid Base64 string has a length that, divided by 4, leaves a remainder of 0, 2 or 3. A remainder of 1 is mathematically impossible: the string has been truncated, most often by a copy-and-paste that stopped at the edge of an input box, or by a database field that was too short. A remainder of 2 or 3 signals nothing worse than missing padding, which is repaired by adding the missing = signs.

Spaces, line breaks, a prefix

A string taken from a PEM file or a MIME header arrives split across lines. A tolerant decoder strips that whitespace; a strict decoder refuses it. Same story for the data:image/png;base64, prefix: it belongs to the URI, not to the payload, and has to be removed before decoding.

Double encoding

If the result of decoding is itself a run of letters and digits ending in one or two = signs, the value was encoded twice — a classic when one application layer encodes what another had already encoded. Simply run the operation again on the result.

The string came out of broken JSON

Base64 payloads very often travel inside a JSON field, where they sit alongside escaped quotation marks and \n sequences. If the response was truncated or badly reassembled, so is the string. Running the document through a JSON formatter lets you confirm that the container is valid first, before hunting for the fault in the content.

Convertu’s encoder and decoder works entirely inside your browser: neither the text nor the file leaves your machine, which matters when the payload is a certificate or an authentication token. It takes files as well as pasted text, up to the maximum size shown on the tool page, recognises both alphabets when decoding without you having to say which is which, and tolerates whitespace, missing padding and the data: prefix. Past a certain size the on-screen preview is truncated so the tab does not freeze, but copying and downloading do cover the complete result.