URL Encoding Explained: RFC 3986, Percent-Encoding, and Common Pitfalls

URL encoding — also called percent-encoding — is one of those web fundamentals that every developer touches daily but few deeply understand. It's why https://example.com/search?q=hello world becomes https://example.com/search?q=hello%20world. It's why name=O'Brien causes server-side bugs if you don't encode the apostrophe. And it's the source of countless subtle bugs and security holes.

This guide explains percent-encoding from scratch — what it is, why it exists, which characters must be encoded, and the most common pitfalls when implementing it. By the end, you'll know exactly when to use encodeURIComponent vs encodeURI (and why this matters for security).

What Is Percent-Encoding?

Percent-encoding is the mechanism for representing characters that have special meaning (or are not allowed) in a URL by encoding them as a % followed by two hexadecimal digits.

Example: a literal space is encoded as %20 because spaces are not allowed in URLs (URLs can't contain unescaped spaces). The character ? is encoded as %3F when it appears in a query value but NOT when it separates the path from the query string.

Percent-encoding is defined in RFC 3986 (the URI standard) and has been the universal mechanism since the early days of the web.

Why URLs Need Encoding

URLs were originally designed for the US-ASCII character set (alphanumerics + a small set of punctuation). But the web is global — your search query might contain Chinese characters, emoji, or accented letters. URLs need a way to represent any character from any alphabet.

Percent-encoding solves this by mapping every possible character to one of the allowed characters. The 95 allowed characters are the unreserved plus reserved characters (defined by RFC 3986).

The Three Character Classes

RFC 3986 divides URL characters into three classes, each treated differently during encoding.

1. Unreserved Characters

These characters are allowed in a URL without encoding:

• Letters: A–Z, a–z (52 characters)
• Digits: 0–9 (10 characters)
• Four punctuation marks: - (hyphen), . (period), _ (underscore), ~ (tilde)

Total: 66 characters. These never need encoding and shouldn't be encoded when they appear in their normal role.

2. Reserved Characters

These characters have special meaning in URLs as delimiters or separators. They must be encoded when they appear in data (not as delimiters):

• : — port separator
• / — path separator
• ? — query separator
• # — fragment separator
• [ ] — IPv6 literal markers
• @ — userinfo separator
• ! $ & ' ( ) * + , ; = — sub-delimiters

When the reserved character is being used as a delimiter, leave it alone. When it appears as data, encode it. So in /search?q=hello+world, the first / is a delimiter (leave it), the ? separates the query (leave it), but the + in the value is technically a sub-delimiter used as data — so strictly speaking, it should be %2B. (Many forms decode + as a space, which is a legacy form-encoding quirk.)

3. Everything Else

All other characters (including spaces, unicode, control characters) MUST be percent-encoded. This includes:

• Space (%20)
• Unicode characters (%E4%B8%AD for Chinese characters)
• ASCII control characters (%00–%1F)
• Special characters: < > " # % { } | / ^ ~ [ ] +

How Percent-Encoding Works (Bytes, Not Characters)

One subtle but critical detail: percent-encoding operates on bytes, not Unicode characters.

For an ASCII character like A (code point 65), the byte is 0x41. Encoded: %41. Easy.

For a Unicode character like é (code point 233), the UTF-8 bytes are 0xC3 0xA9. Encoded: %C3%A9. Note: NOT %E9, which is the Latin-1 encoding. This is a common bug.

For emoji like 🚀 (code point 128640), the UTF-8 bytes are 0xF0 0x9F 0x9A 0x80. Encoded: %F0%9F%9A%80. (4 bytes → 12 percent-encoded characters.)

JavaScript: encodeURIComponent vs encodeURI

JavaScript has two URL encoding functions, and confusing them is the most common pitfall. Here's the difference:

encodeURIComponent

Encodes every reserved character. Use this for component values like query string parameters or path segments.

encodeURIComponent('hello world')
// "hello%20world"

encodeURIComponent('name=O'Brien & friends')
// "name%3DO'Brien%20%26%20friends"

Note that ', !, (, ), ~, * are not encoded by encodeURIComponent (the function is technically non-strict against RFC 3986 but matches form-encoding conventions).

encodeURI

Encodes only characters that are illegal in URIs as delimiters. Use this for complete URLs.

encodeURI('https://example.com/search?q=hello world&filter=new')
// "https://example.com/search?q=hello%20world&filter=new"

encodeURIComponent('https://example.com/search?q=hello world')
// "https%3A%2F%2Fexample.com%2Fsearch%3Fq%3Dhello%20world"

Notice: encodeURI leaves /, ?, & intact because they're valid URL delimiters. encodeURIComponent encodes them too.

The Rule: Pick the Right One

• Encoding a complete URL? Use encodeURI — preserves delimiters.
• Encoding a value that goes into a URL? Use encodeURIComponent — encodes everything sensitive.
• Encoding a form submission? Use neither directly — use URLSearchParams or the browser's form machinery.

The Common Bug: Encoding a URL Twice

If you accidentally encode a URL twice, you get double-encoding. This is one of the most common bugs in production:

const url = 'https://api.example.com/users?name=O'Brien';
// Correct: encode only the value
'https://api.example.com/users?name=' + encodeURIComponent("O'Brien")
// → "https://api.example.com/users?name=O'Brien"  (the apostrophe encoded as %27)

// Bug: encoding the whole URL
encodeURIComponent(url)
// → "https%3A%2F%2Fapi.example.com%2Fusers%3Fname%3DO%27Brien" (BROKEN)

Security: Why This Matters

Improper URL encoding is the root cause of many web vulnerabilities. The three most common attack patterns:

1. XSS via URL Parameter Injection

If your server-side code doesn't properly encode URL parameters before outputting them in HTML, attackers can inject JavaScript:

// Server output, NOT encoded:
<p>Hello, ?name=O'Brien</p>

// Becomes dangerous if rendered unescaped:
<p>Hello, <script>alert(1)</script></p>

Defense: Always use a templating engine that auto-escapes HTML special characters (<, >, &, ", ').

2. SQL Injection via Unescaped URL Parameters

If you take URL parameters and put them directly in a SQL query without parameterization:

// User requests: /users?id=1' OR '1'='1
// Your code: SELECT * FROM users WHERE id = USERINPUT (UNSAFE!)
// → SQL injection!

Defense: Use parameterized queries (prepared statements) — never interpolate user input into SQL.

3. Open Redirects via URL-encoded Payloads

Servers that redirect to user-supplied URLs (e.g., after login) are vulnerable if the URL isn't validated:

// Attacker makes user click:
https://yoursite.com/login?redirect=https%3A%2F%2Fevil.com

// Your code blindly redirects → user goes to attacker's site

Defense: Validate that the redirect URL belongs to your domain (parse and check hostname, not string match).

Content Negotiation: Form vs URL Encoding

HTML forms traditionally use two encoding types:

application/x-www-form-urlencoded

The default. Uses URL encoding but with a quirk: spaces are encoded as + instead of %20. So hello world becomes hello+world.

Both plus signs and %20 decode to spaces in most servers. This causes the famous "plus signs in passwords" bug — if your password has a + and you send it via a form, it becomes a space on the server.

multipart/form-data

Used for file uploads. Each form field is sent as a separate MIME part with its own encoding. No special +-for-space treatment.

Decoding: Watch Out For These

When your server receives a URL-encoded value, decoding it wrong can cause vulnerabilities:

Bug 1: Decoding Twice

If your server stores URL-encoded values and then decodes them when reading, decoding twice causes %2520 to become a literal %20 instead of a space.

Bug 2: Decoding Then Re-encoding

If a value goes through encoding → decoding → re-encoding, you can get case-folding bugs (%2f vs %2F) or double-encoding.

Bug 3: Ignoring Invalid UTF-8

Some decoders silently replace invalid UTF-8 with replacement characters (U+FFFD) or ?. An attacker can use this to bypass input validation that checks for specific byte sequences.

Practical Examples

JavaScript: Build a search URL safely

// User typed into search box:
const query = 'hello world';

// Build the URL safely:
const url = 'https://api.example.com/search?q=' + encodeURIComponent(query);
// "https://api.example.com/search?q=hello%20world"

// For multiple params, use URLSearchParams:
const params = new URLSearchParams();
params.set('q', query);
params.set('limit', '10');
const fullUrl = 'https://api.example.com/search?' + params.toString();
// "https://api.example.com/search?q=hello+world&limit=10"

Python: URL-encoded parsing

from urllib.parse import quote, quote_plus, urlencode

# Component value (use quote)
quote('hello world')  # 'hello%20world'
quote("O'Brien")      # "O%27Brien"

# Form-encoded (use quote_plus)
quote_plus('hello world')  # 'hello+world'

# Build multiple params:
urlencode({'q': 'hello world', 'limit': '10'})
# 'q=hello+world&limit=10'

Node.js: Decode parameters safely

const query = 'q=hello%20world&filter=new%26old';
const params = new URLSearchParams(query);
console.log(params.get('q'));     // 'hello world'
console.log(params.get('filter')); // 'new&old'

Try It Yourself

Need to encode a tricky string? Use our HTML Entities tool for character escaping, or try our free Regex Tester to extract URLs and parameters from messy content.

Building a public API? Encode all values with our JSON Formatter + strict server-side parsing. The most common security bug in modern web APIs is failing to URL-encode user input before using it in a SQL or template context.

Working with JWTs (which use URL-safe Base64)? Read our JWT guide for the related encoding conventions.