Zum Hauptinhalt springen
Architektonische Schaltstelle Unicode 17.0 / WHATWG 6 Kernthemen 10 Schlüsselunterscheidungen

Zeichenkodierungen und Darstellungen

Die Entwickler-Schaltstelle für binäre Textkodierungsformen und Quell-Markup-Escapes. Verstehen Sie, wie abstrakte Unicode-Codepunkte in physische Bytes übersetzt werden, vergleichen Sie Speicher-Kompromisse und inspizieren Sie deterministische Darstellungen in modernen Web- und Betriebssystem-Laufzeitumgebungen.

Kodierungsthemen und Darstellungsformate

Direkter Zugriff auf spezialisierte technische Anleitungen und Spezifikationen. Wählen Sie ein Format, um Byte-Layouts, Transformationsregeln und Codebeispiele zu inspizieren.

Unicode Encoding Form RFC 3629 / Unicode Standard §3.9

UTF-8 Encoding

Codeeinheit / Breite: 8-bit Code Units (1–4 Bytes)

The universal, variable-width encoding for modern web and software systems. 100% backward-compatible with ASCII, using 1 to 4 bytes per Unicode character.

Unicode Encoding Form RFC 2781 / Unicode Standard §3.8

UTF-16 Encoding

Codeeinheit / Breite: 16-bit Code Units (2 or 4 Bytes)

Variable-width encoding using 1 or 2 16-bit code units. Widely utilized in operating system runtime environments (Windows Win32, Java, JavaScript engines) with surrogate pairs.

Unicode Encoding Form Unicode Standard §3.10

UTF-32 Encoding

Codeeinheit / Breite: 32-bit Code Units (4 Bytes Fixed)

Fixed-width encoding using exactly one 32-bit code unit per Unicode scalar value. Offers constant-time indexing for raw code points, but uses quadruple the memory of ASCII.

Markup Escape Syntax WHATWG HTML Living Standard §13 & §14

HTML Entities & References

Codeeinheit / Breite: ASCII Text Tokens

Source-level markup syntax (€, €, €) used in HTML and XML to author characters without collision with document tags or character set limitations.

URI Octet Escaping IETF RFC 3986 / WHATWG URL Standard

URL Percent-Encoding

Codeeinheit / Breite: %HH Hex Byte Triplets

Byte-level URI escape mechanism replacing non-ASCII or reserved octets with percent-encoded byte triplets (%E2%82%AC). Operates on UTF-8 bytes in modern standards.

Coded Character Set The Unicode Standard (ISO/IEC 10646)

Unicode Code Points

Codeeinheit / Breite: Integer Code Points (U+0000–U+10FFFF)

The universal coded character set numbering layer mapping 149,000+ abstract characters to integer code points across 17 planes, independently of byte storage.

Technische Vergleichsmatrix

Technischer Direktvergleich von Zeichenkodierungsformen und Escape-Formaten auf Quellcode-Ebene nach Bittiefe, Byte-Grenzen und primären System-Laufzeitumgebungen.

Format Klassifikation Hauptzweck Codeeinheitsgröße Speichergrenzen Euro (€)-Darstellung Primärer Systemkontext Referenz
UTF-8 Unicode Encoding Form Universal text storage & network serialization 8-bit Code Unit (1 Byte) 1 to 4 bytes (variable) E2 82 AC 3 bytes Web (HTML/CSS/JS), JSON, APIs, text files, Linux/macOS UTF-8 Reference →
UTF-16 Unicode Encoding Form In-memory string representation in runtime engines 16-bit Code Unit (2 Bytes) 1 or 2 code units (2 or 4 bytes) 20AC 1 unit (2 bytes) Windows Win32, Java, JavaScript strings, C# .NET UTF-16 Reference →
UTF-32 Unicode Encoding Form Constant-time code point array indexing 32-bit Code Unit (4 Bytes) 1 code unit (4 bytes fixed) 000020AC 1 unit (4 bytes) Unix wchar_t, internal parser buffers (rare on web) UTF-32 Details →
HTML Entities Markup Escape Syntax Source-level escaping inside HTML/XML documents ASCII Text Tokens Variable string length € or € 6 to 8 ASCII chars HTML source documents, preventing markup tag collisions HTML Entities Guide →
URL Encoding URI Octet Escaping Transmitting non-ASCII/reserved bytes in URIs %HH Hex Triplets 3 ASCII chars per UTF-8 byte %E2%82%AC 9 ASCII chars HTTP query strings, URI paths, form data submission URL Encoding Guide →

UTF-32 verstehen: feste Breite gegen Speicher-Kompromiss

UTF-32 (UCS-4) verwendet genau eine 32-Bit-(4-Byte-)Codeeinheit für jeden Unicode-Skalarwert (0x00000000 bis 0x0010FFFF). Dies garantiert konstanten O(1)-Direktzugriff über den Codepunkt-Index ohne Ersatzpaare.

Warum UTF-32 für den Austausch im Web selten verwendet wird: ASCII-Zeichen, die in UTF-8 nur 1 Byte benötigen, verbrauchen in UTF-32 volle 4 Bytes (eine 300%ige Speicherausweitung). Darüber hinaus garantieren feste 32-Bit-Einheiten nicht, dass 1 Einheit 1 sichtbarem Zeichen entspricht, da komplexe Graphem-Cluster (kombinierende Akzente, Flaggen, Emoji mit Hauttönen) weiterhin mehrere Codepunkte erfordern.

Zeichen- und Byte-Inspektor

Inspizieren Sie Live-Byte-Darstellungen in UTF-8, UTF-16, UTF-32, HTML-Entitäten und URL-Prozentkodierung. Geben Sie ein beliebiges Zeichen ein oder wählen Sie unten einen kuratierten Standard.

Aktualisiert Darstellungen in Echtzeit über UTF-8-Bytes, UTF-16-Surrogatpaare, HTML-Referenzen und URL-Kodierung.

Voreinstellungen:

Euro Sign

U+20AC Dez: 8364 Basic Multilingual Plane (BMP, Plane 0)

Standard BMP character demonstrating 3-byte UTF-8, single 16-bit UTF-16 unit, named HTML entity €, and URL percent-encoded bytes.

UTF-8-Bytesequenz 3 Bytes
E2 82 AC
UTF-16-Codeeinheiten 1 Einheit (2 Bytes)
20AC
UTF-32-Hex 1 Einheit (4 Bytes fest)
000020AC
HTML-Zeichenreferenzen Quell-Escapes
Benannt: €
Dezimal: €
Hex: €
URL-Prozentkodiert RFC-3986-Oktetts
%E2%82%AC
Escapes von Programmiersprachen Compiler-Literale
JavaScript: \u20AC
Python: \u20AC
CSS: \0020AC

Grundkonzepte und Terminologie-Disambiguierung

Wesentliche architektonische Unterscheidungen, die branchenübliche Verwirrungen über Unicode-Repertoires, Codeeinheiten, Surrogatmechanik, Escapes und Normalisierung auflösen.

Fundamental Rule Unicode Standard 17.0 §3.4 & §3.9

Unicode ≠ UTF-8: Repertoire vs. Encoding Form

Unicode is the universal numbered catalog of abstract characters (assigning integer code points U+0000 through U+10FFFF). UTF-8 is the mathematical encoding form that translates those code points into 1 to 4 eight-bit bytes for physical storage and network transport.

Kernregel: Unicode defines WHAT the characters are; UTF-8 defines HOW those characters are stored as bytes.
Text Processing Unicode Standard 17.0 §2.4 & UAX #29

Code Point vs. Code Unit vs. Byte vs. Grapheme Cluster

A code point is a Unicode number (e.g. U+1F600). A code unit is the minimal building block of an encoding form (8-bit for UTF-8, 16-bit for UTF-16, 32-bit for UTF-32). A byte is 8 bits. A grapheme cluster is a user-perceived character on screen that can span multiple code points.

Kernregel: Never assume one code unit equals one character. Combining marks, skin-tone modifiers, and ZWJ emoji join multiple code points into a single grapheme cluster.
Surrogate Mechanics RFC 2781 / Unicode Standard §3.8

UTF-16 Surrogate Pairs: One Character, Two Code Units

Characters outside the Basic Multilingual Plane (> U+FFFF, such as emoji and historical scripts) cannot fit in a 16-bit code unit. UTF-16 encodes them as a surrogate pair: a High Surrogate (0xD800–0xDBFF) followed by a Low Surrogate (0xDC00–0xDFFF). Together, the pair represents a single character.

Kernregel: A surrogate pair is NOT two characters. Individual surrogate code points are invalid on their own and cannot appear in valid UTF-8 or UTF-32.
Memory & Indexing Unicode Standard 17.0 §3.10

UTF-32 Fixed Units vs. Grapheme Clusters

UTF-32 assigns exactly one 32-bit code unit per Unicode scalar value. While this makes code point array indexing fast and constant-time, it does not guarantee one 32-bit unit equals one user-perceived glyph because compound grapheme clusters still require multiple code points.

Kernregel: UTF-32 provides fixed-width code points, NOT fixed-width visual characters.
Web Authoring WHATWG HTML Living Standard §13.1.4

HTML Character References Are Markup Escapes, Not Encodings

HTML entities (numeric decimal €, hex €, and named €) are source-code representations interpreted by the HTML parser. The document itself still possesses an underlying character encoding (almost universally UTF-8). Entities allow authoring characters without syntax collisions.

Kernregel: HTML entities exist in HTML source text; they are not an alternative binary encoding format.
URI Protocol RFC 3986 §2.1 & WHATWG URL Standard

URL Percent-Encoding Operates on Bytes, Not Code Points

RFC 3986 percent-encoding replaces disallowed octets with % followed by two hexadecimal digits (%HH). In modern web architectures, Unicode text is first converted to UTF-8 bytes, and each non-ASCII or reserved byte is then percent-encoded (%E2%82%AC for Euro).

Kernregel: URL encoding escapes bytes, not characters. Context also matters: spaces become %20 in URI paths, but application/x-www-form-urlencoded forms often use +.
Equivalence UAX #15: Unicode Normalization Forms

Unicode Normalization (NFC / NFD) Is Not Encoding

Unicode normalization forms (NFC, NFD, NFKC, NFKD) transform equivalent sequences of code points between precomposed single characters (e.g. é U+00E9) and decomposed combining sequences (e.g. e U+0065 + acute accent U+0301).

Kernregel: Normalization changes which code points are used; character encoding determines how those code points become bytes.
Syntax Escapes ECMAScript §12.8.4 & W3C CSS Syntax §4.3

Programming Escapes (\uXXXX) Are Not Encodings

Notations such as \u20AC in JavaScript/Java, \u{20AC} in PHP/Rust, or \0020AC in CSS are compiler string literal escape syntaxes. They instruct the language runtime which code point to instantiate in memory, but are not binary file encodings.

Kernregel: String escapes are source code conveniences; binary encodings are physical byte formats.
Data Transport IETF RFC 4648

Base64 Is Binary-to-Text, Not a Character Encoding

Base64 (RFC 4648) translates arbitrary raw binary octets into a safe 64-character ASCII representation (A–Z, a–z, 0–9, +, /) to prevent transmission corruption across text-only legacy protocols (such as MIME email).

Kernregel: Character encodings turn text into bytes; Base64 turns arbitrary bytes back into ASCII text.
Byte Order Unicode FAQ: UTF-8, UTF-16, UTF-32 & BOM

UTF-8 BOM Is a Signature, Not an Endian Indicator

Because UTF-8 code units are 8-bit octets, byte order (endianness) does not exist in UTF-8. A Byte Order Mark (0xEF 0xBB 0xBF) in UTF-8 acts solely as an encoding signature. Modern web standards strongly discourage UTF-8 BOMs. Endianness (LE vs. BE) applies strictly to UTF-16 and UTF-32.

Kernregel: UTF-8 never requires a BOM. Endianness matters only when code units exceed 8 bits.

Kodierungs- und Konvertierungswerkzeuge

Spezialisierte client- und serverseitige Hilfsmittel auf CopyCharacter zum Umwandeln von Text, Escapen von Entitäten und Inspizieren von Codepunkten.

Markup Tool

HTML Entity Converter

Encode special characters into HTML named or numeric decimal/hex entities, or decode entity strings back to readable text.

URI Tool

URL Encoder & Decoder

Convert text to RFC 3986 percent-encoded URI strings (or form application/x-www-form-urlencoded), and decode escaped query parameters.

Normalization

Unicode Normalizer

Transform text across Unicode normalization forms (NFC, NFD, NFKC, NFKD) to resolve combining characters and ensure binary equivalence.

Analysis

Unicode Inspector

Analyze strings character-by-character to inspect code points, character categories, blocks, and invisible zero-width characters.

Developer Tool

Unicode Encoder

Convert characters into programming language escape sequences including JavaScript (\uXXXX), Python, CSS, and C/C++.

Developer Tool

Unicode Decoder

Parse raw Unicode escape sequences (\uXXXX, \U0001XXXX, U+XXXX) back into live rendered characters and symbols.

Maßgebliche Standards und Spezifikationen

Alle Zeichendefinitionen, Byte-Transformationen und Entitätsdarstellungen auf CopyCharacter entsprechen strikt den folgenden internationalen Standards.

Unicode Consortium

The Unicode Standard, Version 17.0

Chapter 3 defines conformance and formal Character Encoding Forms (UTF-8 in §3.9, UTF-16 in §3.8, UTF-32 in §3.10).

WHATWG

Encoding Living Standard

Specifies interoperable character encoding algorithms for web browsers, establishing UTF-8 as the mandatory web standard.

IETF

RFC 3629: UTF-8 Transformation Format

Restricts UTF-8 to a maximum of 4 bytes per character, aligning with the 21-bit Unicode codespace (U+0000 to U+10FFFF).

IETF

RFC 2781: UTF-16 Encoding of ISO 10646

Formalizes UTF-16 code unit layout and the mathematical algorithms for computing high and low surrogate pairs.

IETF

RFC 3986: Uniform Resource Identifier (URI)

Defines generic syntax and percent-encoding (%HH) rules for escaping reserved and non-ASCII octets in URIs.

WHATWG

HTML Living Standard §13 & §14

Defines HTML character references (numeric decimal, numeric hex, and the dictionary of 2,231 named character entities).