Indic scripts / Devanagari
Devanagari text in a PDF
Hindi, Marathi, Nepali and Sanskrit are written in a script where the order characters are stored in is not the order they are drawn in.
A font with the characters is not enough
Devanagari needs a shaping engine, and the font has to carry the tables that drive it. Three things happen between your string and the page. Consonant clusters combine into conjunct forms that are single glyphs rather than two letters side by side. The vowel sign for short i is stored after the consonant it modifies and is drawn before it, so the visual order differs from the logical order. And a syllable-initial ra becomes a reph, a mark that is lifted above the following consonant cluster. None of that is in the character data: it lives in the font's GSUB and GPOS tables, and a font without them has every glyph and can assemble none of them.
What the failure looks like
Not tofu. That is what makes it hard to spot: the text renders, every character is present, and the words are wrong. Conjuncts appear as separate letters with a visible virama between them, the short i vowel sits after its consonant instead of before it, and a reader of the language sees something between an accent and nonsense while a reader who does not sees text.
Line breaking
Devanagari uses spaces between words, so ordinary breaking works. The subtlety is that a syllable cluster is a single unit and must never break internally, which is handled correctly by default and broken by any code that truncates a string by character count.
The markup and the CSS
The language attribute is doing real typographic work in every one of these examples. It is what lets the engine choose a regional glyph form, a case mapping or a hyphenation dictionary, and leaving it off is the most common single omission in a multilingual document.
<!-- The lang attribute is what lets the engine select the right
shaping behaviour and the right locale forms. It is not decoration. -->
<p lang="hi" class="devanagari">नमस्ते, आपका बिल तैयार है।</p>
<style>
@font-face {
font-family: "Noto Sans Devanagari";
src: url("https://assets.example.com/fonts/noto-sans-devanagari.woff2")
format("woff2");
/* Do NOT subset this face by unicode-range alone. Subsetting tools that
keep only the code points you used will happily drop the GSUB and GPOS
tables that assemble conjuncts, and the result renders without error. */
}
.devanagari {
font-family: "Noto Sans Devanagari", sans-serif;
/* Devanagari has a headline (the shirorekha) and marks above and below it.
A line-height set for Latin clips the reph on the line above. */
line-height: 1.75;
}</style>The mistake people make
Subsetting the font with a tool configured for Latin. It keeps the code points that appear in your text and discards the layout tables, because for Latin those tables are mostly optional. The subset is smaller, it renders, nothing errors, and every conjunct in the document is broken. Test a subset by rendering a word with a conjunct in it, not by checking the file size.
How to test this without reading the language
Render a string you know is correct, and compare it against a reference rendering from a system that definitely handles the script, such as a browser on a machine with the language installed. Comparing against nothing tells you only that characters appeared, and for most of the scripts on this site characters appearing is not the same as text being right. Where you can, get one native reader to look at one page once; it is faster than any amount of specification reading.
Frequently asked
Why does Devanagari look wrong when every character is there?
Not tofu. That is what makes it hard to spot: the text renders, every character is present, and the words are wrong. Conjuncts appear as separate letters with a visible virama between them, the short i vowel sits after its consonant instead of before it, and a reader of the language sees something between an accent and nonsense while a reader who does not sees text. The underlying reason is that a font is two things: a set of glyphs and a set of layout tables that say how those glyphs combine and position. Character coverage is the first half. Most of the failures on this page are the second half.
Does the lang attribute actually change the rendering?
Yes, and it is not an accessibility nicety here. It selects regional glyph forms where a code point has more than one correct shape, it picks the case mapping for a text transform, it chooses a hyphenation dictionary, and it triggers the locl OpenType feature. A document with no lang attribute gets whatever the font defaults to, which is usually right for one language and wrong for the others.
Can I subset the font to keep the file small?
Carefully, and not with a tool configured for Latin. Subsetters routinely drop the GSUB and GPOS tables that shape and position glyphs, because for Latin those are largely optional. The subset renders without error and every complex feature in it is broken. Test a subset by rendering real text and looking at it, never by checking that the file got smaller.
Related writing systems
Scripts with a related mechanism, and one from each of the other families.
Tamil text in a PDF
Tamil avoids the stacked conjuncts of Devanagari and instead has vowel signs that wrap around the consonant they belong to, which breaks anything that counts characters.
Thai text in a PDF
Thai is written without spaces between words and stacks marks up to three levels above the baseline, so both line breaking and line height need deciding rather than defaulting.
Vietnamese text in a PDF
Vietnamese is Latin with two diacritics stacked on one letter, and most Latin fonts contain the characters while positioning the second mark badly or not at all.
Japanese text in a PDF
Japanese shares code points with Chinese while wanting different glyph shapes for them, and has line-breaking rules that no default implements.
Cyrillic text in a PDF
Most Latin fonts include some Cyrillic, usually the Russian subset, and several languages written in Cyrillic need letters or letter shapes that subset does not have.
Splitting a font stack with unicode-range
The adjacent problem, covered in full there rather than repeated here.
Every writing system covered
The nine that needed a page, and the eight existing pages that cover the rest.
Paste a real string in this script into the playground and compare the output against a reference. It is the only test that means anything if you do not read the language.