Understanding TeX's Paragraph Breaking: How Knuth‑Plass Shapes Your Document

Problem: Uneven lines and overfull boxes

When compiling a TeX document you may notice that some lines are too tight, others too loose, or that TeX reports an overfull \\hbox warning. This happens because TeX must decide where to break each paragraph into lines.

Thesis: The Knuth‑Plass line‑breaking algorithm chooses breakpoints by minimizing a numeric badness score, and you can steer it with a few primitive parameters.

How the algorithm works

TeX scans a paragraph and treats every inter‑word space, hyphenation point, and explicit \\- as a potential breakpoint. For each breakpoint it computes a badness value that reflects how much the line’s glue (the flexible space between words) would have to stretch or shrink to reach the desired line width, plus any penalty you have inserted (e.g., \\hyphenpenalty). The algorithm then runs a dynamic‑programming pass that selects the sequence of breakpoints with the lowest total badness, subject to the current tolerance limits.

Controlling the break

• \\tolerance – maximum badness allowed in the second pass; lower values force tighter lines but increase the chance of overfull boxes.
• \\pretolerance – first‑pass tolerance; if a breakpoint sequence stays under this value TeX accepts it without looking at hyphenation.
• \\hyphenpenalty – penalty added for each hyphenated breakpoint; raising it discourages hyphenation.
• \\exhyphenpenalty – penalty for breaking after an explicit hyphen (e.g., in “self‑contained”).

All four are integer parameters; the default values in plain TeX are \\tolerance=200, \\pretolerance=0, \\hyphenpenalty=50, \\exhyphenpenalty=50.

Worked example: tightening a loose paragraph

Consider the following minimal file saved as test.tex:

\\tolerance=9999
\\pretolerance=9999
\\hsize=3in
This is a short demonstration paragraph that shows how TeX spreads words across a line when the tolerance is very high.
\\end

Run the engine where you have TeX installed (e.g., tex test.tex) with read/write permission in the current directory. The log will show no line‑break messages because the high tolerance lets TeX accept the first‑pass solution.

Now lower the tolerance to force TeX to look for better breakpoints:

\\tolerance=10
\\pretolerance=0
\\hsize=3in
This is a short demonstration paragraph that shows how TeX spreads words across a line when the tolerance is very high.
\\end

Re‑run tex test.tex. In the test.log you will see lines such as:

...\\tolerance=10
...\\pretolerance=0
... line 1-2-3 ...

Inspect the resulting DVI (dvips test.dvi -o test.ps) or PDF (if using pdfTeX) and you will notice that the lines are now tighter, with less inter‑word space, and possibly a hyphenation point if the paragraph were longer.

Trade‑off and limitation

Setting \\tolerance too low can produce overfull boxes when no breakpoint satisfies the badness limit; TeX then protrudes text into the margin and writes an overfull \\hbox warning. Conversely, a very high tolerance yields loose lines that may create visual “rivers” of white space. The algorithm optimizes only the numeric badness; it does not directly avoid rivers or balance column heights, so you may still need manual adjustments such as rewriting sentences, adjusting \\hyphenpenalty, or using micro‑typographic extensions in LuaTeX.

Actionable checklist

1. Add \\tracingparagraphs=1 to your source to get detailed breakpoint messages in the log.
2. Compile with tex -interaction=nonstopmode yourfile.tex (or pdfTeX/XeTeX/LuaTeX as appropriate).
3. Search the log for “@” lines that list each breakpoint and its badness; compare runs after changing a parameter.
4. If you see overfull boxes, raise \\tolerance or increase \\hyphenpenalty to allow more flexible breaking.
5. Keep a copy of the original log; diffing two logs shows exactly which breakpoints changed.

By treating the paragraph breaker as a tunable optimizer rather than a black box, you can predictably steer line tightness, hyphenation frequency, and avoid surprising overfull warnings.