Why Data.Text Beats String in Haskell: A Practical Guide

Problem: The Pain of Haskell’s Default String

In Haskell, String is defined as [Char] – a linked list of Unicode code points. While this is convenient for learning, it has a hidden cost: every string operation traverses a list, leading to O(n²) behavior for concatenation and high memory consumption because each Char is a separate heap object. In a real‑world service that processes logs, user input, or large text files, these inefficiencies quickly become bottlenecks.

Thesis: Data.Text is the De‑Facto Standard for Performance‑Critical Text

The Data.Text library stores text in a compact, contiguous array (either UTF‑8 or UTF‑16, depending on the build). Operations on Text are linear, and the library is heavily optimized by GHC. Replacing String with Text generally yields:

• ~5× faster word‑counting for a 100 MB file
• ≈70% reduction in heap allocation
• Better cache locality and fewer GC pauses

Concrete Example: Word Frequency Counter

Below is a minimal program that reads a file, counts word frequencies, and prints the top 10 words. The code is shown twice: once using String and once using Text.

# String version
import System.Environment (getArgs)
import Data.List (sortOn)
import qualified Data.Map.Strict as M

main :: IO ()
main = do
  [file] <- getArgs
  content <- readFile file
  let words' = words content
      freq   = M.fromListWith (+) [(w,1) | w <- words']
      top10  = take 10 $ sortOn (negate . snd) $ M.toList freq
  mapM_ print top10

# Text version
import System.Environment (getArgs)
import Data.List (sortOn)
import qualified Data.Map.Strict as M
import qualified Data.Text as T
import qualified Data.Text.IO as TIO
import qualified Data.Text.Encoding as TE

main :: IO ()
main = do
  [file] <- getArgs
  content <- TIO.readFile file
  let words' = T.words content
      freq   = M.fromListWith (+) [(w,1) | w <- words']
      top10  = take 10 $ sortOn (negate . snd) $ M.toList freq
  mapM_ print top10

Run each binary with the same input file and compare the results. The Text version should finish in roughly one‑fifth of the time and allocate far fewer bytes.

How to Measure the Difference

Use GHC’s profiler to confirm the gains:

1. Compile with optimizations: ghc -O2 -threaded -rtsopts wordfreq.hs
2. Run with +RTS -s -RTS to see allocation statistics.
3. Optionally, use criterion for a more precise benchmark.

Typical output (values are illustrative):

Metric	String	Text
Execution time	12.4 s	2.3 s
Allocated memory	1.2 GB	0.35 GB

Trade‑Offs and Practical Limits

While Text is faster, it is not a silver bullet:

• Conversion overhead – Converting between String and Text copies data. Use Text.pack and Text.unpack sparingly, ideally only once per data source.
• Library support – Some older packages still expose String APIs. In those cases, wrap the library in a Text layer or use Data.Text.Encoding to stay consistent.
• Lazy vs strict – Data.Text is lazy by default, but you can force strictness with Data.Text.Strict or using BangPatterns to avoid memory leaks in tight loops.

Actionable Takeaway

1. Audit your codebase for String usage. Tools like ghc --make -Werror can surface String types in signatures.

2. Replace String with Text in performance‑critical paths: file I/O, network protocols, log parsing, and data serialization.

3. Profile the changed module with +RTS -s to verify that allocations drop and that conversion costs are negligible.

4. For legacy libraries, write thin adapters that convert to/from Text only once per external boundary.

By following these steps, you’ll unlock significant speed and memory savings while keeping your Haskell codebase modern and maintainable.