Stop Hardcoding Instruction Costs: Using LLVM TargetTransformInfo

The Problem with Static Cost Assumptions

When writing a custom LLVM optimization pass, you often reach a decision point: Is it cheaper to perform this operation in a loop, or should I unroll it? Is a multiplication significantly more expensive than a shift on this target?

The temptation is to hardcode these assumptions—for example, assuming a multiply takes four times longer than an add. However, this approach breaks as soon as your code targets a different architecture. An operation that is "cheap" on x86-64 might be "expensive" on AArch64, or vary wildly between a high-performance server CPU and an embedded microcontroller. Hardcoding these values turns your optimization pass into a liability that may degrade performance on unsupported targets.

The solution is TargetTransformInfo (TTI). TTI is an analysis interface that allows your pass to query the target-specific costs of instructions and architectural preferences without needing to know which CPU the code is actually targeting.

How TTI Abstracts Hardware Costs

TTI does not provide a raw cycle count from a hardware manual. Instead, it provides an InstructionCost object. This is a heuristic value that represents the relative weight of an operation. These costs are categorized into levels such as Tiny, Zero, Base, and Free.

By using these relative values, your pass can make logic-based decisions (e.g., if (costA < costB)) that remain valid across different backends. TTI handles the heavy lifting of mapping the LLVM Intermediate Representation (IR) to the actual capabilities of the target CPU, taking into account flags passed via -mcpu and -mattr.

Key TTI Query Capabilities

• Arithmetic Costs: Use getArithmeticInstrCost to compare the weight of additions, multiplications, or divisions.
• Memory Operations: Use getMemoryOpCost to determine the relative penalty of loads and stores.
• Vectorization: Query preferred vector widths and alignment requirements to avoid generating inefficient SIMD code.
• Loop Preferences: Access getUnrollingPreferences to see if the target architecture favors specific unroll factors.

Implementation: Querying Costs in the New Pass Manager

In the New Pass Manager (NPM), TTI is treated as an analysis. You do not instantiate it yourself; instead, you request it from the AnalysisManager. This ensures that the TTI instance is correctly configured for the current module's target triple.

// Example: Comparing two instructions in a custom NPM pass
PreserveSet preserve; 
auto TTI = AM.getResult<TargetTransformInfo>(F); 

Instruction *instrA = ...; // e.g., a mul instruction
Instruction *instrB = ...; // e.g., a series of shifts/adds

InstructionCost costA = TTI->getInstructionCost(instrA, 1);
InstructionCost costB = TTI->getInstructionCost(instrB, 1);

if (costA > costB) {
    // Perform the transformation to replace A with B
}

Execution Context: This code runs within the run method of your Pass implementation. You must have the AnalysisManager (AM) available in your pass signature. The 1 passed to getInstructionCost represents the number of times the instruction is executed (the trip count), allowing TTI to scale the cost linearly.

Critical Trade-offs and Limitations

While TTI is powerful, it is not a cycle-accurate simulator. Keep these limitations in mind:

• Heuristics vs. Reality: TTI provides estimates. It does not account for real-time hardware effects like cache misses, branch mispredictions, or pipeline stalls.
• Invalid Costs: Some instruction-target combinations are unsupported or cannot be costed. TTI will return InstructionCost::Invalid. Your pass must explicitly check for this state to avoid making incorrect optimization decisions based on a zero or null value.
• Read-Only Nature: TTI is a read-only analysis. It can tell you how expensive an operation is, but it cannot modify the IR. You still need your pass logic to perform the actual mutations.

Verifying TTI Behavior

To verify that your pass is responding correctly to TTI, you should test it against two different target triples. You can do this using the opt tool:

1. Run your pass with -mtriple=x86_64-pc-linux-gnu -mcpu=skylake.
2. Run the same pass with -mtriple=aarch64-linux-gnu -mcpu=cortex-a53.

If your pass is correctly utilizing TTI, you should see different optimization decisions (or different cost logs) for the same input IR, reflecting the architectural differences between x86 and ARM.