JIT and warm-up

Java starts slow and becomes fast. The same method, called with the same input, can take five times longer the first time than the thousandth — not because anything changed in your code, but because the JVM was still deciding how to run it. That process, just-in-time compilation, is why Java services reach speeds close to C in steady state, why they have a latency spike after every deploy, and why a benchmark that runs a loop once measures almost nothing.

The measurements here are from Java 21, in a container limited to 4 CPUs.

#Interpreter, C1, C2

A method's bytecode first runs in the interpreter: correct, and slow. The JVM counts how often each method is called and how often its loops run, and compiles the busy ones to machine code in stages — tiered compilation:

-XX:+PrintCompilation shows it happening. For the method in this lesson's example:

50   83       3       Warmup::priceOrder (58 bytes)
52   86       4       Warmup::priceOrder (58 bytes)
54   83       3       Warmup::priceOrder (58 bytes)   made not entrant

At 50 ms it was compiled at tier 3 — C1 with profiling. Two milliseconds later, with a profile in hand, C2 compiled it at tier 4, and the tier-3 version was made not entrant: new calls go to the better code, and the old code is discarded once nothing is running in it.

#Warm-up, measured

The method sums fifty line items using a small immutable Money record, the way domain code often does:

record Money(long cents, String currency) {
    Money plus(Money o) { return new Money(cents + o.cents, currency); }
}
 
static long priceOrder(long[] lines) {
    Money total = new Money(0, "INR");
    for (long c : lines) total = total.plus(new Money(c, "INR"));
    return total.cents();
}

Called in batches of 20,000, timing each batch. Two runs, nanoseconds per call:

batch:   1    2    3   4   5   6   7   8   9  10   11   12
run 2: 347   80  120  68  58  79  70  69  70 114  232  245
run 3: 322   77  122  68  60  76  72  72  71 116  210  205

The first batch is four to five times slower than the steady state of about 60–70 ns, and it is an average: the calls made before any compilation finished are hidden inside it. Most of the speed arrived by the second batch. The later rises to around 200 ns appear in both runs and have causes the batch timings cannot separate — garbage collection, recompilation, or the machine — which is exactly why careful benchmarks do not rely on hand-written timing loops.

A third run was slower throughout — 1,042 ns for the first batch and 500–800 ns for most of the rest — with nothing changed in the program. Something else on the laptop was competing for the CPU. It is left out of the table above and mentioned here because it is the most useful result of the three: a single timing run on a shared machine can be wrong by a factor of ten.

For comparison, with the JIT disabled entirely:

-Xint (interpreter only): 6348 6498 6223 6645 6300 6453 6231 6256 6247 6244 6176 6180

About 6,200 ns per call, roughly a hundred times slower than compiled code. That is also what a service falls back to if its code cache fills and compilation stops.

#Escape analysis

Look again at priceOrder. Every iteration creates two Money objects: one for the line item, one for the new total. Fifty lines is 101 objects, 24 bytes each: 2,424 bytes per call. Allocation measured per call, at steady state:

default                 : 1224 bytes allocated per call, ~60–70 ns
-XX:-DoEscapeAnalysis   : 2424 bytes allocated per call, ~230–330 ns
-Xint                   : 2424 bytes allocated per call, ~6,200 ns

With escape analysis disabled, or in the interpreter, every object is allocated: 2,424 bytes. With it on, C2 allocated 1,224 bytes — it removed half the allocations, and the method ran several times faster.

Escape analysis is C2 proving that an object never leaves the method — is not stored in a field, returned, or passed to code it cannot see. Such an object does not need to exist on the heap at all: C2 replaces it with its fields as local variables (scalar replacement). Here, the new Money(c, "INR") for each line item is inlined into plus and never escapes, so it is eliminated. The running total is reassigned around the loop, from one iteration to the next, and C2 does not eliminate that one. The count matches: 1,224 bytes is 51 objects.

The practical lesson is not "write code for escape analysis". It is that small, short-lived objects in hot code are often cheaper than they look — and that whether they are eliminated depends on inlining, which depends on method size and call-site shape, so it must be measured rather than assumed.

#Inlining, and deoptimisation

Inlining — copying a called method's body into its caller — is the optimisation that enables most others, including escape analysis. C2 inlines small methods and hot call sites, and for interface or virtual calls it uses the profile: if only one implementation has ever been seen at a call site, it inlines that one, guarded by a cheap type check.

That is a bet, and bets can lose. A method that calls Discount.apply a few million times with only NoDiscount, and then meets a second implementation:

36   81       3       Warmup::applyAll (46 bytes)
36   82       4       Warmup::applyAll (46 bytes)
37   81       3       Warmup::applyAll (46 bytes)   made not entrant
--- a second implementation arrives
55   82       4       Warmup::applyAll (46 bytes)   made not entrant
55   87 %     4       Warmup::applyAll @ 13 (46 bytes)

The C2 version (compile id 82) was made not entrant the moment the second type arrived: the guard failed, execution deoptimised back to the interpreter, and the method was compiled again (id 87 — the % marks an on-stack replacement compile of the loop) with the new profile. Deoptimisation is normal and usually brief. It becomes visible as a latency blip when a rarely used code path — the first refund, the first request from a new client type — hits a hot method that had been optimised for everything else.

#The spike after every deploy

A freshly started service runs its hot paths in the interpreter and C1, at many times its eventual speed, while C2 compiles in the background on CPU the requests also need. The result is a well-known pattern: p99 latency spikes for the first minutes after each deploy or restart, then settles.

Ways to soften it:

• Warm up before taking traffic. Send representative requests to a new instance before its readiness check passes, so the hot paths are compiled when real users arrive.
• Roll out gradually, so new instances take a small share of traffic while they warm.
• Give the container enough CPU at startup — compilation is CPU work, and a tightly limited container compiles slowly.
• Reduce the work: class data sharing archives cut class loading, and Java's newer ahead-of-time features reduce warm-up further.

#Honest benchmarks: JMH

Every measurement in this lesson has problems a real benchmark must avoid: warm-up mixed into results, the JIT possibly eliminating work whose result is unused, garbage collection landing in one batch and not another, a single JVM run on a noisy machine. JMH, the Java Microbenchmark Harness from the OpenJDK project, handles these: it runs separate warm-up and measurement iterations, forks fresh JVMs so profiles from one benchmark do not pollute another, provides Blackhole to consume results so the JIT cannot discard the work, and reports error margins.

The rule to carry: a Java benchmark without warm-up, forks and repetition measures the JIT's progress, not your code.