Bytecode and the JIT

"Java is slow" was true in 1997 and is a misunderstanding today. The reason it stopped being true is the JIT — the just-in-time compiler — and understanding roughly what it does explains three things you will meet in production: why a service is slower in its first minute, why a microbenchmark that runs once is worthless, and why the JVM sometimes throws away code it already compiled. This lesson is the pipeline from interpreter to C2, the numbers that drive it, what an optimised method actually looks like, and how to watch all of it happen.

#Interpretation first

When the JVM starts executing a method, it interprets the bytecode: reads an instruction, does what it says, reads the next. Interpretation is slow — perhaps 10 to 50 times slower than native code — but it starts immediately and costs nothing to set up. For code that runs once (application startup, a configuration parser), interpretation is the right choice; compiling it would take longer than running it.

HotSpot's interpreter is a template interpreter: at startup it generates a small machine-code routine for each of the roughly 200 bytecodes, so "interpreting" is jumping between those templates. While it runs it also profiles: every method has an invocation counter, every loop has a back-edge counter, and every branch and call site records what it saw.

#Hot code gets compiled

When a counter crosses a threshold, the method is hot, and the JVM hands it to a compiler thread that produces real machine code for this CPU. Subsequent calls run the compiled version. HotSpot has two compilers and uses both, in what is called tiered compilation:

A loop on its own is a different threshold. On-stack replacement fires at 60,000 back-edges for tier 3 and 40,000 for tier 4 — and yes, tier 4's is the lower of the two: a loop hot enough to matter reaches C2 sooner than it reaches C1. Every number here came from java -XX:+PrintFlagsFinal -version, and they are the same on 8, 21 and 25.

They are also a floor rather than a constant. HotSpot scales every threshold by how long its compiler queues are (Tier3LoadFeedback, Tier4LoadFeedback), so a JVM that is busy compiling raises its own bar for what counts as hot — which is part of why start-up is slower than the table suggests, exactly when ten thousand classes are loading at once.

A hot method typically goes interpreter → C1 (tier 3) → C2 over its first thousands of calls. A hot loop inside a method that was called once gets the same treatment through on-stack replacement: the JVM compiles the loop and jumps into the compiled code mid-iteration, which is why a main with one giant loop still ends up native. Compiled code lives in the code cache (240 MB reserved by default with tiering), and a cache that fills up stops the compiler with a warning that is easy to miss and turns a fast service slow.

#What C2 does with a profile

The JIT is not just "compile to native". Because it runs while the program runs, it has information a static compiler never has: which branches are actually taken, which types actually flow through a call site, which methods are actually called from where. It uses that aggressively.

• Inlining. A call to a small method is replaced by the method's body. Getters, setters, equals, lambda bodies — gone as calls. This is the single most important optimisation, because it exposes the others. The limits are numbers you can look up: a method under 35 bytes of bytecode is inlined almost always; a hot one under 325 bytes usually; a bigger one, never, however hot. That is the mechanism behind "keep methods small": not style, but the inliner's budget.
• Monomorphic call sites. If a List reference at a given call site has only ever been an ArrayList, C2 compiles a direct call to ArrayList.get with a cheap type check, instead of a virtual dispatch. Two observed types is still fine (bimorphic). Three or more is megamorphic: a real virtual call through a table, and no inlining through it. An interface with many implementations flowing through one hot call site is slower for that reason alone.
• Escape analysis. If an object never leaves the method that created it (after inlining, which is why inlining comes first), C2 may not allocate it on the heap at all — its fields become registers. new Point(x, y) in a tight loop can cost nothing.
• Dead code and branch elimination. A branch that the profile says is never taken is compiled as an "uncommon trap" — a jump back into the interpreter if it ever happens. A null check on a value that was never null becomes an implicit check that relies on the CPU's page fault.
• Intrinsics. Math.sqrt, System.arraycopy, String.indexOf, CRC32, AES: replaced by hand-written machine code or a single instruction, regardless of what the Java source says.

#Walkthrough: what happened to total()

double total() {
    double sum = 0;
    for (Line line : lines) sum += line.price() * line.qty();   // lines is an ArrayList<Line>
    return sum;
}

After a few thousand calls, C2 has seen that lines was always an ArrayList, that Line.price() and Line.qty() are trivial getters, and that the loop never threw. What it emits, described in prose because the assembly is a page:

1. A type guard: is lines an ArrayList? If not, uncommon trap.
2. The ArrayList iterator inlined, then its hasNext and next inlined, then the Iterator object itself removed by escape analysis: what remains is an index loop over the backing array with a bounds check.
3. price() and qty() inlined to two field loads.
4. A multiply-add per element, unrolled four at a time, possibly vectorised on a CPU with AVX.
5. The modCount check the iterator does for ConcurrentModificationException hoisted out of the loop.

The source called three methods per element and allocated an iterator; the machine code calls nothing and allocates nothing. Now imagine a second call site passes a LinkedList. The guard in step 1 fails, the method deoptimises, the profile becomes bimorphic, and the next compilation carries two paths. Pass a third implementation and the call goes megamorphic and stays a virtual call. Nothing in the source changed; the shape of the data did.

#Deoptimisation

C2 compiled code on the assumption that the profile holds. When the assumption breaks — a new subclass shows up at a call site that was monomorphic, a branch that was never taken is taken, a class is loaded that invalidates a "this method has no overrides" assumption — the JVM deoptimises: throws the compiled code away, reconstructs the interpreter's frame from the compiled one (that reconstruction is why compiled code carries debug info at every safepoint), and falls back to interpreting, then recompiles with the new profile.

This is normal and mostly invisible. It becomes visible when it happens in a pattern, such as a service that handles one request type for a minute, then a burst of another, and looks slower for a moment while it recompiles. A method that repeatedly deoptimises is eventually marked not compilable and stays interpreted, which is the worst outcome and shows up as "this one endpoint is ten times slower and nobody knows why".

#Why microbenchmarks lie

long start = System.nanoTime();
for (int i = 0; i < 1_000_000; i++) compute(i);
System.out.println(System.nanoTime() - start);

This measures a mixture of interpretation, C1 code, C2 code and compilation itself, in unknown proportions. Run it twice in the same process and the second number is very different. If compute has no observable side effect, C2 may delete the loop entirely and report a time of nearly zero. If the loop count is a constant, C2 may unroll or fold it in ways your production code never sees.

The tool that exists for this is JMH (Java Microbenchmark Harness). It runs warm-up iterations until compilation settles, forks fresh JVMs so one benchmark's profile cannot pollute another, and gives you a Blackhole to consume results so they cannot be optimised away. If a number was not produced by JMH or an equivalent, treat it as an anecdote.

#Seeing it happen

java -XX:+PrintCompilation -jar app.jar | head -50

    141   23       3       java.lang.String::hashCode (60 bytes)
    892  318       4       java.util.HashMap::getNode (150 bytes)
   1204  411 %     4       com.acme.Order::total @ 9 (48 bytes)

Columns: milliseconds since start, compile id, tier, method and bytecode size. % marks an on-stack replacement of a loop, with @ 9 the bytecode offset it entered at. You will see String::hashCode and HashMap::getNode early, because everything calls them. made not entrant on a later line is a deoptimisation — and it comes with a reason, which is the part worth reading:

40   8     3   Hot::total (35 bytes)   made not entrant: not used
43  11 %   3   Hot::main @ 28          made not entrant: OSR invalidation of lower level
64  13 %   4   Hot::main @ 28          made not entrant: uncommon trap

not used means the tier-3 version was superseded by tier 4, and OSR invalidation of lower level means C1's loop compilation was replaced by C2's. Both are the ladder working. Only uncommon trap is a broken assumption, and it is the only one worth investigating. Treating every made not entrant as a problem is how people conclude their JVM is thrashing when it is warming up normally. Add -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining to watch inlining decisions, including the ones that were refused with too big or callee is too large — a @Transactional service method of 300 lines will not be inlined into anything. For a running service, Java Flight Recorder (jcmd <pid> JFR.start) records compilations and deoptimisations with no restart and little overhead.

#Try it yourself

static int fib(int n) { return n < 2 ? n : fib(n - 1) + fib(n - 2); }
long t = System.nanoTime(); fib(30); System.out.println(System.nanoTime() - t);

#Misconceptions

• "The JIT compiles the whole program eventually." It compiles the methods that get hot, which in a typical service is a few thousand out of a hundred thousand. Everything else stays interpreted, cheaply.
• "Compiled code is final." It carries assumptions and is discarded when one breaks. A method can be compiled, deoptimised and recompiled many times in its life.
• "Small methods are a style preference." They are the inliner's budget. A method over the size limit is never inlined, which also blocks escape analysis and specialisation inside it.
• "Interfaces are free." Monomorphic and bimorphic call sites are; a megamorphic one is a real virtual call and an optimisation boundary.
• "System.nanoTime around a loop is a benchmark." It is a measurement of a mixture. JMH, or nothing.

#Going deeper

• Aleksey Shipilëv's "JVM Anatomy Quarks": short, exact posts on the JIT, with the assembly.
• The JMH samples in the OpenJDK repository, which are the tutorial.
• JEP 483 and JEP 515 (Project Leyden): ahead-of-time class loading and method profiles, in Java 24 and 25.
• -XX:+PrintCompilation, -XX:+PrintInlining, and JFR.start on a service you run.
• The HotSpot source for the numbers above: src/hotspot/share/compiler/compilationPolicy.cpp, specifically call_predicate_helper and loop_predicate_helper. (It was TieredThresholdPolicy until JDK 17 merged it here.)