Virtual threads

Virtual threads (Java 21, JEP 444) are the biggest change to how a Java server is built since the servlet: a thread that costs a few hundred bytes of heap instead of a megabyte of stack, so a service can have one per request, per connection, per task, and block in each without running out. The thread-per-request model, the simplest way to write a server, stops being the model you scale away from. This lesson is what a virtual thread is made of, what it means that it is "mounted" on a carrier, what pinning was and how Java 24 removed most of it, and the three habits from the platform-thread era that must be unlearned.

#The problem they solve

A platform thread wraps an OS thread: about 1 MB of reserved stack, a kernel-scheduled entity, expensive to create and to context-switch. A server with a 200-thread pool can have 200 requests in flight; the 201st waits, even if all 200 are blocked on a database call and the CPU is idle. The alternative was reactive programming, which avoids blocking by turning every call into a callback chain, at the cost of readable code and stack traces.

Virtual threads keep the blocking style and remove the cost:

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    for (int i = 0; i < 100_000; i++) {
        executor.submit(() -> {
            Thread.sleep(Duration.ofSeconds(1));   // blocks — cheaply
            return fetch(i);
        });
    }
}   // waits for all tasks

100,000 tasks, each sleeping a second, complete in about a second on a laptop. The same with platform threads would exhaust memory or take a very long time.

#Under the hood: a continuation, a carrier, and a park

A virtual thread is a Continuation object (an internal class) holding its stack as a chain of heap-allocated frames plus a VirtualThread wrapper. When it runs, the JDK scheduler, a ForkJoinPool in FIFO mode with parallelism equal to the core count, mounts it on a carrier platform thread: the continuation's frames are copied lazily onto the carrier's real stack and execution proceeds as normal JIT-compiled code. When the virtual thread blocks in Thread.sleep, Socket.read, Lock.lock, BlockingQueue.take, Future.get, or any other JDK blocking call, the call ends in VirtualThread.park(): the continuation yields, its live frames are copied back to the heap, and the carrier is free to mount another virtual thread. When the I/O completes (a poller thread watches the file descriptors), the virtual thread is unparked and submitted to the scheduler again, to be mounted on whichever carrier is free next, not necessarily the same one.

What this costs: a park and unpark copies frames (a few hundred nanoseconds to a microsecond), so a virtual thread that blocks a million times a second is a poor fit, and a CPU-bound task gains nothing, since it never yields. What it does not cost: memory per thread beyond the live frames, and kernel involvement, since the OS sees only the carriers.

A virtual thread has no ThreadGroup of its own, is always a daemon, has normal priority that cannot be changed, and Thread.currentThread() inside one returns the VirtualThread, never the carrier, which is invisible to your code.

#Pinning: what it was and what remains

A virtual thread cannot yield if its frames cannot be copied to the heap. Two things prevent it:

1. Native frames: a blocking call inside JNI or a native method. The virtual thread stays mounted, the carrier is stuck, and the scheduler compensates by temporarily adding a carrier (up to jdk.virtualThreadScheduler.maxPoolSize, default 256).
2. synchronized and Object.wait (Java 21–23): the JVM tracked monitors on the carrier's stack, so a virtual thread blocking inside a synchronized block or on a monitor pinned its carrier. Under a pool of, say, eight carriers, eight virtual threads blocked inside synchronized on a slow database call froze the entire application: no carrier was free to run anything.

Java 24 (JEP 491) rewrote monitor handling so that a virtual thread blocking on or inside a synchronized block unmounts like any other blocking call. On Java 24 and later, the synchronized advice below is about cleanliness, not correctness; on Java 21, it is about liveness. The diagnostic is the same on both: -Djdk.tracePinnedThreads=full (Java 21–23), or the jdk.VirtualThreadPinned JFR event, which fires whenever a virtual thread parks while pinned and records the stack.

#Using them

Thread.startVirtualThread(() -> work());
Thread.ofVirtual().name("worker-", 0).start(() -> work());
Thread.ofVirtual().unstarted(() -> work());
 
ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();   // one thread per task, no queue

Spring Boot 3.2+: spring.threads.virtual.enabled=true puts Tomcat's request handling and @Async on virtual threads; the same property configures Jetty, Kafka listener containers and @Scheduled.

#Three habits to unlearn

1. Do not pool virtual threads. A pool exists to amortise creation cost. A virtual thread costs nothing to create. newFixedThreadPool(200) of virtual threads is a queue with a 200-limit for no reason; newVirtualThreadPerTaskExecutor() is the right shape.

2. Bound with semaphores, not pools. A pool limited concurrent access to a resource as a side effect. With virtual threads, limit it explicitly:

private final Semaphore dbSlots = new Semaphore(50);
void query() throws InterruptedException {
    dbSlots.acquire();
    try { jdbc.query(...); } finally { dbSlots.release(); }
}

The connection pool already does this for databases; the semaphore is for everything else that used to be protected by "there are only 200 threads".

3. Avoid ThreadLocal for per-request state, or at least stop caching in it. A ThreadLocal on a million virtual threads is a million copies. Pass context explicitly, or use ScopedValue (final in Java 25, JEP 506):

static final ScopedValue<User> CURRENT = ScopedValue.newInstance();
ScopedValue.where(CURRENT, user).run(() -> handle(request));   // visible to callees, immutable, bounded

A scoped value is bound for the dynamic extent of run, inherited by child threads under structured concurrency, and cannot be mutated or leaked past its scope.

#Walkthrough: the Java 21 service that froze

A Spring Boot 3.2 service turned on virtual threads and, an hour into peak traffic, stopped responding without any error logged.

public synchronized boolean tryAcquire(String key) {
    var bucket = buckets.computeIfAbsent(key, k -> loadFromRedis(k));   // network call, ~2 ms
    return bucket.tryConsume();
}

1. On Java 21, every request thread is virtual. tryAcquire is synchronized; inside it, loadFromRedis blocks on a socket. The virtual thread cannot unmount while holding the monitor: pinned.
2. Eight cores, eight carriers. When Redis slowed to 100 ms under a failover, eight virtual threads were inside tryAcquire, each pinning a carrier. Every other virtual thread, including the ones that would have released Redis connections, could not be scheduled. The scheduler's compensation did not help, because pinning on synchronized did not trigger it (only native frames did).
3. A thread dump (jcmd <pid> Thread.dump_to_file -format=json) showed the carriers inside tryAcquire and 40,000 virtual threads parked. JFR showed jdk.VirtualThreadPinned events, tens of thousands, all with this stack.
4. The fix was to replace synchronized with a ReentrantLock (which unmounts on contention even on Java 21) and, better, to move the network call out of the lock: computeIfAbsent with a cheap constructor, and a separate load with a per-key lock. Under Java 24 the freeze would not have happened, but the network call inside a lock would still have been a bad idea.
5. The team added -Djdk.tracePinnedThreads=full to the staging JVM and treated any output as a build failure until they upgraded.

The same code on a platform-thread pool would have degraded gracefully; on virtual threads it stopped. That is the one way virtual threads are less forgiving: the carriers are few, and anything that holds one holds the application.

#Structured concurrency (Java 25, JEP 505)

Virtual threads make fan-out cheap; structured concurrency makes it safe. A StructuredTaskScope treats a set of subtasks as one unit: they start together, and the scope does not close until all have completed or been cancelled.

try (var scope = StructuredTaskScope.open()) {
    Subtask<User>  user  = scope.fork(() -> users.find(id));
    Subtask<List<Order>> orders = scope.fork(() -> orderRepo.forUser(id));
    scope.join();                        // waits; if either fails, the other is cancelled and join throws
    return new Profile(user.get(), orders.get());
}

No leaked threads, cancellation flows down, and a thread dump shows the tree. It replaces the CompletableFuture fan-out for the common "call two things and combine" shape.

#When not to use them

• CPU-bound work: no blocking, no benefit; the parallelism is the core count either way.
• Frequent, fine-grained blocking in a hot loop, where park/unpark overhead is measurable.
• Code that pins: JNI, synchronized around blocking calls on Java 21–23, or a library that does either. Check with the JFR event before flipping the switch.

The best case is a service whose threads spend most of their time waiting on the network, which is most services.

#Try it yourself

synchronized void refresh() { token = http.fetchToken(); }   // 200 ms network call

#Misconceptions

• "Virtual threads make code faster." They make blocking cheap, which raises throughput for I/O-bound work. A CPU-bound task runs at the same speed with the same parallelism.
• "Pool them like platform threads." Creation is nearly free; a pool only adds a queue. Bound resources with semaphores.
• "synchronized is fine, it always was." On Java 21–23 it pins the carrier and can freeze the service; on Java 24+ it unmounts. Either way, keep blocking calls out of critical sections.
• "A virtual thread is a green thread or a coroutine you must yield from." The JDK's blocking calls yield for you; there is no await, and ordinary blocking code just works.
• "Thread dumps will be useless with a million threads." jcmd Thread.dump_to_file groups them by scope and executor; structured concurrency makes the tree readable.

#Going deeper

• JEP 444 (virtual threads), JEP 491 (synchronize without pinning, Java 24), JEP 505 (structured concurrency, Java 25), JEP 506 (scoped values, Java 25).
• Ron Pressler, "State of Loom", the design essay, and the Continuation/VirtualThread sources in java.base.
• The JDK "Virtual Threads" guide in the core libraries documentation, including the section on ThreadLocal and pinning diagnostics.
• JFR event reference: jdk.VirtualThreadPinned, jdk.VirtualThreadSubmitFailed.
• Spring Framework's "Virtual threads" documentation, and the spring.threads.virtual.enabled property reference.