Streams

A stream is a pipeline: a source, zero or more transformations, and one terminal operation that makes the whole thing run. Written well, a stream says what is being computed and hides the loop. Written badly, it is a loop with worse stack traces. The difference is knowing what the pipeline does underneath: the chain of stages the terminal operation builds, the spliterator that feeds it, and the optimisations that occasionally skip your code entirely.

#The shape

List<String> topCustomers = orders.stream()                       // source
        .filter(o -> o.status() == PAID)                          // intermediate
        .collect(groupingBy(Order::customerId, summingLong(Order::total)))  // terminal
        .entrySet().stream()
        .sorted(Map.Entry.<String, Long>comparingByValue().reversed())
        .limit(5)
        .map(Map.Entry::getKey)
        .toList();

Sources: collection.stream(), Stream.of(...), Arrays.stream(arr), IntStream.range(0, n), Files.lines(path), Stream.iterate, Stream.generate.

Intermediate operations return a new stream and do nothing yet: filter, map, flatMap, sorted, distinct, limit, skip, peek, takeWhile, dropWhile, mapToInt, and since Java 24, gather.

Terminal operations run the pipeline and produce a result: collect, toList, forEach, reduce, count, sum, min, max, anyMatch, allMatch, findFirst, findAny, iterator.

#Laziness

Nothing happens until the terminal operation. Then elements flow through the pipeline one at a time, not stage by stage:

Stream.of("a", "bb", "ccc")
      .filter(s -> { System.out.println("filter " + s); return s.length() > 1; })
      .map(s -> { System.out.println("map " + s); return s.toUpperCase(); })
      .findFirst();
// filter a, filter bb, map bb — "ccc" is never touched

findFirst is short-circuiting: it stops the source once it has an answer. So is limit, anyMatch, allMatch, noneMatch. This is why Stream.iterate(1, i -> i * 2).limit(10) works on an infinite stream, and why sorted() — which must see everything before it can emit anything — breaks that laziness and buffers the whole stream.

A stream can be consumed once. Calling a second terminal operation on it throws IllegalStateException.

#Under the hood: stages, sinks and the spliterator

Each intermediate call returns a new ReferencePipeline stage object linked to the previous one: the pipeline is a linked list of stages, each remembering only what it does. Nothing runs. The terminal operation walks that list backwards and wraps each stage in a Sink, so the result is a chain of sinks where filter's sink calls map's sink calls the collector's sink. Then it asks the source for a Spliterator and calls forEachRemaining(sink), pushing one element at a time down the chain. That push model is why elements flow individually, why a short-circuit can stop the source mid-way (the sink reports cancellationRequested()), and why stack traces show a dozen frames per stage: each element passes through every sink's accept.

Stages are stateless (filter, map) or stateful (sorted, distinct, limit, skip). A stateful stage may need to buffer everything before emitting anything: sorted collects every element into an array, sorts, then pushes them all downstream, which is what makes it a barrier to laziness and to short-circuiting.

The spliterator carries characteristics that let the pipeline skip work: SIZED (the source knows its count), SORTED, DISTINCT, ORDERED. Since Java 9, count() on a SIZED source with no size-changing stages returns the size directly without traversing, sorted() on a SORTED source is a no-op, and distinct() on a DISTINCT one too. That optimisation has a visible side effect: a peek in such a pipeline never runs, because no element was ever pushed.

Parallel streams use the other half of Spliterator: trySplit(). The source is split recursively into chunks, each chunk runs the same sink chain on a ForkJoinPool worker, and the results are merged with the collector's combiner. An ArrayList or an array splits evenly in O(1); a LinkedList or Stream.iterate cannot split well and parallelises badly; a HashSet splits by bucket range. The merge is the hidden cost: collect(toList()) on a parallel stream builds many lists and concatenates them, and an ORDERED stream must preserve encounter order across chunks, which is why forEachOrdered is slower than forEach.

#Collectors

collect is where most of the power is. The ones to know:

.toList()                                       // Java 16+, unmodifiable
.collect(toSet())
.collect(toMap(Order::id, Function.identity()))            // throws on duplicate keys
.collect(toMap(Order::customerId, Order::total, Long::sum)) // merge function for duplicates
.collect(groupingBy(Order::customerId))                    // Map<String, List<Order>>
.collect(groupingBy(Order::customerId, counting()))        // Map<String, Long>
.collect(groupingBy(Order::customerId, TreeMap::new, mapping(Order::id, toList())))
.collect(partitioningBy(o -> o.total() > 100))             // Map<Boolean, List<Order>>
.collect(joining(", ", "[", "]"))
.collect(summarizingLong(Order::total))                    // count, sum, min, avg, max
.collect(teeing(counting(), summingLong(Order::total), (n, sum) -> sum / n))

toMap throwing on duplicates is a feature: silently keeping one would hide a data problem. Supply a merge function when duplicates are expected. A Collector is four functions, supplier, accumulator, combiner and finisher, and Collector.of(...) lets you write one when the built-ins do not fit; Stream.toList() differs from collect(toList()) in returning an unmodifiable list, which is the right default for a result.

#Walkthrough: the peek that stopped logging

A batch job counted records and logged each one with peek:

long n = records.stream()
        .peek(r -> log.debug("processing {}", r.id()))
        .count();

1. On Java 8 the pipeline traversed every element: peek ran, the log filled, count summed. The team relied on the log for an audit trail.
2. On Java 9 and later, records is an ArrayList, its spliterator is SIZED, and peek does not change the size. count() reads spliterator.getExactSizeIfKnown() and returns it. No element is pushed; the peek lambda never runs; the audit log is empty; the count is correct.
3. Nobody noticed for a month, because the number was right and the log was only read during an audit.
4. The Javadoc for peek says it "exists mainly to support debugging" and that the action "may not be invoked" when the element count can be computed without traversal. The audit was a side effect in a pipeline whose semantics are the result, not the path.
5. The fix is to make the log part of the result path: forEach(r -> { log.debug(...); n++ }) in a loop, or map the ids and log the collected list. Side effects belong in terminal operations or loops, never in peek.

The general version: the stream API promises the result, and reserves the right to compute it without running every stage. Any code that depends on a stage running is depending on a detail the next JDK may optimise away.

#Primitive streams

IntStream, LongStream, DoubleStream avoid boxing and add sum, average, range, summaryStatistics. Move between them with mapToInt/mapToObj/boxed:

long total = orders.stream().mapToLong(Order::total).sum();       // no Long objects
IntStream.rangeClosed(1, 100).filter(i -> i % 3 == 0).count();

orders.stream().map(Order::total).reduce(0L, Long::sum) boxes every element; mapToLong(...).sum() does not.

#flatMap, Optional and gatherers

flatMap turns each element into a stream and concatenates: orders.stream().flatMap(o -> o.lines().stream()) is every line of every order. Since Java 10 it is lazy with respect to short-circuiting (flatMap(...).findFirst() no longer drains the first inner stream). Optional has stream() too, so .map(this::lookup).flatMap(Optional::stream) drops the empties — cleaner than filter(isPresent).map(get).

Gatherers (Java 24, JEP 485) are custom intermediate operations, the thing collect could never be: Gatherers.windowFixed(3) groups elements in threes, windowSliding(2) gives overlapping pairs, fold and scan carry state through the stream, mapConcurrent(n, fn) maps with bounded concurrency on virtual threads. A Gatherer is an initializer, an integrator that receives each element and a downstream, and an optional finisher, so "running total" and "deduplicate consecutive" stop being loops.

#When to use a loop instead

• The body has side effects (writing to a database, mutating state). forEach with side effects is a loop wearing a costume, and peek is for debugging only.
• You need to break out early on a condition that takeWhile/findFirst do not express.
• You need an index. IntStream.range(0, n).mapToObj(i -> ...) works but reads worse than for.
• You need to throw a checked exception.
• Performance in a hot inner loop, measured. Streams add a small fixed overhead (the stage and sink objects, a few virtual calls per element); for a million elements it is negligible, for a million calls over ten elements it may not be.

A stream is right when the code is a transformation — filter, map, group, reduce — and reads as such.

#Parallel streams

.parallel() splits the source across the common ForkJoinPool (size = cores − 1) and merges. It helps for CPU-bound work over large, cheaply splittable sources (arrays, ArrayList, ranges) with no shared mutable state. It hurts, or does nothing, for small collections, I/O-bound work, LinkedList or Stream.iterate sources, and anything that synchronises. And it shares one pool with every other parallel stream in the JVM — a blocking call inside a parallel stream stalls unrelated code.

#Stack traces and debugging

A stream stack trace shows a dozen internal frames per stage, one per sink. To debug, extract lambdas into named methods (then the frame has a name), use peek temporarily, or set a breakpoint inside a block lambda. Prefer several short pipelines assigned to named variables over one thirty-line chain.

#Try it yourself

List<Integer> xs = List.of(3, 1, 2);
xs.stream().peek(x -> System.out.print("p" + x + " ")).count();
System.out.println();
xs.stream().sorted().peek(x -> System.out.print("s" + x + " ")).findFirst();
System.out.println();
Stream.iterate(1, i -> i + 1).filter(i -> i % 2 == 0).map(i -> i * 10).limit(2).forEach(System.out::print);

#Misconceptions

• "A stream processes stage by stage." It processes element by element through a chain of sinks, which is what makes short-circuiting and infinite sources work.
• "peek is a safe place for logging." It is a debugging hook the pipeline may skip when the result can be computed without traversal. Since Java 9, count() on a sized source is that case.
• "collect(toList()) and toList() are the same." The first returns a mutable ArrayList; the second an unmodifiable list. Choose the second for results.
• "Parallel streams make things faster." They make CPU-bound work over splittable sources faster, on a shared pool, with a merge cost. I/O inside one stalls the JVM's other parallel streams.
• "Streams are slow." They add a few virtual calls and a handful of objects per pipeline. What is slow is the N+1 query the stream made look declarative.

#Going deeper

• java.util.stream package Javadoc, "Stream operations and pipelines" and the "Side-effects" section, where peek's licence to skip is written.
• Spliterator Javadoc: characteristics, trySplit, and why sources parallelise differently.
• AbstractPipeline.wrapSink and copyInto in the JDK source, the sink chain in fifty lines.
• JEP 485, Stream Gatherers, and Viktor Klang's "Gatherers" talk for the design.
• Brian Goetz, "State of the Lambda: Libraries Edition", the original design document for streams.