A recursive function that walks a tree works fine in every test, then throws `StackOverflowError` in production on one particular customer's data. The recursive call is the very last statement in the method — doesn't that make it a tail call, and shouldn't the JVM optimise that away?
No — the JVM never eliminates tail calls, in any circumstance, regardless of where the recursive call sits in the method. Every call, recursive or not, pushes a full frame onto the thread's stack, and that frame stays there until the call returns; writing the recursive call as the last statement doesn't change that. What almost certainly happened is that this particular customer's tree is not balanced the way the test trees were — a balanced tree of a million nodes is only about twenty levels deep and recurses fine, but a degenerate tree (effectively a long chain, one child per node) recurses as deep as it has nodes, and a default JVM stack overflows somewhere around 45,000 frames. The fix isn't finding a missing base case — the recursion is correct — it's recognising that recursion depth was never a safe choice for an input shape that could be a long chain, and either converting the walk to an explicit loop with a heap-allocated stack, or bounding the depth and falling back before it's reached.