Concurrency and Performance
Open in the Padhaiko app5.1 Overview of multithreading
Multithreading allows a single Java process to execute multiple independent paths of control (threads) concurrently—improving CPU utilization on multi-core processors and keeping servers/UIs responsive while waiting on I/O.
| Aspect | Process | Thread |
|---|---|---|
| Memory Space | Isolated virtual address space | Shares Heap and Method Area/Metaspace within the same process |
| Private State | Own heap, stack, file descriptors, PID | Own JVM Stack, Program Counter (PC), and local variables |
| Creation & Context Switch | Heavyweight (OS page tables, IPC needed) | Lightweight (shared memory; faster switch) |
| Fault Isolation | Crash does not corrupt other processes | An unhandled exception or heap leak affects the shared JVM |
| Communication | Pipes, sockets, shared memory (IPC) | Direct via shared heap objects (requires synchronization) |
- Concurrency vs. Parallelism: Concurrency means structuring a program to manage multiple tasks with overlapping lifetimes (interleaved on 1+ cores). Parallelism means literally executing multiple tasks at the exact same instant on multiple CPU cores.
- Three Classic Ways to Define Work in Java:
- Extend
Thread: Overridesrun(). Disfavored because Java allows single class inheritance only and couples the task to the thread mechanism. - Implement
Runnable(Preferred for void tasks): Functional interfacevoid run(), passed tonew Thread(task)or anExecutorService. - Implement
Callable<V>: Functional interfaceV call() throws Exception; returns a typed result viaFuture<V>and propagates checked exceptions.
- Extend
- Java 21 Virtual Threads: Lightweight JVM-managed threads (
Thread.ofVirtual().start(task)) multiplexed onto a small pool of OS carrier threads. Unlike platform threads (~1 MB stack each), millions of virtual threads can wait on blocking I/O cheaply.
5.2 Thread lifecycle and synchronization
Thread Lifecycle (java.lang.Thread.State)
A Java thread moves through six well-defined states exposed by thread.getState():
new Thread() ──> [ NEW ]
│
t.start()
v
[ RUNNABLE ] <─────────────────────────────┐
/ │ \ │
monitor lock / wait() \ sleep(ms) / join(ms) │ notify() / lock acquired /
contended / join() \ │ timeout elapsed
v v v │
[ BLOCKED ] [ WAITING ] [ TIMED_WAITING ] ──────────┘
│
run() finishes / uncaught exception
v
[ TERMINATED ]Race Conditions, Monitor Locks, and Visibility
A race condition occurs when correctness depends on the unpredictable interleaving of multiple threads accessing shared mutable state. Even count++ is not atomic—it compiles into three separate steps: (1) read count, (2) add 1, (3) write count. Two threads running count++ concurrently can read the same value and overwrite each other's update.
Synchronization Mechanisms in Java
synchronized(Intrinsic Monitor Lock): Every Java object has a monitor lock. Marking a method orsynchronized (lockObj) { ... }block ensures mutual exclusion (only one thread executes inside at a time) and memory visibility (changes made before releasing the lock are flushed to main memory and visible to the next thread acquiring it).volatileKeyword: Guarantees visibility and prevents instruction reordering across threads (reads/writes bypass per-core CPU caches directly against main memory). Crucially,volatiledoes not make compound operations likecount++atomic.ReentrantLock(java.util.concurrent.locks): Explicit lock supportingtryLock(timeout)(avoids indefinite blocking), interruptible locking (lockInterruptibly()), and fairness policies; always release in afinallyblock.- Cooperation (
wait(),notify(),notifyAll()): Must be called inside asynchronizedblock on the monitor object, always guardingwait()inside awhile (!condition)loop to handle spurious wakeups.
Deadlock and Prevention
Deadlock occurs when two or more threads wait forever for locks held by each other (requires all four Coffman conditions: mutual exclusion, hold-and-wait, no preemption, and circular wait). The simplest, most reliable prevention strategy in Java is consistent global lock ordering—if Thread A and Thread B both acquire locks in canonical order (lock1 before lock2, e.g., sorted by account ID), circular wait is impossible.
5.3 Executor framework and thread pools
Spawning new Thread(task).start() per incoming HTTP request is dangerous under load: OS platform thread creation is expensive, each stack consumes significant memory, and unbounded threads lead to OutOfMemoryError and CPU thrashing.
The java.util.concurrent Executor Framework decouples task submission from thread execution mechanics:
| Factory / Class | Behavior | Best Use Case | Caution |
|---|---|---|---|
Executors.newFixedThreadPool(n) | Fixed n worker threads + unbounded LinkedBlockingQueue | Steady CPU/bounded concurrent workloads | Unbounded queue can grow indefinitely if tasks arrive faster than n threads drain them |
Executors.newCachedThreadPool() | Creates threads on demand, reuses idle threads (60 s TTL) | Short-lived asynchronous bursts | Unbounded max threads can exhaust memory under heavy load |
Executors.newSingleThreadExecutor() | 1 worker thread executing tasks sequentially in FIFO order | Sequential logging or serialized state mutation | No parallelism |
Executors.newScheduledThreadPool(n) | Supports delayed and periodic tasks (scheduleAtFixedRate) | Heartbeats, cache eviction, periodic polling | Long tasks delay subsequent runs if pool is small |
new ThreadPoolExecutor(...) | Explicit corePoolSize, maxPoolSize, bounded queue, RejectedExecutionHandler | Production services needing backpressure | Tune queue size and rejection policy carefully |
Executors.newVirtualThreadPerTaskExecutor() | Spawns one lightweight virtual thread per task (Java 21+) | High-concurrency blocking I/O (HTTP, JDBC) | Do not pool virtual threads; avoid pinning inside long synchronized I/O blocks |
import java.util.concurrent.*;
public class PoolDemo {
public static void main(String[] args) throws Exception {
ExecutorService pool = Executors.newFixedThreadPool(4);
try {
Future<Integer> future = pool.submit(() -> {
Thread.sleep(100);
return 42;
});
System.out.println("Result = " + future.get());
} finally {
pool.shutdown(); // stops accepting new tasks, finishes queued work
if (!pool.awaitTermination(5, TimeUnit.SECONDS)) {
pool.shutdownNow(); // interrupts running workers if timeout expires
}
}
}
}5.4 Completable future and parallel streams
Asynchronous Composition with CompletableFuture<T>
Classic Future.get() blocks the calling thread and lacks clean callback chaining. CompletableFuture<T> (Java 8+) enables non-blocking, composable asynchronous pipelines (backed by ForkJoinPool.commonPool() by default, or a custom Executor):
| Method | Role | Analogy |
|---|---|---|
supplyAsync(Supplier<T>) / runAsync(Runnable) | Start async computation returning a value / void | Launch background job |
thenApply(fn) / thenApplyAsync(fn) | Transform result T -> U synchronously / async | Stream.map |
thenCompose(fn) | Chain dependent async stage T -> CompletableFuture<U> | Stream.flatMap |
thenCombine(other, biFn) | Combine results of two independent futures in parallel | Parallel join |
thenAccept(consumer) | Consume final result with no return value | Terminal sink |
allOf(f1, f2, ...) / anyOf(...) | Wait for all futures or fastest future | Fan-out / fan-in |
exceptionally(fn) / handle((res, ex) -> ...) | Recover from exceptions inside the async pipeline | Async try-catch |
import java.util.concurrent.CompletableFuture;
public class AsyncOrderSummary {
public static void main(String[] args) {
CompletableFuture<String> userFuture =
CompletableFuture.supplyAsync(() -> "User#101");
CompletableFuture<Double> priceFuture =
CompletableFuture.supplyAsync(() -> 1200.0);
CompletableFuture<String> summary = userFuture
.thenCombine(priceFuture, (user, price) -> user + " total=" + (price * 1.13))
.exceptionally(ex -> "Fallback summary: " + ex.getMessage());
System.out.println(summary.join());
}
}Parallel Streams
Calling collection.parallelStream() (or .stream().parallel()) splits the data source using a Spliterator and processes chunks concurrently across ForkJoinPool.commonPool() using work-stealing (idle worker threads steal queued sub-tasks from busy workers).
- Use Parallel Streams when: the dataset is large, per-element computation is CPU-bound, operations are stateless, have no side effects, and reductions are associative (e.g., summing or finding max).
- Avoid Parallel Streams when: the dataset is small (splitting/merging overhead exceeds gains), tasks perform blocking I/O (starves
ForkJoinPool.commonPool()shared across the entire JVM), or code mutates shared non-thread-safe collections (ArrayList).
5.5 Best practices for thread-safe programming
- Prefer Immutability: Immutable objects (
finalfields, Javarecords,List.of(...),List.copyOf(...)) cannot change state after construction and are automatically thread-safe with zero locks. - Thread Confinement & Statelessness: Keep variables local to a method stack frame or isolate per-thread state with
ThreadLocal<T>(always call.remove()in thread pools to avoid memory leaks). - Use Atomic Variables (
java.util.concurrent.atomic):AtomicInteger,AtomicLong, andLongAdderachieve lock-free thread safety via CPU CAS (Compare-And-Swap) instructions (incrementAndGet(),compareAndSet()). - Use Concurrent Collections: Replace
HashMap(unsafe) andCollections.synchronizedMap(coarse global lock) withConcurrentHashMap(fine-grained CAS + per-bucket locking),CopyOnWriteArrayList(read-heavy observer lists), andBlockingQueue(ArrayBlockingQueue/LinkedBlockingQueuefor producer-consumer pipelines). - Minimize Lock Scope and Never Nest Arbitrary Locks: Hold locks only around the minimal critical section, never invoke external callbacks or network I/O while holding a lock, and enforce consistent lock acquisition ordering to prevent deadlocks.
5.6 Performance of multi-threaded program
Adding threads or CPU cores does not scale throughput linearly forever. Performance is bounded by serial bottlenecks, synchronization overhead, and hardware limits.
Amdahl's Law
Amdahl's Law and Maximum Speedup
If a fraction P of a program's execution time can be parallelized across N processors (while the remaining fraction 1 - P is strictly serial—such as lock-protected initialization or sequential I/O), the theoretical maximum speedup S(N) is:
S(N) = 1 / ((1 - P) + (P / N))
As the core count N grows toward infinity, P / N -> 0, capping speedup at S_max = 1 / (1 - P).
Thread Pool Sizing Guidelines
Let N_cores = Runtime.getRuntime().availableProcessors():
- CPU-Bound Tasks (image processing, encryption, matrix math):
N_threads = N_cores(orN_cores + 1). Creating far more threads than cores wastes time on context switching without adding CPU capacity. - I/O-Bound Tasks (database queries, HTTP calls, disk reads): While a thread blocks waiting on I/O for time
Wand uses CPU for timeC, optimal platform thread count is estimated by:N_threads = N_cores * (1 + W / C)(In Java 21+, virtual threads replace large platform thread pools for blocking I/O.)
Common Concurrency Performance Bottlenecks
- Lock Contention: When many threads compete for one
synchronizedmonitor or singleAtomicLonghotspot, threads stall waiting or retrying CAS. Fix withLongAdder(striped cells), lock striping (ConcurrentHashMap), or lock-free immutability. - Context Switching Overhead: Saving/restoring registers and warming CPU L1/L2 caches degrades throughput when runnable threads vastly outnumber cores.
- False Sharing: Two unrelated
volatile/atomic variables written by different cores happen to sit on the same 64-byte CPU cache line, triggering constant cache-line invalidation across cores.
Worked Examples
5.1
Suppose 80% (P = 0.80) of a Java batch job can run in parallel, while 20% (1 - P = 0.20) is strictly sequential.
- Using Amdahl's Law, calculate the speedup
S(N)on 4 CPU cores and on 16 CPU cores. - What is the theoretical maximum speedup even with infinitely many CPU cores?
- For a separate I/O-bound service running on 8 cores where each request spends 45 ms waiting on database/network I/O (
W = 45) and 5 ms computing on the CPU (C = 5), estimate the ideal platform thread pool size.
Apply Amdahl's Law S(N) = 1 / ((1 - P) + P / N) with 1 - P = 0.20 and P = 0.80:
- For
N = 4cores:S(4) = 1 / (0.20 + 0.80 / 4) = 1 / (0.20 + 0.20) = 1 / 0.40 = 2.50x. - For
N = 16cores:S(16) = 1 / (0.20 + 0.80 / 16) = 1 / (0.20 + 0.05) = 1 / 0.25 = 4.00x.
Find the asymptotic limit as N -> infinity:
S_max = 1 / 0.20 = 5.00x.
Compute the I/O-bound platform thread pool size for N_cores = 8, W = 45 ms, C = 5 ms: N_threads = 8 * (1 + 45 / 5) = 8 * (1 + 9) = 8 * 10 = 80 threads.
- Speedup is 2.50x on 4 cores and 4.00x on 16 cores.
- Theoretical maximum speedup is 5.00x.
- Estimated optimal platform thread pool size for the I/O-bound service is 80 threads.
5.2
A Spring service builds a dashboard response by fetching three independent pieces of data via CompletableFuture.supplyAsync(...) on separate threads:
- User profile service: 120 ms
- Order history service: 210 ms
- Loyalty points service: 90 ms
Once all three complete (thenCombine / allOf), a final CPU aggregation step takes 15 ms. Compare the total wall-clock latency of executing the three calls sequentially versus concurrently with CompletableFuture, and calculate the time saved.
Calculate sequential execution latency (each call waits for the previous one to finish before starting): T_seq = 120 + 210 + 90 + 15 = 435 ms.
Calculate concurrent CompletableFuture execution latency:
- The three independent I/O calls run concurrently, so the fan-out stage finishes when the slowest branch completes:
max(120, 210, 90) = 210 ms. - Adding the
15 msaggregation step afterallOf:T_async = 210 + 15 = 225 ms. - Time saved:
435 - 225 = 210 ms(a435 / 225 = 1.93xspeedup).
Sequential execution takes 435 ms, whereas concurrent CompletableFuture execution finishes in 225 ms, saving 210 ms.
5.3
How should a student answer the past exam question: "Explain why volatile alone cannot fix a race condition on counter++, and contrast synchronized vs. AtomicInteger"?
Decompose counter++ to Show the Race Condition: Show that counter++ consists of three distinct operations: read, increment, and write. Even with volatile (which guarantees every read sees the latest value in main memory), Thread A and Thread B can both read 5 simultaneously, compute 6, and write 6, losing one increment.
Contrast synchronized vs. AtomicInteger:
synchronized: Uses mutual-exclusion monitor locking (pessimistic locking) so only one thread executes the read-modify-write block at a time.AtomicInteger: Uses lock-free hardware CAS (Compare-And-Swap) viaincrementAndGet()(optimistic concurrency), typically delivering higher throughput for simple counters without thread blocking.
A complete exam answer breaks counter++ into read-modify-write steps to prove volatile provides visibility without atomicity, then contrasts synchronized (mutual-exclusion monitor locks) with AtomicInteger (lock-free hardware Compare-And-Swap via incrementAndGet()).
Exercises
Practice Exercises — Concurrency and Performance
List the six states in java.lang.Thread.State and describe the exact transitions between RUNNABLE, BLOCKED, WAITING, and TIMED_WAITING. Also explain why implementing Runnable or Callable<V> is preferred over extending Thread.
The Six Thread.State Values and Transitions:
NEW:Threadobject instantiated,start()not yet called.RUNNABLE: Executing on CPU or ready in OS scheduler queue afterstart().BLOCKED: Waiting to acquire an intrinsicsynchronizedmonitor lock held by another thread; returns toRUNNABLEwhen the lock is acquired.WAITING: Parked indefinitely viaObject.wait(),Thread.join(), orLockSupport.park(); wakes onnotify(),notifyAll(), or target thread termination.TIMED_WAITING: Parked for a bounded duration viaThread.sleep(ms),wait(ms), orjoin(ms); returns toRUNNABLEon timeout or notification.TERMINATED:run()has completed or exited with an uncaught exception.
Why Runnable / Callable<V> Is Preferred: Implementing Runnable or Callable<V> decouples the task logic from the thread execution engine (allowing submission to an ExecutorService thread pool), preserves the class's ability to extend another superclass, and (Callable<V>) allows returning typed results and throwing checked exceptions.
The six states are NEW, RUNNABLE, BLOCKED (waiting for a monitor lock), WAITING (indefinite wait/join), TIMED_WAITING (bounded sleep/wait), and TERMINATED. Prefer Runnable/Callable<V> over extending Thread to keep single inheritance free, return results (Callable), and reuse tasks with thread pools.
What causes a deadlock in multithreaded Java code? Write a small two-account transfer(Account from, Account to, double amount) example showing how consistent lock ordering prevents deadlock.
Deadlock Conditions: Deadlock happens when two threads each hold one monitor lock while waiting indefinitely for a lock held by the other (circular wait). For example, transfer(A, B, 100) locks A then tries to lock B, while a concurrent transfer(B, A, 50) locks B then tries to lock A.
Deadlock-Free Solution Using Canonical Lock Ordering:
class Account {
final int id;
double balance;
Account(int id, double balance) { this.id = id; this.balance = balance; }
}
static void transfer(Account from, Account to, double amount) {
Account first = (from.id < to.id) ? from : to;
Account second = (from.id < to.id) ? to : from;
synchronized (first) {
synchronized (second) {
if (from.balance >= amount) {
from.balance -= amount;
to.balance += amount;
}
}
}
}Because both threads lock the smaller id first, circular wait is mathematically impossible.
Deadlock arises when threads acquire shared locks in conflicting orders, creating a circular wait. Ordering locks consistently by a unique key (e.g., locking min(from.id, to.id) before max(from.id, to.id)) breaks the circular-wait condition and prevents deadlock.
Explain why ExecutorService thread pools are preferred over creating raw new Thread(...) per request. Contrast newFixedThreadPool, newCachedThreadPool, and Java 21's newVirtualThreadPerTaskExecutor, and show the standard graceful shutdown pattern.
Why Thread Pools Beat Unbounded new Thread(): Thread pools reuse a controlled set of worker threads, eliminating repeated OS thread allocation/teardown costs, bounding memory used by thread stacks, and providing task queuing/backpressure so traffic spikes do not crash the JVM.
Pool Types and Graceful Shutdown:
newFixedThreadPool(n): Boundednplatform threads with a shared queue (steady predictable load).newCachedThreadPool(): Expands dynamically and reclaims idle threads after 60 s (short bursts, risky under unbounded load).newVirtualThreadPerTaskExecutor()(Java 21): Spawns cheap JVM-managed virtual threads per task without pooling, ideal for massive concurrent blocking I/O.- Graceful shutdown: Call
pool.shutdown(), wait withpool.awaitTermination(timeout, unit), and callpool.shutdownNow()only if tasks fail to finish in time.
Thread pools reuse workers to avoid thread-creation overhead and OOM crashes. newFixedThreadPool bounds worker count, newCachedThreadPool scales dynamically for short bursts, and newVirtualThreadPerTaskExecutor handles high-concurrency I/O with lightweight virtual threads. Always shut down pools via shutdown() -> awaitTermination() -> shutdownNow().
A computation has a 75% parallelizable portion (P = 0.75) and a 25% serial portion (1 - P = 0.25).
- Using Amdahl's Law, calculate the speedup on a 6-core CPU and the maximum theoretical speedup as cores approach infinity.
- On the same 6-core machine, an I/O-bound service spends 60 ms waiting on external APIs and 15 ms computing on CPU per request. Compute the recommended platform thread pool size.
Amdahl's Law Speedup:
- For
P = 0.75,1 - P = 0.25,N = 6:S(6) = 1 / (0.25 + 0.75 / 6) = 1 / (0.25 + 0.125) = 1 / 0.375 = 8 / 3 = 2.67x. - Maximum theoretical speedup (
N -> infinity):S_max = 1 / (1 - P) = 1 / 0.25 = 4.0x.
I/O-Bound Thread Pool Sizing: With N_cores = 6, wait time W = 60 ms, and compute time C = 15 ms: N_threads = 6 * (1 + 60 / 15) = 6 * (1 + 4) = 6 * 5 = 30 threads.
- Speedup on 6 cores is 2.67x (
8/3), and the theoretical upper limit is 4.0x. - Recommended I/O-bound platform thread pool size is 30 threads.
Contrast CompletableFuture with Parallel Streams in Java, and explain why blocking I/O inside parallelStream() hurts overall JVM application performance. State three best practices for thread-safe Java programming.
CompletableFuture vs. Parallel Streams:
CompletableFuture: Designed for asynchronous I/O and task orchestration (supplyAsync,thenCombine,exceptionally), allowing customExecutorpools tailored to I/O wait times.- Parallel Streams: Designed for CPU-bound data-parallel loops over large in-memory collections, splitting work across
ForkJoinPool.commonPool(). - Why Blocking I/O Hurts
parallelStream(): Every parallel stream in the JVM shares the same smallForkJoinPool.commonPool()(sized tocores - 1). Blocking those worker threads on network/DB calls starves unrelated parallel streams and async tasks across the entire application.
Three Thread-Safety Best Practices:
- Immutability: Use
records,finalfields, and unmodifiable collections so shared state never mutates. - Thread-Safe Primitives & Collections: Prefer
AtomicInteger/LongAdderandConcurrentHashMap/BlockingQueueover manual locking orHashMap. - Minimize Lock Scope & Order Locks: Keep
synchronizedblocks small, avoid I/O inside locks, and acquire multiple locks in a fixed global order.
CompletableFuture orchestrates asynchronous (often I/O-bound) stages on custom executors, whereas Parallel Streams split CPU-bound collection pipelines across ForkJoinPool.commonPool(), which gets starved if workers block on I/O. Key thread-safety practices include favoring immutable objects, using Atomic*/ConcurrentHashMap, and keeping locks minimal and consistently ordered.