PadhaikoAll subjects

Concurrency and Performance

Java Programming (ENCT 327)

Open in the Padhaiko app

5.1 Overview of multithreading

Multithreading allows a single Java process to execute multiple independent paths of control (threads) concurrently—improving CPU utilization on multi-core processors and keeping servers/UIs responsive while waiting on I/O.

AspectProcessThread
Memory SpaceIsolated virtual address spaceShares Heap and Method Area/Metaspace within the same process
Private StateOwn heap, stack, file descriptors, PIDOwn JVM Stack, Program Counter (PC), and local variables
Creation & Context SwitchHeavyweight (OS page tables, IPC needed)Lightweight (shared memory; faster switch)
Fault IsolationCrash does not corrupt other processesAn unhandled exception or heap leak affects the shared JVM
CommunicationPipes, sockets, shared memory (IPC)Direct via shared heap objects (requires synchronization)

5.2 Thread lifecycle and synchronization

Thread Lifecycle (java.lang.Thread.State)

A Java thread moves through six well-defined states exposed by thread.getState():

Java Thread Lifecycle States (Thread.State)Text
  new Thread() ──> [ NEW ]
                      │
                 t.start()
                      v
                 [ RUNNABLE ] <─────────────────────────────┐
                 /    │     \                               │
   monitor lock /   wait()   \ sleep(ms) / join(ms)         │ notify() / lock acquired /
   contended   /    join()    \                             │ timeout elapsed
              v       v        v                            │
        [ BLOCKED ] [ WAITING ] [ TIMED_WAITING ] ──────────┘
                      │
              run() finishes / uncaught exception
                      v
                [ TERMINATED ]

Race Conditions, Monitor Locks, and Visibility

A race condition occurs when correctness depends on the unpredictable interleaving of multiple threads accessing shared mutable state. Even count++ is not atomic—it compiles into three separate steps: (1) read count, (2) add 1, (3) write count. Two threads running count++ concurrently can read the same value and overwrite each other's update.

Definition

Synchronization Mechanisms in Java

  1. synchronized (Intrinsic Monitor Lock): Every Java object has a monitor lock. Marking a method or synchronized (lockObj) { ... } block ensures mutual exclusion (only one thread executes inside at a time) and memory visibility (changes made before releasing the lock are flushed to main memory and visible to the next thread acquiring it).
  2. volatile Keyword: Guarantees visibility and prevents instruction reordering across threads (reads/writes bypass per-core CPU caches directly against main memory). Crucially, volatile does not make compound operations like count++ atomic.
  3. ReentrantLock (java.util.concurrent.locks): Explicit lock supporting tryLock(timeout) (avoids indefinite blocking), interruptible locking (lockInterruptibly()), and fairness policies; always release in a finally block.
  4. Cooperation (wait(), notify(), notifyAll()): Must be called inside a synchronized block on the monitor object, always guarding wait() inside a while (!condition) loop to handle spurious wakeups.

Deadlock and Prevention

Deadlock occurs when two or more threads wait forever for locks held by each other (requires all four Coffman conditions: mutual exclusion, hold-and-wait, no preemption, and circular wait). The simplest, most reliable prevention strategy in Java is consistent global lock ordering—if Thread A and Thread B both acquire locks in canonical order (lock1 before lock2, e.g., sorted by account ID), circular wait is impossible.


5.3 Executor framework and thread pools

Spawning new Thread(task).start() per incoming HTTP request is dangerous under load: OS platform thread creation is expensive, each stack consumes significant memory, and unbounded threads lead to OutOfMemoryError and CPU thrashing.

The java.util.concurrent Executor Framework decouples task submission from thread execution mechanics:

Core ExecutorService Thread Pool Configurations
Factory / ClassBehaviorBest Use CaseCaution
Executors.newFixedThreadPool(n)Fixed n worker threads + unbounded LinkedBlockingQueueSteady CPU/bounded concurrent workloadsUnbounded queue can grow indefinitely if tasks arrive faster than n threads drain them
Executors.newCachedThreadPool()Creates threads on demand, reuses idle threads (60 s TTL)Short-lived asynchronous burstsUnbounded max threads can exhaust memory under heavy load
Executors.newSingleThreadExecutor()1 worker thread executing tasks sequentially in FIFO orderSequential logging or serialized state mutationNo parallelism
Executors.newScheduledThreadPool(n)Supports delayed and periodic tasks (scheduleAtFixedRate)Heartbeats, cache eviction, periodic pollingLong tasks delay subsequent runs if pool is small
new ThreadPoolExecutor(...)Explicit corePoolSize, maxPoolSize, bounded queue, RejectedExecutionHandlerProduction services needing backpressureTune queue size and rejection policy carefully
Executors.newVirtualThreadPerTaskExecutor()Spawns one lightweight virtual thread per task (Java 21+)High-concurrency blocking I/O (HTTP, JDBC)Do not pool virtual threads; avoid pinning inside long synchronized I/O blocks
Java
import java.util.concurrent.*;

public class PoolDemo {
    public static void main(String[] args) throws Exception {
        ExecutorService pool = Executors.newFixedThreadPool(4);
        try {
            Future<Integer> future = pool.submit(() -> {
                Thread.sleep(100);
                return 42;
            });
            System.out.println("Result = " + future.get());
        } finally {
            pool.shutdown(); // stops accepting new tasks, finishes queued work
            if (!pool.awaitTermination(5, TimeUnit.SECONDS)) {
                pool.shutdownNow(); // interrupts running workers if timeout expires
            }
        }
    }
}

5.4 Completable future and parallel streams

Asynchronous Composition with CompletableFuture<T>

Classic Future.get() blocks the calling thread and lacks clean callback chaining. CompletableFuture<T> (Java 8+) enables non-blocking, composable asynchronous pipelines (backed by ForkJoinPool.commonPool() by default, or a custom Executor):

Essential CompletableFuture Methods
MethodRoleAnalogy
supplyAsync(Supplier<T>) / runAsync(Runnable)Start async computation returning a value / voidLaunch background job
thenApply(fn) / thenApplyAsync(fn)Transform result T -> U synchronously / asyncStream.map
thenCompose(fn)Chain dependent async stage T -> CompletableFuture<U>Stream.flatMap
thenCombine(other, biFn)Combine results of two independent futures in parallelParallel join
thenAccept(consumer)Consume final result with no return valueTerminal sink
allOf(f1, f2, ...) / anyOf(...)Wait for all futures or fastest futureFan-out / fan-in
exceptionally(fn) / handle((res, ex) -> ...)Recover from exceptions inside the async pipelineAsync try-catch
Java
import java.util.concurrent.CompletableFuture;

public class AsyncOrderSummary {
    public static void main(String[] args) {
        CompletableFuture<String> userFuture =
                CompletableFuture.supplyAsync(() -> "User#101");
        CompletableFuture<Double> priceFuture =
                CompletableFuture.supplyAsync(() -> 1200.0);

        CompletableFuture<String> summary = userFuture
                .thenCombine(priceFuture, (user, price) -> user + " total=" + (price * 1.13))
                .exceptionally(ex -> "Fallback summary: " + ex.getMessage());

        System.out.println(summary.join());
    }
}

Parallel Streams

Calling collection.parallelStream() (or .stream().parallel()) splits the data source using a Spliterator and processes chunks concurrently across ForkJoinPool.commonPool() using work-stealing (idle worker threads steal queued sub-tasks from busy workers).


5.5 Best practices for thread-safe programming

  1. Prefer Immutability: Immutable objects (final fields, Java records, List.of(...), List.copyOf(...)) cannot change state after construction and are automatically thread-safe with zero locks.
  2. Thread Confinement & Statelessness: Keep variables local to a method stack frame or isolate per-thread state with ThreadLocal<T> (always call .remove() in thread pools to avoid memory leaks).
  3. Use Atomic Variables (java.util.concurrent.atomic): AtomicInteger, AtomicLong, and LongAdder achieve lock-free thread safety via CPU CAS (Compare-And-Swap) instructions (incrementAndGet(), compareAndSet()).
  4. Use Concurrent Collections: Replace HashMap (unsafe) and Collections.synchronizedMap (coarse global lock) with ConcurrentHashMap (fine-grained CAS + per-bucket locking), CopyOnWriteArrayList (read-heavy observer lists), and BlockingQueue (ArrayBlockingQueue / LinkedBlockingQueue for producer-consumer pipelines).
  5. Minimize Lock Scope and Never Nest Arbitrary Locks: Hold locks only around the minimal critical section, never invoke external callbacks or network I/O while holding a lock, and enforce consistent lock acquisition ordering to prevent deadlocks.

5.6 Performance of multi-threaded program

Adding threads or CPU cores does not scale throughput linearly forever. Performance is bounded by serial bottlenecks, synchronization overhead, and hardware limits.

Amdahl's Law

Definition

Amdahl's Law and Maximum Speedup

If a fraction P of a program's execution time can be parallelized across N processors (while the remaining fraction 1 - P is strictly serial—such as lock-protected initialization or sequential I/O), the theoretical maximum speedup S(N) is:

S(N) = 1 / ((1 - P) + (P / N))

As the core count N grows toward infinity, P / N -> 0, capping speedup at S_max = 1 / (1 - P).

Thread Pool Sizing Guidelines

Let N_cores = Runtime.getRuntime().availableProcessors():

Common Concurrency Performance Bottlenecks


Worked Examples

Example 5.1

5.1

Suppose 80% (P = 0.80) of a Java batch job can run in parallel, while 20% (1 - P = 0.20) is strictly sequential.

  1. Using Amdahl's Law, calculate the speedup S(N) on 4 CPU cores and on 16 CPU cores.
  2. What is the theoretical maximum speedup even with infinitely many CPU cores?
  3. For a separate I/O-bound service running on 8 cores where each request spends 45 ms waiting on database/network I/O (W = 45) and 5 ms computing on the CPU (C = 5), estimate the ideal platform thread pool size.

Apply Amdahl's Law S(N) = 1 / ((1 - P) + P / N) with 1 - P = 0.20 and P = 0.80:

  • For N = 4 cores: S(4) = 1 / (0.20 + 0.80 / 4) = 1 / (0.20 + 0.20) = 1 / 0.40 = 2.50x.
  • For N = 16 cores: S(16) = 1 / (0.20 + 0.80 / 16) = 1 / (0.20 + 0.05) = 1 / 0.25 = 4.00x.

Find the asymptotic limit as N -> infinity:

S_max = 1 / 0.20 = 5.00x.

Compute the I/O-bound platform thread pool size for N_cores = 8, W = 45 ms, C = 5 ms: N_threads = 8 * (1 + 45 / 5) = 8 * (1 + 9) = 8 * 10 = 80 threads.

Answer:
  1. Speedup is 2.50x on 4 cores and 4.00x on 16 cores.
  2. Theoretical maximum speedup is 5.00x.
  3. Estimated optimal platform thread pool size for the I/O-bound service is 80 threads.
Example 5.2

5.2

A Spring service builds a dashboard response by fetching three independent pieces of data via CompletableFuture.supplyAsync(...) on separate threads:

Once all three complete (thenCombine / allOf), a final CPU aggregation step takes 15 ms. Compare the total wall-clock latency of executing the three calls sequentially versus concurrently with CompletableFuture, and calculate the time saved.

Calculate sequential execution latency (each call waits for the previous one to finish before starting): T_seq = 120 + 210 + 90 + 15 = 435 ms.

Calculate concurrent CompletableFuture execution latency:

  • The three independent I/O calls run concurrently, so the fan-out stage finishes when the slowest branch completes: max(120, 210, 90) = 210 ms.
  • Adding the 15 ms aggregation step after allOf: T_async = 210 + 15 = 225 ms.
  • Time saved: 435 - 225 = 210 ms (a 435 / 225 = 1.93x speedup).
Answer:

Sequential execution takes 435 ms, whereas concurrent CompletableFuture execution finishes in 225 ms, saving 210 ms.

Example 5.3

5.3

How should a student answer the past exam question: "Explain why volatile alone cannot fix a race condition on counter++, and contrast synchronized vs. AtomicInteger"?

Decompose counter++ to Show the Race Condition: Show that counter++ consists of three distinct operations: read, increment, and write. Even with volatile (which guarantees every read sees the latest value in main memory), Thread A and Thread B can both read 5 simultaneously, compute 6, and write 6, losing one increment.

Contrast synchronized vs. AtomicInteger:

  • synchronized: Uses mutual-exclusion monitor locking (pessimistic locking) so only one thread executes the read-modify-write block at a time.
  • AtomicInteger: Uses lock-free hardware CAS (Compare-And-Swap) via incrementAndGet() (optimistic concurrency), typically delivering higher throughput for simple counters without thread blocking.
Answer:

A complete exam answer breaks counter++ into read-modify-write steps to prove volatile provides visibility without atomicity, then contrasts synchronized (mutual-exclusion monitor locks) with AtomicInteger (lock-free hardware Compare-And-Swap via incrementAndGet()).


Exercises

Practice Exercises — Concurrency and Performance

Question 1

List the six states in java.lang.Thread.State and describe the exact transitions between RUNNABLE, BLOCKED, WAITING, and TIMED_WAITING. Also explain why implementing Runnable or Callable<V> is preferred over extending Thread.

The Six Thread.State Values and Transitions:

  • NEW: Thread object instantiated, start() not yet called.
  • RUNNABLE: Executing on CPU or ready in OS scheduler queue after start().
  • BLOCKED: Waiting to acquire an intrinsic synchronized monitor lock held by another thread; returns to RUNNABLE when the lock is acquired.
  • WAITING: Parked indefinitely via Object.wait(), Thread.join(), or LockSupport.park(); wakes on notify(), notifyAll(), or target thread termination.
  • TIMED_WAITING: Parked for a bounded duration via Thread.sleep(ms), wait(ms), or join(ms); returns to RUNNABLE on timeout or notification.
  • TERMINATED: run() has completed or exited with an uncaught exception.

Why Runnable / Callable<V> Is Preferred: Implementing Runnable or Callable<V> decouples the task logic from the thread execution engine (allowing submission to an ExecutorService thread pool), preserves the class's ability to extend another superclass, and (Callable<V>) allows returning typed results and throwing checked exceptions.

Answer:

The six states are NEW, RUNNABLE, BLOCKED (waiting for a monitor lock), WAITING (indefinite wait/join), TIMED_WAITING (bounded sleep/wait), and TERMINATED. Prefer Runnable/Callable<V> over extending Thread to keep single inheritance free, return results (Callable), and reuse tasks with thread pools.

Question 2

What causes a deadlock in multithreaded Java code? Write a small two-account transfer(Account from, Account to, double amount) example showing how consistent lock ordering prevents deadlock.

Deadlock Conditions: Deadlock happens when two threads each hold one monitor lock while waiting indefinitely for a lock held by the other (circular wait). For example, transfer(A, B, 100) locks A then tries to lock B, while a concurrent transfer(B, A, 50) locks B then tries to lock A.

Deadlock-Free Solution Using Canonical Lock Ordering:

Java
class Account {
    final int id;
    double balance;
    Account(int id, double balance) { this.id = id; this.balance = balance; }
}

static void transfer(Account from, Account to, double amount) {
    Account first  = (from.id < to.id) ? from : to;
    Account second = (from.id < to.id) ? to   : from;
    synchronized (first) {
        synchronized (second) {
            if (from.balance >= amount) {
                from.balance -= amount;
                to.balance += amount;
            }
        }
    }
}

Because both threads lock the smaller id first, circular wait is mathematically impossible.

Answer:

Deadlock arises when threads acquire shared locks in conflicting orders, creating a circular wait. Ordering locks consistently by a unique key (e.g., locking min(from.id, to.id) before max(from.id, to.id)) breaks the circular-wait condition and prevents deadlock.

Question 3

Explain why ExecutorService thread pools are preferred over creating raw new Thread(...) per request. Contrast newFixedThreadPool, newCachedThreadPool, and Java 21's newVirtualThreadPerTaskExecutor, and show the standard graceful shutdown pattern.

Why Thread Pools Beat Unbounded new Thread(): Thread pools reuse a controlled set of worker threads, eliminating repeated OS thread allocation/teardown costs, bounding memory used by thread stacks, and providing task queuing/backpressure so traffic spikes do not crash the JVM.

Pool Types and Graceful Shutdown:

  • newFixedThreadPool(n): Bounded n platform threads with a shared queue (steady predictable load).
  • newCachedThreadPool(): Expands dynamically and reclaims idle threads after 60 s (short bursts, risky under unbounded load).
  • newVirtualThreadPerTaskExecutor() (Java 21): Spawns cheap JVM-managed virtual threads per task without pooling, ideal for massive concurrent blocking I/O.
  • Graceful shutdown: Call pool.shutdown(), wait with pool.awaitTermination(timeout, unit), and call pool.shutdownNow() only if tasks fail to finish in time.
Answer:

Thread pools reuse workers to avoid thread-creation overhead and OOM crashes. newFixedThreadPool bounds worker count, newCachedThreadPool scales dynamically for short bursts, and newVirtualThreadPerTaskExecutor handles high-concurrency I/O with lightweight virtual threads. Always shut down pools via shutdown() -> awaitTermination() -> shutdownNow().

Question 4

A computation has a 75% parallelizable portion (P = 0.75) and a 25% serial portion (1 - P = 0.25).

  1. Using Amdahl's Law, calculate the speedup on a 6-core CPU and the maximum theoretical speedup as cores approach infinity.
  2. On the same 6-core machine, an I/O-bound service spends 60 ms waiting on external APIs and 15 ms computing on CPU per request. Compute the recommended platform thread pool size.

Amdahl's Law Speedup:

  • For P = 0.75, 1 - P = 0.25, N = 6: S(6) = 1 / (0.25 + 0.75 / 6) = 1 / (0.25 + 0.125) = 1 / 0.375 = 8 / 3 = 2.67x.
  • Maximum theoretical speedup (N -> infinity): S_max = 1 / (1 - P) = 1 / 0.25 = 4.0x.

I/O-Bound Thread Pool Sizing: With N_cores = 6, wait time W = 60 ms, and compute time C = 15 ms: N_threads = 6 * (1 + 60 / 15) = 6 * (1 + 4) = 6 * 5 = 30 threads.

Answer:
  1. Speedup on 6 cores is 2.67x (8/3), and the theoretical upper limit is 4.0x.
  2. Recommended I/O-bound platform thread pool size is 30 threads.
Question 5

Contrast CompletableFuture with Parallel Streams in Java, and explain why blocking I/O inside parallelStream() hurts overall JVM application performance. State three best practices for thread-safe Java programming.

CompletableFuture vs. Parallel Streams:

  • CompletableFuture: Designed for asynchronous I/O and task orchestration (supplyAsync, thenCombine, exceptionally), allowing custom Executor pools tailored to I/O wait times.
  • Parallel Streams: Designed for CPU-bound data-parallel loops over large in-memory collections, splitting work across ForkJoinPool.commonPool().
  • Why Blocking I/O Hurts parallelStream(): Every parallel stream in the JVM shares the same small ForkJoinPool.commonPool() (sized to cores - 1). Blocking those worker threads on network/DB calls starves unrelated parallel streams and async tasks across the entire application.

Three Thread-Safety Best Practices:

  1. Immutability: Use records, final fields, and unmodifiable collections so shared state never mutates.
  2. Thread-Safe Primitives & Collections: Prefer AtomicInteger / LongAdder and ConcurrentHashMap / BlockingQueue over manual locking or HashMap.
  3. Minimize Lock Scope & Order Locks: Keep synchronized blocks small, avoid I/O inside locks, and acquire multiple locks in a fixed global order.
Answer:

CompletableFuture orchestrates asynchronous (often I/O-bound) stages on custom executors, whereas Parallel Streams split CPU-bound collection pipelines across ForkJoinPool.commonPool(), which gets starved if workers block on I/O. Key thread-safety practices include favoring immutable objects, using Atomic*/ConcurrentHashMap, and keeping locks minimal and consistently ordered.

← Spring Boot and Modern Web Development (part 2: 4.3–4.4)