PadhaikoAll subjects

Edge Computing, Emerging Trends and Case Studies

Distributed and Cloud Computing (ENCT 411)

Open in the Padhaiko app

9.1 Edge and Fog Computing: Architecture and the IoT–Edge–Cloud Continuum

Traditional cloud computing routes every sensor reading across wide-area networks to centralized data centers, causing unacceptable WAN latency (), backbone bandwidth saturation, and privacy exposure for real-time IoT workloads.

Definition

Edge Computing vs. Fog Computing

The Three-Tier IoT–Edge–Cloud Continuum

Three-Tier IoT - Edge/Fog - Cloud ContinuumText
[Tier 1: IoT / End Devices]   Sensors, wearables, autonomous vehicles, PLC actuators
         | (BLE / Wi-Fi / 5G / LoRaWAN: < 5 ms)
[Tier 2: Edge / Fog Layer]    Gateways, Multi-Access Edge Computing (MEC), local micro-DCs
         | (Filtered aggregates, models, anomalies over WAN: 50-150 ms)
[Tier 3: Central Cloud]       Hyperscale Regions (S3 data lakes, long-term ML training)
AspectEdge / Fog ComputingCentralized Cloud Computing
Proximity & LatencyPhysically near data source ()Remote regional data centers ()
Bandwidth ConsumptionLow (filters/aggregates raw video/sensor streams locally)High (raw data shipped upstream over WAN)
Offline AutonomyOperates during WAN disconnects (local safety loops)Unavailable when WAN link drops
Compute & Storage CapacityConstrained by power, thermal, and physical spaceVirtually elastic (thousands of GPUs, petabyte storage)

9.2 Task Offloading, Migration, and Edge–Cloud Collaboration Models

Computational Task Offloading

Resource-constrained mobile or IoT devices (Mobile Edge Computing, MEC) decide dynamically whether to execute a task locally or offload it to a nearby edge server or distant cloud:

where is workload (CPU cycles), is CPU frequency (cycles/s), is data size (bits), and is link rate (bps).

Live Service Migration Across Edge Nodes

When a mobile user (e.g., connected car) moves between 5G cell towers, its stateful edge container/VM must migrate to the new MEC host using:

  1. Cold Migration: Stop container, copy full state/memory, restart (simple, noticeable downtime).
  2. Pre-Copy Live Migration: Copy memory pages iteratively while container keeps running, then pause briefly for final dirty-page transfer (minimal downtime).
  3. Post-Copy / Stateless Migration: Move execution immediately and fetch missing memory pages on demand (or keep state in a shared regional store).

Edge–Cloud Collaboration Models


9.3 AI-Driven Cloud Optimization and Distributed Systems for Large-Scale AI

9.3.1 ML-Based Auto-Scaling and Cloud Optimization

Traditional threshold auto-scalers (e.g., "add Pods if CPU ") react after a traffic spike arrives, leaving users exposed to cold-start latency. AI-driven cloud optimization uses machine learning proactively:

9.3.2 Distributed Systems for Large-Scale AI Training and Inference

Modern foundation models exceed the memory and compute of any single GPU, requiring distributed parallelization across GPU clusters connected via NVLink and RDMA/InfiniBand:

  1. Data Parallelism (DP): Every GPU holds a full replica of the model and processes a different mini-batch shard. After backward pass, GPUs synchronize gradients via Ring-AllReduce (communication cost independent of cluster size : bytes for model size ) before updating weights.
  2. Tensor (Intra-Layer) Parallelism: Splits individual large matrix multiplications (attention heads, MLP layers) across GPUs within a single node over ultra-fast NVLink ().
  3. Pipeline (Inter-Layer) Parallelism: Assigns contiguous groups of transformer layers to sequential GPU stages and streams micro-batches through the pipeline to minimize idle "pipeline bubbles".
  4. Distributed Inference Serving: Separates prefill (prompt compute-bound) and decode (token-generation memory-bandwidth-bound) phases, using KV-cache paging (PagedAttention), continuous batching, and model quantization across GPU replicas.

9.4 Hybrid and Multi-Cloud Strategies

AspectHybrid CloudMulti-Cloud
CompositionCombines private (on-premises) infrastructure + one or more public cloudsUses two or more public cloud providers (e.g., AWS + GCP + Azure)
Primary DriversData sovereignty, legacy mainframe integration, regulated workloads, cloud burstingAvoiding vendor lock-in, best-of-breed services (e.g., GCP BigQuery + AWS EC2), geo-resilience
Key ChallengesWAN latency between on-prem and cloud; unified IAM/security perimeterCross-cloud data egress fees, heterogeneous APIs, unified observability

Architectural Enablers


9.6 Case Studies: Google, Amazon, and Netflix

9.6.1 Google Distributed Infrastructure

Google pioneered hyperscale commodity distributed systems:

9.6.2 Amazon: Dynamo and AWS Global Infrastructure

9.6.3 Netflix Microservices Architecture

Netflix transitioned from a single-datacenter monolith (2008 DB outage) to an elastic, fault-isolated cloud-native architecture on AWS ( microservices):


Worked Out Examples

Example 9.1

9.1

A smart-factory inspection camera must run an object-detection task requiring CPU cycles on a () image frame (output bounding-box result is negligible, ). Compare the total completion latency across three execution tiers:

  1. Local Device: CPU speed ().
  2. Edge MEC Server: CPU speed (), uplink rate (), base propagation RTT ().
  3. Central Cloud: CPU speed (), WAN uplink rate (), WAN RTT ().

Which tier meets a real-time factory safety deadline?

1. Local Execution Latency:

2. Edge Offloading Latency:

3. Central Cloud Offloading Latency:

Only Edge MEC () meets the deadline—Cloud has the fastest CPU () yet loses due to the WAN upload!

Answer:
  1. Local = , Edge = , Cloud = .
  2. Only Edge MEC () satisfies the real-time deadline.
Example 9.2

9.2

An -GPU cluster trains a model of parameter gradient size using Data Parallelism with Ring-AllReduce. Each GPU computes forward + backward pass for its local micro-batch in , and the inter-GPU network provides effective Ring-AllReduce bandwidth ().

  1. Calculate the Ring-AllReduce communication volume transferred per GPU per step using .
  2. Compute the communication time and total step time (assuming non-overlapped communication), and determine the computation-to-total-time efficiency.

1. Ring-AllReduce Volume per GPU:

2. Communication Time, Step Time, and Efficiency:

(This explains why real distributed frameworks like PyTorch DDP overlap gradient AllReduce with backward computation and rely on high-speed NVLink/InfiniBand fabrics.)

Answer:
  1. Ring-AllReduce transfers per GPU per step.
  2. , total step time = , compute efficiency .
Example 9.3

9.3

Case Study Comparison (Exam Synthesis): Compare how Google Spanner and Amazon Dynamo sit on opposite ends of the CAP Theorem and how Netflix applies fault isolation in its microservices architecture.

1. Google Spanner (CP — External Consistency) vs. Amazon Dynamo (AP — Always Writable):

  • Spanner (CP): Uses Paxos replication groups per shard + Two-Phase Commit (2PC) across shards + TrueTime hardware atomic/GPS clocks (wait-out uncertainty window ). Guarantees external consistency (linearizable ACID) globally, sacrificing availability if a majority quorum is unreachable.
  • Dynamo (AP): Uses Consistent Hashing, Sloppy Quorums / Hinted Handoff, and Vector Clocks so shopping-cart writes succeed even during partitions—trading strict consistency for always-on write availability and resolving sibling versions on read.

2. Netflix Fault Isolation and Chaos Engineering:

  • Circuit Breakers & Bulkheads: Wrap every remote service call so slow or failing downstream microservices trip open immediately and return degraded fallback UI (e.g., static recommendations) rather than exhausting thread pools across the whole platform.
  • Open Connect & Chaos Monkey: Offloads streaming bandwidth to ISP-embedded CDN nodes and injects random instance failures in production to prove resilience continuously.
Answer:

Google Spanner prioritizes global ACID/external consistency (CP) via Paxos + TrueTime, whereas Amazon Dynamo maximizes write availability (AP) via consistent hashing, sloppy quorums, and vector clocks. Netflix prevents cascading microservice outages using circuit breakers, graceful fallbacks, ISP-edge CDNs, and Chaos Engineering.


Exercises

Practice Exercises — Edge Computing, Emerging Trends and Case Studies

Question 1

Compare Edge Computing, Fog Computing, and Cloud Computing within the IoT–Edge–Cloud continuum, explaining four benefits of pushing computation to the edge.

Continuum Roles:

  • Edge: Processes data right at/adjacent to end sensors/MEC base stations.
  • Fog: Forms a multi-tier intermediate compute/network fabric bridging edge gateways and the cloud.
  • Cloud: Provides elastic central storage and heavy batch/ML training.

Four Benefits of Edge Processing:

  1. Sub- ultra-low latency for real-time control.
  2. Reduced WAN uplink bandwidth costs via local filtering/aggregation.
  3. Continued offline autonomy during WAN disconnects.
  4. Improved data privacy/sovereignty by keeping sensitive raw feeds local.
Answer:

Edge computes right at the data source, Fog spans intermediate network tiers, and Cloud handles heavy centralized analytics—reducing latency, WAN bandwidth, outage risk, and privacy exposure.

Question 2

Distinguish between binary and partial task offloading, and describe the three live migration strategies for moving edge containers across hosts. Numerical part: An AR headset needs cycles on a () frame. Local CPU runs at , while an edge server runs at over a link with . Compare local vs. edge completion times.

Offloading and Migration:

  • Binary vs. Partial Offloading: Binary moves the whole task remotely or runs it locally; partial splits a task DAG between local and edge nodes.
  • Migration: Cold (stop-copy-restart), Pre-Copy (iterative memory copy while running, short pause), and Post-Copy (move execution first, demand-page memory later).

Latency Calculation:

Answer:

Edge offloading drops completion time from (local) to ()—nearly faster.

Question 3

Explain how ML-based predictive auto-scaling improves upon reactive threshold scaling, and compare Data Parallelism, Tensor Parallelism, and Pipeline Parallelism in distributed AI training. Numerical part: Calculate the Ring-AllReduce communication volume per GPU per step for an -GPU Data-Parallel job with gradient payload .

ML Auto-Scaling & Parallelism Strategies:

  • Predictive Auto-Scaling: Forecasts traffic ahead of spikes using time-series ML to pre-warm instances and eliminate cold-start latency.
  • Data Parallelism: Replicates the full model per GPU and averages gradients across data shards via AllReduce.
  • Tensor Parallelism: Splits individual layer matrix multiplications across GPUs inside a fast NVLink node.
  • Pipeline Parallelism: Places sequential layer groups on different GPUs and streams micro-batches across stages.

Ring-AllReduce Volume (, ):

Answer:

Predictive ML scaling eliminates cold-start lag; Data, Tensor, and Pipeline parallelism scale model training across shards, intra-layer matrices, and sequential layers. Ring-AllReduce transfers per GPU per step.

Question 4

Differentiate between Hybrid Cloud and Multi-Cloud strategies. State two key technical or economic challenges of multi-cloud deployments and how Kubernetes and IaC help address them.

Hybrid vs. Multi-Cloud:

  • Hybrid Cloud: Integrates private on-premises infrastructure with public cloud(s) for data residency and cloud bursting.
  • Multi-Cloud: Uses two or more public clouds simultaneously for best-of-breed services, vendor independence, and resilience.

Challenges & Enablers:

  • Challenges: High cross-cloud data egress fees, divergent provider APIs/IAM models, and operational complexity.
  • Enablers: Kubernetes provides a uniform container runtime API across clouds, while Terraform/OpenTofu IaC standardizes network/compute provisioning across providers.
Answer:

Hybrid links private and public clouds; Multi-Cloud spans multiple public clouds. Kubernetes and IaC mitigate provider API fragmentation, though egress fees and cross-cloud IAM remain key challenges.

Question 5

Summarize the key architectural innovations in:

  1. Google Spanner (TrueTime + Paxos),
  2. Amazon Dynamo (consistent hashing + sloppy quorums + vector clocks), and
  3. Netflix Microservices (service discovery, circuit breakers, Open Connect CDN).

Google Spanner and Amazon Dynamo:

  • Google Spanner: Achieves global linearizable ACID (external consistency) using per-shard Paxos + 2PC + TrueTime GPS/atomic-clock uncertainty waits.
  • Amazon Dynamo: Achieves always-writable high availability (AP) via consistent hashing, sloppy quorums / hinted handoff, Merkle trees, and vector clocks for sibling reconciliation.

Netflix Microservices:

  • Uses Eureka + API gateway for dynamic routing, circuit breakers with graceful fallbacks and Chaos Engineering to isolate downstream failures, and Open Connect ISP-embedded CDN appliances for global streaming scale.
Answer:

Spanner combines Paxos and TrueTime for global ACID (CP); Dynamo pairs consistent hashing, sloppy quorums, and vector clocks for always-on writes (AP); Netflix isolates microservice failures via circuit breakers, Chaos Engineering, and ISP-edge Open Connect CDNs.

← Fault Tolerance and Reliability