Virtualization, Containers and Cloud-Native Systems
Open in the Padhaiko app7.1 Virtualization: Full, Para-, and Hardware-Assisted Virtualization
System Virtualization
Virtualization creates an isolated software abstraction (Virtual Machine, VM) of physical hardware (CPU, RAM, disk, NIC) via a Virtual Machine Monitor (VMM) or Hypervisor, allowing multiple independent Guest OS instances to share one physical host safely.
On x86 CPUs, classic Popek-Goldberg "trap-and-emulate" fails for certain sensitive-but-unprivileged instructions (e.g., SGDT, SIDT, POPF) that fail silently instead of trapping when run outside Ring 0. Three CPU virtualization techniques solve this:
- Full Virtualization (Binary Translation):
- The Guest OS runs completely unmodified in Ring 1 (or Ring 3), while the hypervisor runs in Ring 0.
- At runtime, the hypervisor scans guest kernel instruction blocks and dynamically translates unsafe sensitive instructions into safe hypervisor calls (binary translation), caching translated blocks.
- Pros/Cons: Runs any OS unmodified, but software translation incurs noticeable CPU overhead.
- Paravirtualization:
- The Guest OS kernel is explicitly modified (recompiled) to replace sensitive non-virtualizable instructions with direct hypercalls to the hypervisor (analogous to system calls).
- Pros/Cons: Eliminates binary translation overhead and achieves near-native I/O via shared ring buffers (e.g., Xen PV,
virtio), but requires modifying the guest OS (cannot run closed-source OSs unmodified).
- Hardware-Assisted Virtualization (Intel VT-x / AMD-V):
- Adds hardware CPU support (VMX Root Mode for the hypervisor and VMX Non-Root Mode for the guest OS), letting an unmodified Guest OS run directly at Ring 0 inside non-root mode while sensitive operations automatically trigger a VM Exit trap to the hypervisor (EPT / NPT hardware page tables also eliminate shadow-page-table overhead).
| Technique | Guest OS Modification? | Mechanism for Sensitive Instructions | Performance Overhead | Examples |
|---|---|---|---|---|
| Full Virtualization | No (unmodified) | Dynamic binary translation + caching | Moderate | Early VMware Workstation/ESX |
| Paravirtualization | Yes (patched kernel) | Explicit hypercalls to hypervisor | Low | Classic Xen PV, virtio drivers |
| Hardware-Assisted | No (unmodified) | CPU root/non-root modes + hardware VM Exit + EPT | Very low (near-native) | KVM, VMware ESXi, Xen HVM |
7.2 Hypervisors: Type 1 and Type 2; Server, Storage, and Network Virtualization
7.2.1 Type 1 (Bare-Metal) vs. Type 2 (Hosted) Hypervisors
| Aspect | Type 1 (Bare-Metal) Hypervisor | Type 2 (Hosted) Hypervisor |
|---|---|---|
| Execution Layer | Runs directly on physical hardware (no general-purpose host OS underneath) | Runs as an application on top of a Host OS (Windows, macOS, Linux) |
| Performance & Latency | Highest (direct hardware scheduling and DMA/SR-IOV passthrough) | Lower (double scheduling + Host OS I/O layer overhead) |
| Security & Isolation | Minimal Trusted Computing Base (small attack surface) | Depends on full Host OS security |
| Examples & Use Cases | VMware ESXi, Xen, Microsoft Hyper-V, KVM (Linux kernel acts as Type 1) | VirtualBox, VMware Workstation, QEMU (user-mode) — desktop dev/testing |
- KVM (Kernel-based Virtual Machine): Turns the Linux kernel itself into a Type 1 hypervisor (
/dev/kvmmodule) using VT-x/AMD-V, while QEMU emulates peripheral devices. - Xen: Microkernel Type 1 hypervisor booting an initial privileged control domain (Dom0) that manages unprivileged guest domains (DomU).
- VMware ESXi: Proprietary bare-metal Type 1 enterprise hypervisor (
vmkernel) widely used in private clouds.
7.2.2 Server, Storage, and Network Virtualization
- Server Virtualization: Partitions physical CPU sockets/cores, NUMA memory, and PCIe devices across isolated VMs with live migration (vMotion / KVM live migration transferring dirty RAM pages iteratively before final cutover).
- Storage Virtualization: Abstracts physical LUNs/NVMe drives into logical storage pools, thin-provisioned virtual disks (
.vmdk,.qcow2), and software-defined distributed volumes (Ceph RBD, vSAN). - Network Virtualization (NFV & Overlays): Creates virtual NICs (
vNIC), virtual switches (Open vSwitch), and VXLAN / GENEVE L2-over-L3 tunnels so isolated tenant subnets span any physical rack without re-cabling VLANs.
7.3 Containers: Namespaces, cgroups, and Containers vs. Virtual Machines
Unlike VMs that virtualize hardware and boot a full guest OS kernel per VM, containers implement OS-level virtualization: multiple isolated user-space processes share the single host Linux kernel.
Core Linux Kernel Primitives Behind Containers
- Linux Namespaces (Isolation — "What a Process Can See"):
pid: Isolated process ID tree (container has its ownPID 1).net: Independent network stack, IP addresses, routing tables, andeth0veth pairs.mnt: Isolated filesystem mount points (pivot_root+ OverlayFS layered image).ipc/uts/user: Separate SysV IPC, hostname, and UID/GID mapping (rootless containers).
- Control Groups —
cgroups(Resource Accounting & Limits — "What a Process Can Use"):- Enforce hard quotas on CPU shares/cores (
cpu.max), memory (memory.max, OOM killer scope), block I/O (io.max), and process counts (pids.max) so one noisy container cannot starve the host.
- Enforce hard quotas on CPU shares/cores (
- Union / Layered Filesystems (OverlayFS): Stack immutable read-only image layers beneath one thin writable container layer via Copy-on-Write, enabling instant image reuse.
| Aspect | Virtual Machines (VMs) | Containers (Docker / containerd) |
|---|---|---|
| Isolation Boundary | Hardware-level (separate Guest OS + kernel per VM) | Process-level (shared host Linux kernel via namespaces + cgroups) |
| Image / Disk Footprint | Gigabytes (full OS + libraries + app) | Tens to hundreds of megabytes (shared image layers + app) |
| Startup Time | Tens of seconds to minutes (full OS boot) | Milliseconds to |
| Resource Overhead & Density | High (duplicated OS RAM/CPU); VMs/host | Minimal; containers/host |
| Security Isolation | Strongest (hardware VT-x boundary; runs Windows & Linux) | Kernel-shared (mitigated via seccomp, AppArmor, gVisor/Kata) |
7.4 Kubernetes: Architecture, Pods, Deployments, Services, Scaling, and Self-Healing
Kubernetes (K8s) is a declarative container orchestration platform that automates scheduling, service discovery, rolling updates, auto-scaling, and self-healing across clusters.
Kubernetes Cluster Architecture
+---------------------- CONTROL PLANE ----------------------+
| kube-apiserver <--> etcd (Raft cluster state store) |
| kube-scheduler | kube-controller-manager |
+---------------------------+-------------------------------+
| (HTTPS / Watch API)
+------------------+------------------+
v v
+--- WORKER NODE 1 ---+ +--- WORKER NODE 2 ---+
| kubelet | kube-proxy| | kubelet | kube-proxy|
| containerd: [Pods] | | containerd: [Pods] |
+---------------------+ +---------------------+- Control Plane Components:
kube-apiserver: Central REST/gRPC gateway validating and persisting cluster objects.etcd: Distributed Raft key-value store holding the authoritative cluster state.kube-scheduler: Watches unscheduled Pods and binds them to optimal nodes (filtering + scoring CPU/RAM/affinity).kube-controller-manager: Runs reconciliation loops ensuring actual state == desired state.
- Worker Node Components:
kubelet: Node agent ensuring containers in assigned Pods are running and passing health probes.kube-proxy: Configuresiptables/IPVSrules for Service virtual IPs.- Container Runtime (CRI):
containerd/ CRI-O pulling images and launching containers.
Core Abstractions, Scaling, and Self-Healing
- Pod: The smallest deployable unit—one or more tightly coupled containers sharing a single network namespace (
localhost), IP address, and shared volumes. Pods are ephemeral. - Deployment & ReplicaSet: Declaratively manage stateless Pod replicas (
replicas: 5), supporting zero-downtime rolling updates (maxSurge,maxUnavailable) and instant rollbacks. - Service: Provides a stable virtual IP (ClusterIP), node port (NodePort), or cloud LoadBalancer + internal DNS name (
orders-svc.default.svc.cluster.local) over dynamic Pod IPs selected by labels. - Scaling: Horizontal Pod Autoscaler (HPA) adjusts Pod
replicasbased on CPU/RAM or custom RPS metrics:
while Cluster Autoscaler adds/removes worker VMs.
- Self-Healing:
livenessProberestarts deadlocked containers;readinessProberemoves unready Pods from Service endpoints;ReplicaSetimmediately reschedules Pods on healthy nodes if a worker node fails.
7.5 Microservices Architecture: API Gateway, Service Discovery, and Service Mesh
Microservices decompose a monolithic application into small, independently deployable services organized around business capabilities, each owning its private database and communicating over lightweight APIs (REST/gRPC/events).
- API Gateway: Single external entry point handling TLS termination, request routing, authentication/JWT verification, rate limiting, and response aggregation—shielding internal services from clients.
- Service Discovery: Tracks dynamic ephemeral container IP:port endpoints via:
- Client-Side Discovery: Clients query a service registry (Netflix Eureka, Consul) and load-balance locally.
- Server-Side / DNS Discovery: Clients call a fixed DNS/Virtual IP (Kubernetes
Service+CoreDNS+kube-proxy), and the platform routes to healthy Pods.
- Service Mesh (Istio, Linkerd): Offloads cross-cutting network concerns out of application code into a transparent sidecar proxy (e.g., Envoy) injected into every Pod:
- Data Plane (Sidecar Proxies): Intercepts all Pod traffic to enforce mTLS encryption, retries, timeouts, circuit breaking, canary traffic splitting (), and distributed tracing/metrics.
- Control Plane (
istiod): Pushes routing policies, TLS certificates, and telemetry config dynamically to all sidecars.
7.6 Cloud-Native Application Design: Twelve-Factor App, DevOps, CI/CD, and IaC
The Twelve-Factor App Methodology (Key Principles)
Designed for SaaS and containerized microservices:
- Codebase: One codebase tracked in version control (Git), many deploys.
- Dependencies: Explicitly declare and isolate dependencies (
requirements.txt,package.json, container image). - Config: Store environment-specific config (DB URLs, secrets) in environment variables, never hardcoded in source.
- Backing Services: Treat databases, caches, and queues as attached replaceable resources via URLs.
- Build, Release, Run: Strictly separate immutable build artifacts, release config combination, and stateless execution.
- Processes: Execute the app as stateless, share-nothing processes (persist state only in backing stores).
- Port Binding, Concurrency, Disposability: Self-contained HTTP port binding; scale out via process model; fast startup and graceful shutdown (
SIGTERM) for container orchestration. - Dev/Prod Parity, Logs as Event Streams (
stdout), Admin Processes as one-off containers.
DevOps, CI/CD, and Infrastructure as Code (IaC)
- DevOps: Cultural and engineering alignment unifying development and operations via automation, shared ownership, and observability.
- CI/CD Pipelines:
- Continuous Integration (CI): Automated build, unit/integration tests, security scan, and immutable container image push on every commit.
- Continuous Delivery / Deployment (CD): Automated canary/blue-green rollout to staging and production clusters (GitOps with ArgoCD/Flux).
- Infrastructure as Code (IaC): Provision VPCs, Kubernetes clusters, IAM roles, and databases from declarative versioned files (Terraform, OpenTofu, Pulumi) rather than manual console clicks—making environments reproducible, auditable, and drift-free.
Worked Out Examples
7.1
A physical server has of usable RAM (assume host hypervisor/OS reserves in both cases? Wait—let's distinguish Type-1 VM Guest OS overhead vs. shared Host OS Container overhead clearly!). Let the server have total physical RAM and run identical microservice workloads, each requiring RAM for the application itself:
- VM Deployment: Each VM runs a full Guest OS consuming RAM overhead in addition to the application (assume hypervisor host overhead is negligible).
- Container Deployment: A single shared Host Linux OS consumes RAM once, and each container runs only the application with negligible per-container OS overhead.
Calculate:
- Maximum number of instances that fit under VMs vs. Containers.
- Effective application RAM utilization percentage in both deployments.
1. VM Deployment Capacity ( Guest OS + App ):
2. Container Deployment Capacity ( Shared Host OS + App/Container): Remaining RAM after the single shared Host OS :
(Note: For smaller microservices—e.g., apps—the Guest OS advantage of containers is even more dramatic: higher density!)
- VMs: instances; Containers: instances (plus spare).
- Useful application RAM utilization: (VMs) vs. (Containers).
7.2
A Kubernetes Deployment currently runs Pod replicas with a Horizontal Pod Autoscaler (HPA) targeting average CPU utilization.
- During a flash sale, the average CPU utilization across the Pods rises to . Using the Kubernetes HPA scaling formula, compute the new
desiredReplicas(always rounding up with ceiling). - Later, a new container image is rolled out on a Deployment with
replicas: 10,maxSurge: 3, andmaxUnavailable: 2. Compute the maximum total Pods allowed during the rollout and the minimum healthy available Pods guaranteed during the rollout.
1. HPA Desired Replicas Calculation:
2. Rolling Update Bounds (replicas = 10, maxSurge = 3, maxUnavailable = 2):
- Maximum Total Pods (Old + New):
- Minimum Available Healthy Pods:
Thus up to new Pods can be created immediately ( surge + replacing terminated old Pods) while keeping at least healthy Pods serving traffic.
- HPA scales from to Pods.
- Rolling update allows at most total Pods and guarantees at least available Pods.
7.3
Cloud-Native Microservices Design: An online ticketing platform is migrating from a legacy VM monolith to a cloud-native Kubernetes microservices architecture.
- Explain how an API Gateway, Kubernetes Service/CoreDNS, and a Service Mesh (Istio + Envoy sidecars) collaborate for an incoming ticket-booking request.
- Identify three Twelve-Factor App violations common in legacy monoliths (hardcoded DB passwords, local
/tmp/sessionfiles, manual SSH server edits) and state the cloud-native fix for each.
1. Request Path Across Gateway, Discovery, and Service Mesh:
- API Gateway: Terminates external HTTPS, validates the user's JWT token, enforces rate limiting, and routes
/bookto the internalbooking-svc. - Service Discovery (
CoreDNS+Service): Resolvesbooking-svcto healthy Pod endpoints dynamically as HPA scales Pods up/down. - Service Mesh (Envoy Sidecars): Intercepts Pod-to-Pod gRPC calls (
bookingpayment), establishing mutual TLS (mTLS), applying retry/circuit-breaker rules, and exporting distributed traces automatically.
2. Fixing Legacy Monolith Anti-Patterns via Twelve-Factor Rules:
- Hardcoded DB passwords Factor III (Config): Inject credentials via environment variables or Kubernetes
Secrets. - Local
/tmp/sessionfiles Factor VI (Stateless Processes) & IV (Backing Services): Store user sessions in an external Redis cluster so any Pod can serve any request and Pods remain disposable (Factor IX). - Manual SSH edits on VMs Factor V (Build/Release/Run) & IaC: Use immutable Docker images built by CI/CD and declarative Terraform/Kubernetes manifests in Git.
The API Gateway handles edge auth/rate limits, K8s Service/DNS tracks dynamic Pod IPs, and the Service Mesh enforces mTLS, circuit breaking, and observability via sidecars. Twelve-Factor compliance moves secrets to env config, sessions to external backing stores, and server changes to immutable CI/CD images + IaC.
Exercises
Practice Exercises — Virtualization, Containers and Cloud-Native Systems
Compare Full Virtualization, Paravirtualization, and Hardware-Assisted Virtualization (Intel VT-x/AMD-V), and distinguish between Type 1 and Type 2 hypervisors with two examples of each.
Virtualization Techniques:
- Full Virtualization: Runs unmodified Guest OS using runtime binary translation of non-virtualizable instructions.
- Paravirtualization: Modifies Guest OS kernel to issue explicit hypercalls for higher performance.
- Hardware-Assisted (VT-x/AMD-V): CPU adds root/non-root execution modes and automatic VM Exit traps plus EPT memory virtualization so unmodified Guest OSs run at near-native speed.
Type 1 vs. Type 2 Hypervisors:
- Type 1 (Bare-Metal): Runs directly on physical hardware with minimal TCB and highest performance (e.g., VMware ESXi, Xen, KVM, Hyper-V).
- Type 2 (Hosted): Runs as a user-space program inside a Host OS (e.g., VirtualBox, VMware Workstation).
Full uses binary translation, Para uses modified guest hypercalls, and Hardware-assisted uses CPU root/non-root modes for unmodified guests. Type 1 hypervisors (ESXi, Xen, KVM) run directly on bare metal, whereas Type 2 hypervisors (VirtualBox, VMware Workstation) run on top of a Host OS.
Explain the roles of Linux namespaces and cgroups in containerization, and list three differences between Containers and Virtual Machines. Numerical part: A server runs microservice instances. Under VMs, each VM adds Guest OS RAM overhead; under Containers, a single Host OS uses RAM once with zero per-container OS overhead. How many instances fit in each, and what is the percentage increase in capacity?
Namespaces, cgroups, and VM vs. Container:
- Namespaces isolate visibility (
pid,net,mnt,ipc,uts,user), whilecgroupsenforce CPU, RAM, and I/O resource limits. - Containers share the host kernel (MBs, millisecond startup, higher density), whereas VMs virtualize hardware and boot separate Guest OS kernels (GBs, slower boot, stronger hardware isolation).
Density Calculation ( Host):
- VMs ( per VM): .
- Containers ( usable at each): .
- Capacity Increase: .
Namespaces isolate process views and cgroups cap resource usage on a shared kernel. On RAM, containers fit vs. VMs ( more capacity).
Describe the functions of kube-apiserver, etcd, kube-scheduler, kubelet, and kube-proxy in Kubernetes, and distinguish between a Pod, a Deployment, and a Service. Numerical part: An HPA with target average CPU utilization currently runs Pods at average CPU. What desiredReplicas does HPA compute? If load later drops to across those Pods, what does it scale down to?
Kubernetes Components and Objects:
- Control Plane:
kube-apiserver(API gateway),etcd(Raft state store),kube-scheduler(places Pods on nodes). Node:kubelet(manages Pod containers/probes),kube-proxy(network routing). - Objects: Pod (ephemeral container group sharing IP), Deployment (declarative replica count + rolling updates), Service (stable virtual IP + DNS load balancing over Pods).
HPA Calculations:
- Scale out ( Pods at vs. target): .
- Scale in ( Pods at vs. target): .
kube-apiserver, etcd, and kube-scheduler run the control plane while kubelet and kube-proxy operate worker nodes for Pods, Deployments, and Services. HPA scales and later scales down to .
In a microservices architecture, explain the distinct responsibilities of an API Gateway, Service Discovery, and a Service Mesh (control plane vs. sidecar data plane).
Three Microservices Infrastructure Layers:
- API Gateway: Handles north-south external traffic—TLS termination, authentication, rate limiting, and path routing.
- Service Discovery: Maps logical service names to dynamic healthy backend Pod/container IP addresses via registries (Eureka/Consul) or DNS/virtual IPs (Kubernetes Service).
- Service Mesh: Handles east-west internal service-to-service traffic using injected sidecar proxies (Envoy data plane) managed by a control plane (
istiod) to provide transparent mTLS, retries, circuit breaking, traffic splitting, and distributed tracing.
An API Gateway manages external edge routing/auth, Service Discovery resolves dynamic backend IPs, and a Service Mesh injects sidecar proxies for transparent mTLS, resilience policies, and observability between internal services.
Explain six key principles of the Twelve-Factor App, and describe how DevOps, CI/CD, and Infrastructure as Code (IaC) enable reliable cloud-native delivery.
Six Twelve-Factor App Principles:
- Codebase: One Git repo per app, many deploys.
- Dependencies: Explicitly declared and isolated.
- Config: Stored in environment variables, not code.
- Backing Services: Databases/queues attached via URLs.
- Stateless Processes: Share-nothing execution; state externalized.
- Disposability & Logs: Fast startup, graceful
SIGTERMshutdown, logs streamed tostdout.
DevOps, CI/CD, and IaC:
- DevOps aligns engineering and operations around automation and shared reliability.
- CI/CD automates testing, container image builds, and zero-downtime canary/rolling deployments.
- IaC (Terraform/OpenTofu) defines cloud networks, clusters, and IAM in versioned declarative code, eliminating manual configuration drift.
Key Twelve-Factor rules (single codebase, explicit dependencies, env config, attached backing services, stateless processes, fast disposable instances) pair with DevOps, automated CI/CD pipelines, and declarative IaC to deliver reproducible, zero-downtime cloud deployments.