Implementation and Testing
Open in the Padhaiko app3.1 Implementation and deployment of the project
Implementation in Project II transforms your detailed architecture into a working, integrated, reproducible, and deployable system (Product Track) or a complete, repeatable experimental pipeline (Research Track). Unlike early seventh-semester scripts, eighth-semester implementation must handle real error paths, clean configuration, and consistent environment packaging.
3.1.1 Incremental Implementation Sequence
Rather than coding isolated UI pages or standalone notebooks until the last week, build vertically in thin end-to-end slices:
- Core Data & Environment Setup: Initialize the database migrations, dataset loaders, or hardware firmware skeleton with pinned dependency manifests (
requirements.txt,package.json,platformio.ini) and.env.exampletemplates (never commit API keys or database passwords to Git). - Core Business Logic / Model Pipeline: Implement domain services, REST/WebSocket handlers, or training/evaluation scripts with structured error handling, meaningful status codes, and rotating log files.
- Interface & Hardware Integration: Connect frontend screens or physical sensors/actuators incrementally to real backend endpoints, replacing static mock JSON early.
- Hardening & Packaging: Add authentication guards, input sanitization, graceful offline/timeout fallbacks, and one-command startup scripts (
docker compose upor single-command build/flash).
3.1.2 Deployment Strategies Across Project Tracks
"Deployment" means placing the completed system into an operational environment where target users or evaluators can actually run and verify it.
| Project Type | Practical Deployment Target | Key Deployment Checklist |
|---|---|---|
| Web / SaaS Platform | Docker Compose on VPS/cloud VM or local campus server behind Nginx reverse proxy | HTTPS/TLS, environment variables, database backup script, health-check endpoint (/health) |
| Mobile Application | Signed Android APK installed on physical test devices + hosted backend API | Offline state handling, API base-URL configuration, permission prompts, crash logging |
| ML / Research Pipeline | Reproducible CLI + lightweight demo web UI (Gradio/FastAPI) or documented inference package | Fixed random seeds, saved model checkpoints, eval.py script reproducing thesis tables |
| Embedded / IoT Hybrid | Flashed microcontroller in protective enclosure + local gateway or cloud telemetry server | Watchdog timer enabled, brownout recovery, Wi-Fi/MQTT auto-reconnect, calibration constants |
3.2 Testing and validation of the developed system (unit testing, integration testing, and user testing)
Testing proves that your implementation works as specified; validation proves that the system actually solves the real-world problem for its intended users or research hypothesis. ENCT 461 explicitly requires three complementary testing layers: unit testing, integration testing, and user testing.
Unit, Integration, and User Testing
- Unit Testing: Verifies individual functions, classes, API validators, firmware drivers, or data preprocessing routines in isolation, checking normal inputs, boundary values, and error paths using automated assertions and mocks.
- Integration Testing: Verifies that combined modules interact correctly across boundaries—such as API-to-database transactions, frontend-to-backend contracts, microcontroller-to-sensor bus communication, or dataset-to-model inference pipelines.
- User Testing (User Acceptance Testing / UAT): Evaluates the complete system with representative end users (or domain stakeholders) performing realistic tasks to verify usability, workflow fit, clarity, and real-world acceptance.
3.2.1 Designing a Rigorous Test Case Matrix
For Chapter 5 of your final report, examiners expect structured test evidence rather than a single sentence claiming "everything worked." Document test IDs, preconditions, inputs, expected outputs, actual results, and pass/fail status across all three layers.
| Test Layer | What Is Verified | Typical Tools / Methods | Evidence for Final Report |
|---|---|---|---|
| Unit Testing | Pure functions, schema validators, auth helpers, feature extractors, sensor parsers | pytest, unittest, Jest, JUnit, Unity (C/embedded) | Automated test runner output + pass count and edge cases |
| Integration Testing | Auth + DB CRUD flows, MQTT-to-DB pipeline, edge-to-cloud sync, inference API | TestClient / Supertest, Postman/Newman, Docker test DB, hardware-in-the-loop | Integration test matrix with HTTP codes, payloads, and failure recovery checks |
| User Testing (UAT) | Task completion rate, error recovery by non-authors, UI clarity, field usability | Guided task sheet (5–15 target users) + System Usability Scale (SUS) survey | Task-success table, SUS score summary, qualitative feedback & UI fixes made |
def test_claim_amount_rejects_negative_and_zero():
"""Unit test: boundary check on validation helper."""
assert validate_dosage_ml(0.5) is True
assert validate_dosage_ml(0.0) is False
assert validate_dosage_ml(-2.0) is False
def test_telemetry_ingest_persists_to_db(test_client, db_session):
"""Integration test: API endpoint validates, stores, and retrieves sensor packet."""
payload = {"device_id": "node-01", "temp_c": 4.2, "humidity": 61.5}
res = test_client.post("/api/v1/telemetry", json=payload)
assert res.status_code == 201
fetched = test_client.get("/api/v1/telemetry/node-01/latest")
assert fetched.status_code == 200
assert fetched.json()["temp_c"] == 4.23.2.2 Conducting Credible User Testing (UAT)
Even a strong technical backend fails user validation if real operators find the workflow confusing:
- Recruit 8 to 20 representative users outside your three-member project group (students, clinicians, farmers, teachers, or company staff matching your target domain).
- Define 4 to 6 concrete tasks (without coaching users step-by-step on where to click) and record completion success, time taken, and points of confusion.
- Collect structured feedback using a brief 5-point Likert questionnaire or the standard 10-item System Usability Scale (SUS), then fix the top usability friction points and log the before/after improvement.
3.3 Performance evaluation and improvement
The ENCT 461 syllabus emphasizes systematic performance evaluation, critical comparison against existing solutions or baseline methods, and iterative improvement based on findings.
3.3.1 Choosing Appropriate Evaluation Metrics
Select quantitative metrics that genuinely reflect your system's goal—never cherry-pick a misleading metric (such as raw accuracy on an imbalanced 95:5 dataset).
| Aspect | Product-Oriented Evaluation Metrics | Research-Oriented Evaluation Metrics |
|---|---|---|
| Primary Quality Metrics | Task success rate, uptime/reliability, error rate, SUS usability score | Macro-F1, Precision/Recall, PR-AUC / ROC-AUC, RMSE/MAE, BLEU/WER, IoU/mAP |
| Speed & Resource Metrics | API latency (p50, p95, p99 in ms), throughput (req/s), memory/CPU, battery life | Inference latency (ms/sample), FLOPs/parameter count, training time, VRAM footprint |
| Comparison Standard | Legacy manual workflow, existing open-source/commercial tool, or unoptimized v1 build | Published baseline models, standard heuristics, and ablation variants of your own method |
| Failure / Error Analysis | Load-test saturation point, offline-sync recovery time, security scan findings | Confusion matrix, worst-performing subclasses, noise/domain-shift sensitivity |
3.3.2 Baseline Comparison, Ablation, and Iterative Improvement Loop
- Benchmark a clean baseline first: Measure your initial version or standard reference baseline on the exact same test split, hardware, or workload.
- Profile to locate the true bottleneck: Never guess what is slow or inaccurate. Use database
EXPLAIN ANALYZE, PythoncProfile, browser DevTools Network/Performance tabs, or per-class confusion matrices to find the dominant bottleneck. - Apply targeted improvements one variable at a time:
- Database/Backend: Add composite indexes, eliminate N+1 queries, introduce Redis/in-memory caching, or move heavy exports to async background jobs.
- ML/Edge Inference: Apply class-weighted loss or focal loss, domain-specific data augmentation, hyperparameter tuning, or ONNX / INT8 post-training quantization.
- Embedded/Hardware: Replace blocking
delay()calls with timer interrupts, add digital low-pass/median filtering for sensor spikes, or optimize deep-sleep duty cycles.
- Re-measure and document the delta: Report the exact before-and-after numbers under identical test conditions (for example, p95 query latency reduced from 480 ms to 32 ms, or macro-F1 improved from 0.71 to 0.84 after adding temporal features).
Designing a Three-Layer Test Matrix for a Product-Track Web & Mobile System
A group has implemented a Clinic Appointment & Prescription Triage Web/Mobile System with a React/Android frontend, FastAPI backend, and PostgreSQL database. Design a structured test plan covering unit testing, integration testing, and user testing (UAT) that produces defensible Chapter 5 evidence.
Define Unit Tests (isolated logic and boundary validation):
- Test slot-generation logic for overlapping doctor schedules, lunch breaks, and invalid past timestamps.
- Test role-permission guards (ensuring a patient token cannot query another patient's prescription ID) and password-hashing/JWT expiry helpers.
Define Integration Tests (end-to-end service and DB contracts):
- Run an ephemeral PostgreSQL test container with
pytest: verifyPOST /appointmentscreates a booking inside an ACID transaction and prevents double-booking the same slot under two concurrent requests. - Verify offline-queued clinic sync: submit two offline prescriptions from the Android client, restore connectivity, and confirm idempotency keys prevent duplicate records.
Design User Acceptance Testing (UAT) with real non-team participants:
- Recruit 10 volunteers (6 acting as patients, 4 acting as reception/clinic staff) to execute 5 scripted tasks (register, book next available slot, reschedule, view prescription QR, cancel).
- Measure task completion rate, average time per task, and SUS questionnaire score; record that 3 of 6 patients initially missed the "Reschedule" button, then move it to the primary appointment card and re-verify.
The group pairs automated pytest unit tests (slot rules, RBAC), containerized transactional integration tests (concurrency and offline sync idempotency), and a 10-user task-based UAT with SUS scoring and documented UI fixes.
Conducting Baseline Comparison, Ablation, and Performance Improvement (Research Track)
A Research-Track group built an Acoustic Cough & Breathing Classifier for Early Respiratory Screening on an imbalanced dataset (70% healthy, 20% mild, 10% severe). Their initial CNN baseline on raw MFCCs reached 79% overall accuracy, yet examiners note that overall accuracy hides poor detection of the minority severe class. Outline a systematic performance evaluation and improvement workflow.
Replace misleading overall accuracy with class-sensitive evaluation metrics: Compute macro-averaged F1, per-class Precision and Recall, a 3x3 confusion matrix, and one-vs-rest PR-AUC under 5-fold subject-independent cross-validation (ensuring recordings from the same patient never split across train and test folds).
Run error analysis and diagnose the bottleneck: Inspecting the confusion matrix reveals that the baseline achieves 0.91 recall on Healthy but only 0.42 recall on Severe because severe samples represent just 10% of the training batches and short silence segments dilute MFCC averages.
Apply controlled improvements via an ablation study:
- Baseline: Raw MFCC + 2D-CNN + standard cross-entropy.
- Ablation A (+Class-Weighted Focal Loss): Penalizes missed minority Severe cases.
- Ablation B (+Log-Mel Spectrogram + Delta Features + SpecAugment): Preserves fine spectral/temporal dynamics.
- Ablation C (+Pretrained Audio Backbone Fine-Tuning / Lightweight Separable CNN): Tests representation capacity and measures CPU/edge inference latency (ms per clip).
Document exact macro-F1, Severe-class recall, and inference latency across all four rows on identical test folds.
The group switches from raw accuracy to subject-independent macro-F1 and per-class recall, diagnoses minority-class collapse via a confusion matrix, and executes a step-by-step ablation study isolating focal loss, log-Mel/SpecAugment features, and lightweight architecture trade-offs.
Viva-Style Questions: Implementation and Testing
What is the difference between unit testing, integration testing, and user testing, and why are all three needed in Project II?
Roles of the three testing layers:
- Unit testing checks individual functions or components in isolation (fast, catches logic/edge-case bugs early).
- Integration testing checks data flow and contracts between connected modules (DB queries, APIs, sensor buses, model pipelines).
- User testing (UAT) checks whether real target users can complete actual tasks comfortably without confusion or workflow breakdowns.
Why none can replace the others: Passing all unit tests does not guarantee two modules agree on JSON field names or database transactions (integration bug); passing integration tests does not guarantee real users understand the interface or trust the system's workflow (usability/validation gap).
Unit tests verify isolated functions, integration tests verify inter-module contracts and data pipelines, and user tests validate real-world usability and workflow fit; all three together prove both technical correctness and practical usefulness.
What does "reproducible implementation" mean for both a Product-Track software project and a Research-Track ML project?
For Product Track: Anyone (teammates or evaluators) can clone the Git repository, follow a documented setup using pinned dependency files or Docker Compose, seed test data, and run the exact same working application without hidden local files or manual database tweaks.
For Research Track: Dataset split indices, preprocessing scripts, hyperparameters, and random seeds are fixed and versioned so running the evaluation script reproduces the reported metrics, confusion matrices, and comparison tables consistently.
Reproducibility means pinned environments, versioned configuration/seeds, and documented setup/evaluation scripts so a clean machine reproduces the exact running system (Product) or reported benchmark tables (Research).
Why is reporting only "accuracy" or "average response time" often criticized during ENCT 461 performance evaluation?
Hidden failure modes in averages:
- On imbalanced datasets (e.g., 92% normal, 8% fault), a trivial classifier predicting only "normal" gets 92% accuracy while failing completely on the critical minority class.
- On backend APIs, a low average latency (e.g., 45 ms) can hide severe tail latency (p95/p99 spikes of 3,000 ms under concurrent load or cache misses).
Better evaluation practice: Pair summary metrics with macro-F1 / per-class recall / confusion matrices for ML tasks, and p50/p95/p99 latency plus concurrent throughput for software systems.
Single averages mask minority-class failure on imbalanced data and tail-latency spikes under load; pairing macro-F1/per-class recall or p95/p99 latency reveals true system behavior.
What is an ablation study, and how does it strengthen Topic 3.3 (Performance evaluation and improvement)?
Definition: An ablation study systematically removes, swaps, or modifies one component at a time (such as a preprocessing step, loss term, feature group, or caching layer) while holding all other settings constant, and measures the resulting change in performance.
Value in Project II: It provides scientific proof of why your proposed improvement works—separating genuine contributions from accidental complexity—and directly answers examiner questions such as "Did module X really help, or would a simpler baseline perform just as well?"
An ablation study isolates one component at a time under identical conditions to prove quantitatively which design choices actually drive performance gains versus added complexity.
How should a team handle negative or disappointing test results discovered during Topic 3.3 evaluation (for instance, a model performing worse under low light or an endpoint slowing down under 100 concurrent users)?
Diagnose and iterate honestly: Document the failure honestly as a finding, profile or inspect error cases to identify the root cause (e.g., lack of low-light training samples or missing DB connection pooling), apply a targeted engineering fix, and re-measure on the same benchmark.
Report remaining limits transparently: If a constraint remains (for example, hardware memory caps batch size or extreme darkness still degrades accuracy below threshold), state it clearly in Scope/Limitations rather than hiding or fabricating numbers—evaluators reward rigorous failure analysis far more than suspiciously perfect claims.
Log the failure honestly, profile the root cause, apply and benchmark a targeted improvement, and document any remaining boundary conditions transparently in Limitations rather than hiding unfavorable cases.