Software Quality, Assurance, Maintenance
Open in the Padhaiko app7.1 Quality concepts
Software Quality (Pressman / IEEE)
Software quality is conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics expected of all professionally developed software (such as maintainability, reliability, and usability).
Modern quality management distinguishes three complementary ideas:
- Quality of Design: How well the specification and architectural design satisfy customer needs (grade of features, tolerances, UX).
- Quality of Conformance: How faithfully the implemented code matches the design and engineering specifications.
- Cost of Quality: Total cost incurred to achieve quality or suffered from failing to achieve it, divided into:
- Prevention Costs: Quality planning, formal technical reviews, training, test automation infrastructure.
- Appraisal Costs: Inspections, static analysis, unit/integration/system testing.
- Internal Failure Costs: Rework, re-testing, and debugging defects found before shipment.
- External Failure Costs: Customer support, hotfixes, warranty claims, lost reputation, and liability after release (often higher than prevention).
Defect Removal Efficiency (DRE)
A standard metric measuring how effectively reviews and testing catch errors before release is Defect Removal Efficiency ():
where is the number of errors found before software delivery to end users and is the number of defects discovered after delivery (during a fixed post-release window). Higher (approaching ) indicates strong early quality filtering.
7.2 Quality attributes
Quality attributes make abstract "software quality" measurable and testable. Two foundational taxonomies appear across Institute of Engineering syllabi:
1. McCall's Software Quality Factors (11 Factors in 3 Perspectives)
| Perspective | Quality Factors | Core Meaning |
|---|---|---|
| Product Operation (Using it) | Correctness, Reliability, Efficiency, Integrity, Usability | Does it compute right answers, stay up, use minimal CPU/RAM, resist unauthorized access, and feel easy to learn? |
| Product Revision (Changing it) | Maintainability, Flexibility, Testability | How easily can engineers locate and fix bugs, add new features, and test changes? |
| Product Transition (Moving it) | Portability, Reusability, Interoperability | How easily can it run on new hardware/OS, reuse modules in other systems, and exchange data with external systems? |
2. ISO/IEC 25010 Product Quality Model (8 Characteristics)
- Functional Suitability: Completeness, correctness, and appropriateness of functions.
- Performance Efficiency: Time behavior (latency), resource utilization, and capacity.
- Compatibility: Co-existence and interoperability with other systems.
- Usability: Learnability, operability, accessibility, and user error protection.
- Reliability: Maturity, availability, fault tolerance, and recoverability.
- Security: Confidentiality, integrity, non-repudiation, accountability, and authenticity.
- Maintainability: Modularity, reusability, analyzability, modifiability, and testability.
- Portability: Adaptability, installability, and replaceability.
7.3 Reviews, inspections and QA concepts
| Aspect | Quality Control (QC) | Quality Assurance (QA) |
|---|---|---|
| Nature | Product-oriented & reactive (inspects/tests deliverables to find defects) | Process-oriented & proactive (audits and improves the process to prevent defects) |
| Activities | Reviews, code inspections, unit/system/acceptance testing | Defining standards, process audits, metric tracking, training, tool enforcement |
| Goal | Catch and fix defects before release | Ensure the development process consistently produces high-quality software |
Software Quality Assurance (SQA) & Formal Technical Reviews (FTR)
An SQA group acts as the customer's internal advocate, performing independent process audits, ensuring technical reviews occur, tracking defect metrics, and verifying compliance with standards.
A Formal Technical Review (FTR)—which includes structured walkthroughs and Fagan inspections—is a peer-conducted quality filter applied to requirements, design models, and code. Its objectives are:
- Uncover errors in function, logic, or implementation in any representation of software.
- Verify that the artifact conforms to predefined standards and requirements.
- Achieve uniform, maintainable software developed across teams.
7.4 ISO standards, CMMI levels
Organizations demonstrate process maturity and quality governance using two major international frameworks:
1. ISO 9000 / ISO 9001 Quality Management System (with ISO/IEC 90003 for Software)
- ISO 9001 certifies that an organization maintains a documented, repeatable, auditable Quality Management System (QMS)—not just ad-hoc heroics.
- Core principles: Customer focus, leadership, engagement of people, process approach, continuous improvement, evidence-based decision-making, and relationship management.
- Core motto: "Document what you do, do what you documented, and prove it through records."
2. Capability Maturity Model Integration (CMMI)
Developed by the Software Engineering Institute (SEI), CMMI evaluates how mature and predictable an organization's engineering processes are across five staged maturity levels:
| Level | Name | Characteristics |
|---|---|---|
| Level 1 | Initial (Ad-hoc) | Processes are unpredictable, poorly controlled, and reactive; success depends on individual heroics rather than repeatable practice. |
| Level 2 | Managed | Basic project management tracks cost, schedule, and requirements at the project level; practices are repeatable on similar projects. |
| Level 3 | Defined | Engineering and management processes are standardized, documented, and tailored from an organization-wide standard process across all teams. |
| Level 4 | Quantitatively Managed | Processes are measured and controlled using statistical and quantitative metrics (e.g., defect density, variance limits, velocity bounds). |
| Level 5 | Optimizing | Organization focuses on continuous process improvement driven by quantitative feedback, root-cause defect prevention, and piloting innovative technologies. |
7.5 Software maintenance and its types
Software Maintenance (IEEE)
Software maintenance is the modification of a software product after delivery to correct faults, improve performance or other attributes, or adapt the product to a modified operating environment.
Maintenance is classically divided into four categories (Swanson / IEEE 1219):
| Maintenance Type | Purpose | Typical Share of Effort | Concrete Example |
|---|---|---|---|
| 1. Corrective | Diagnose and fix residual defects (bugs) discovered after release | Fixing a floating-point rounding bug in tax calculation | |
| 2. Adaptive | Modify software to keep working in a changed hardware, OS, database, or regulatory environment | Updating an app for Android 16 APIs or a new VAT tax rule | |
| 3. Perfective | Enhance features, performance, UI usability, or business reports requested by users | Adding dark mode, CSV export, or optimizing query speed by | |
| 4. Preventive | Proactively restructure/refactor code to improve maintainability and prevent future failures | Refactoring a monolithic legacy parser into clean modules and adding unit tests |
7.6 Maintenance effort and lifecycle considerations
Post-release maintenance typically consumes of total software lifecycle cost. Understanding what drives maintenance effort helps engineers design systems that age gracefully.
Belady–Lehman Maintenance Effort Model
Belady and Lehman modeled annual software maintenance effort as:
where:
- = productive effort (actual analysis, design, and coding of the change),
- = an empirical organization/project constant,
- = complexity caused by poor structure, tight coupling, and lack of modularity,
- = familiarity / documentation degree (understanding of the system by maintainers).
Key Engineering Insight: Because sits inside an exponential, high complexity () combined with poor documentation/familiarity () causes non-productive maintenance overhead to explode exponentially!
Lehman's Laws of Software Evolution (Key Principles)
- Continuing Change: An operational real-world system must continually adapt or become progressively less useful.
- Increasing Complexity: As a system evolves, its internal complexity grows unless deliberate preventive maintenance (refactoring) is invested to reduce it.
- Declining Quality: Quality appears to decline unless the system is rigorously maintained and adapted to its changing environment.
Lifecycle Considerations & Reengineering
To contain lifecycle maintenance cost:
- Design for Maintainability Upfront: Enforce high cohesion, low coupling, automated regression suites, and living documentation.
- Reverse Engineering: Analyze existing legacy binaries/code to recover design abstractions, data models, and control flow.
- Software Reengineering: Examine and alter an existing legacy system to reconstitute it in a new, more maintainable form (restructuring code, normalizing databases, adding automated tests) without discarding core business logic.
Worked Examples
7.1
During the development of a university billing portal, Team A finds errors through reviews and testing before launch, and customers report defects in the first months after release. After adopting formal code inspections and automated unit tests in Release 2, Team A finds errors pre-release and customers report only defects post-release.
- Compute the Defect Removal Efficiency () for Release 1 and Release 2.
- Explain how preventive/appraisal investments shifted the Cost of Quality between the two releases.
Part 1 — Calculate for Release 1 and Release 2: Using :
Part 2 — Cost of Quality Shift:
- improved from in Release 1 to in Release 2.
- Investing slightly more in pre-release appraisal (inspections and unit tests) sharply reduced expensive post-release external failure costs.
7.2
- A cloud attendance service achieves a Mean Time To Failure () of and a Mean Time To Repair () of (). Calculate its steady-state Availability percentage.
- Over one year, a software team logs corrective bug fixes ( each), perfective feature enhancements ( each), and adaptive OS/API updates ( each). Compute the total annual maintenance hours and identify which maintenance category dominates.
Part 1 — Availability Calculation:
Part 2 — Annual Maintenance Effort Breakdown:
- Corrective:
- Perfective:
- Adaptive:
Perfective maintenance (, or ) dominates, matching typical industry maintenance patterns.
- Steady-state availability is .
- Total annual maintenance effort is , dominated by perfective maintenance ().
7.3
How to Answer a Past Exam Question: "Compare ISO 9001 and CMMI, and explain the Belady–Lehman maintenance effort equation ."
Part 1 — ISO 9001 vs. CMMI: Explain that ISO 9001 is a binary (certified / not certified) quality management system standard applicable to any industry ("document what you do, do what you documented"), whereas CMMI is a software/systems engineering maturity model with 5 staged levels (Initial Managed Defined Quantitatively Managed Optimizing) providing a continuous improvement roadmap.
Part 2 — Belady–Lehman Equation Interpretation: State every symbol ( productive effort, empirical constant, structural complexity, maintainer familiarity/documentation) and emphasize why exponential growth in occurs whenever poor modularization () coincides with missing documentation ().
An exam-ready answer contrasts ISO 9001 (binary QMS certification) against CMMI (5-level maturity progression) and defines all terms in to show how low modularity () and weak documentation () drive exponential maintenance overhead.
Exercises
Practice Exercises — Software Quality, Assurance & Maintenance
Define software quality and explain the four categories of Cost of Quality (Prevention, Appraisal, Internal Failure, and External Failure).
Software Quality Definition: Software quality is conformance to explicitly stated functional/performance requirements, documented standards, and implicit professional expectations (usability, maintainability, reliability).
Four Categories of Cost of Quality:
- Prevention: Quality planning, reviews, standards, training, CI setup.
- Appraisal: Inspections, static analysis, and testing before release.
- Internal Failure: Rework and re-testing defects caught prior to shipment.
- External Failure: Customer support, emergency patches, liability, and reputation loss after shipment.
Software quality means meeting explicit requirements, standards, and implicit expectations. Cost of Quality comprises Prevention, Appraisal, Internal Failure, and External Failure costs—with external failures being the costliest by far.
List McCall's three quality perspectives (with key factors in each) and name the eight characteristics of the ISO/IEC 25010 software product quality model.
McCall's Three Perspectives:
- Product Operation: Correctness, Reliability, Efficiency, Integrity, Usability.
- Product Revision: Maintainability, Flexibility, Testability.
- Product Transition: Portability, Reusability, Interoperability.
ISO/IEC 25010 Eight Quality Characteristics: Functional Suitability, Performance Efficiency, Compatibility, Usability, Reliability, Security, Maintainability, and Portability.
McCall groups factors into Product Operation, Product Revision, and Product Transition; ISO/IEC 25010 defines eight characteristics: Functional Suitability, Performance Efficiency, Compatibility, Usability, Reliability, Security, Maintainability, and Portability.
Distinguish between Quality Assurance (QA) and Quality Control (QC), and list the five CMMI maturity levels with one defining characteristic of each.
QA vs. QC: QA is proactive and process-oriented (ensuring the engineering process prevents defects via standards, audits, and training), whereas QC is reactive and product-oriented (testing and inspecting deliverables to detect defects).
Five CMMI Maturity Levels:
- Initial: Ad-hoc, chaotic, dependent on heroics.
- Managed: Basic project-level cost, schedule, and requirement tracking.
- Defined: Standardized, documented organization-wide process tailored per project.
- Quantitatively Managed: Statistical and metric-driven process control.
- Optimizing: Continuous process improvement via quantitative feedback and innovation.
QA improves the process proactively; QC inspects the product reactively. The five CMMI levels are 1. Initial, 2. Managed, 3. Defined, 4. Quantitatively Managed, and 5. Optimizing.
Describe the four types of software maintenance (Corrective, Adaptive, Perfective, and Preventive) with a concrete example of each, and explain what software reengineering is.
Four Types of Maintenance:
- Corrective: Fixing post-release bugs (e.g., patching a null-pointer crash).
- Adaptive: Adapting to new OS/hardware/regulatory environments (e.g., migrating from Python 3.8 to 3.12).
- Perfective: Adding user-requested enhancements or performance gains (e.g., adding export filters).
- Preventive: Refactoring internal structure to prevent future decay (e.g., splitting a God class and adding tests).
Software Reengineering: Reengineering analyzes a legacy system (reverse engineering) and restructures/modernizes its code, architecture, or data schema into a maintainable form while preserving core business behavior.
The four maintenance types are Corrective (bug fixes), Adaptive (environment changes), Perfective (new features/performance), and Preventive (proactive refactoring). Reengineering reorganizes legacy code and data into a clean, maintainable form without rewriting business logic from scratch.
- A software product uncovers defects prior to release and defects during its first year in production. Calculate its Defect Removal Efficiency (). If rises to and falls to in the next release, compute the new .
- A server application has and . Compute its Availability percentage.
- Using the Belady–Lehman equation , explain two engineering practices that reduce lifecycle maintenance effort .
Part 1 — Defect Removal Efficiency (): Release 1:
Release 2:
Part 2 — Availability Percentage:
Part 3 — Reducing :
- Modular refactoring & low coupling lower structural complexity , shrinking the exponent .
- Clear documentation, clean code, and automated tests raise maintainer familiarity , further reducing the exponential non-productive overhead.
- increases from to .
- Availability .
- Reducing structural complexity (modularization) and increasing familiarity (documentation and automated tests) exponentially reduces non-productive maintenance overhead .