Software quality


In the context of software engineering, software quality refers to two related but distinct notions:
  • Software's functional quality reflects how well it complies with or conforms to a given design, based on functional requirements or specifications. That attribute can also be described as the fitness for the purpose of a piece of software or how it compares to competitors in the marketplace as a worthwhile product. It is the degree to which the correct software was produced.
  • Software structural quality refers to how it meets non-functional requirements that support the delivery of the functional requirements, such as robustness or maintainability. It has a lot more to do with the degree to which the software works as needed.
Many aspects of structural quality can be evaluated only statically through the analysis of the software's inner structure, its source code, at the unit level, and at the system level, which is in effect how its architecture adheres to sound principles of software architecture outlined in a paper on the topic by Object Management Group.
Some structural qualities, such as usability, can be assessed only dynamically. Other aspects, such as reliability, might involve not only the software but also the underlying hardware, therefore, it can be assessed both statically and dynamically.
Using automated tests and fitness functions can help to maintain some of the quality related attributes.
Functional quality is typically assessed dynamically but it is also possible to use static tests.
Historically, the structure, classification, and terminology of attributes and metrics applicable to software quality management have been derived or extracted from the ISO 9126 and the subsequent ISO/IEC 25000 standard. Based on these models, the Consortium for IT Software Quality has defined five major desirable structural characteristics needed for a piece of software to provide business value: Reliability, Efficiency, Security, Maintainability, and Size.
Software quality measurement quantifies to what extent a software program or system rates along each of these five dimensions. An aggregated measure of software quality can be computed through a qualitative or a quantitative scoring scheme or a mix of both and then a weighting system reflecting the priorities. This view of software quality being positioned on a linear continuum is supplemented by the analysis of "critical programming errors" that under specific circumstances can lead to catastrophic outages or performance degradations that make a given system unsuitable for use regardless of rating based on aggregated measurements. Such programming errors found at the system level represent up to 90 percent of production issues, whilst at the unit-level, even if far more numerous, programming errors account for less than 10 percent of production issues. As a consequence, code quality without the context of the whole system, as W. Edwards Deming described it, has limited value.
To view, explore, analyze, and communicate software quality measurements, concepts and techniques of information visualization provide visual, interactive means useful, in particular, if several software quality measures have to be related to each other or to components of a software or system. For example, software maps represent a specialized approach that "can express and combine information about software development, software quality, and system dynamics".
Software quality also plays a role in the release phase of a software project. Specifically, the quality and establishment of the release processes, configuration management are important parts of an overall software engineering process.

Motivation

Software quality is motivated by at least two main perspectives:
  • Risk management: Software failure has caused more than inconvenience. Software errors can cause human fatalities. The causes have ranged from poorly designed user interfaces to direct programming errors, see for example Boeing 737 case or Unintended acceleration cases or Therac-25 cases. This resulted in requirements for the development of some types of software, particularly and historically for software embedded in medical and other devices that regulate critical infrastructures: " see Java programs stalling for one third of a second to perform garbage collection and update the user interface, and they envision airplanes falling out of the sky.". In the United States, within the Federal Aviation Administration, the FAA Aircraft Certification Service provides software programs, policy, guidance and training, focus on software and Complex Electronic Hardware that has an effect on the airborne product. Certification standards such as DO-178C, ISO 26262, IEC 62304, etc. provide guidance.
  • Cost management: As in any other fields of engineering, a software product or service governed by good software quality costs less to maintain, is easier to understand and can change more cost-effective in response to pressing business needs. Industry data demonstrate that poor application structural quality in core business applications, customer relationship management results in cost, schedule overruns and creates waste in the form of rework. Moreover, poor structural quality is strongly correlated with high-impact business disruptions due to corrupted data, application outages, security breaches, and performance problems.
  • *CISQ reports on the cost of poor quality estimates an impact of:
  • **$2.08 trillion in 2020
  • **
  • *IBM's Cost of a Data Breach Report 2020 estimates that the average global costs of a data breach:
  • **$3.86 million

    Definitions

ISO

Software quality is the "capability of a software product to conform to requirements." while for others it can be synonymous with customer- or value-creation or even defect level. Software quality measurements can be split into three parts: process quality, product quality which includes internal and external properties and lastly, quality in use, which is the effect of the software.

ASQ

uses the following definition: Software quality describes the desirable attributes of software products. There are two main approaches exist: defect management and quality attributes.

NIST

Software Assurance covers both the property and the process to achieve it:
  • confidence that software is free from vulnerabilities, either intentionally designed into the software or accidentally inserted at any time during its life cycle and that the software functions in the intended manner
  • The planned and systematic set of activities that ensure that software life cycle processes and products conform to requirements, standards, and procedures

    PMI

The Project Management Institute's PMBOK Guide "Software Extension" defines not "Software quality" itself, but Software Quality Assurance as "a continuous process that audits other software processes to ensure that those processes are being followed." whereas Software Quality Control means ''"taking care of applying methods, tools, techniques to ensure satisfaction of the work products toward quality requirements for a software under development or modification."''

Other general and historic

The first definition of quality in recorded history is from Shewhart in the beginning of 20th century: "There are two common aspects of quality: one of them has to do with the consideration of the quality of a thing as an objective reality independent of the existence of man. The other has to do with what we think, feel or sense as a result of the objective reality. In other words, there is a subjective side of quality."
Kitchenham and Pfleeger, further reporting the teachings of David Garvin, identify five different perspectives on quality:
  • The transcendental perspective deals with the metaphysical aspect of quality. In this view of quality, it is "something toward which we strive as an ideal, but may never implement completely". It can hardly be defined, but is similar to what a federal judge once commented about obscenity: "I know it when I see it".
  • The user perspective is concerned with the appropriateness of the product for a given context of use. Whereas the transcendental view is ethereal, the user view is more concrete, grounded in the product characteristics that meet user's needs.
  • The manufacturing perspective represents quality as conformance to requirements. This aspect of quality is stressed by standards such as ISO 9001, which defines quality as "the degree to which a set of inherent characteristics fulfills requirements".
  • The product perspective implies that quality can be appreciated by measuring the inherent characteristics of the product.
  • The final perspective of quality is value-based. This perspective recognizes that the different perspectives of quality may have different importance, or value, to various stakeholders.
Tom DeMarco has proposed that "a product's quality is a function of how much it changes the world for the better." This can be interpreted as meaning that functional quality and user satisfaction are more important than structural quality in determining software quality.
Another definition, coined by Gerald Weinberg in Quality Software Management: Systems Thinking, is "Quality is value to some person."

Other meanings and controversies

One of the challenges in defining quality is that "everyone feels they understand it" and other definitions of software quality could be based on extending the various descriptions of the concept of quality used in business.
Software quality also often gets mixed-up with Quality Assurance
or Problem Resolution Management
or Quality Control
or DevOps.
It does overlap with these areas, but it is distinctive as it does not solely focus on testing but also on processes, management, improvements, assessments, etc.