7.1 Software Quality

• What is Quality

  • "a characteristic or attribute of something"

  There are 3 types of quality:

  1. Quality of design: requirements, specifications, and design of system
  2. Quality of conformance: an issue that focuses on implementation
  3. User Satisfaction: complaint product + good quality + delivery within budget and schedule
• Definition of Software Quality

  An effective software process applied in a manner that createsa useful product that provides measurable value for those whoproduce it and those who use it.
• Advantages of providing useful products

  1. Greater software product revenue
  2. Better profit when an application supports a business process
  3. Improved availability of information is crucial for the business
• Software Quality – Effective Process

  • establishes infrastructure that helps building a high-quality software product
  • The management aspects of process create the checks and balances that help avoid project chaos (usually what causes poor quality).
  • Software eng practices help the developer analyze the problem and design a solid solution (needed for building high quality software)
  • Related to management and technical reviews (umbrella activities)
• Software Quality – Useful Product

  • useful products deliver content, functions, and features that the end-user wants
  • Delivers this stuff in a reliable error free way
  • Always satisfies requirements that were explicitly stated by stakeholders
  • Satisfies implicit requirements that all high-quality software has
• Software Quality – Adding Value

  • By adding value for the user and developer of a software, high quality software provides benefits for the software organization and the end-users.
  • software organization gains added value because highquality software needs less maintenance effort, bug fixes, and reduced customer support.
  • The users gains added value because the application provides a useful capability in a way that expedites some business process
• Quality in Use

  1. Effectiveness: Accuracy and completeness with which users achieve goals
  2. Efficiency: Resources expended to achieve user goals completely with desired accuracy
  3. Satisfaction: Usefulness, trust, pleasure, comfort
  4. Freedom from risk: reducing economic, health, safety, and environmental risks
  5. Context coverage: Completeness, flexibility
• Product Quality

  1. Functional suitability: Complete, correct, appropriate.
  2. Performance efficiency: Timing, resource use, capacity
  3. Compatibility: Coexistence, interoperability.
  4. Usability: Appropriateness, learnability, operability, error protection, aesthetics, accessibility.
  5. Reliability: Maturity, availability, fault tolerance
  6. Security: Confidentiality, integrity, authenticity
  7. Maintainability: Reusability, modifiability, testability
  8. Portability. Adaptability, installability, replaceability
• Qualitative Quality Assessment

  • focus on the complete software product and can be used as an indication of the quality of an application
  • Your team might decide to create a user survey and a set of structured tasks for users to perform for each quality factor you want to assess
  • You might observe the users while they perform these tasks and have them complete the questionnaire when they finish
  • For some quality factors it may be important to test the software in the wild (or in the production environment)
• Quantitative Quality Assessment

  • the eng community tries to develop precise measures for software quality
  • Internal code attributes can sometimes be described quantitatively using software metrics
  • If software metric values for a code fragment are outside the range of acceptable values, it might mean there is a quality problem
  • Metrics represent indirect measures; we never really measurequality but rather some manifestation of quality
  • The complicating factor is the accuracy of the relationship between the variable that is measured and the quality of software.
• What are Reviews

  • A meeting conducted by technical people
  • A technical assessment of a work product created during the software engineering process
  • A way to ensure software quality
  • A training ground
• What are NOT reviews

  • A project summary or progress assessment
  • A meeting intended solely to impart information
  • A mechanism for political or personal reprisal
• Cost Impact of Software Defects

  • Error: quality problem found before the software is released to end users
  • Defect: quality problem found after the software is released to end-users
  • There is a difference between errors and defects because they have different economic, business, psychological, and human impact
  • 50-65% of software defects are seen through design activities
  • Review activities uncover about 75% of design flaws
  • The sooner you find a defect the cheaper it is to fix it.
• Defect Amplification and Removal

  1. Defect Amplification: how a defect introduced earlier in the software eng work flow (in requirements modeling) and undetected usually turns into multiple errors in design and more errors in construction
  2. Defect Propagation: the impact of an undiscovered error on future development or product behavior
  3. Technical Debt: the costs incurred by not finding and fixing effects early or not updating documentation after software changes
• A Model of Revies includes

  1. Planning and preparation
  2. Meeting structure
  3. Correction and verification
  4. Roles individuals play
• Informal Reviews

  • Benefit: immediately discovering errors and better work product quality
  • These include:
  • A simple desk check of a software engineering work product with a colleague
  • A casual meeting (with more than 2 people) to review a work product
  • The review-oriented aspects of pair programming which encourages continuous review as work is created
• Formal Technical Reviews


  The purpose of FTR is:

  • To uncover errors in function, logic, or implementation for any representation of the software
  • To verify that the software under review meets its requirements
  • To ensure that the software has been represented according to predefined standards
  • To achieve software that is developed in a uniform manner
  • To make projects more manageable
• Review Meeting

  • 3-5 people (typically) should be involved in the review.
  • You should prepare in advance but prep should require no more than two hours of work for each person
  • The review meeting should NOT last more than 2 hours
  • Focus is on a work product (ex: a portion of a requirements model, a detailed component design, source code for a component)
• Review Players

  • Producer - the person who has developed the work product
  • Review leader - evaluates the product for readiness, generates copies of product materials, and distributes them to two or three reviewers for advance preparation and facilitates the meeting discussion
  • Reviewer(s) - expected to spend between one and two hours reviewing the product, making notes, and otherwise becoming familiar with the work.
  • Recorder - reviewer who records (in writing) all important issues raised during the review.
• Review Outcome


  At the end of the review, all attendees of the FTR must decide whether to

  1. Accept the product without further modification
  2. Reject the product due to severe errors (once corrected, another review must be performed)
  3. Accept the product provisionally (minor errors have been encountered and must be corrected, but no additional review will be required)
• Review Reporting and Record Keeping

  • During the FTR, the recorder records all issues raised and summarizes these in a review issues list to serve as an action list for the producer.
  • A formal technical review summary report is created that answers three questions: What was reviewed?, Who reviewed it?, What were the findings and conclusions?
  • You should establish a follow-up procedure to ensure that items on the issues list have been properly corrected
• Review Guidelines

  1. Review the product, not the producer.
  2. Set an agenda and maintain it.
  3. Limit debate and rebuttal
  4. Enunciate problem areas, but don’t try to solve every problem noted
  5. Take written notes.
  6. Limit the number of participants and insist upon advance preparation
  7. Develop a checklist for each product that is likely to be reviewed.
  8. Allocate resources and schedule time for FTRs
  9. Conduct meaningful training for all reviewers
  10. Review your early reviews.
• Postmortem Evaluations (PME)

  • PME is a way to determine what went right and what went wrong with the software eng process and practices used for a specific project
  • PME us attended by members of the software team and stakeholders who examine the software project, focusing on excellences (achievements and positive experiences) and challenges (problems and negative experiences)
  • Intent is to extract lessons learned from the challenges and excellences and to suggest improvements to process and practice moving forward