Software testing is defined as the process of executing software to determine its functionality. The goal is to demonstrate to the developers as well as the users that the software meets the technical and business requirements (Homè̀s, 2012). Standards on the other hands are documents outlining guidelines, specifications, characteristics, or requirements to ensure that products are consistent and meet their purpose. There are national and international standards that guide the development and testing processes for software. In Australia, the AS/NZS ISO/IEC 25062:2006 is a standard that provides User Interface (UI) designers or software usability professionals procedures for testing the usability of a computer program. The standard was developed from the international standard, ISO/IEC 25062:2006, Software engineering—Software product Quality Requirements and Evaluation (SQuaRE)—Common Industry Format (CIF), for usability test reports. An example of a universal standard discussed in this document is the IEEE 829 - 2008 standard, also referred to as IEEE 829 (IEEE Standard for Software and System Test Documentation, 2008). This is the Institute of Electrical and Electronics Engineers (IEEE) Standard for Software and system test documentation. It is a revision of the IEEE Std 829 – 1998. This standard specifies the form and type of processes, and documents that organization should use to control the different types of software testing.
Australian/New Zealand Usability Testing Standard
The AS/NZS ISO/IEC 25062:2006 is a product of the Standards Australia and Standards New Zealand Committee. It is copyrighted under the joint task force Standards Australia/Standards New Zealand that was responsible for its preparation (Software engineering—Software product Quality Requirements and Evaluation (SQuaRE)—Common Industry Format (CIF) for usability test reports AS/NZS, 2006). It applies to software engineers in the two countries. Software development companies mostly perform several usability tests to gauge the user experience and satisfaction. Their goal is to produce software that is easy to use, acceptable, appealing, and flexible to the users. AS/NZS ISO/IEC 25062:2006 is used to report the results of the usability tests. Regarding scope, it is recommended for use by usability professionals in software development firms or suppliers, technical experts and managers within the customer organizations. Usability experts refer to the standard in generating usability reports that can be interpreted by any user or client firms. The organizations or users can examine the reports and then confer with the AS/NZS ISO/IEC 25062:2006 standard to check that they meet international standards for usability. Software technicians and managers in the organization can base their business decisions to purchase the products on the reports generated. They not only evaluate the usability results presented in the reports but the technical merit and suitability of the product.
Key terms of AS/NZS ISO/IEC 25062:2006 include usability, efficiency, effectiveness, user satisfaction, accessibility, the context of use among others. These terms are used throughout the standard’s reports. Usability means the extent to which product in use achieves the set goals effectively, efficiently with user satisfaction in a given context of use (Software engineering—Software product Quality Requirements and Evaluation (SQuaRE)—Common Industry Format (CIF) for usability test reports AS/NZS, 2006). On the other hand, effectiveness is used to refer to the accuracy with which users of a software product can achieve specified goals besides completeness. Efficiency in the context of usability is used to refer to the production in relation to amount of resources used in attaining the set objectives (Myers, Sandler, & Badgett, 2012; Wasson, 2015). With satisfaction, this is the positive attitudes and freedom of the user from unwarranted discomfort following interaction with an application. A context of use is the environment in which a particular computer program is used. The environment comprises of the hardware, software, users among other physical and social environment factors that may influence the performance of the software (Wasson, 2015). About accessibility, this refers to the ease with which users can interact with a product regardless of the environment or the ability of a person. There are other terms captured in the AS/NZS ISO/IEC 25062:2006 standard like goals, tasks, and user which are easily understood.
The primary purpose of the standard is to integrate usability with other decision-making processes in an organization, for example, procurement and upgrade of applications, and office automation. Developers, UI designers, software engineers, and organizations all have a standard reference for improvement of usability. This is relevant to usability testing as it can help organizations deploy software that is easy to use, efficient with an enjoyable user experience which is likely to affect the acceptability in the market.
IEEE 829 – 2009 Standard
As already mentioned, IEEE 829 – 2009 is an IEEE copyright Standard for Software and System Test Documentation that came into effect in the year 2008. The revision of earlier standard was meant to address the issue of integrity levels of software. According to IEEE Standard for Software and System Test Documentation (2008), IEEE 829 – 2009 also changed the focus to the testing processes from the document-focus of IEEE Std 829 – 1998. It also provided a method of choosing contents and documents for any test conducted ("IEEE 829 Documentation and how it fits in with testing", n.d.).
About the scope, the standard covers all software-based systems. It should be applied to software still in development, or already developed. Also, the testing processes and documents standards apply to all Commercial-Off-the-Shelf (COTS) systems. The standard stipulates the software testing environments and recommends examination of other parts of the systems that interact with the software (IEEE Standard for Software and System Test Documentation, 2008). It helps testers to identify the processes to use in addition to tasks to be performed and documents for recording results during testing. As a result, it makes it easy to determine software attributes like correctness, accuracy, completeness, testability, and consistency. IEEE 829 – 2009 can be referenced by developers, software suppliers, testers, software technicians, auditors, the legal system, managers and local users in organizations as well.
Key terms to understand in the standard include checkout, component, anomaly, integration testing, development, and operational testing, integrity level, and quality among others. Checkout is a test conducted after installation of an application to ensure that it performs as required (Craig & Jaskiel, 2002). A component is a single part of a system while anomaly is any error found in system functionality. In an integrated testing, different software and hardware components are combined and tested to check whether they can work together without any hitch (IEEE Standard for Software and System Test Documentation, 2008). Development testing is performed on new software to ensure that it meets desired criteria. On the other hand, operational testing is conducted in the real environment in which the program is expected to work. Integrity level is used to refer to the extent to which a computer program conforms to a set of expected software characteristics. Quality in the IEEE 829 – 2009 has been used to refer to the degree to which a program can meet specification requirements.
According to Chen (n.d), one of the main goals of IEEE 829 – 2009 was to set up a universal framework for software testing processes in support of the software life cycle. Besides, the standard helps to identify the inputs and outputs of each test conducted. Furthermore, it recommends appropriate contents, and documents to be used for each type of test conducted. This standard is relevant for testing as it helps to organize the testing procedures and reports which make it easy to interpret even for individuals without technical know-how. It is also easy for testers to trace the source of errors in a system with clearly defined procedures.
Comparison between AS/NZS ISO/IEC 25062:2006 and IEEE 829 – 2009
An apparent commonality between the AS/NZS ISO/IEC 25062:2006 and IEEE 829 – 2009 is that both are applied in the entire life-cycle of the software. From development to implementation and maintenance, the standards provide the testing procedures to be followed to ensure that the products meet the specified requirements. Secondly, both also improve the usability of an application. Besides, the two recommend user involvements in the testing procedures. This involvement will provide testers and developers feedback on where to make improvements. Regarding usage or scope, both the standards can be used by software developers, suppliers, client organizations, and the users in determining that the software meets specification requirements.
A definite difference between AS/NZS ISO/IEC 25062:2006 and IEEE 829 – 2009 is that besides being more focused on testing processes after review of the earlier standard, IEEE 829 – 2009 also provides a clear format of documentation to use in each test. AS/NZS ISO/IEC 25062:2006, on the other hand, is more concerned about the report generated after usability tests. Another difference is that AS/NZS ISO/IEC 2506 appears to address software concerns at the Application Program Interface level. API influence the usability as it deals with the Graphical User Interface. The IEEE 829 – 2009, on the other hand, addresses all the technical aspects of the software from processing to GUI that will make an application meet the functional requirements. We can also note that since AS/NZS ISO/IEC 2506 is about usability test, it addresses the black-box testing technique whereby users do not have the knowledge of how a system works. IEEE 829 – 2009 on the hand involves a combination of both white-box and black-box testing techniques; it can be used by users with or without in-depth knowledge of how a system works.
Conclusion
Even though Australia and New Zealand Standards made a revision of the ISO/IEC 25062:2006, the AS/NZS ISO/IEC 2506 does not have much difference and can still be applied internationally. AS/NZS ISO/IEC 2506 and IEEE 829 – 2009 standards outline the importance of software testing. Testing improves efficiency, user experience in addition to helping software engineers find, and fix bugs. Having a single standard that guides software development and testing procedures make it easy to deploy software across different regions. Furthermore, a standard also helps to ensure the quality, reliability, and safety of products. For software-based companies and client organizations, the testing standards can be used as strategic tools for costs reduction through minimization of errors and wastes while at the same time improve productivity.
References
Chen, N. IEEE std 829-2008 and Agile Process– Can They Work Together? (1st ed.). California, USA. Retrieved from http://worldcomp-proceedings.com/proc/p2013/SER3559.pdf
Craig, R. & Jaskiel, S. (2002). Systematic software testing. Boston: Artech House.
Homè̀s, B. (2012). Fundamentals of software testing. Hoboken, NJ: ISTE/Wiley.
IEEE 829 Documentation and how it fits in with testing. Coleyconsulting.co.uk. Retrieved 4 May 2016, from http://www.coleyconsulting.co.uk/IEEE829.htm
IEEE Standard for Software and System Test Documentation. (2008) (1st ed.). New York, NY 10016-5997, USA. Retrieved from http://dis.unal.edu.co/~icasta/ggs/Documentos/Normas/829-2008.pdf
Myers, G., Sandler, C., & Badgett, T. (2012). The art of software testing. Hoboken, N.J.: John Wiley & Sons.
Software engineering—Software product Quality Requirements and Evaluation (SQuaRE)— Common Industry Format (CIF) for usability test reports AS/NZS. (2006) (1st ed.). Retrieved from https://infostore.saiglobal.com/Store/PreviewDoc.aspx?saleItemID=497671
Wasson, C. (2015). System Engineering Analysis, Design, and Development: Concepts, Principles, and Practices. John Wiley & Sons.
