Chapter 15:
Discuss the advantages and disadvantages of analytical evaluation.
One advantage of the analytical method is that it can be quickly completed and does not require the participation of users (Rogers et al, 2007). This is especially useful when users cannot be easily accessed or when their involvement will take too much time and be too expensive. On the other hand, its disadvantage is that the results can be considered guesses – such as the time hypothetical users need to complete a given task or the possible usability issues they might encounter during their interaction with the product. This then requires that the usability expert be capable of putting themselves in the shoes of another type of user who’s different from themselves.
An inspection method, the heuristic model can be performed at any stage of the design phase. Moreover, it can be used for the evaluation of devices that are used by people on-the-move, as it does not impose the difficulties that come with usability testing and field studies. However, although the evaluation is based on a set of heuristics, there’s still a possibility that the evaluator won’t be able to catch all of the problems. There can also be disagreement among the experts on the set of heuristics to be used. In addition, there’s a tendency for the evaluator to focus on a single usability aspect, which may result in other aspects being missed. It is then recommended that several evaluators be involved, although this can become very expensive.
Another method of the inspection approach is the cognitive walkthrough. It focuses on the identification of specific users’ problem in a highly detailed manner. Its narrow focus makes it useful for certain kinds of systems, such as applications that involve complex operations for the completion of tasks. Some of its disadvantages, though, are that it requires “a good understanding of the cognitive processes involved” (Rogers et al., p. 703) and is very time-consuming and tedious to perform.
Another type of walkthrough, the pluralistic walkthrough has the advantage of having a strong focus on the tasks of users at a detailed level where the steps taken are considered. This level of analysis can be useful for systems such as safety-critical systems where a usability problem that occurs in a single step can be critical to the application’s efficiency or safety. In addition, this method is appropriate for participatory design practices where a multi-disciplinary team is involved, which allows for various opinions and expertise to be raised. On the other hand, its disadvantages are that it may prove to be quite difficult to get all the experts together at once, that they need to proceed at the pace of the slowest, and that only a limited number of paths or scenarios can often be explored due to time constraints.
A benefit of the predictive methods is that the quantitative predictions allow for the easy comparison of various specifications, prototypes, and interfaces. Om the other hand, its disadvantage is that it has a very limited scope in that only routine computer-based tasks can be evaluated with these methods. In addition, the method only predicts expert performance and does not account for errors nor does it account for the participants’ individual differences. It is useful only for predicting predictable behavior, which makes it inappropriate for the evaluation f real-world tasks, which are unpredictable in nature.
References
Rogers, Y., Sharp, H., & Preece, J. (2007). Chapter 15: Analytical evaluation. In
Interaction design: Beyond human-computer interaction (2nd ed.) (684-747). Hoboken,
NJ: John Wiley & Sons
