Web Accessibility Guideline Aggregation for Older Users and Its Validation

Web site-evaluation methodologies and validation engines take the view that all accessibility guidelines must be met to gain compliance. Problems exist in this regard, as contradictions within the rule set may arise, and the type of impairment or its severity is not isolated. The Barrier Walkthrough (BW) method goes someway to addressing these issues, by enabling barrier types derived from guidelines to be applied to different user categories such as motor or visual impairment, etc. However, the problem remains of combinatorial explosion of possibilities when one has to consider users with multiple disabilities. In this paper, a simple set theory operation is used to create a validation scheme for older users by aggregating barrier types specific to motor impaired and low-vision users, thereby creating a new “older users” category from the results of this set union. To evaluate the feasibility and validity of this aggregation approach, two BW experiments were conducted. The first experiment evaluated the aggregated results by focusing on quality attributes and showed that aggregation generates data whose quality is comparable to the original one. However, this first experiment could not test for validity, as the older users category was not included. To remedy this deficiency, another BW experiment was conducted with expert judges who evaluated a web page in the context of older users. In this second experiment, it was found that there is no significant difference between the aggregated and the manually evaluated (by experts) barrier scores, and that the same barriers are identified using experts and aggregation, even though there are differences in how severity scores are distributed. From these results, it is concluded that the aggregation of barriers is a viable alternative to expert evaluation, when the target of that aggregation could not be evaluated manually or it would not be feasible to do so. It is also argued that aggregation is a technique that can be used in combination with other evaluation methods, like user testing or subjective assessments.