Examining the Effects of Hypertext Shape on User Performance

By Michael L. Bernard¹,²

Studies examining the depth and breadth of hypertext structures have consistently found that increasing its depth correspondingly decreases its search efficiency. This is typically reflected in increased user search time, disorientation, and error, along with reduced satisfaction (e.g., Jacko, & Salvendy, 1996; Kiger, 1984; Larson & Czerwinski, 1998; Snowberry, Parkinson, & Sisson, 1983; Zaphiris, 2000). For example, Snowberry, et al. (1983), who examined four structures that consisted of 64 menu item choices on a single level (64); four menu items per level at a depth of three levels (4 x 4 x 4); eight menu items per level at a depth of two levels (8 x 8); and binary menu items at a depth of six levels (2 x 2 x 2 x 2 x 2 x 2), found that as the degree of hypertext depth increased from one to six levels, the rate of user errors rose from 4.0% to 34%. Likewise, Kiger (1984) found that increasing depth from two to six levels increased its user error rate from 2.2% to 12.5%. Participants also favored the shallowest structure over the deepest ones (Norman, 1990). It has therefore been recommended by numerous researchers that the design emphasis should be placed on reducing the overall depth of a hierarchy by correspondingly increasing the overall degree of menu item breadth.

However, because there is a practical limit to the degree of breadth (as well as depth) within a hypertext, compromises must take place within the design of a structure. To do this, depending on its contents, most hypertexts have several levels of depth with varying degrees of breadth for each level (i.e., having generally expanded breadths at some levels, while having constricted breadths at other levels). Consequently, it is more relevant to hypertext design to concurrently examine the overall shape of a structure by assessing its breadth at each hierarchical level.

Unfortunately, very few studies have examined the 'shape' of hypertext structures by varying the amounts of breath over several levels of depth. The most notable exception to this is a study conducted by Norman and Chin (1988). This study examined hypertext shape by assessing five different tree structures of varying breadths, while keeping the depth invariant at four levels. The five structures examined were: a constant breadth (4 x 4 x 4 x 4) structure, comprising of four menu items choices for each level; a decreasing (8 x 8 x 2 x 2) structure, with eight menu items in the first and second levels and two menu items in the third and fourth levels; an increasing (2 x 2 x 8 x 8) structure, with two menu items in the first and second levels and eight menu items in the third and fourth levels; a concave (8 x 2 x 2 x 8) structure, with eight items in the first and fourth levels and two items in the second and third levels; and a convex (2 x 8 x 8 x 2) structure, with two items in the first and fourth levels and eight items in the second and third levels.

Participants were instructed to search a simulated electronic commerce hypertext for either explicitly named items, or items implied in a scenario situation in which they were to find the most appropriate answer—such as to search for a gift item "for a pilot always on the go with time tables to meet." In searching for explicitly named items, search times were similar across all structures. However, for implicit targets the different shapes did have a significant effect on participants' search time and search performance. For these targets, participants using the concave (8 x 2 x 2 x 8) structure took less time, searched fewer nodes to find the target items, and used fewer 'Back' commands, suggesting a lower degree of disorientation than the other menu tree shapes. For explicit targets, the increasing (2 x 2 x 8 x 8) structure facilitated slightly less navigational disorientation than the other structures.

The convex (2 x 8 x 8 x 2) structure produced the poorest performance for implicit targets. Participants using this structure had slower search time, searched more nodes (web pages) than the other structures, and used the Back command significantly more often than the concave structure.

Yet, as interesting as the Norman and Chin (1988) study is, it did not address the important interplay between hypertext shape and depth. To address this issue, the present study examined six different hypertext shapes at different levels of depth. Moreover, unlike the method used by Norman and Chin, where the number of terminal level nodes remained invariant (256 nodes) across conditions while the total number varied from 294 to 456 nodes, this study sought to have approximately equal number of nodes across hypertext conditions—since smaller structures are generally easier to search than large structures. In the present study, the general questions asked were 1) which factor, hypertext breadth or depth, has the greatest effect on user performance, and 2) which type of shape promotes the greatest user performance when general hypertext size remains relatively constant?

Method

Participants

One hundred and twenty undergraduate students between 18 and 52 (M = 22) years of age volunteered to participate in this study. Participants reported using the Web at least once per month (96.7 percent reported using the Web a few times per month or more and 79.8 percent reported using the Web two or more hours per week). No significant differences in reported computer usage and anxiety, as well as Web use, were found among participants within the different conditions.

Experimental Task

Participants were assigned to one of six hypertext conditions. Both the search tasks and the hypertext conditions were ordered by means of a Latin square design. Each task required them to search the presented hypertext for a specific merchandise item that would most appropriately satisfy the context of the search task. The number of nodes was approximately equal (varying from 330 nodes across conditions by ten or fewer nodes).

All participants were given the same search tasks (24 total), which resembled typical directed browsing tasks that reflected 'real-world' hypertext searches. Each task scenario had only one intended target, which was a terminal node. Similar to Norman and Chin (1988), search tasks were both explicit and implicit in nature (12 explicit and 12 implicit tasks).

The hypertext structures consisted of constant, decreasing, increasing, concave, and variable shapes. The depth and breadth of the hypertext conditions varied from a depth of two to six levels and a breadth of two to 27 menu items per node. The hypertext conditions are as follows: (12 x 27), (11 x 5 x 5), (4 x 4 x 4 x 4), (6 x 2 x 2 x 12), (3 x 2 x 2 x 2 x 12), and (2 x 3 x 2 x 3 x 2 x 3). The structural layout of each hypertext condition is presented in Figure 1.

Figure 1. The structural representations of each hypertext condition.

Dependent Variables

The dependent variables were search efficiency and search time (time taken to find the correct information). Search efficiency was measured by examining the number of deviations from the optimal path and the number of total back-page presses or 'commands' taken to reach the targeted node. The optimal path is the pre-established, shortest route to a specific targeted node that satisfies a search task. Deviations from this path consist of unintended detours, which indicate navigational disorientation. This was measured by comparing the total number of pages accessed in reaching the target node minus the shortest or 'ideal' amount of paging needed to reach this node per search task for each respective hypertext condition (total # of pages accessed — total # of paging needed to acquire relevant nodes).

Materials

A Pentium II based PC computer, using a 60 Hz, 96dpi 17-inch high-resolution RGB monitor with a resolution of 1024 x 768 pixels was used. The computer operating system used was Microsoft's Windows XP. The website was saved locally in order to insure equal download time for all searches.

Procedure

Participants were assigned to one of six hypertext conditions. Each participant was then given, one at a time, randomly assigned explicit and implicit search tasks. They then searched the hypertext until they found the most appropriate answer to the task statement. To search, participants could use the menu item links located on each respective page, or use the 'Forward' or 'Back' page button located on the browser's menu bar. They could also select a 'homepage' link that was located at the top-left side of the screen in order to return to the parent page at any time during the search. If a participant selected an item that was not designated as the target, he or she would be informed that the item was 'incorrect' and instructed to search again for an item that best satisfies the search task. For each task, participants searched until they found the correct task information, or until the allotted time (5 minutes) expired.

Results

Comparison of Explicit and Implicit Task Types

A 2 x 3 MANOVA was used to examine the implicit and explicit task scores for search time and search efficiency (deviations from optimal path and back-page commands). The results revealed significant differences in performance between two task types in that participants searching with explicit tasks had faster search times and had a higher search performance than when searching with implicit tasks [F (1, 113) = 47.05, p < .001], which is consistent with results of Norman and Chin (1988). The task types did not, however, significantly interact with the three conditions (p = .35). Therefore the three hypertext conditions were examined across both implicit and explicit tasks.

Comparing Hypertext Conditions

Analysis of participants' search efficiency indicated a significant difference between hypertext conditions for deviations from the optimal navigational path and total amount of back-page commands, respectively [F (5, 114) = 30.08, p < .001; F (5, 114) = 48.61, p < .001], as well as for search time [F (5, 114) = 30.08, p < .001]. Post hoc analysis revealed that the (12 x 27) and (11 x 5 x 5) conditions had significantly fewer deviations from the optimal navigational path and less back-page usage than all but the (6 x 2 x 2 x 12) structure. The (2 x 3 x 2 x 3 x 2 x 3) condition had significantly greater navigational deviations than all other conditions (See Figures 2 and 3).

Analysis of search time also revealed that both the (12 x 27) and the (11 x 5 x 5) conditions had significantly faster search than all but the (6 x 2 x 2 x 12) structure. The (2 x 3 x 2 x 3 x 2 x 3) condition had significantly longer search time than all other conditions (see Figure 4). Together, the measurements of search efficiency and search times suggest that the (12 x 27) and the (11 x 5 x 5) conditions were more informationally accessible than all but the (6 x 2 x 2 x 12) structure.

Figure 2. Deviations from optimal path (per search task)

Figure 3. Number of back-page commands (per search task)

Figure 4. Search time (per search task)

Discussion

Predicting search efficiency

The results of this study paint a more complex picture of hypertext performance than has been previously observed. That is to say, with regard to hypertext structure, depth alone may not be the sole, or even the greatest determinate in predicting search performance. In fact, as this study has shown, the shape of a hypertext structure had at least as much to do with search efficiency than its depth. Indeed, the (4 x 4 x 4 x 4) structure was found to be not only less efficient than hypertext shapes of the same depth (i.e., the (6 x 2 x 2 x 12) structure), but structures that were deeper, such as the (3 x 2 x 2 x 2 x 12) structure. As discussed, much has been said about hypertext depth, in that the greater the depth, the less informationally efficient the structure should be (e.g., Jacko & Salvendy, 1996; Snowberry, et al., 1983). However, what seems to be occurring in this study is that the participants' search efficiency is at least in part, determined by the properties related to the overall shape of the hypertext structure. These properties, then, act to either help facilitate or impede hypertext efficiency by altering the general complexity of the structure. Accordingly, having an inefficient shape will decrease a hypertext's search efficiency.

The results of this study also support the findings of Jacko and Salvendy (1996), Snowberry et al. (1983), and others, which state that broader and shallower hypertext structures are, on a whole, generally superior in search efficiency than narrower and deeper structures. Namely, broad categorical groupings, which was seen with the (12 x 27) condition, and to a lesser extent the (11 x 5 x 5) condition, facilitated participant navigation that had fewer deviations from the optimum path and back-page commands, as well as faster search times than the more narrow and deeper structures.

Hypertext Shape Efficiency

Moreover, together with the findings of Norman and Chin (1988), it is asserted that concave shapes (i.e., (6 x 2 x 2 x 12)) are more navigationally efficient than relatively constant shapes (i.e., (4 x 4 x 4 x 4)) of the same size and depth. Norman and Chin argue that the concave structure is an optimal design because having a larger percentage—and thus more defined—descriptor items at the beginning of a search helps the user form a more exact match between the concept related to the target item and the actual target item itself. At the terminal level, broad menus reduce the overall information uncertainty, since at this level the target items are more explicitly defined. This notion is also associated with the concept of information scent. That is, the more explicit the association between the initial, descriptor items and the targeted, terminal items, the greater the scent. For this reason, a search at the terminal level should benefit from a maximum amount of scent.

Conversely, nodes in the middle level of the tree structure are, in part, designed to ultimately direct the user to the terminal level to access the targeted item. It is argued by Norman and Chin (1988) that the breadth of items at this level should be constricted, and limited to only directing the user to the appropriate target item. Thus, it is maintained that broad menus at the middle level will only increase the likelihood of users' choosing the wrong path to the target item. This was found to be also true in the present study.

Implications for Website Design

Typically when individuals evaluate the information accessibility of a website, it mostly involves the appraisal of the site's physical interface. However, as discussed above, its general shape can play just as, or even more important role in its overall information accessibility — especially for large structures. Furthermore, assessing solely its depth may not provide a truly accurate prediction as to its accessibility. That is to say, an inefficient shape (i.e., a structure with a large percentage of menu choices within the center of the structure) with a relatively shallow depth might be less informationally accessible than a more efficient shape at greater depths. From these results, as well as Norman and Chin's (1988) study, it is further suggested that websites which have several or more levels of depth attempt to give the user the greatest number of choices at both the top and terminal levels of the site, while constricting the choices between these levels.

¹Note: This research is part of a larger dissertation study that examined a metric for predicting the accessibility of information within hypertext structures

References

Jacko, J., A., & Salvendy, G. (1996). Hierarchical menu design: breadth, depth, and task complexity. Perceptual and Motor Skills, 82, 1187-1201.

Kiger, J. I. (1984). The depth/breadth tradeoff in the design of menu-driven interfaces. International Journal of Man-Machine Studies, 20, 201-213.

Larson, K. & Czerwinski, M. (1998). Web page design: Implications of memory, structure and scent from information retrieval. Proceedings of the Association for Computing Machinery's CHI '98, 18-23.

Norman, K., L. (1990). The psychology of menu selection: Designing cognitive control of the human/computer interface. Norwood, NJ: Ablex Publishing co.

Norman, K., L. & Chin, J. P. (1988). The effect of tree structure on search performance in a hierarchical menu selection system. Behaviour and Information Technology, 7, 51-65.

Snowberry, K., Parkinson, S., & Sisson, N. (1983). Computer display menus. Ergonomics, 26, 699-712.

Zaphiris, P. (2000). Depth Vs Breadth in the Arrangement of Web Links, Proceedings of the 44th Annual Meeting of the Human Factors and Ergonomics Society, 139-144.

²Note: Michael Bernard is currently employed at Sandia National Laboratories.