Olympics 2012 Goes Mobile: Who Wins the Gold for Best App?

By M. H. Phan, J. R. Jardina, S. D. Jansen, & B. S. Chaparro

Summary. The Olympics is one of the most renowned sporting events with billions of followers around the world. In order to aid the Olympics-viewing experience many Olympic apps were developed for various mobile devices. In this study four popular Olympic iPad apps (i.e., NBC Olympics, NBC Live Extra, Reuters, and London 2012 Results) were evaluated across five dimensions: Aesthetics, Multipurpose Content, Execution and Error, Ease of Use, and Usefulness. Results revealed that Reuters was the best-looking app while NBC Olympics was the best in providing multipurpose content and a low number of errors. In addition, NBC Live Extra was found to be the most useful Olympics app with its live, continuous coverage and was the easiest to use. However, London 2012 Results app won the title of the best overall app for consistently performing well across all five dimensions. Suggestions for improving future Olympic apps are discussed.

INTRODUCTION

With 3,500 hours of live-coverage, mobile media played an important role in bringing information to users around the world during the 2012 Olympics (Baker, 2012; Laird, 2012). Tablet apps and websites distributed much of the coverage and featured many kinds of information (e.g., pictures, news articles, recap videos, etc.) targeted at different user interests (Horn, 2012; Sung, 2012; Upbin, 2012).

In a review of the official 2012 Olympic site (http://www.london2012.com) Nielsen (2012) reported the site to be a marked improvement from the 2002 olympic site but criticized it for having a “scattered and uncoordinated internet presence”. He also noted that the correpsonding iPad Olympic 2012 app sometimes used poor text formatting for its stories.

This article reports on the findings of a usability evaluation of four popular Olympic apps used on an iPad: NBC Olympics, NBC Live Extra, Reuters, and the London 2012 Results. The NBC Olympics app featured basic information such as news and highlights; NBC Live Extra presented live-streaming video; Reuters displayed photo galleries of each event; and London 2012 Results showed real-time event results (Horn, 2012).

METHOD

Four members of the Software Usability Research Lab used all four of the Olympic apps installed on an iPad (2012 version) during the Olympic games (July 27, 2012 – August 12, 2012) and served as evaluators. Each evaluator was instructed to keep a daily log of their usage time for each app and corresponding thoughts regarding their likes and dislikes. A post-survey was administered to gather perceptions on five dimensions: Aesthetics, Multipurpose Content, Execution and Error, Ease of Use, and Usefulness. Evaluators rated each app on each dimension on a scale from 1 (lowest) to 10 (highest). The five dimensions were defined as follows:

  • Aesthetics – The overall appearance or the look-and-feel of the app.
  • Multipurpose Content – The app’s ability to provide a variety of content for different tasks (e.g., keep track of medal count, accessing athletes’ info, etc.).
  • Execution & Error – The app’s ability to carry out each task smoothly and efficiently, and with minimum amount of error.
  • Ease of Use – The app’s ease of use for completing tasks without seeking assistance or giving up.
  • Usefulness – The app’s ability to provide the information desired and more..

RESULTS

The inter-rater agreement between different evaluators was high to very high (Cronbach’s Alpha 0.889 to 0.931). Total scores were determined per app by summing the mean of all five dimension ratings, and then multiplying by 2 to generate a score between 0 and 100. Figure 1 shows the overall standings of the apps collapsed across all dimensions: 1. London 2012 Results 2. NBC Live Extra 3. NBC Olympics, and 4. Reuters London 2012.

Results of the Olympic app ratings

Figure 1. Results of the Olympic app ratings based on their total score across all dimensions.

Table 1. The Summary of the Ranking and Mean Score of Each App per Dimension

Dimension Live Extra London 2012 Results NBC Olympics Reuters
Aesthetics bronze
 
6.50
silver
 
7.50
4th
 
5.75
gold
 
8.25
Multipurpose Content silver
 
6.25
silver
 
6.25
gold
 
7.25
bronze
 
4.00
Execution & Error 4th
 
6.00
silver
 
7.25
gold
 
8.00
bronze
 
6.50
Ease of Use gold
 
7.75
silver
 
7.50
bronze
 
6.75
4th
 
4.75
Usefulness gold
 
8.25
silver
 
7.50
bronze
 
6.50
4th
 
2.75

Aesthetics: Which app was the best looking?

The photos displayed on the Reuters app were spectacular, plentiful, and arranged in a colorful, scrollable timeline. Users were able to view many photos of a single event and share them with others. The NBC app paled in comparison in aesthetics and sometimes appeared to have cluttered images or dull pages. The London 2012 Results app did not show photos but had a clean interface, which contributed to an overall positive aesthetic experience. The Live Extra app displayed high quality video but the truncated clip titles and deemphasized navigational menu weakened the overall look and feel.

Table 2. The Ranking and Mean Score of Each App on the Aesthetics Dimension

Reuters London 2012 Results Live Extra NBC Olympics
gold
 
8.25
silver
 
7.50
bronze
 
6.50
4th
 
5.75

Reuters app aesthetics

Figure 2. High quality pictures on the Reuters app bolstered its aesthetic ratings.

NBC Olympics aesthetics

Figure 3. Crowded images and dullness left the NBC Olympics app with no medal for aesthetics.

Multipurpose Content: Which app provided the most information in a variety of media?

The Reuters app mainly provided photos of each event, but offered very little information regarding the athletes or the events (Figure 5). The NBC Olympics app, on the other hand, allowed users to view pictures, watch videos, check results and schedules, and perform many other functions (Figure 4).

Table 3. The Ranking and Mean Score of Each App on the Multipurpose Content Dimension

NBC Olympics London 2012 Results Live Extra Reuters
gold
 
7.25
silver
 
6.25
silver
 
6.25
bronze
 
4.00

Screenshot of the functions on the NBC Olympics app

Figure 4. Multiple functions on the NBC Olympics app provided a good overview of all aspects.

Screenshot of the Reuters app

Figure 5. The Reuters app’s primary function was to show pictures and little else and thus scored low as a multipurpose content app.

Execution & Error: Which app performed the best and without error?

In comparison to the other apps, NBC Olympics had the lowest number of errors. The majority of the links worked and rarely were there dead links or pop-up error messages from the app. Additionally, the highlight videos on the NBC Olympics were well-executed. The highlight videos were short, but impactful and informative. Conversely, Live Extra generated a lot of errors from the app itself as well as from the live videos. For example, some videos would take a long time to load (over 2 minutes) or did not load at all. Also, some videos that were listed under live events would only show an empty stadium, an off-air screen, or an NBC Olympics logo. This made it confusing to know whether the event had not yet started or had already ended. Figure 6 shows a screenshot of the Videos section on the NBC Olympics app and Figure 7 shows some of the error messages from the Live Extra app.

Table 4. The Ranking and Mean Score of Each App on the Execution and Error Dimension

NBC Olympics London 2012 Results Reuters Live Extra
gold
 
8.00
silver
 
7.25
bronze
 
6.50
4th
 
6.00

Screenshot of the errors on the NBC Olympics app

Figure 6. The NBC Olympics app rarely generated errors.

Screenshot of the errors on the Live Extra app

Figure 7.The Live Extra app frequently experienced technical errors.

Ease of Use: Which app was easiest to use?

Out of the four Olympics apps, Live Extra was rated the easiest to use since the navigation system was straightforward and the videos were usually one tap away from being accessed (See Figure 8). Reuters was rated the most difficult app to use since there were so many pictures to navigate and there was no search function. Additionally, the icons below each image were not labeled and were not intuitive as to their function (See Figure 9). In short, the way the app was structured combined with the vast amount of pictures contributed to users being overwhelmed and/or “lost”.

Table 5. The Ranking and Mean Score of Each App on the Ease of Use Dimension

Live Extra London 2012 Results NBC Olympics Reuters
gold
 
7.75
silver
 
7.50
bronze
 
6.75
4th
 
4.75

Screenshot of the navigation on the Live Extra app

Figure 8. The Live Extra app was easy to navigate and use.

Screenshot of the navigation on the Reuters app

Figure 9. The Reuters app was difficult to navigate and it was difficult to find certain information.

Usefulness: Which app was the most useful for viewing events and learning results?

Timeliness of the the information (results and video) was an important contributor to the usefulness dimension. The Live Extra app was deemed to be the most useful primarily due to its real-time coverage of all events. Users could quickly tune in and watch an event as it occurred. The Live Extra app was especially useful for watching the more obscure sports that television viewers would rarely see on television (e.g., trials of sports like handball and pentathlon). The second most useful app was the London 2012 Results app since it presented the results of each event in real time. The Reuters app was reported to be the least useful simply because it did not provide the depth of information found in the other apps.

Table 6. The Ranking and Mean Score of Each app on the Usefulness Dimension

Live Extra London 2012 Results NBC Olympics Reuters
gold
 
8.25
silver
 
7.50
bronze
 
6.50
4th
 
2.75

Screenshot of the usefulness of the Live Extra app

Figure 10. The Live Extra app had many useful features. The app allowed users to select their favorite sports (left) and the schedule showed live and upcoming events (right).

Screenshot of the usefulness of the Reuters app

Figure 11. The Reuters app provided minimal useful information regarding the events and athletes.

All-Around: Which app was the best overall?

Table 7. The All-Around Ranking and Score of Each App

London 2012 Results Live Extra NBC Olympics Reuters
gold
 
72.00
silver
 
69.50
bronze
 
68.50
4th
 
52.50

London 2012 Results – This app was given the All-Around gold due to its ease of use and ability to provide a quick results recap. While this app did not dominate in any one area, it was consistently reliable and satisfying to use. However, it left the user wanting a more detailed breakdown in areas such as medal count and race results.

  • What did it do best?
    • Bright and colorful interface.
    • Color coded sections (e.g., medals, athletes) and schedule made it easy to understand/use.
    • Important information was one tap away on front page (e.g., medal count, athlete, and sports).
  • What challenges did it have?
    • Some navigation wasn’t intuitive and was easily missed (e.g., grayed horizontal scroll icons in sports section and dark navigation bar on bottom of app).
    • The medals did not break down to the same level each time (e.g., the medal counter on front page breaks down to country, but the medals section breaks down to medals by sport). Users wanted a more detailed breakdown (e.g., medals by country, then sport, then athlete with detailed race results).
    • Lack of multipurpose information (e.g., no videos) and prompt news updates
    • Profile pictures of the athletes varied in quality (Figure 12).
    • Organization of lists not consistent (e.g., medalists listed by athlete with most medals to least instead of alphabetically and “overall” medals listed by country according to the number of gold not “overall” medals).

    Live Extra – This app was very useful and easy to use. There were a lot of videos to watch, but some had technical difficulties and other issues due to poor labeling of videos. The evaluators also found the repeated advertisements to be annoying.

    • What did it do best?
      • Lots of videos to watch: highlights, live events, and events that weren’t on TV.
      • Videos were high quality and looked impressive on the iPad retina display.
      • Recap videos allowed the user to catch up on multiple sports.
      • Marking favorites allowed users to keep track of events of most interest.
    • What challenges did it have?
      • Many errors with the live videos.
      • The format and organization of the schedule section made it difficult to use.
      • Truncation of video clip titles made it difficult to determine what it was.
      • Users forced to switch between landscape (when watching videos) to portrait orientation (for main menu).

    NBC Olympics – This app was very helpful for staying up-to-date with the Olympics since it has a variety of coverage (e.g., news, highlight videos, images, medal count, etc.). However, its dull-looking and at times confusing layout prevented the users from obtaining the information that they wanted to seek.

    • What did it do best?
      • Had the most diverse updates in regard to the Olympics (e.g., recaps and highlights videos, athletes’ twitter updates, etc.).
      • Content can be viewed either in Landscape or portrait orientation.
      • Offered a “Primetime Companion” which allowed users to follow-up on televised coverage.
      • Important updates (e.g., News, Videos, Images, and medal count) are easily accessible on the front-page.
    • What challenges did it have?
      • The front-page structure were difficult to navigate.
      • The option to make video videos bigger is “hidden”.
      • Useful sections such as “Primetime Companion” and “Medal” sections were “hidden”.
      • Some useful sections such as “Athletes” and “Schedules” sections were poorly structured and not intuitive.

    Reuters – This app was reported to be the least useful overall. While the pictures featured were outstanding in quality and were fun to view, the sheer volume of images made it difficult to search for a particular athlete or picture and made it easy to get lost.

    • What did it do best?
      • Picture quality and variety was excellent.
      • Timeline metaphor was easy to use.
      • Easy access to portfolio and bio of photographers.
      • Olympic records was easy to find in one table.
    • What challenges did it have?
      • Overall navigation of app was not clear.
      • Too many pictures.
      • Difficult to find pictures by athlete.
      • Limited information on sports or athletes available.
      • Some picture titles were mislabeled or misleading.

    RECOMMENDATIONS

    Though the Olympics are over, results from this study have helped us to generate the following recommendations for future game or sporting event apps:

    • Aesthetics
      • Main navigation should stand out from the rest of the content (e.g., brighter and more vibrant in terms of color) and be obvious upon first exposure. First impressions may dictate whether the app is used again.
      • Images are important, but it shouldn’t be the only focus of any Olympic app; users want to be able to drill down to detail from a photo.
      • Profile images of the athletes should be consistent across apps.

    Athletes' photos on London 2012 Result app
    Figure 12. Photos of some athletes on the London 2012 Results app varied considerably from profile photos on other apps (e.g., NBC Olympics).

    • Multipurpose Content
      • The “Medal” results are extremely important to the Olympic Games and it should be well-organized and flexible:
        • Users should have the option to easily view the medal count either by country, athlete/medalist, or by sport.
        • Medal counts should be listed by country according to the number of “overall” medals and not the number of gold (or allow both).

    Medal count screenshot across apps

    Figure 13. Medal count tables for each of the apps: (1) Reuters (organized by overall gold counts), (2) London 2012 Results (organized by overall gold counts), and (3) NBC Olympics (organized by overall medal counts).

    • Whenever possible, users should be able to customize the app settings (e.g., time zone, font size, the frequency in which notifications will be sent, option to track all events of a favorite athlete or country, option to build a list of favorite sporting events to monitor).

    • Execution & Error
      • The major title of all video clips should be succinct and should not be truncated.
      • Screenshot of a truncated video title in the Live Extra app

        Figure 14. Example of truncated video title in the Live Extra app. It is not clear which event this represents.

      • Avoid dead links or links that lead to empty pages.
      • Allow users a quick access to return to a working page when users encounter dead links or error messages.
      • In cases of videos featuring live events, there should be a visible message to inform users whether such live events have just recently showed, are on-break time, or recently ended.
      • The image on the Live Extra app that was displayed when there was no live video

        Figure 15. The image on the Live Extra app that was displayed when there was no live video.

    • Ease of Use
      • “Results and Schedules” is undoubtedly important information for any Olympics app and it should be easily viewed through button taps and gestures (e.g., swipe left/right to move forward/backward on dates and swipe down/up to move forward/backward on the time frames).
      • Content should be viewable in both landscape and portrait orientations.
      • The main screen on the Live Extra app was only viewable in portrait mode

         Figure 16. The main screen on the Live Extra app was only viewable in portrait mode.

      • All buttons should be large in size to facilitate accurate tapping.

    Screenshot of small and large buttons example

    Figure 17. The buttons on the London 2012 Results app were large and obvious (bottom), while the buttons on the Reuters app were small and ambiguous (top).

    • Usefulness
      • Video and photos should be optimized to load quickly and updated frequently.
      • A search option should be available for sections such as Videos, Athletes, etc.
      • In addition to basic demographics, each athlete’s page should be continuously updated with results information.

    An example of an athlete's page, without any results displayed

     Figure 18. An example of an athlete’s page, without any results displayed.

    REFERENCES

    Baker, N. (2012). Apps for 2012 summer Olympic games abound. Retrieved from http://www.reuters.com/article/2012/07/30/us-oly-apps-idUSBRE86T13K20120730

    Horn, L. (2012). Which Olympics app is right for you? Retrieved from http://gizmodo.com/5929664/which-olympics-app-is-right-for-you

    Laird, S. (2012). 2012 Olympics: How to watch 3,500 hours of NBC coverage online. Retrieved from http://mashable.com/2012/06/28/2012-olympics-nbc/

    Nielsen, J. (2012). Official Olympic Website: UI Silver – but UX DQ. Retrieved from http://www.useit.com/alertbox/olympics-ui-vs-ux.html

    Sung, D., (2012). Best Olympic apps for following the London 2012 games. Retrieved from http://www.pocket-lint.com/news/46667/best-olympic-apps-london-2012

    Upbin, B. (2012). The best apps for the London 2012 Olympics. Retrieved from http://www.forbes.com/sites/bruceupbin/2012/07/27/the-london-2012-olympics-app-and-tools-guide/

Tagged with: , , , , , , , , , , , , , ,
Posted in Usability News
Subscribe to SURL

Want to receive notifications when SURL has new articles? Please enter your name and email address to subscribe to our website.

Popular Topics in Usability News
Log in/Sign Up
%d bloggers like this: