Card Sorting from A–Z: Part VII – How to Analyze Results of a Closed Card Sort

This is part of a series of posts which is based on a 3-hour hands-on workshop I offer on this topic. Be sure and check out the preceding posts:

Card Sorting from A–Z: Overview
Card Sorting form A-Z: Part I – Why and When
Card Sorting from A-Z: Part II – How to Prepare for a Card Sort
Card Sorting from A–Z: Part III – How to Implement an Open Card Sort
Card Sorting from A–Z: Part IV – How to Analyze Results of an Open Card Sort
Card Sorting from A–Z: Part V – How to Make Recommendations Based on Findings
Card Sorting from A–Z: Part VI – How to Implement a Closed Card Sort

 

How to Conduct Analysis of a Closed Card Sort

Analysis of a closed card sort is a bit different that that for an open one. With a closed card sort you will be looking for:

  • Whether articles have made it in to the categories you expected them to based on the open card sort
  • How many articles have been miscategorized into those categories as well.

 

Analyzing the Data

     

  1. The first step in analyzing the data is to plug all of the information into a spreadsheet, or if you have done an online sort, to download the Excel-compatible file.
  2.  

    ;
    Download analysis spreadsheet

     

  3. Next, you will want to start to determine which categories did the best in capturing all of their related items, and which categories recieved items errantly placed into them. While there are of course no right or wrong answers, you will have made certain assumptions based on your open card sort. Now you are testing that these assumptions are true – for example if everyone in your open sort placed an item such as career opportunities into an “About the Library” category, and therefore you determined that an “About the Library” category was necessary, you now will want to make sure than everyone in this closed sort, also placed career opportunities into the “About the Library” category.
  4.  

  5. One way to go about this analysis is to make a list of each category, and then, looking at the spreadsheet, determine which content items have been unexpectedly placed into which categories. Your list may look something like this:
  6.  

    Closed Card Sort Analysis (4 Participants)

     

    About the Library Miscategorized – 0

     

    My Account Miscategorized – 0

     

    Collections Miscategoried – 5

    Digital Audiobook Collection x3

    Hot Titles

    Electronic Books

     

    Online Resources Miscategorized – 0

     

    Events Miscategorized – 1

    Teen Anime Clubs

    Almost nothing in here, may want to consider whether an Events category is necessary.

     

    Programs & Services Miscategorized – 2

    Volunteer at the library

    Library News Feeds

     

    Recommended Miscategorized – 0

     

    Misc. – 3

    Library News Feeds

    Pathfinders x2

     

     

  7. A list such as the one above tallies the errant articles or items which have been placed into each category, it takes a bit of a closer inspection, however to determine how successful each category was in capturing all of its related items. If you list each of the errant items under your total tally, you will then begin to see which categories didn’t do so well in that area. For example, at first glance, the Online Resources category seems very succesful – its miscategorized items are at 0, however, it didn’t do so well capturing all of its related items when you look closer – Library News feeds has been placed in 2 other categories, Electronic Books and Digital Audiobooks have both been placed in another category by multiple participants.
  8.  

  9. Your next task is to bring your analysis to this closer, more focused level. So your next document, expands upon the first one and may look something like this:

 

Data Analysis

 

  • About the Library
    • This category was one of two most successful categories in the study, it captured all of its items and had no items misplaced into it.

 

  • My Account
    • This category tied About the Library for most successful category, it captured all of its items and had no items misplaced into it.

 

  • Collections
    • The Collections category did well capturing its articles, however, they were also the largest catch-all category, receiving even more errant articles than were put into the Misc. category.

 

  • Online Resources
    • This was the least successful category in the study.
    • Although the category did not receive any miscategorized items placed into it, it had several outlying articles which were mislabeled:
      • Library News Feeds were placed in the Programs & Services category by nearly one-quarter of the respondents, and into the Misc. category by another one-quarter.
      • Digital Audiobook Collection was placed in the Collections category by 75% of participants.
      • Electronic Books was placed into the Collections category by nearly half of all participants.

 

  • Events
    • The Events category was successful capturing all of its items, plus one errant item, however, the study indicates that there may not be enough content items to justify keeping this category.

 

  • Programs & Services
    • Programs & Services followed My Account as one of the most successful categories although it did have 2 errant items placed into it.

 

  • Recommended
    • This category did well however it did have a couple outlying articles which were misplaced.
      • Hot Titles was categorized in Collections by ¼ of all participants.
      • Pathfinders was placed in the Misc. category by more than half of participants.

 

  • Misc.
    • The Misc. category was dominated by two items which did not appear to have a clear category:
      • Pathfinders
      • Library News Feeds

 

Next: How to Make Recommendations After a Closed Sort