Text on XYZ.com

A HOT Evaluation by YourNameGoesHere

 

Scannability

 

Purpose

To determine the extent to which the text on this site achieves the scannability needed for effective online presentation.

Scannability opens up the text, allowing people to see how it is organized, and encouraging them to jump in at the spot they find most interesting. 

To achieve scannability, though, a writer must adopt several different strategies, and within those broad approaches, a number of tactics.  Making an article scannable takes more effort than just writing the article. The strategies are:

Strategy 1. The Titles Signal the Content, Fast.

Strategy 2. Headings provide meaningful information about the way you have organized the material.

Strategy 3. Important text stands out.

Strategy 4. Sets of items appear in bulleted or numbered lists.

The strategies and tactics for scannability appear here as a set of guidelines based on research and actual experience, documented in books, journal articles, and online styleguides. (References appear at the end of each strategy). Each guideline, then, provides a method for a writer to follow, or a heuristic.

In this evaluation we test the text against these guidelines.  This, then, is a heuristic evaluation.

Method

Here’s how to perform a Heuristic Online Text (HOT) evaluation.

1. Save this file with a name that includes

q       The site you are analyzing

q       The aspect you are evaluating (brevity, in this case)

q       Initials

q       A period

q       A suffix indicating the file type (doc for Word files, htm for HTML files)

Examples: ibmbrevityjp.doc, yahoobrevityds.htm

2. Go to the site, and locate a fairly typical page that has several paragraphs of running text.

Running text is actual content, not labels, menu items, or link descriptions.

3. In this file, type the title of the page, under Sample #1, below.

The title appears in the title bar of the window (not including ads for your browser).

4. Copy the text on the page, including all headings, and paste it into this file after the subject.

5. Return to the page and copy the URL for that page, then paste that into this file, in the line right after the paragraph.

The URL is the address of the page.

6. Type today’s date on the next line, to show when you collected the sample.

7. Repeat this process, collecting titles and content from at least 5 pages.

If possible, find pages with different kinds of content.

Tip: You may want to print out your samples, so you can look at their text on paper as you work onscreen.

8. Apply the HOT Evaluation to the text samples you have collected, filling out the evaluation form.

If a strategy or tactic seems irrelevant, omit it from your evaluation.  Note that this will change the total possible points.

Samples

Sample # 1

Subject:

Paragraph:

URL to the page:

Date investigated:

Sample # 2

Subject:

Paragraph:

URL to the page:

Date investigated:

Sample # 3

Subject:

Paragraph:

URL to the page:

Date investigated:

Sample # 4

Subject:

Paragraph:

URL to the page:

Date investigated:

Sample # 5

Subject:

Paragraph:

URL to the page:

Date investigated:

 


Evaluation

Strategy 1. The Titles Signal the Content, Fast.

Definition: A title appears in the window's title bar. Within the window, you see headings and subheads.

 

Critical information appears first, in each title.

Test

In each sample title, the distinguishing information appears before any repeated information, such as the company name, or department name.

YES=1, NO=0.

Impression

Overall, did the titles start off with the text that indicates the most important topics on this page? 

YES=1, NO=0.

Example

Which title seemed slow to give up critical information (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

Each title makes sense out of context.

Test

In looking at these titles in a search results list, or a menu, I could tell what they are talking about.

YES=1, NO=0.

Impression

Overall, did the titles stand on their own, describing the content of their pages, even if you did not know what section of the site these pages appear in? 

YES=1, NO=0.

Example

Which title seemed least informative (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

The beginning of each title is unique.

Test

In each sample title, the first few words are unique. They do not appear in any other titles.

YES=1, NO=0.

Impression

Overall, did the titles start off with the text that distinguishes this page from others? 

YES=1, NO=0.

Example

Which title seemed similar to others, or the same (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

The major heading echoes the title.

Test

In each sample page, the highest-level heading uses the same terms as the title, or repeats the title.

YES=1, NO=0.

Impression

Overall, did the titles seem to match the major headings? 

YES=1, NO=0.

Example

Which title seemed at odds with the major heading (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

The organizational identification appears last, if at all.

Test

In each sample title, the repeated identification (of company, department, section) appears after the information that is unique to this page, so users can spot the differences between pages in search results and menus.

YES=1, NO=0.

Impression

When looking at the first 40-60 characters, I always saw information that described this particular page before I saw any repeated identification.

YES=1, NO=0.

Example

Which title put corporate identity before page content (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

Titles on similar topics, or within a section of related topics, show parallel structure.

Test

In each title in the group, the grammatical form, and phrasing, is the same, or very similar.

YES=1, NO=0.

Impression

Overall, did the sample titles show parallelism of structure and phrasing, when the pages were similar? 

YES=1, NO=0.

Example

Which title seemed out of step (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

Each title faithfully reflects the content of the page.

Test

Each sample title describes the major topic of the page accurately.

YES=1, NO=0.

Impression

Overall, did the titles describe their pages in a way that made sense before you read the pages—and afterward? 

YES=1, NO=0.

Example

Which title seemed at odds with the content of the page (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

References

 

See: Ameritech (1997), Berners-Lee (1995), Bricklin (1998), Dumas (1988), Frisse (1987), Levine (1997), Lynch & Horton (1997), Nielsen (1996, 1999f, 2000b), Rosenfeld & Morville (1998), Uncle Netword (1999b), Spyridakis (2000), Waite (1982), Wallace (1985) in the bibliography at

http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf

 

 


Strategy 2. Headings provide meaningful information about the way you have organized the material.

A heading announces every change of topic.

Test

In each sample page, a heading of some kind appears whenever the subject matter changes.  Headings, then, act as advance organizers, tipping the user off to the transition, and the next topic.

YES=1, NO=0.

Impression

Overall, did headings introduce each important change of topic? 

YES=1, NO=0.

Example

Which page had a number of changes of topic without any headings alerting the user to those transitions (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

Subheads introduce subtopics.

Test

In each sample page, if there are subtopics, they are introduced by subordinate headings.

YES=1, NO=0.

Impression

Overall, did lower level headings appear in front of topics that relate to the previous paragraphs, but stand out, as distinct subtopics? 

YES=1, NO=0.

Example

Which page had subtopics without subheads (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

Pages use no more than three levels of headings.

Test

In each sample page, there are no more than three levels of heading within the main content.

YES=1, NO=0.

Impression

Overall, did your eye perceive the difference between levels of heading easily? 

YES=1, NO=0.

Example

Which page had more than three levels of headings (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

Each heading makes sense.

Test

Each heading expresses the content of the section that follows. The headings are not puns, jokes, metaphorical excursions, or phrases that make you guess what the authors meant.

YES=1, NO=0.

Impression

Overall, did headings accurately and concisely sum up the content of the following sections? 

YES=1, NO=0.

Example

Which heading seemed least meaningful?.

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

On each page, the sequence of headings makes sense.

Test

In each sample page, the whole set of headings, taken as a sequence, expresses the basic content of the page. A user, looking only at the headings, gets an outline of the content.

YES=1, NO=0.

Impression

Overall, did headings signal the structure of each page, showing what the page is about, and how the page is organized? 

YES=1, NO=0.

Example

Which page had headings that did not, as a group, offer an outline of the page (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

The set of headings, taken as a structure, echoes some familiar schema, or pattern.

Test

In each sample page, the headings indicate the way the page has been organized, and, if that follows a familiar pattern, the headings are phrased in a way that reminds the user of that schema.

YES=1, NO=0.

Impression

Overall, did the headings remind the user of some familiar pattern of organization, such as chronological order, familiar to unfamiliar, important to unimportant, cause and effect, problem and solution? 

YES=1, NO=0.

Example

Which page had headings that gave the least indication of a familiar pattern (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

 

References

 

See: Ameritech (1997), Bricklin (1998), Hartley and Trueman (1983),  IBM Ease of Use (1999), Kaiser (2000), Lorch & Lorch (1985, 1995), Lynch (2000), Lynch & Horton (1997), Killian (1999), Mayer, Dyck & Cook (1984), Morkes & Nielsen (1997), Nielsen (1997a, 1997b, 1998b, 1999d, 1999f, 2000b), Spyridakis (2000), Sun (2000), Van Dijk & Kintsch (1983), Williams (2000) in the bibliography at

http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf

 

 

Strategy 3. Important text stands out.

In almost every paragraph, at least one important word or phrase has been emphasized.

Test

In most paragraphs, at least one critical word or phrase has been boldfaced, or otherwise highlighted. (A link counts as an emphasized phrase).

YES=1, NO=0.

Impression

Overall, did paragraphs include some text that stood out because of its formatting? 

YES=1, NO=0.

Example

Which page had paragraphs with little or no emphasis (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

 

Link text is written and placed in a position where it tends to emphasize its content.

Test

In a paragraph containing one or more links, the linktext says something meaningful about the topic, and therefore helps the user figure out what the paragraph is about, as a whole.

YES=1, NO=0.

Impression

Overall, did links speak meaningfully to the topic of the paragraph? 

YES=1, NO=0.

Example

Which links seemed least illuminating? (As emphasized text, they did not do their job of articulating the content of the paragraph).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

No paragraph has more than three or four boldfaced terms and links.

Test

In most paragraphs, the emphasized words, phrases, or links stand out, because they are not surrounded with a lot of other emphasized text.

YES=1, NO=0.

Impression

Overall, did paragraphs seem relatively calm, even when they included emphasized text? 

YES=1, NO=0.

Example

Which paragraph had the most emphasize items?  How many?
(Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

Taken as a set, the emphasized terms, phrases, or links within the article give you a good overview of the content.

Test

In most pages, the emphasized text could be used as a digest of the major points, or topics.

YES=1, NO=0.

Impression

Overall, did pages light up with significant words and phrases, tipping you off to the point of the page, and, possibly, its organization? 

YES=1, NO=0.

Example

Which page was poorly described by the emphasized text  (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

References

 

See: America Online (2001), Lynch (2001), Morkes & Nielsen (1997, 1998), Nielsen (1997a, 1997b)in the bibliography at

http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf


Strategy 4. Sets of items appear in bulleted or numbered lists.

Any set of three or more items in a row appear in a list.

Test

Whenever there are three or more items in a row, they are broken out in a list, so people can scan through the material quickly.

YES=1, NO=0.

Impression

Overall, did most sets of items appear in lists? 

YES=1, NO=0.

Example

Which paragraph had a long set of items without breaking them out as a list (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

The sentence introducing the list, and the list, appear at the end of the paragraph.

Test

When paragraphs include lists, the lists always appear at the end of the paragraphs.

YES=1, NO=0.

Impression

Overall, did most lists appear at the end of the paragraph? 

YES=1, NO=0.

Example

Which paragraph had a list in the middle of the paragraph (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

 

If the list items must be read, or performed, in order, they are introduced by numbers.

Test

Whenever there are items that must be read or done in order, the list items begin with numbers.

YES=1, NO=0.

Impression

Overall, did most sequential lists have numbering, rather than bullets? 

YES=1, NO=0.

Example

Which sequential list appears with bullets (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

 

Optional items, without any important sequence, appear with bullets.

Test

Whenever there are optional items in a list, they begin with bullets.

YES=1, NO=0.

Impression

Overall, did most lists of options appear with bullets? 

YES=1, NO=0.

Example

Which non-sequential list lacks bullets (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

 

Similar list items start in the same way.

Test

When a set of items are similar, they begin with the same kind of phrase, or use the same grammatical form, to help people compare one with another.

YES=1, NO=0.

Impression

Overall, did most lists seem consistently phrased? 

YES=1, NO=0.

Example

Which list seemed to be the most inconsistent (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

When list items have two parts, they are separated graphically.

Test

When each list item has two parts (a term and definition, a link and a description, say), the two parts are separated by punctuation such as a dash, a colon, or a paragraph return, or by other formatting..

YES=1, NO=0.

Impression

Overall, when list items contained two elements, I could quickly distinguish one from the other.

YES=1, NO=0.

Example

Which paragraph had items with two parts, but failed to distinguish them (if any)? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

Long lists are organized into sections.

Test

Whenever a list extends beyond nine or ten items, the items are grouped by object, topic, or action, and if the list is particularly long, those groupings are indicated by headings. (Not necessary if the list must be arranged in alphabetical or numeric order).

YES=1, NO=0.

Impression

If there was a long list, was it broken up into sections? 

YES=1, NO=0.

Example

Which long list went on and on, without any sectioning, even though the organization was not alphabetical or numeric? (Sample #   ).

Comments

If this guideline does not seem to apply, indicate that here. 

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

 

References

 

See: Bricklin (1998), Brusaw, et al (1997), Hackos & Stevens (1996), IBM (1999), Kaiser (2000), Killian (1999), Lohse & Spiller (1998), Morkes & Nielsen (1997, 1998), Nielsen (1997a, 1997b), Price & Korman (1993), Tarutz (1992) in the bibliography at

http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf

 

 


Total Score

Assigning a grade to text is always a bit arbitrary.  But counting up the points for these sample texts, we reach this diagnosis:

Total Points:

Total Possible:

Percentage:

Interpretation

90-100%: Excellent scannability.

75-89%: Pretty scannable, but occasionally dense.

60-74%: Could use some opening up.

45-59%: Dense.

25-44%: A jungle of text.

0-24: Impenetrable.

Overall Conclusions

In a few paragraphs, summarize your most important observations—both positive and negative.

Major Recommendations

List the top three problems with the text, and in a sentence or two, summarize what you would recommend as solutions.