Text on XYZ.com

A HOT Evaluation by YourNameGoesHere

 

Cognitive Burdens

 

Purpose

To determine the extent to which the text on this site reduces cognitive burdens on the visitor.

Don’t make me think, when I use or read your text. 

To avoid making the reader think unnecessarily, about tangential issues, you need to adopt a number of strategies and tactics, which appear here as a set of guidelines based on research and actual experience, documented in books, journal articles, and online styleguides. (References appear at the end of each strategy). Each guideline, then, provides a method for a writer to follow, or a heuristic.

In this evaluation we test the text against these guidelines.  This, then, is a heuristic evaluation.

Method

Here’s how to perform a Heuristic Online Text (HOT) evaluation.

1. Save this file with a name that includes

q       The site you are analyzing

q       The aspect you are evaluating (brevity, in this case)

q       Initials

q       A period

q       A suffix indicating the file type (doc for Word files, htm for HTML files)

Examples: ibmcogburdenjp.doc, yahoocognitiveds.htm

2. Go to the site, and locate a fairly typical page that has at least one paragraph of running text.

Running text is actual content, not labels or menu items.

3. In this file, type the subject of the page, under Sample #1, below.

The subject appears in the title bar of the window (not including ads for your browser) or in the major heading at the top of the page. Use whichever best articulates what the page is about.

4. Copy the paragraph and paste it into this file after the subject, breaking the paragraph up into its individual sentences, each on its own line, so the sentences are separated for analysis.

5. Return to the page and copy the URL for that page, then paste that into this file, in the line right after the paragraph.

The URL is the address of the page.

6. Type today’s date on the next line, to show when you collected the sample.

7. Repeat this process, collecting paragraphs from at least 5 pages.

If possible, find pages with different kinds of content. Also, for consistency, pick the same number paragraph on each page; for instance, pick the second paragraph on every page.

Tip: You may want to print out your samples, so you can look at their text on paper as you work onscreen.

8. Apply the HOT Evaluation to the text samples you have collected, filling out the evaluation form.

If a strategy or tactic seems irrelevant, omit it from your evaluation.  Note that this will change the total possible points.

Samples

Sample # 1

Subject:

Sentences from paragraph:

URL to the page:

Date investigated:

Sample # 2

Subject:

Sentences from paragraph:

URL to the page:

Date investigated:

Sample # 3

Subject:

Sentences from paragraph:

URL to the page:

Date investigated:

Sample # 4

Subject:

Sentences from paragraph:

URL to the page:

Date investigated:

Sample # 5

Subject:

Sentences from paragraph:

URL to the page:

Date investigated:

Evaluation

Most sentences have one main clause, and no subordinate clauses.

A clause contains a subject and a verb. 

Examples:

·         The batter hit the ball.

·         When I go to the store

·         If I like the dress

·         Who is my best friend

·         That I chose

·         Which is the brightest red of all

If the clause is introduced with a word like when, because, while, who, that, which, or a similar word, the clause is subordinate to the main clause.  Tip: If you can remove the clause and still have a complete sentence, you have spotted a subordinate clause.

 

Test

I found only one or two sentences that contained subordinate clauses.

YES=1, NO=0.

Impression

Overall, the sentences seemed simple, because they contained very few subordinate clauses, or none.

YES=1, NO=0.

Example

Which sentence seemed particularly complex, if any?

Comments

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

References

See: Creaghead and Donnelly (1982), Galitz (1985), Heckel (1984), Horton (1990), Isakson and Spyridakis (1999), Kilian (1999), Larkin and Burns (1977), Lynch & Horton (1997), Rayner, Carlson, and Frazier (1983), Roemer & Champanis (1982), Spyridakis (2000) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf

 

There are very few nouns made up to replace perfectly good verbs or adjectives.

Test

I found only one or two sentences that contained nominalizations—nouns made out of perfectly good verbs or adjectives, to make the prose sound more elegant, or formal.

YES=1, NO=0.

Impression

Overall, most nouns seemed simple, not made up out of verbs, to avoid actually using the verb.

YES=1, NO=0.

Example

Which nouns seemed a way for the writer to avoid using the original verb?

Comments

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

References

See: Bush & Campbell (1995), Horton (1990), Price & Korman (1993), Tarutz (1992), Waite (1982), Williams (1990)) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf

 

There are no strings of three or more nouns in a row.

Test

I found no noun trains, where all the nouns build up to a single idea (as opposed to a list of items).

YES=1, NO=0.

Impression

Overall, the text was free of noun strings where I wondered how to put the nouns together, to figure out which nouns formed pairs modifying some other noun.

YES=1, NO=0.

Example

The worst noun string.

Comments

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

References

See: Bush & Campbell (1995), Horton (1990), Price & Korman (1993), Tarutz (1992), Waite (1982), Williams (1990) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf

 

There are no ambiguous phrases that the reader must ponder and debate.

Tests

I found no familiar terms used in a strange new way

YES=1, NO=0.

For every pronoun, I found the noun referred to, right before the pronoun, without any other nouns in between.

YES=1, NO=0.

No sentences began with a solo “This” or “That” referring vaguely to the whole previous sentence, or an idea suggested, but not spelled out, earlier.

YES=1, NO=0.

References to other elements within the page, or on other pages, do not assume that I am reading sequentially (first, second, before, after), or that the element is in a particular spatial relation to the mention (above, below, to the right, to the left).

YES=1, NO=0.

 

Impression

Overall, the text was free of ambiguities.

YES=1, NO=0.

Example

The worst ambiguity I found (if any).

Comments

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

References

See: Horton (1990), Kilian (1999), Levine (1997), Price & Korman (1993), Rosenfeld (1999), Tarutz (1992) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf

 

The agent—the person, program, or thing carrying out the action—appears as the grammatical subject of the sentence..

Tests

The subject of the sentence is, in fact, the person, program, or thing that performs the action described in the main verb.

YES=1, NO=0.

There are no verbs in passive voice (“The ball was hit.”). 

YES=1, NO=0.

If the agent acts on an object, the object appears after the verb. 

YES=1, NO=0.

 

Impression

Overall, the text followed the standard English order: subject, verb, object.

YES=1, NO=0.

Example

This sentence hid the agent, or obscured the action:

Comments

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

References

See: Broadbent (1978), Flower, Hayes, and Swarts (1983), Herriot (1970), Horton (1990), Kilian (1999), Kintsch (1993), Spyridakis (2000), Tarutz (1992), Williams (1990) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf

 

All statements are positive.

Tests

The writer avoids explicit negatives, such as not, no, never.

YES=1, NO=0.

The writer avoids implicit negatives, in words such as impossible, unlikely, inattentive. 

YES=1, NO=0.

The writer avoids putting together several explicit or implicit negatives in the same sentence.

YES=1, NO=0.

 

Impression

Overall, the text seemed straightforward and declarative.

YES=1, NO=0.

Example

This most confusingly negative sentence.

Comments

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

References

See: Boomer (1975), Chase & Clark (1972), Clark & Chase (1972), Dewer (1976), Hackos & Stephens (1996), Herriot (1970), Horton (1990), Simpson & Casey (1988), Whitaker & Stacey (1981), Wickens (1984) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf

 

Scrolling is kept to a minimum.

Tests

I rarely had to scroll down more than one or two screens, to see all the content on the page.

YES=1, NO=0.

I never had to scroll horizontally. 

YES=1, NO=0.

The most important content appeared above the fold, at the top of the first window.

YES=1, NO=0.

Even when some content was out of sight, I had a good idea what I would find, when I eventually scrolled down to that text.

YES=1, NO=0.

 

Impression

Overall, the pages seemed to put key content into the first window, without making me scroll a lot to find important information. I was never in doubt about what was out of sight, lower down, on the page.

YES=1, NO=0.

Example

This page with the most scrolling was:.

Comments

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

References

See: Black & Elder (1997), Dillon (1994), IBM (1999), Farkas and Farkas (2000), Levine (1997), Lovelace and Southall (1983), Lynch (2000), Microsoft (2000), Nielsen (1997, 1999f), Rothkopf (1971) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf

 

Long documents even if broken up into a series of short pages also appear on a single page, or in a single file, for saving or printing.

Tests

Long documents appeared in a printer-friendly version.

YES=1, NO=0.

Long documents appeared in a downloadable PDF version. 

YES=1, NO=0.

 

Impression

When a page might extend over several printed sheets, the site offered me the opportunity to get an HTML or PDF file formatted for printing.

YES=1, NO=0.

Example

The longest page was x pages long when printed out.

Comments

If you have further thoughts, or reflections, about the way the text follows, or ignores, this guideline, please put those observations here.

References

See: IBM (1999), Kilian (1999), Levine (1997), Lynch & Horton (1997) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf

 


Total Score

Assigning a grade to text is always a bit arbitrary.  But counting up the points for these sample texts, we reach this diagnosis:

Total Points:

Total Possible:

Percentage:

Interpretation

90-100%: Very easy to understand.

75-89%: Pretty clear, with occasional rough spots.

60-74%: Could use some editing to make the text easier to comprehend.

45-59%: Confusing.

25-44%: A tangled mess.

0-24: Laughably incoherent.

Overall Conclusions

In a few paragraphs, summarize your most important observations—both positive and negative.

Major Recommendations

List the top 3 problems with the text, and in a sentence or two, summarize what you would recommend as solutions.