Facebook
Edutopia on Facebook
Twitter
Edutopia on Twitter
Google+
Edutopia on Google+
Pinterest
Edutopia on Pinterest Follow Me on Pinterest
WHAT WORKS IN EDUCATION The George Lucas Educational Foundation

Technical Writing: How Usability Prepares Students for Essay-Scoring Software

Technical Writing: How Usability Prepares Students for Essay-Scoring Software

Related Tags: STEM
More Related Discussions
0 Replies 17 Views
T. R. Girill Society for Technical Communication/Lawrence Livermore National Lab. trgirill@acm.org Technical Writing: How Usability Prepares Students for Essay-Scoring Software Now that essay questions are starting to appear on standardized tests, the high cost of human readers has accelerated the push to develop software that scans each student essay and outputs an evaluation just as a real person would. In fact, one recent foundation-funded study compared no less than eight such programs that automate essay scoring (Mark D. Shermis and Ben Hammer, "Contrasting State-of-the-art Automated Scoring of Essays: Analysis," April 2012, online at http://dl/dropbox.com/u/44416236/NCME%202012%20Paper3_29_12.pdf ). If this is the future that your students face, how can you prepare them to draft nonfiction science text that diverse essay-scoring software would rate highly? How Text Looks to a Software Robot Automated essay scoring usually focuses on one of four kinds of text features, relying on that feature to index or reflect overall essay quality (a few programs try to meld several feature types from this same set). The kinds are: Linguistic Features. Some programs look for likely linguistic mistakes (such as subject/verb disagreement in sentences or inappropriate use of English articles ('a' or 'the')), or count and compare such linguistic features as prepositions used or vocabulary diversity. These are just the hardest English features for many non-native speakers to deploy fluently and flawlessly. Statistical Features. During the 1980s, many "readability formulas" tried to estimate text quality by calculating such quantitative features as average sentence length (in words) or average word length (in syllables). Some essay-scoring programs today also rely on a similar statistical approach. Logical Features. When you read a text you are concerned about what it asserts (the logical propositions that it contains) and how those assertions fit together (the logical argument that the text advances). Some software likewise tries to identify propositions and arguments, even when expressed in diverse linguistic forms. Historical Features. This longitudinal or developmental approach to automated essay rating is the most indirect: the program scans the same 12 million words (but only about 80,000 unique words) of reading material that a typical high-school student would encounter in their lifetime, then compares their essay to that corpus looking for revealing trends. The Usability Response So how can your students prepare themselves for these feature- based rating schemes without obsessively, mechanically trying to manipulate each feature that the software might check? Fortunately, there is a strategic alternative. Readers of this technical-literacy series know that several decades of empirical usability research have yielded text-design techniques that help humans construct prose that other humans can really use to solve "far-transfer" science and engineering problems. Real people value such prose because they find it so effective. But, as a side benefit, it also ends up with just the features that essay-scoring software rates highly (but not installed in an artificial way). One crude but revealing indicator of this overlap between the usability approach to text engineering and the "computational needs" of automated essay rating appears in the index to Karen Schriver's influential handbook "Dynamics in Document Design" (Wiley, 1997). I easily found half a dozen index entries that reflected longitudinal/developmental topics in her book, seven that addressed linguistic features, and 12 index entries on readability/statistical issues. Schriver's entire 55-page Chapter 3 ("How documents engage readers' thinking and feeling") explores the impact of evidence and propositional clarity on text quality. So the strategic way of "beating the essay software" is to learn about and practice writing techniques that yield usable, effective technical text from the inside out. Your students are unlikely to hear the word "usability" in language arts class, however....that burden will likely fall to you. Want a quick review of usability issues and research? See "Usability: How Technical Writing Succeeds" online at http://www.ebstc.org/TechLit/handbook/usability.html

Comments (0 Replies)Sign in or register to postSubscribe to comments via RSS

article STEM Everywhere: Science, Technology, Engineering, and Math in the Real World

Last comment 1 week 5 days ago in STEM

blog Informal Learning: From STEM Clubs to Careers in Science

Last comment 1 week 5 days ago in STEM

blog Computer Science: The Future of Education

Last comment 6 days 9 hours ago in STEM

blog How to Design Right-Sized Challenges

Last comment 5 days 1 hour ago in PBL Planning

Discussion Science Fair Mentor

Last comment 1 week 4 days ago in Health

Sign in and Join the Discussion! Not a member? Register to join the discussion.