Usability

Updated May 17, 2013

The Usability page focuses on major national policy reports/documents and selected research articles going back to 1999.  Resources are selected with an emphasis on usability as a software selection tool and as an adjunct to system design.   As always, if you have comments and suggestions, please submit them using the contact form.

Books
Tidwell  J.  Designing Interfaces, 2^nd Edition. Sebastopol, CA : O’Reilly Media; 2011  [New]

Microsoft Inductive User Interface Guideline. Microsoft; 2001 [New]

Reports
Agency for Healthcare Research and Quality
EHR Usability Toolkit: A Background Report on Usability and Electronic Health Records, 2011
Electronic Health Record Usability: Vendor Practices and Perspectives, 2010
Electronic Health Record Usability: Interface Design Considerations , 2009
Electronic Health Record Usability: Evaluation and Use Case Framework, 2009

Institute of Medicine
Comparative User Experiences of Health IT Products: How User Experiences Would Be Reported and Used, 2012

National Institute of Standards and Technology
A Human Factors Guide to Enhance EHR Usability of Critical User Interactions when Supporting Pediatric Patient Care, 2012
Creating Usable Electronic Health Records (EHRs): A User-Centered Design Best Practices Workshop,  2012
Technical Evaluation, Testing and Validation of the Usability of Electronic Health Records Literature, 2012
Customized Common Industry Format Template for Electronic Health Record Usability Testing, 2011
NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records, 2011

Posts
EHR Design
Moving Beyond Paper-based Thinking in EHR Design
Making Clinical Concepts Computable
Rethinking the Design of Electronic Health Record Systems
How Does EHR Design Affect Implementation and Optimization?

Testing
Enhancing Usability Testing, EHR Selection, and Informatics Research with Prototyping Tools
An Interesting Approach to EHR Selection and Usability Testing

Tools
Usability as an EHR Selection Tool
NIST Usability Resources—A Goldmine for Developers

Software
University of Texas Health Science Center at Houston
turf:EHR Usability Toolkit  - Usability evaluation tool for EHRs developed as part of the Strategic Healthcare IT Advanced Research Projects (SHARP) program.  [New]

Key Articles
Flanagan ME, Saleem JJ, Millitello LG, Russ AL, Doebbeling BN. Paper- and computer-based workarounds to electronic health record use at three benchmark institutions. J Am Med Inform Assoc. 2013 Mar 14.  [New]

Saleem JJ, Russ AL, Justice CF, Hagg H, Ebright PR, Woodbridge PA, Doebbeling BN. Exploring the persistence of paper with the electronic health record. Int J Med Inform. 2009 Sep;78(9):618-28. [New]

Middleton B, Bloomrosen M, Dente MA, Hashmat B, Koppel R, Overhage JM, PayneTH, Rosenbloom ST, Weaver C, Zhang J. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc. 2013 Jan 25 [E]

Zhang J, Walji MF. TURF: toward a unified framework of EHR usability. J Biomed Inform. 2011 ec;44(6):1056-67.

Articles [New April 19, 2013 – May 17, 2013]  [E] = e-publish date
2013
Waterson P, Hoonakker PL, Carayon P. Special issue on human factors and the implementation of health information technology (HIT): Comparing approaches across nations. Int J Med Inform. 2013 May;82 (5):277-80.

Kushniruk AW, Bates DW, Bainbridge M, Househ MS, Borycki EM. National efforts  to improve health information system safety in Canada, the United States of America and England. Int J Med Inform. 2013 May;82 (5):e149-60.

Patterson ES, Zhang J, Abbott P, Gibbons MC, Lowry SZ, Quinn MT, Ramaiah M, Brick D. Enhancing electronic health record usability in pediatric patient care: a scenario-based approach. Jt Comm J Qual Patient Saf. 2013 Mar;39(3):129-35.

Flanagan ME, Saleem JJ, Millitello LG, Russ AL, Doebbeling BN. Paper- and computer-based workarounds to electronic health record use at three benchmark institutions. J Am Med Inform Assoc. 2013 Mar 14. [E]

Keenan G, Yakel E, Dunn Lopez K, Tschannen D, Ford YB. Challenges to nurses’ efforts of retrieving, documenting, and communicating patient care information. J Am Med Inform Assoc. 2013 Mar 1;20(2):245-51.

Walji MF, Kalenderian E, Tran D, Kookal KK, Nguyen V, Tokede O, White JM, Vaderhobli R, Ramoni R, Stark PC, Kimmes NS, Schoonheim-Klein ME, Patel VL. Detection and characterization of usability problems in structured data entry interfaces in dentistry. Int J Med Inform. 2013 Feb;82(2):128-38.

Middleton B, Bloomrosen M, Dente MA, Hashmat B, Koppel R, Overhage JM, PayneTH, Rosenbloom ST, Weaver C, Zhang J. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc. 2013 Jan 25. [E]

Kopanitsa G, Hildebrand C, Stausberg J, Englmeier KH. Visualization of Medical Data Based on EHR Standards. Methods Inf Med. 2013 Jan 14;52(1):43-50.

McGreevey JD. Order sets in electronic health records: principles of good practice. Chest. 2013 Jan 1;143(1):228-35.

2012
Horsky J, Schiff GD, Johnston D, Mercincavage L, Bell D, Middleton B. Interface design principles for usable decision support: A targeted review of best practices for clinical prescribing interventions. J Biomed Inform. 2012 Dec;45(6):1202-16.

Russ AL, Weiner M, Russell SA, Baker DA, Fahner WJ, Saleem JJ. Design and implementation of a hospital-based usability laboratory: insights from a Department of Veterans Affairs laboratory for health information technology. Jt Comm J Qual Patient Saf. 2012 Dec;38(12):531-40.

Kannampallil TG, Franklin A, Mishra R, Almoosa KF, Cohen T, Patel VL. Understanding the nature of information seeking behavior in critical care: Implications for the design of health information technology. Artif Intell Med. 2012 Nov 26. [E]

Li AC, Kannry JL, Kushniruk A, Chrimes D, McGinn TG, Edonyabo D, Mann DM. Integrating usability testing and think-aloud protocol analysis with “near-live”  clinical simulations in evaluating clinical decision support. Int J Med Inform. 2012 Nov;81(11):761-72.

Terner A, Lindstedt H, Sonnander K. Predefined headings in a multiprofessional electronic health record system. J Am Med Inform Assoc. 2012 Nov 1;19(6):1032-8.

Holmes C, Brown M, Hilaire DS, Wright A. Healthcare provider attitudes towards the problem list in an electronic health record: a mixed-methods qualitative study. BMC Med Inform Decis Mak. 2012 Nov 11;12(1):127. [E]

Wanderer JP, Rao AV, Rothwell SH, Ehrenfeld JM. Comparing two anesthesia information management system user interfaces: a usability evaluation. Can J Anaesth. 2012 Oct 5. [E]

Neri PM, Pollard SE, Volk LA, Newmark LP, Varugheese M, Baxter S, Aronson SJ,  Rehm HL, Bates DW. Usability of a novel clinician interface for genetic results.  J Biomed Inform. 2012 Oct;45(5):950-7.

Knijnenburg SL, Kremer LC, Versluys AB, Braam KI, Mud MS, van der Pal HJ, Caron HN, Jaspers MW. Evaluation of a patient information website for childhood cancer survivors. Support Care Cancer. 2012 Sep 25. [E]

Urda D, Ribelles N, Subirats JL, Franco L, Alba E, Jerez JM. Addressing critical issues in the development of an Oncology Information System. Int J Med Inform. 2012 Sep 13. [E]

Hollin I, Griffin M, Kachnowski S. How will we know if it’s working? A multi-faceted approach to measuring usability of a specialty-specific electronic  medical record. Health Informatics J. 2012 Sep;18(3):219-32.

Hou SI. Health Literacy Online: A Guide to Writing and Designing Easy-to-Use Health Web Sites. Health Promot Pract. 2012 Sep;13(5):577-80.

Stricker E, Pfeffer S, Trick M, Rall M, Maier T. Standardized research on ergonomics, usability and workflow analysis using high fidelity simulation labs.  Biomed Tech (Berl). 2012 Aug 31.  [E]

Ybarra ML, Biringi R, Prescott T, Bull SS. Usability and Navigability of an HIV/AIDS Internet Intervention for Adolescents in a Resource-Limited Setting. Comput Inform Nurs. 2012 Aug 22. [E]

Avansino J, Leu MG. Effects of CPOE on Provider Cognitive Workload: A Randomized Crossover Trial. Pediatrics. 2012 Aug 13. [E]

van der Peijl J, Klein J, Grass C, Freudenthal A. Design for risk control: The role of usability engineering in the management of use-related risks. J Biomed Inform. 2012 Aug;45(4):795-812.

Love JS, Wright A, Simon SR, Jenter CA, Soran CS, Volk LA, Bates DW, Poon EG. Are physicians’ perceptions of healthcare quality and practice satisfactionaffected by errors associated with electronic health record use? J Am Med Inform Assoc. 2012 Jul 1;19(4):610-4.

Poulter T, Bath PA. The Use and Usability of EPR systems in Oncology. StudHealth Technol Inform. 2012;180:398-402.

Marcilly R, Bernonville S, Riccioli C, Beuscart-Zephir MC. Patientsafety-oriented usability testing: a pilot study. Stud Health Technol Inform.2012;180:368-72.

Kopanitsa G, Tsvetkova Z, Veseli H. Analysis of Metrics for the UsabilityEvaluation of EHR Management Systems. Stud Health Technol Inform.2012;180:358-62.

López-García P, Boeker M, Illarramendi A, Schulz S. Usability-driven pruning of large ontologies: the case of SNOMED CT. J Am Med Inform Assoc. 2012 Jun 1;19(e1):e102-e109.

Fritz F, Balhorn S, Riek M, Breil B, Dugas M. Qualitative and quantitative evaluation of EHR-integrated mobile patient questionnaires regarding usability and cost-efficiency. Int J Med Inform. 2012 May;81(5):303-13.

Bakhshi-Raiez F, de Keizer NF, Cornet R, Dorrepaal M, Dongelmans D, Jaspers MW. A usability evaluation of a SNOMED CT based compositional interface terminology for intensive care. Int J Med Inform. 2012 May;81(5):351-62.

Lindblom K, Gregory T, Wilson C, Flight IH, Zajac I. The impact of computer self-efficacy, computer anxiety, and perceived usability and acceptability on the efficacy of a decision support tool for colorectal cancer screening. J Am Med Inform Assoc. 2012 May-Jun;19(3):407-12.

Yen PY, Bakken S. Review of health information technology usability study  methodologies. J Am Med Inform Assoc. 2012 May-Jun;19(3):413-22.

Menon A, Korner-Bitensky N, Chignell M, Straus S. Usability testing of two e-learning resources: methods to maximize potential for clinician use. J Rehabil Med. 2012 Apr;44(4):338-45.

Yadrich DM, Fitzgerald SA, Werkowitch M, Smith CE. Creating patient and family education web sites: assuring accessibility and usability standards. Comput Inform Nurs. 2012 Jan;30(1):46-54.

Alsos OA, Das A, Svanæs D. Mobile health IT: the effect of user interface and form factor on doctor-patient communication. Int J Med Inform. 2012 Jan;81(1):12-28.

Chan AJ, Islam MK, Rosewall T, Jaffray DA, Easty AC, Cafazzo JA. Applying usability heuristics to radiotherapy systems. Radiother Oncol. 2012 Jan;102(1):142-7.

Darby J, Black J, Morrison D, Buising K. An information management system for patients with tuberculosis: Usability assessment with end-users. Stud Health Technol Inform. 2012;178:26-32.

Kopanitsa G, Tsvetkova Z, Veseli H. Analysis of metrics for the usability evaluation of electronic health record systems. Stud Health Technol Inform. 2012;174:129-33.

2011
Zopf-Herling KM. Enhancing usability in EMR screen design. Comput Inform Nurs. 2011 Dec;29(12):679-91.

Zachariah M, Phansalkar S, Seidling HM, Neri PM, Cresswell KM, Duke J, Bloomrosen M, Volk LA, Bates DW. Development and preliminary evidence for the validity of an instrument assessing implementation of human-factors principles in medication-related decision-support systems–I-MeDeSA. J Am Med Inform Assoc. 2011 Dec;18 Suppl 1:i62-72.

Zhang J, Walji MF. TURF: toward a unified framework of EHR usability. J Biomed Inform. 2011 ec;44(6):1056-67.

Holden RJ. What stands in the way of technology-mediated patient safety improvements?: a study of facilitators and barriers to physicians’ use of electronic health records. J Patient Saf. 2011 Dec;7(4):193-203.

Carayon P, Cartmill R, Blosky MA, Brown R, Hackenberg M, Hoonakker P, Hundt AS, Norfolk E, Wetterneck TB, Walker JM. ICU nurses’ acceptance of electronic health records. J Am Med Inform Assoc. 2011 Nov-Dec;18(6):812-9.

Viitanen J, Hyppönen H, Lääveri T, Vänskä J, Reponen J, Winblad I. National questionnaire study on clinical ICT systems proofs: physicians suffer from poor usability. Int J Med Inform. 2011 Oct;80(10):708-25

Harrington L, Porch L, Acosta K, Wilkens K. Realizing electronic medical record benefits: an easy-to-do usability study. J Nurs Adm. 2011Jul-Aug;41(7-8):331-5.

Carrington JM, Effken JA. Strengths and limitations of the electronic health record for documenting clinical events. Comput Inform Nurs. 2011 Jun;29(6):360-7.

Harrington C, Wood R, Breuer J, Pinzon O, Howell R, Pednekar M, Zhu M, Zhang J. Using a unified usability framework to dramatically improve the usability of an EMR Module. AMIA Annu Symp Proc. 2011;2011:549-58.

Hall SA, Kushniruk AW, Borycki EM. Usability analysis of the tele-nursing call management software at HealthLink BC. Stud Health Technol Inform. 2011;164:208-12.

2010
Belden J. Three bright ideas for improving electronic health record usability. Fam Pract Manag. 2010 Jul-Aug;17(4):7-9.

Saitwal H, Feng X, Walji M, Patel V, Zhang J. Assessing performance of an Electronic Health Record (EHR) using Cognitive Task Analysis. Int J Med Inform. 2010 Jul;79(7):501-6.

Corrao NJ, Robinson AG, Swiernik MA, Naeim A. Importance of testing for usability when selecting and implementing an electronic health or medical record  system. J Oncol Pract. 2010 May;6(3):120-4

Boone E. EMR usability: bridging the gap between nurse and computer. Nurs Manage. 2010 Mar;41(3):14-6.

Webster C, Copenhaver M. Process-aware EHR BPM systems: two prototypes and a  conceptual framework. Stud Health Technol Inform. 2010;160(Pt 1):106-10.

Koppel R, Kreda DA. Healthcare IT usability and suitability for clinical needs: challenges of design, workflow, and contractual relations. Stud Health Technol Inform. 2010;157:7-14.

2009
Zheng K, Padman R, Johnson MP, Diamond HS. An interface-driven analysis of user interactions with an electronic health records system. J Am Med Inform Assoc. 2009 Mar-Apr;16(2):228-37. .

Meredith J. Electronic patient record evaluation in community mental health.  Inform Prim Care. 2009;17(4):209-13.

Senathirajah Y, Bakken S. Architectural and usability considerations in the development of a Web 2.0-based EHR. Stud Health Technol Inform. 2009;143:315-21.

2008
Moore BJ, Wright JA, Watson B, Friedman RH, Adams WG. Usability testing of an electronic health record form to support physician-based counseling and self-management of overweight children. AMIA Annu Symp Proc. 2008 Nov 6:1061.

Schnipper JL, Linder JA, Palchuk MB, Einbinder JS, Li Q, Postilnik A, Middleton B. “Smart Forms” in an Electronic Medical Record: documentation-based clinical decision support to improve disease management. J Am Med Inform Assoc. 2008 Jul-Aug;15(4):513-23.

Wu RC, Orr MS, Chignell M, Straus SE. Usability of a mobile electronic medical record prototype: a verbal protocol analysis. Inform Health Soc Care. 2008 Jun;33(2):139-49. PubMed PMID: 18604757.

Alsos OA. Attention and usability issues in mobile health information systems at point-of-care. Stud Health Technol Inform. 2008;136:877-8.

Jaspers MW, Peute LW, Lauteslager A, Bakker PJ. Pre-post evaluation ofphysician s’ satisfaction with a redesigned electronic medical record system. Stud Health Technol Inform. 2008;136:303-8.

2007
Whitten P, Buis L, Mackert M. Factors impacting providers’ perceptions regarding a midwestern university-based EMR. Telemed J E Health. 2007 Aug;13(4):391-7.

Kjeldskov J, Skov MB, Stage J. A longitudinal study of usability in health care – does time heal? Stud Health Technol Inform. 2007;130:181-91.

Zheng K, Padman R, Johnson MP. User interface optimization for an electronic  medical record system. Stud Health Technol Inform. 2007;129(Pt 2):1058-62.

2006
Santiago O, Li Q, Gagliano N, Judge D, Hamann C, Middleton B. Improving EMR Usability: a method for both discovering and prioritizing improvements in EMR workflows based on human factors engineering. AMIA Annu Symp Proc. 2006:1086.

Drury BM. Ambulatory EHR functionality: a comparison of functionality lists.  J Healthc Inf Manag. 2006 Winter;20(1):61-70.

Møller-Jensen J, Lund Pedersen I, Simonsen J. Measurement of the clinical usability of a configurable EHR. Stud Health Technol Inform. 2006;124:356-61.

Lilholt LH, Pedersen SS, Madsen I, Nielsen PH, Boye N, Andersen SK, Nøhr C. Development of methods for usability evaluations of EHR systems. Stud Health Technol Inform. 2006;124:341-6.

Angermo LM, Ruland CM. Increasing the usability of nursing intervention classification in EHR’s. Stud Health Technol Inform. 2006;122:985-6.

2005
Rose AF, Schnipper JL, Park ER, Poon EG, Li Q, Middleton B. Using qualitative studies to improve the usability of an EMR. J Biomed Inform. 2005 Feb;38(1):51-60.

2002
Rodriguez NJ, Murillo V, Borges JA, Ortiz J, Sands DZ. A usability study of physicians interaction with a paper-based patient record system and a graphical-based electronic patient record system. Proc AMIA Symp. 2002:667-71.

Krall MA, Sittig DF. Clinician’s assessments of outpatient electronic medical record alert and reminder usability and usefulness requirements. Proc AMIA Symp.2002:400-4.

2000
Langlotz CP, Meininger L. Enhancing the expressiveness and usability of structured image reporting systems. Proc AMIA Symp. 2000:467-71.

1999
Sittig DF, Kuperman GJ, Fiskio J. Evaluating physician satisfaction regarding user interactions with an electronic medical record system. Proc AMIA Symp. 1999:400-4.