The VHA Health Informatics Human Factors Engineering (HFE) works with development teams to ensure that mobile applications (Apps) meet usability standards and “acceptable use” criteria. Our reviews are meant to ensure a minimum level of usability by identifying issues that would lead to dissatisfaction among end users. When a request for a compliance review is received, the response typically takes the form of a heuristic evaluation (HE), where a human factors expert evaluates an App against a pre-defined set of widely accepted usability heuristics (considered standards of best practices in both business and academia) and reports issues found, categorized as high, medium or low impact.
HEs are most appropriate for Apps, such as education or progress tracking tools, that do not involve private or vital medical information. In some cases, HFE may recommend or require additional evaluations, such as usability testing. Usability testing is commonly conducted with Apps that require users to enter personal information or Apps where patient safety could be a potential issue. The final determination on the type of testing to be pursued is made in collaboration with the App development team.
There is not a pass/fail that accompanies the heuristic evaluation; the purpose of a heuristic evaluation is to identify issues that might be expected to impact use in a negative way. Therefore, we generally recommend addressing all "high-impact" issues if at all possible, and, as resources allow, tackling the "medium-impact" issues as well. We work iteratively (typically via conference calls) with App development teams to track any changes made as well as the any reasons that changes are not being pursued for each of the findings.
The HFE User Interface (UI) Design Confirmation Review is a separate compliance review requirement.
In addition to the compliance review that occurs at the end of the App development process, App development teams are invited to contact us when ideas for Apps are registered. By engaging with HFE early in the App development process, you can ensure that you product will be intuitive, engaging, and well received by your end-users. The potential for HFE engagement throughout the development process is summarized below.
The following steps outline the process for seeking iterative expert reviews from HFE prior to the final compliance review:
- App Developers may request a review of a completed set of mock-ups (wireframes) at any time before or during development. Developers will provide the mock-ups and associated documentation to HFE and schedule a meeting to walk through the mock-ups and their intended functionality. Typically, the minimal state will be 1/3 complete for a larger App and 1/2 complete for a smaller App. This notification will include the date that the build will be available. Tasking for HFE contractors (if necessary) will be approved by HFE.
- HFE will conduct an initial evaluation of the App. This evaluation will typically result in annotated screenshots and a findings table. The document is returned to developers within as close to one week as possible.
- Business owner and developers will review the evaluation and document a response for each finding. Responses will be discussed during a call among HFE, business owner and App development team.
- This process will repeat at intervals appropriate to the App’s development; ideally, HFE will review after each significant change.
In addition to expert reviews, HFE highly recommends that development teams consider involving end users as often as possible.
HFE is managed under Emerging Health Technologies.
For more information pertaining to Human Factors Engineering and associated methods, please refer to our Human Factors Engineering SharePoint site.
- mHIMSS: Selecting a Mobile App: Evaluating the Usability of Medical Applications: http://www.mhimss.org/resource/selecting-mobile-app-evaluating-usability-medical-applications
- Jakob Nielsen: Ten Usability Heuristics http://www.useit.com/papers/heuristic/heuristic_list.html
- BJ Fogg: Persuasive Technology: Using Computers to Change What We Think and Do