Manage Learn to apply best practices and optimize your operations.

Behavioral eManipulation: Attacking the care delivery workflow to deliver harm

Here, we introduce and analyze the novel security concept of Behavioral eManipulation, the attack methodology with which an attacker uses technical exploits to influence human behavior in order to arrive at an adverse outcome.

Technical exploit + human behavior = Adverse outcome

Behavioral eManipulation entails tricking a competent professional into making an error that he otherwise might not make. This concept is relevant to scenarios where all of the following conditions are present:

  • The workflow relies on technology that could be exploited,
  • The workflow relies on humans, whose behavior could be influenced, and
  • There is potential for a tangible or intangible adverse outcome resulting from modified data, stolen assets, denial of service or system availability, etc.

A high-profile example of Behavioral eManipulation was Stuxnet, where an Iranian nuclear centrifuge was attacked while simultaneously the alarms to alert technicians of mechanical issues were silenced, thereby preventing the necessary human intervention. This denial of human interaction resulted in severe damage to the centrifuge, which would take years to repair.

Other adverse outcomes might include scenarios such as revenue prevention, profit erosion, negative customer/user experience, customer attrition/exodus, stock price decline, brand damage or physical harm. For illustrative purposes, we narrow scope to focus on the lattermost of those outcomes, by analyzing the specific workflow of delivery of healthcare. We do this for a few reasons: first, it provides compelling examples for illustration; second, it involves the most valuable possible asset (human safety); and third, it is an industry that is not adequately preparing itself to address this concept. In the context of the healthcare delivery workflow, the formula for Behavioral eManipulation can be summarized as follows:

Technical exploit + human behavior = Patient harm

Research context
The foundation for this research is based upon the key findings of “Hacking Hospitals,” a 24-month study of hospitals across the United States, that published exploitable technical vulnerabilities inherent in medical devices and their supporting networks, business shortcomings inherent within the healthcare industry that lead to the introduction of these exploitable technical vulnerabilities, and a blueprint for how to solve these issues. We focused the following body of research on this concept of Behavioral eManipulation to extrapolate the ways in which exploitable security vulnerabilities manifest themselves in the real-world delivery of care. For this phase of the research, we consulted renowned trauma surgeon Konstantinos M. Triantafillou, MD, a practicing orthopaedic trauma surgeon, certified by the American Board of Orthopaedic Surgery, who serves patients at University of Tennessee Medical Center, a Level I trauma center. Over the course of the 14 months following the publication of “Hacking Hospitals,” through interviews and exploit mapping exercises, we have used Dr. Triantafillou’s perspective as an actively practicing trauma surgeon to help parse out the unlikely theoretical outcomes from the high-risk areas on which the security community should focus.

Assets
Core to this paper and the concept of Behavioral eManipulation is the assumption that the most important asset that the healthcare ecosystem must protect is patient safety. The key finding of the seminal “Hacking Hospitals” is that the healthcare industry is focused primarily on protecting patient data, with inadequate protections of patient safety; the concept of Behavioral eManipulation reinforces this assumption by demonstrating how this misalignment of the security mission in healthcare could undermine the Hippocratic Oath, a concept which most healthcare professionals have sworn to uphold. This oath advocates that first and foremost, medical professionals shall do no harm. The integrity of this oath can be undermined through Behavioral eManipulation.

Medical devices: Active versus passive
Much of the healthcare-oriented security research to date has focused on active medical devices, which are those that actively do something to the patient, such as delivering medication or modifying the heartbeat. Some groundbreaking findings related to active medical devices have been published in the areas of insulin pumps, drug infusion pumps and pacemakers. It’s fairly straightforward to understand how manipulating active devices might result in harm to a patient. However, prior to “Hacking Hospitals,” minimal attention was paid to passive medical devices, which are not doing something to the patient, but instead are reacting to the patient. Examples of passive medical devices include patient monitors, imaging and lab equipment. While the direct correlation between device input and patient outcome in the context of active medical devices makes it easy to understand how manipulating active medical devices could harm a patient, what is lesser understood is how similar patient safety outcomes could arrive in the context of passive medical devices. Behavioral eManipulation shows how this is possible, by using such devices to attack the workflow; the workflow in turn delivers the adverse outcome rather than the device directly.

Attack scenarios

Methodologies
Behavioral eManipulation is best understood in the context of conventional cyberattack methods:

  • Technical exploits attack systems. This is where an attacker finds and exploits a security vulnerability (or vulnerabilities) in an application or network. A well-known example of this is when an American teenager hacked the International Space Station’s life support systems, causing $41,000 to repair the damage.
  • Social engineering attacks people. This is where an attacker takes advantage of human susceptibility to exploit the human. Perhaps the most famous example of all time is the original Trojan Horse, where a seemingly defeated Greek army left a trophy for the seemingly triumphant Trojans, who then unwittingly brought the trophy inside the walls only to later find it filled with invading Greeks.
  • Behavioral eManipulation attacks workflows. This is where an attacker exploits a device or system that an unwitting human victim relies upon to make decisions, and then the workflow delivers the adverse outcome as a result of reliance on that bad information. Examples of this include the variety of scenarios described below in this section.

Behavioral eManipulation has a few discrete phases to the attack flow:

  1. Understand the workflow
  2. Manipulate the data
  3. Rely on humans to unwittingly deliver the adverse outcome
    1. In the context of healthcare, phase 1 is actually a mitigating factor, it requires many years of medical training for aspiring physicians to learn the many care delivery workflows. However, while that limits the potential pool of viable attackers who can use this attack method efficiently, it does not prevent any attacker from acquiring this knowledge.
    2. Phase 2 is where most security research to date has focused.
    3. Not including accidental exploits, phase 3 is only possible when the combination of phases 1 and 2 have concluded.

Diagnostic attack scenarios: Overview
Most medical errors are communication errors. Behavioral eManipulation is essentially an amplification of communication errors, thereby compounding medical errors. In a healthcare context, Behavioral eManipulation manifests itself most dramatically in diagnostic workflows, which are the parts of the care delivery workflow that require a physician, nurse or other medical professional to interpret data in order to diagnose a patient’s health issue, for purposes of prescribing a treatment plan. Such diagnostic scenarios largely fall into one of two categories:

  1. Workflows that have failsafes built in, also known as “system of checks,” or
  2. Workflows that do not have failsafes and rely heavily on the physician to interpret the data.

Each of these diagnostic attack scenarios has different implications to patient safety if the information upon which the physician bases decisions is manipulated.

Diagnostic attack scenarios: Systems of checks
Diagnostic attack scenarios with systems of safety checks are typically those which involve biographical data, systems or other information which are known to deliver harm if the workflow is flawed. An ever-growing body of statistics demonstrates that errors related to blood and allergies are known to have harmful or even fatal implications; for that reason, healthcare providers have numerous safety checks in place to ensure such errors are minimized. If, in a typical care workflow, the physician notes an anomaly, there are procedures in place that would trigger certain actions1 by the physician to verify the anomaly before prescribing a potentially harmful procedure or dosage. Through Behavioral eManipulation, an attacker would trigger anomalies that require physicians to re-perform diagnostics, order new tests, and/or consult additional medical professionals. Time is critical in the kinds of settings where such diagnostics would take place (e.g., acute care settings, operating rooms, etc.), and such additional steps all require critical time. As a result, in some cases time impedes care, while in other cases the physician may lose trust in the diagnostics system – assuming it to be faulty – and revert to other diagnostic methods that circumvent the established system of checks. All of these conditions dramatically increase the likelihood of medical error that could result from such manipulation of the care delivery workflow.

Proper workflow

  1. Patient produces data
  2. Physician reviews data
  3. Physician consults duplicative sources for data (failsafe)
  4. Physician makes diagnosis
  5. Physician administers treatment plan

Manipulated workflow

  1. Patient produces data
  2. Physician reviews data
  3. Attacker exploits security vulnerability*
  4. Attacker manipulates duplicative data sources
  5. Physician consults duplicative data sources (failsafe)
  6. Physician notes anomalies
  7. Physician measures patient again
  8. Physician reviews new data
  9. Physician consults/compares with duplicative sources for data (failsafe)
  10. Physician notes anomalies
  11. Physician measures patient again
  12. Physician reviews new data
  13. Physician proceeds on bad data, or loses trust in system
  14. Physician makes diagnosis
  15. Physician administers invalid treatment plan

*Example technical exploit: As outlined in “Hacking Hospitals,” we discovered a privilege escalation security vulnerability in lobby check-in kiosks that allowed an attacker to take the kiosk out of kiosk mode, gain administrative control, pivot to other systems on the same subnet and manipulate the data accessible by those other systems. One such system was a bloodwork system.

Diagnostic attack scenarios: Data interpretation
By contrast, some attack scenarios do not have safety checks to them and rely solely on the physician’s experience and interpretation of data to process the diagnosis and resulting treatment plan. Examples of these would pertain to monitoring equipment or lab results. The physician is unlikely to question the validity of the information coming off such systems unless the readouts fall unusually outside of expected parameters. As there are not additional failsafes in place, if acting upon manipulated information, the physician operating in the care delivery workflow would likely unwittingly deliver harm.

Proper workflow

  1. Patient produces data
  2. Physician reviews data
  3. Physician makes diagnosis
  4. Physician administers treatment plan

Manipulated workflow

  1. Patient produces data
  2. Attacker exploits security vulnerability*
  3. Physician reviews data
  4. Physician makes diagnosis
  5. Physician administers invalid treatment plan

*Example technical exploit: As outlined in “Hacking Hospitals,” we discovered an authentication bypass vulnerability that, when combined with a remote code execution exploit, allowed a remote attacker to take administrative control over a patient bedside monitor. This enabled the attacker to manipulate the data feeding the screen, thereby triggering invalid physician response or preventing valid and necessary physician response.

Remedies

The safety implications introduced with the attack concept of Behavioral eManipulation are significant; they usher in a new area of how the security community and the device manufacturing community should think about security vulnerabilities. However, many of the recommended solutions to the situation are either long-proven approaches or are new takes on long-proven approaches.

Return on investment: Training physicians versus securing devices
Many security professionals advocate for training physicians as a necessary solution to the many security vulnerabilities that plague healthcare. However, training of physicians is unlikely to lead to a considerable improvement on security posture. This is due to the fact that physician behavior is a byproduct of decades of medical training that prioritizes delivery of care in the most time-efficient manner. The most important aspect of this is that physicians are trained to recognize patterns; when they notice irregularities is when they investigate further for possible intervention. Compound this with the fact that medical professionals already invest extreme hours per week on the core medical training required to become a licensed and effective physician and there is simply insufficient time to teach a new behavior on something that is not aligned with either pattern recognition or the core medical foundation of their education. Given the confluence of these reasons, it is our position that the most effective model for addressing the workflow-related security implications introduced by this article is by focusing not on physician training, but rather on securing the devices themselves. By eliminating the workflow problems at the source of the data manipulation, the entire concept of Behavioral eManipulation is thwarted. What follows are some effective approaches for securing medical devices.

Think like an attacker thinks
First and foremost, it should go without saying that if devices have the ability to be manipulated to arrive at an outcome of harm, then they should first be investigated for how they will be attacked. This is commonly known as a security assessment, or in some vernacular as penetration tests. However, these approaches are significantly different, do not share the same goal and do not both best remediate security risk. Vendors of connected medical devices – especially of passive medical devices, which until now have not historically been considered relevant to patient safety in a security context – should not be satisfied with cursory approaches to security like automated scanning, commodity penetration testing, baseline compliance and deferring risk to the hospital. Instead, vendors should:

  • Apply the same level of rigor to security assessment as a sophisticated adversary would;
  • Engage in white box security assessments, not just black or gray box;
  • Investigate for not just known and common vulnerabilities, but also custom and zero-day vulnerabilities; and
  • Work with proven, research-focused, services-only consulting firms.

Security models: Threat model + trust model
Over the years of executing security assessments and performing security research, we’ve found that many companies either aren’t familiar with the concepts of both threat modeling and trust modeling, or of those who do, very few have both implemented. This inherently undermines a successful security mission, as both are required in order to deploy a successful security program. A threat model is an exercise through which an organization identifies what it wants to protect (i.e., its assets), who is it trying to defend against (i.e., its adversaries), and the areas against which attacks will be launched (i.e., its attack surfaces). By contrast, a trust model is an exercise through which an organization defines who and what it trusts, why it trusts those people and systems, and how access based on that trust is provisioned, revoked and validated for authorization and authentication. All device vendors should:

  • Define and implement their threat model
  • Define and implement their trust model
  • Update both frequently
  • Work with trusted third-party security experts to investigate how well the actual design and implementation of the systems adheres to both security models

Recognize role of compliance
The Health Insurance Portability and Accountability Act (HIPAA) governs much of what the healthcare industry does related to security. However, many of the issues inherent with Behavioral eManipulation are actually out of HIPAA’s scope. When it comes to medical devices in particular, the Food and Drug Administration (FDA) has published guidelines that outline security approaches relevant to medical device security. However, there is common and widespread misunderstanding about these guidelines and whether they require new rounds of FDA approval – and these approvals typically take many years. In some cases, these guidelines are not enforceable (an issue that this author investigated in a separate analysis).

In either the case of HIPAA or FDA, it is important to note that compliance typically does an adequate job of establishing the baseline requirements for the foundation of a security program; however, compliance should not be seen as the entire security program unto itself. Effective security organizations recognize the role of compliance as being important to satisfying stakeholder needs, but will go beyond the outlined minimum if delivering a robust security posture is important. To be effective in this domain, organizations should:

  • Define what a successful outcome of the security model looks like,
  • Define the delta between compliance and the desired outcome, and
  • Mobilize security investments accordingly to address the delta between regulatory compliance and security mission.

Summary

Behavioral eManipulation is the combination of technical exploit and human behavior to arrive at adverse outcomes. It represents an evolution in long-standing attack models, and should be immediately incorporated into defense schema across all organizations in all industries, with the baseline condition that those organizations have something worth protecting. For the purposes of this article, we narrowed the scope of the analysis to focus on patient harm issues related to exploited passive medical devices, but the principles of attacking workflows are relevant across virtually every industry, such as the Stuxnet example provided earlier. We advocate for suppliers of applications, devices and systems to work with security experts to identify and remediate any workflow-related security risks that may not have historically been considered in threat models and trust models for your systems. This research will first be presented at RSA Conference USA 2018.

Konstantinos M. Triantafillou, M.D. is a practicing orthopaedic trauma surgeon, certified by the American Board of Orthopaedic Surgery. He serves patients at University of Tennessee Medical Center, a Level 1 trauma center. Dr. Triantafillous is active in advancing the field of orthopaedic surgery through research, and pursues his interest in medical security due to concerns from a practitioner’s perspective about the vulnerability in medical devices deployed across surgical, acute care and in-patient settings. He is widely published across medical journals and has presented medical research at conferences such as the American Academy of Orthopaedic Surgeons. He pursued his BA, MD and residency at Georgetown University, and his fellowship in orthopaedic traumatology at the renowned Campbell Clinic.

1 For the purposes of this article, we intentionally leave details of the care workflows opaque, as it would be irresponsible to equip malicious entities with an attack blueprint on well-established medical workflows that are unlikely to change quickly. Such details can be disclosed under signed agreements – contact authors for more information.

All IoT Agenda network contributors are responsible for the content and accuracy of their posts. Opinions are of the writers and do not necessarily convey the thoughts of IoT Agenda.

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

-ADS BY GOOGLE

SearchCIO

SearchSecurity

SearchNetworking

SearchDataCenter

SearchDataManagement

Close