Assessing Reliable Metrics and Measures of Effectiveness for Insider Threat Detection and Mitigation

  (Symposium II)


Symposium Leaders

COL Ronald C. Dodge JR  (primary contact)

United States Military Academy at West Point

606 Thayer Road, Room 105, West Point, NY 10996, USA

Phone: +1-845-938-5569 (direct)

Phone: +1-845.938.3615 (organization)


Dawn M. Cappelli


Aaron J. Ferguson, Ph.D.


Go to  HICSS Conference Site

This full-day symposium will focus on research in the area of Metrics & Measures of Effectiveness (MMoEs) for insider threat detection and mitigation. According to Green (2001), an Insider Threat Detection Metric/Measure of Effectiveness (MMOE) measures how a best practice, process, and/or technology performs its functions within its environment. It is generally an aggregation of Measures of Performance (MOPs), e.g., probability of detection, false positive rate, attribution granularity. MMoEs have nine critical attributes. Insider Threat Detection Metric/Measures of Effectiveness (MMOE) are:

  1. 1.Mission Oriented – assumes a malicious insider’s mission is Theft of IP, IT Sabotage, or Fraud.

  2. 2.Discriminatory – can reliably identify real differences between a malicious insider and a non-malicious insider.

  3. 3.Measurable - Can be computed or estimated. You cannot measure intent but you can measure actions and attempt to impute intent.

  4. 4.Quantitative - Can be assigned numbers or ranked.

  5. 5.Objective - Defined or derived, independent of subjective opinion

  6. 6.Appropriate - Relates to acceptable standards and analysis objectives.

  7. 7.Sensitive - Reflects changes in system variables.

  8. 8.Independent- Mutually exclusive with respect to other measures.

  9. 9.Simple - Easily understood by the user.

According to Greitzer and Frincke (2010), the two most significant challenges in developing a predictive analytic methodology for insider threat detection are: (1) defining precursors--events prior to the attack)--in terms of observable cyber and psychosocial indicators; and (2) developing a methodology that integrates these indicators. What’s even harder is identifying and defining metrics and measures of effectiveness that determine: (a) the probability of success of these precursors predicting an insider attack; or (b) whether the methodology used is reliable across different organizational domains. Researchers from government, industry, and academia have proposed promising technical solutions for detecting insider threat activity; often based on violation of some policy, e.g., unauthorized use of removable media. However, those tasked with paying for these solutions want some insight into a solution/approach’s Return on Investment (ROI), and to date, this area of research has been thin. Identifying insider threat MMoEs is critical as organizations seek to prevent fraud, theft of intellectual property, sabotage, and espionage. Novel research involving new methods or insight into developing MMoEs for insider threat detection and defense is appropriate for this symposium.

Ron C. Dodge is an active duty Colonel in the Army and is an Associate Professor in the Electrical Engineering and Computer Science department at the United States Military Academy.  He is the Associate Dean for Information and Education Technology and teaches operating systems and security courses.  Ron’s current research focuses are information warfare, virtualization, security protocols, and performance planning and capacity management.  He is a frequent speaker at national and international IA conferences.


“We are beginning to more deeply understand how insider threats can be detected at an early enough stage.”

Organizers: Ron C. Dodge (lead), Dawn M. Cappelli, & Aaron J. Ferguson

This is the 3rd full-day symposium on Insider Threat at HICSS. The symposium will discuss current methods and tools available for insider threat detection and mitigation and conduct tabletop exercises designed to explore the complex insider problem

Data loss and services disruption has emerged as a significant concern to government and industry.  Government and commercial organizations have been bombarded with products and services promising to “find the next PVT Manning” and “find the next malicious insider.” However, MMoEs for these “solutions” have not been clearly and consistently defined.   The first half of the day will be a series of selected papers on MMoEs for insider threat detection and mitigation and the second half will consist of panel discussions. This symposium will solicit participation from practitioners, students, educators, and researchers.

COL. Ronald C. Dodge

The HICSS-47  version of the symposium had to be suspended due to the lack of funding for its contributors