HMT: A Deeper Dive

Human-Machine Teaming

The HMT team is focused on optimizing human-machine interaction by developing technology and systems that learn user task patterns, assess human cognitive state, and adapt the system to user needs. Our goal is to create unique human-machine interactions that enhance human performance.

Adaptive Training

Our adaptive training systems support students learning complex psychomotor tasks. We use machine learning for unsupervised pattern recognition in high-dimensional streaming data. Our training systems: (1) infer users’ tasks based on observable actions; (2) learn to classify successful and unsuccessful patterns; (3) identify opportunities to improve user proficiency; (4) plan tailored, scaffolded raining to improve proficiency; and (5) provide objective performance metrics to users and supervisor(s).

Projects include:

  • CAMBIO: Cognitive Adaptation of Management Behavior of Information via Observation
  • MALT: Multi-task Adaptive Learning Tutor
  • SEATRAIN: System for Embedded ASW Training Using Artificial Intelligence

Proactive Decision Support

Our proactive decision support tools assist users in planning and executing complex tasks. We use natural language processing, reinforcement learning, graph convolutional neural networks, genetic algorithms, and optimization. Applications include rapid and optimal asset allocation to achieve tactical goals, detecting and reacting to adversarial strategies, and achieving and maintaining situation awareness.

Projects include:

  • InfoCog: Info-cognitive Proactive Decision Support
  • MARSHAL:Mission Asset Recommendation System with Historical Ability Learning

Workflow Discovery and Management

Our user workflow support tools improve information gathering, workflow, and task prioritization. We use natural language processing (NLP), machine learning, pattern learning, and pattern matching, to detect workflows, user preferences, and user actions.

Examples include:

  • A novel document search capability that accounts for user roles and responsibilities in the search algorithm. This enables tailoring search results to a user’s functional needs, thereby reducing the time required to locate and understand the most relevant information.
  • Automatic discovery of user workflows and priorities allows us to: detect what tasks a user is executing and where they are in the workflow, preposition data and analytics to support a given task, alert a user to higher priority tasks.
  • Tools for assessing cognitive state (e.g., stress), combined with task priorities, allow our systems to throttle the amount of information provided to a user.

Projects include:

  • STARTER: Semantic Targeting using Analyst Role, Topics and Entity Recognition
  • CAMBIO: Cognitive Adaptation of Management Behavior of Information via Observation