I am very excited to highlight three recent archival papers from my group on the theme of co-adaptation between human and machine learners.

These papers are a major milestone in the transition my research has taken since finishing my PhD: whereas my work in grad school and first few years as faculty largely concerned modeling and control of legged locomotion, my postdoc and recent years as faculty shifted focus to the human/machine interface (my CAREER proposal represents the bridge between these two stages).

Of course I’m biased, but I regard each of these as landmark studies – I think we’ve really hit on important and timely themes, and I think the experimental work (led by Ben Chasnov and Momona Yamagami) is absolutely top-notch. Below I’ll provide some context and informal interpretations that don’t appear in the manuscripts; if you’re interested in the technical content and relationship to the broader literature, you’ll just have to read the papers :D

Biosignal-based co-adaptive user-machine interfaces for motor control

The first paper to appear is a review that Amy Orsborn was invited to contribute to a special issue of Current Opinion in Biomedical Engineering. The special issue was curated by Helen Huang and Greg Sawicki, and Amy very graciously invited me to pitch in on the paper together with our co-advised student Maneeshika Madduri.

Our paper reviews recent literature on the co-adaptation problem across multiple fields including neuroengineering, motor control, and human-computer interaction. Perhaps unsurprisingly, we propose game theory as a framework for understanding and shaping outcomes when adaptive humans and machines interact. I can assert unreservedly that this is a fantastic paper since I am neither the first nor last author :)

Human adaptation to adaptive machines converges to game-theoretic equilibria

The second paper arose from my longstanding collaboration with Lillian Ratliff and our co-advised student Ben Chasnov. We have worked on learning in games for some time: I entered into this space through a grad school collaboration with Lillian that continued with Ben taking the lead, and we’ve received both collaborative and individual grants that explore the topic in different domains (NSF CPS #1836819, NSF CAREER #1844729,2045014). Of course, Lillian is the authority on the topic.

Particularly in light of recent advances in ML (e.g. RL and LLM), there is a lot of anxiety about the power of “AI agents” to emulate and influence human behavior. There’s some evidence that algorithms can influence elections. But we are not aware of scientific literature that systematically studies the strategic interaction between algorithms and individual people (this review provides a timely summary of the state of the field).

To fill this gap, we conceived the simplest possible instantiation of a continuous game – scalar actions and quadratic costs – and tested different machine learning algorithms against a human “opponent”. I’m using scare quotes to qualify the adversarial terminology, because we actually think the most interesting applications of our algorithms will be in assistive or augmentative capacities, e.g. designing rehabilitative interventions using exoskeletons or providing decision support to people in multi-agent settings.

We find remarkable agreement between interaction outcomes and a constellation of game theory equilibria, establishing a new framework for predicting how individuals respond to algorithms.

Co-adaptation improves performance in a dynamic human-machine interface

The third paper represents the final aim of Momona Yamagami’s thesis. Building on Momona’s wonderful experimental work studying how people learn to control dynamic machines using different modalities and different limbs, we test what happens when an adaptive agent is inserted in-the-loop with the human and machine.

A note on nomenclature: we decided to term the other adaptive agent the “interface”, and the non-adaptive dynamics the “machine”. Although we’ve only run experiments with simulated machines up to this point, in real-world deployment this subsystem would represent the immutable physical plant, i.e. the robot or vehicle or assistive device being controlled. We think our terminology is useful and appropriate since the interface between the human and the (fixed/static) machine is what learning algorithms can modify on-the-fly.

We adopted a linear systems paradigm since (a) ourselves and others (starting with McRuer et al.) have shown how useful it can be for modeling human behavior and (b) it provides a powerful and flexible toolkit for synthesizing a robust/optimal controller (the interface in our case). We used nonminimum-phase machine dynamics from a wonderful study out of Jesse Hoagg’s group because these dynamics are fundamentally hard to control, so an adaptive interface may genuinely help – rather than hinder.

Since humans are thought to use optimization in motor control, we start by modeling the human and interface as optimization-based agents. In the linear systems context, this model naturally leads to the assumption that both agents solve optimal control problems. But we show that there is a fundamental problem with this model, since there appears to be no “equilibrium” solution the interaction could converge to – the essential problem is that controller complexity explodes, which we regard as an instance of an infinite regress that has long been studied in game theory.

Acknowledging instead that both the human and interface have bounded rationality, it seems implausible that this infinite regress would arise in reality. So as a starting point, we finitely-parameterize the interface and conduct experiments to see how humans respond. Since the paper’s title contains spoilers, it should come as no surprise that we find that the adaptive interface confers benefits all around – from the perspective of the human, interface, and whole closed-loop system.