It is obvious that animals outperform robots at running – really, any legged locomotion task involving significant momentum. But what causes this performance gap? Could it be better actuators? sensors? “compute”-ors? The answer to this question is important for determining the most fruitful lines of research for roboticists interested in closing the performance gap. This observation motivated my co-authors and I to write a review paper that definitively answers the question.

Spoiler: it’s not the parts that gives animals the advantage – it must be something about how the parts assemble into the whole.

For historical accuracy, I should point out that the observation initially motivated two of the co-authors to take up the challenge: Tom Libby and Max Donelan. Max was on sabbatical in Berkeley in 2014, so he had time to think big thoughts. Tom was a PhD student and Director of the CiBER center at the time, so he had the motivation to go after weighty problems. These two intellectual heavyweights set themselves on the monumental task of systematically comparing biological and engineering technologies at the level of every individual component but also subsystem and whole-system levels across scales spanning ants and cockroaches to cheetahs and elephants. The original vision was to create a “datasheet” containing a comprehensive comparison of every known metric – plus definitions of new, better metrics and corresponding experiments to characterize them in biomechanical and electromechanical locomotors.

Aaaand .. it went about as well as could be expected.

Which is to say: it went sloooowly. And dauntingly. Overwhelmingly humblingly challengingly. Ego-crushing panic-inducing existentially dreadfully.

That may be overstating the sitch a lil bit. I’m definitely projecting more than a lil, as those were the feels I personally felt once I weaseled my way into the project years later. But you get the idea: it was a big project that required a tremendous amount from its workers.

But back to my weaseling. I was lucky enough to recruit Tom as a postdoc through a (now sadly defunct) Institute for Neuroengineering (called “UWIN” :) in 2017. The “AvM” project (“animals v. machines”) was still in the mix, but so were a half dozen other wonderfullly fascinating projects – both old and new – that were on his plate. In the intervening years, the AvM project scope had been dialed back to “merely” a mega-review, rather than a mega-review-plus-half-dozen-PhD-theses as originally conceived. But it was still too much ground for a pair of researchers to cover.

To put a finer point on it, by this time the scope had been dialed down to focus on 5 subsystems deemed critical for running: power, frame, sensing, actuation, and control. So all that was needed was expertise in5 different Departments: energy systems, material science, sensory neuroscience, kinesiology / biomechanics, and control theory. I genuinely believe Tom has such astonishing breadth that he could have covered all this ground himself. But doing so to the degree of rigor sought by the team would require review of hundreds of papers to convince onesself that you weren’t missing some critical detail that would invalidate the paper’s whole premise.

Observing Tom grapple with all this from the perspective of a postdoc (co-)advisor, I made the very sage and selfless observation that what they were missing was … me! In particular, I felt I could offer two key benefits: I could handle the control subsystem, and I could lower their standards help enforce a reasonable project scope and timeline.

Given that this was in 2019 and you, dear reader, are being regaled with this delightful tale in or after the year 2024, I clearly delivered on no more than half of my promises.

I think my real contribution to the project occured two years of frustrated false starts later when I declared that what we were really missing was … more experts! I had actually made this suggestion many times before. In fact, I’d suggested it to Tom before I joined the project, which is in all likelihood the only reason I have the privilege of writing this today. But – to my recollection – Max resisted bringing even more people in for the longest time. (Probably because he regretted the mistake he and Tom already made with bringing in someone new …)

But after Max became the BPK Chair, he had to acknowledge that something needed to change if we were ever to release this monster into the world. So after a little deliberation we agreed that what was missing was … our friends! We had decided that assigning one expert to each subsystem would be most effective. Between the three of us, Max had power covered, Tom could handily handle actuation, and I could muddle through control. So we were missing frame and sensing. Fortunately for us, our numero uno choices for each subsystem readily agreed to join the project, so we now had Kaushik Jayaram on frame and Simon Sponberg on sensing. An interesting historical note is that we all had a strong connection to the biomechanics group at Berkeley, and in particular with Bob Full, a towering figure in the integrative study of movement: Bob was Max’s sabbatical host, the founder of the center Tom directed, the PhD advisor to Kaushik and Simon, and a cherished mentor and collaborator to me. This project is built on Bob’s shoulders.

With this fresh injection of energy and renewed purpose, we made rapid progress … until we didn’t. Although we’d significantly decreased the workload on each of us individually, the mammoth scope of the endeavor continued to conflict with our many other obligations. It was just too hard to squeeze in thinking such big thoughts and making such sweeping claims among teaching, advising, grantwriting, service, and life.

It’s at this point where the story gets a lot less interesting and therefore quickly wraps up. The project had lain dormant for many months when I received an email notice about a Special Issue on Legged Locomotion in Science Robotics with a deadline 6 weeks out. We’d been targeting SciRob since getting positive feedback on a pre-submission inquiry 5 years prior (lol). And putting these ideas into a Special Issue that the community would be more likely to see was an opportunity we couldn’t afford to miss. I happened to have the good fortune of being on sabbatical at that moment, so I had the time in addition to the motivation to close. So we made it happen.

It’s amazing what a time constraint can do :)

It’s also amazing what a space constraint can do: the original conception was a 10,000 word, 200 cite monolith, but Science Robotics advises a svelte 5,000 words and 75 cites. Not wanting to antagonize the editor or reviewers, we brought our S-tier pithiness to the problem. I regard brevity as my gift, so it was a delightful challenge to boil the ideas down to their bones and serve up only the delicious marrow from with- .. this analogy is getting a little thin and macabre, so let’s move on …

I want to talk a bit informally about the ideas in the paper and give context for some of the decisions and considerations that went into the final product. I’ll work through the sections in order.

When considering System Performance, the original conception was to commit to a specific set of metrics and quantify performance of a suite of robot and animal runners – to create a “datasheet” of sorts that the community could continue to build out over the years. However, there two major problems with this idea: one scientific, and one sociological.

The scientific challenge is that the metrics we have for concepts like range, agility, and robustness are inadequate to capture what seems intuitively clear. One grand idea we tossed around was the conjecture that any metric for these concepts could be computed from the reachable set, that is, the set of states that can be achieved by a control system through an admissible input signal in a given distribution of environments and a given parameterization of designs. We ultimately abandoned mentioning this idea because, although potentially interesting, there’s no currently practical way to compute this set (and Bellman tells us there can’t be in general).

The sociological challenge is that we did not want to dunk on our colleagues, or get into endless debates about why we chose the specific metric we did and why their robot did so poorly with respect to it. We figured that no reasonable person would challenge the assertion that animals outperform robots in their range, agility, and robustness (however you define these terms) – what would surely be controversial is how existing robots stack up relative to one another. So we opted for the qualitative / coarsely-quantized comparison in the first Figure.

Regarding the central conclusion, that the difference in performance of parts does not explain the difference in performance of wholes, there are some caveats.

If you were building a cyborg to run as far as possible completely power autonomous, using metabolism would give an order-of-magnitude advantage in range over gas power (nearly two orders-of-magnitude w.r.t. batteries). So along that solitary dimension, defined in that specific way, the difference in the part does explain the difference in the whole. But as soon as you allow that there may be gas stations or electrical outlets along the way, this advantage disappears.

The biological distribution of sensors throughout a body is quite compelling from an agility and robustness perspective: richly sensing terrain or other interactions with the environment could be a real boon for those dimensions of performance. But the “simulated cyborg” thought experiment from the Discussion convinces us that, even in the presence of perfect state information about the locomotor and environment, we still lack the tools to integrate that information to make a high-performing runner.

Finally, there are a couple of points to make about biological and engineered controllers. To make the most apples-to-apples comparison, we looked at natural and artificial spiking neural networks. Of course robot controllers can be implemented using conventional von Neumann architectures. But there are no proof-of-concept high-performing controllers in that paradigm to compare to those in animals, and the comparison is difficult to make at a component level: although we can pack upwards of hundreds of billions of transistors into a chip (comparable to the number of neurons in the human brain), it seems clear that a single transistor has less computational power than an individual neuron, and we are not aware of any rigorous attempt to quantify their relative computational power. Even the comparison between natural and artificial spiking neural networks is probably unfair in the sense that ANN dynamics are vastly simpler (e.g. piecewise-linear) than their biological counterparts (NNN?). But it’s the best comparison we can make at present, and including these factors would only tip the outcome even further in biology’s favor.

HOWever, even allowing that brains can, in principle, implement vastly more complex transformations than chips (at any scale – cockroaches have more neurons and synapses than the biggest neural ICs), it is important to remember that the brain is doing a whole lot more than locomotion. I keep returning to the example we cite in the paper (citation 90) of a parasitic wasp that lyses more than 7000 of its approximately 7400 neurons during pupation. The upshot is that there are autonomous flyers that can identify and infect hosts using fewer than 400 neurons !!! If you gave me 400 neurons, I think I’d struggle to invert a pendulum ..

My takeaway from this example is that we could be doing a lot more (robust and agile behavior) with a lot less (computational power) if only (a) we had the right bodies and (b) we knew what to do with them.

The Discussion covers a lot of ground that doesn’t need to be retrod here. But there is one point I want to dwell on a bit more, because I personally find it very interesting and compelling: the need for better metrics. This problem came up a few paragraphs ago when I discussed the challenge in defining what we mean by “agility” and “robustness”. One way to view the results of our paper is that we are focusing on the wrong metrics when we evaluate performance at the subsystem level, as these are evidently not predictive of system performance. What’s needed are metrics for the integration of multiple components or subsystems – and these metrics must capture something about the whole-system behavior we seek. The reason good metrics could be so powerful is that the endeavor of engineering is driven by “specs”, i.e. performance criteria. Once you tell me how my artifact is going to be evaluated, I can bring the powerful machinery of prototyping, optimization, learning, et al. to bear on squeezing that metric for everything it’s got. In the absence of metrics, engineering becomes art.

As a final note for the history books, I want to acknowledge where this paper fits in my intellectual and academic trajectory. I got my start in research in the summer before my first year of undergrad working with Eric Klavins, who began his career in robotics before switching to synbio. In fact, Eric got his PhD with a luminary in legged robotics, Dan Koditschek, and it was through this connection (certainly not merit) that I had the tremendous good fortune to do an REU at UPenn the summer after my sophomore year. The REU was my first exposure to the interdisciplinary world of legged locomotion, and I was completely enraptured. (Actually, for historical accuracy, I have to acknowledge that my very first exposure to this world was as a high school student when I was part of the inaugural cohort of students at the Summer Institute for Mathematics at the University of Washington, where the inimitable Tom Daniel gave an afternoon lecture on biolocomotion that included a very memorable demo on passive dynamic walking. So I suppose I was primed to become enraptured.)

Biolocomotion was the driver behind my applications to grad school and fellowships, and legged locomotion in particular ended up as the focus of my PhD thesis. My postdoc took me in a completely new direction – human-in-the-loop control – so when I started my faculty position there were two main areas of focus. Over time, legged locomotion has shrunk from the dominant theme at the beginning to now, where I have only one PhD student in this area, and they will graduate in six months. So this review represents the closure of a major chapter in my career – a very satisfying closure to be sure, but bittersweet nonetheless.

With that, I’ll stop – this commentary has already run almost half as many words as you’ll find in the paper :)