Karen Leung

selected publications

1. 

   Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approach

   Ivanovic, B.*, Leung, K.*, Schmerling, E., and Pavone, M.

   IEEE Robotics and Automation Letters, 2021

   Abstract PDF Bib

   Human behavior prediction models enable robots to anticipate how humans may react to their actions, and hence are instrumental to devising safe and proactive robot planning algorithms. However, modeling complex interaction dynamics and capturing the possibility of many possible outcomes in such interactive settings is very challenging, which has recently prompted the study of several different approaches. In this work, we provide a self-contained tutorial on a conditional variational autoencoder (CVAE) approach to human behavior prediction which, at its core, can produce a multimodal probability distribution over future human trajectories conditioned on past interactions and candidate robot future actions. Specifically, the goals of this tutorial paper are to review and build a taxonomy of state-of-the-art methods in human behavior prediction, from physics-based to purely data-driven methods, provide a rigorous yet easily accessible description of a data-driven, CVAE-based approach, highlight important design characteristics that make this an attractive model to use in the context of model-based planning for human-robot interactions, and provide important design considerations when using this class of models.

   @article{IvanovicLeungEtAl2020,
     bibtex_show = {true},
     author = {Ivanovic, B.* and Leung, K.* and Schmerling, E. and Pavone, M.},
     title = {Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approach},
     journal = {{IEEE Robotics and Automation Letters}},
     volume = {6},
     number = {2},
     pages = {295--302},
     year = {2021},
     note = {In Press},
     arxiv = {2008.03880},
     keywords = {Journal},
     img = {IvanovicLeungEtAl2020.jpg},
     selected = {true},
     category = {interaction}
   }

2. 

   On Infusing Reachability-Based Safety Assurance within Planning Frameworks for Human-Robot Vehicle Interactions

   Leung, K., Schmerling, E., Zhang, M., Chen, M., Talbot, J., Gerdes, J. C., and Pavone, M.

   Int. Journal of Robotics Research, 2020

   Abstract PDF Bib

   Action anticipation, intent prediction, and proactive behavior are all desirable characteristics for autonomous driving policies in interactive scenarios. Paramount, however, is ensuring safety on the road—a key challenge in doing so is accounting for uncertainty in human driver actions without unduly impacting planner performance. This paper introduces a minimally-interventional safety controller operating within an autonomous vehicle control stack with the role of ensuring collision-free interaction with an externally controlled (e.g., human-driven) counterpart while respecting static obstacles such as a road boundary wall. We leverage reachability analysis to construct a real-time (100Hz) controller that serves the dual role of (1) tracking an input trajectory from a higher-level planning algorithm using model predictive control, and (2) assuring safety through maintaining the availability of a collision-free escape maneuver as a persistent constraint regardless of whatever future actions the other car takes. A full-scale steer-by-wire platform is used to conduct traffic weaving experiments wherein two cars, initially side-by-side, must swap lanes in a limited amount of time and distance, emulating cars merging onto/off of a highway. We demonstrate that, with our control stack, the autonomous vehicle is able to avoid collision even when the other car defies the planner’s expectations and takes dangerous actions, either carelessly or with the intent to collide, and otherwise deviates minimally from the planned trajectory to the extent required to maintain safety.

   @article{LeungSchmerlingEtAl2020,
     bibtex_show = {true},
     author = {Leung, K. and Schmerling, E. and Zhang, M. and Chen, M. and Talbot, J. and Gerdes, J. C. and Pavone, M.},
     title = {On Infusing Reachability-Based Safety Assurance within Planning Frameworks for Human-Robot Vehicle Interactions},
     journal = {{Int.\ Journal of Robotics Research}},
     year = {2020},
     volume = {39},
     issue = {10--11},
     pages = {1326--1345},
     arxiv = {2012.03390},
     keywords = {Journal},
     img = {LeungSchmerlingEtAl2020.png},
     selected = {true},
     category = {interaction}
   }

3. 

   Backpropagation through signal temporal logic specifications: Infusing logical structure into gradient-based methods

   Leung, K., Aréchiga, N., and Pavone, M.

   Int. Journal of Robotics Research, 2022 (In press)

   Abstract PDF Bib

   This paper presents a technique, named STLCG, to compute the quantitative semantics of Signal Temporal Logic (STL) formulas using computation graphs. STLCG provides a platform which enables the incorporation of logical specifications into robotics problems that benefit from gradient-based solutions. Specifically, STL is a powerful and expressive formal language that can specify spatial and temporal properties of signals generated by both continuous and hybrid systems. The quantitative semantics of STL provide a robustness metric, i.e., how much a signal satisfies or violates an STL specification. In this work, we devise a systematic methodology for translating STL robustness formulas into computation graphs. With this representation, and by leveraging off-the-shelf automatic differentiation tools, we are able to efficiently backpropagate through STL robustness formulas and hence enable a natural and easy-to-use integration of STL specifications with many gradient-based approaches used in robotics. Through a number of examples stemming from various robotics applications, we demonstrate that STLCG is versatile, computationally efficient, and capable of incorporating human-domain knowledge into the problem formulation.

   @article{LeungArechigaEtAl2021,
     bibtex_show = {true},
     author = {Leung, K. and Ar\'{e}chiga, N. and Pavone, M.},
     title = {Backpropagation through signal temporal logic specifications: Infusing logical structure into gradient-based methods},
     year = {2022},
     journal = {{Int.\ Journal of Robotics Research}},
     keywords = {In press},
     arxiv = {2008.00097},
     selected = {true},
     img = {LeungArechigaEtAl2021.png},
     category = {logic}
   }

4. 

   Towards the Unification and Data-Driven Synthesis of Autonomous Vehicle Safety Concepts

   Bajcsy, A.*, Leung, K.*, Schmerling, E., and Pavone, M.

   Abstract PDF Bib

   As safety-critical autonomous vehicles (AVs) will soon become pervasive in our society, a number of safety concepts for trusted AV deployment have been recently proposed throughout industry and academia. Yet, agreeing upon an "appropriate" safety concept is still an elusive task. In this paper, we advocate for the use of Hamilton Jacobi (HJ) reachability as a unifying mathematical framework for comparing existing safety concepts, and propose ways to expand its modeling premises in a data-driven fashion. Specifically, we show that (i) existing predominant safety concepts can be embedded in the HJ reachability framework, thereby enabling a common language for comparing and contrasting modeling assumptions, and (ii) HJ reachability can serve as an inductive bias to effectively reason, in a data-driven context, about two critical, yet often overlooked aspects of safety: responsibility and context-dependency.

   @inproceedings{BajcsyLeungEtAl2021,
     bibtex_show = {true},
     author = {Bajcsy, A.* and Leung, K.* and Schmerling, E. and Pavone, M.},
     title = {Towards the Unification and Data-Driven Synthesis of Autonomous Vehicle Safety Concepts},
     year = {preprint},
     img = {BajcsyLeungEtAl2021.png},
     selected = {true},
     arxiv = {2107.14412},
     category = {safety}
   }