These are my academic publications.

For the bean counters, there are 176 of them. You can see who's citing them on Google Scholar, Semantic Scholar, or ACM. Note that 7 are still under my deadname, and thousands cite me by my deadname. Publishers refuse to fix these citations, so please cite me Amy J. Ko, regardless of how you might find my prior work in reference lists and scholarly search engines. And if you're reviewing a paper that cites me incorrectly, please demand they fix it.

Filter by publication topic and awards.

AI educationUI miningassessmentdebuggingdesigndiversityeditorserrorsexpertisefamiliesfeedbackhelpinterestissue trackingjusticelearningmethodspedagogyproblem solvingprogram readingprogrammingprogramming systemsstudiesteamstoolstutorialsverification

best paperdiversity + inclusion awardhonorable mentionmost influential papermost influential paper honorable mention

2024– Informatics program chair; Associate Dean for Academics 🔗

Mara Kirdani-Ryan, Amy J. Ko (2024)
Neurodivergent Legitimacy in Computing Spaces
ACM Transactions on Computing Education
Finds that computing learning spaces legitimize hyper-focus, deep ``special'' interests, and high organization, and that fitting these expectations was frequently required for persistence.

▸ cite ⋅ doi ⋅ 🔗

Chris Perdriau, Meron Solomon, Amy J. Ko (2024)
Recruiting Practices in Informal CS Learning
ACM Transactions on Computing Education
Discovers 18 different recruting practices, most with low levels of cultural compentence, resulting in inequities in participation.

▸ cite ⋅ doi ⋅ 🔗

Anael Kuperwajs Cohen, Alannah Oleson, Amy J. Ko (2024)
Factors Influencing the Social Help-Seeking Behavior of Introductory Programming Students in a Competitive University Environment
ACM Transactions on Computing Education
Communication style, type of question being asked, cheating policy, competition, and feelings intimidation shape who students ask for help and whether they do at all.

▸ cite ⋅ doi ⋅ 🔗

★ honorable mention
Rotem Landesman, Jean Salac, Jared Lim, Amy J. Ko (2024)
Integrating Philosophy Teaching Perspectives to Foster Adolescents' Ethical Sensemaking of Computing Technologies
ACM International Computing Education Research Conference (ICER)
Students expressed their ethical sensemaking by considering multiple perspectives, questioning the status quo, wrestling with dissonance between their principles and actions, and rejecting the good/bad binary.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Eman Sherif, Jayne Everson, Megumi Kivuva, Mara Kirdani-Ryan, Amy J. Ko (2024)
Exploring the Impact of Assessment Policies on Marginalized Students' Experiences in Post-Secondary Programming Courses
ACM International Computing Education Research Conference (ICER)
Reveals ten distinct ways policy and students' lives interacted to create or heighten inequities, which significantly shaped marginalized students' lives.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper
Jayne Everson, Megumi Kivuva, Camilo Montes de Haro, Amy J. Ko (2024)
Culture-centric computational embroidery
ACM Technical Symposium on Computer Science Education (SIGCSE), Experience Report Track
Students enthusiastically engaged with the programming aspects of the course and sought to make complicated and beautiful work that interwove their diverse cultures and identities.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Hongwen Guo, Mo Zhang, Amy J. Ko, Min Li, Benjamin Zhou, Jared Lim, Paul Pham, Chen Li (2024)
Measuring students' programming skills via online practice
Computer Science Educational Data Mining Workshop
Examines fairness concerns and tradeoffs in scoring students' solutions to programming problems.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Alannah Oleson, Amy J. Ko, Richard Ladner (2024)
Teaching Accessible Computing
Online
A book for computing educators who want to integrate accessible computing concepts and skils into computer science courses.

▸ cite ⋅ doi ⋅ 🔗

Jean Salac, Lena Armstrong, Megumi Kivuva, Jayne Everson, Amy J. Ko (2024)
How Economically-Marginalized Adolescents of Color Negotiate Critical Pedagogy in a Computing Classroom
ACM Transactions on Computing Education
Finds tensions between participants' computing attitudes, knowledge, self-efficacy, and social consciousness, suggesting pathways for scaffolding the critical examination of technology in secondary education

▸ cite ⋅ 🔗

Rotem Landesman, Jean Salac, Jared Lim, Amy J. Ko (2024)
"There Will Always be a Yes and No Side": Facilitating Ethical Sensemaking Around Technology with Teens
ISLS 2024
Our preliminary analyses identified launchpads for ethical sensemaking, instances where students leveraged their lived experiences, community discussion, and ethical scaffolding to begin reasoning about moral dilemmas, and (2) expressions of ethical sensemaking, signals within students’ reasoning processes that indicated critical sensemaking was taking place

▸ cite ⋅ 🔗

2023– Sabbatical (Winter/Spring/Summer) 🔗

Jean Salac, Rotem Landesman, Stefania Druga, Amy J. Ko (2023)
Scaffolding children's sensemaking around algorithmic fairness
ACM Interaction Design for Children
Finds that youth use a tiered human lens and then technological lens to reason through fairness, often invoking identity.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Benjamin Xie, Jared Lim, Paul Pham, Min Li, Amy J. Ko (2023)
Developing novice programmers' self-regulation skills with code replays
ACM International Computing Education Research Conference (ICER)
After watching code replays, learners more frequently interpreted problem prompts and planned their solutions, two crucial self-regulation behaviors that novices often overlook.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Lena Armstrong, Jayne Everson, Amy J. Ko (2023)
Navigating a blackbox: Students' experiences and perceptions of automated hiring
ACM International Computing Education Research Conference (ICER)
Finds that while manys students deemed automation a 'necessary evil' to combat scale, many struggled with the inequity automated hiring processes perpetuated

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper
Jean Salac, Alannah Oleson, Lena Armstrong, Audrey Le Meur, Amy J. Ko (2023)
Funds of knowledge used by adolescents of color in scaffolded sensemaking around algorithmic fairness
ACM International Computing Education Research Conference (ICER)
Finds that students use their cultural knowledge and experiences to situate their interpretation of three algorithms and how this can lead to teaching about algorithmic fairness (including challenging the notion of 'average use' in design

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Anne Beitlers, Jayne Everson, Brett Wortzman, Dan Gallagher (2023)
Proposing, preparing, and teaching an equity- and justice-centered secondary pre-service CS teacher education program
ACM Technical Symposium on Computer Science Education (SIGCSE), Experience Report Track
Describes the administrative, fundraising, and pedagogical backstory behind the CS creation of a pre-service program.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper
Jenny Liang, Maryam Arab, Minhyuk Ko, Amy J. Ko, Thomas D. LaToza (2023)
A qualitative study on the implementation design decisions of developers
ACM/IEEE International Conference on Software Engineering (ICSE)
Implementation design decisions, rather than being a natural outcome from higher levels of design, require constant monitoring and reconsideration of higher level design choices, such as requirements and architecture.

▸ cite ⋅ doi ⋅ 🔗

Mara Kirdani-Ryan, Amy J. Ko, Emilia A. Borisova (2023)
"Taught to be automata": Examining the departmental role inshaping initial career choices of computing students
Computer Science Education
Finds that unspoken departmental capitalist norms and values strongly shape not only student career plans, but advising practices, teaching, and student identity.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Leah Perlmutter, Jean Salac, Amy J. Ko (2023)
"A field where you will be accepted": Belonging in student and TA interactions in post-secondary CS education
ACM International Computing Education Research Conference (ICER)
Finds student and TA conceptions and narratives of belonging aligned with the three basic needs for wellness as described in Self-Determination Theory: relatedness, competence, and autonomy.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Megumi Kivuva, Keith O'Hara, Amy J. Ko (2023)
Exploring identity through computing integration in a Spanish language and literature class
IEEE Conference on Research in Equity and Sustained Participation in Engineering, Computing, and Technology (RESPECT)
Offers promising ideas for how to integrate computing concepts into language learning and language arts classes.

▸ cite ⋅ pdf ⋅ 🔗

Jayne Everson, Anne Beitlers, Amy J. Ko (2023)
What do you really mean by rigor? Deconstructing the definition in CS Teaching
American Education Research Association
Finds widely varying and inconsistent definitions of rigor amongst CS educators and students across primary, secondary, and post-secondary settings.

▸ cite ⋅ 🔗

2022– Informatics program chair; gender transition; sabbatical (Summer/Fall) 🔗

Alannah Oleson, Meron Solomon, Chris Perdriau, Amy J. Ko (2022)
Teaching inclusive design skills with the CIDER assumption elicitation technique
ACM Transactions on Computer-Human Interaction
Contributes a teaching method for helping students identify design assumptions that exclude.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Stefania Druga, Nancy Otero, Amy J. Ko (2022)
The Landscape of Teaching Resources for AI Education
ACM Conference on Innovation and Technology in Computer Science Education
Finds that most AI education resources do not consider teacher needs and do not align with emerging AI literacy standards.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Stefania Druga, Tom Ball, Amy J. Ko (2022)
How families design and program games: a qualitative analysis of a 4-week online in-home study with a cellular-automata programming platform
ACM Interaction Design for Children
Finds that inter-generational collaboration patterns and use of resources are distinct from individual and pair programming.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Alannah Oleson + Benjamin Xie, Jean Salac, Jayne Everson, Megumi Kivuva, Amy J. Ko (2022)
A decade of demographics in computing education research: A critical review of trends in collection, reporting, and use
ACM International Computing Education Research Conference (ICER)
Finds that most computing education research studied adults, did not mention how demographics data was gathered, and used imprecise and sometimes hegemonic aggregate terms about diversity.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Jayne Everson, Amy J. Ko (2022)
“I would be afraid to be a bad CS teacher”: Factors Influencing Participation in Pre-Service Secondary CS Teacher Education
ACM International Computing Education Research Conference (ICER)
Finds that teacher candidates displayed many conceptions of justice that motivated them to teach CS, including repairing past wrongs in education, improving representation, and expanding literacy, but fears of added course preparation and opportunity costs were signficant deterrents.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Maryam Arab, Thomas D. LaToza, Jenny Liang, Amy J. Ko (2022)
An Exploratory Study of Sharing Strategic Programming Knowledge
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Finds that developers can share strategies for programming, but they struggle in choosing a level of detail and understanding the diversity of the potential audience, requiring substantial feedback, but also that developers struggle to give feedback on strategies.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Stefania Druga, Fee Christoph, Amy J. Ko (2022)
Family as a Third Space for AI Literacies: How Do Children and Parents Learn about AI Together?
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Finds that parents mediate children's learning by taking on different roles, including mentor, student, collaborator, and tinkerer, and that family learning promotes new framings of consumer devices in homes.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper
Jayne Everson, Megumi Kivuva, Amy J. Ko (2022)
“A key to reducing inequities in like, AI, is by reducing inequities everywhere first”: Emerging Critical Consciousness in a Co-Constructed Secondary CS Classroom
ACM Technical Symposium on Computer Science Education (SIGCSE), Research Track
Finds that before there is space for critical consciousness about CS, teachers have to navigate issues of trust, positionality, and inequitable systems of education.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Mara Kirdani-Ryan, Amy J. Ko (2022)
The House of Computing: Integrating Counternarratives into Computer Systems Education
ACM Technical Symposium on Computer Science Education (SIGCSE), Experience Report Track
Teaching computer systems through the lens of sociopolitical and historical choices can open space for student reexamination of technical knowledge and values.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ diversity + inclusion award
Benjamin Xie, Alannah Oleson, Jayne Everson, Amy J. Ko (2022)
Surfacing Equity Issues in Large Computing Courses with Peer-Ranked, Demographically-Labeled Student Feedback
Proceedings of the ACM on Human-Computer Interaction
Developed and evaluated tool that contextualizes student feedback for teaching teams to identify equity issues.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2021– Informatics program chair; COVID-19; gender transition 🔗

Stefania Druga, Amy J. Ko (2021)
How Do Children’s Perceptions of Machine Intelligence Change when Training & Coding Smart Programs?
ACM Interaction Design for Children
Discovers that children, after building and training small machine learned programs, come to view smart devices as less intelligent, shifting agency from the device to developers.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Kyle Thayer, Sarah Chasins, Amy J. Ko (2021)
A Theory of Robust API Knowledge
ACM Transactions on Computing Education
Contributes a novel theory of robust API knowledge, including knowledge about the domain that an API models, knowledge of the semantics of API functionality, and knowledge of API usage patterns.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Benjamin Xie, Matt Davidson, Baker Franke, Emily McLeod, Min Li, Amy J. Ko (2021)
Domain Experts’ Interpretations of Assessment Bias in a Scaled, Online Computer Science Curriculum
ACM Learning at Scale
Finds systemtic test gender and bias in Code.org's assessments, but barriers to interpreting psychometric results.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper
Chenglong Wang, Yu Feng, Rastislav Bodik, Isil Dillig, Alvin Cheung, Amy J. Ko (2021)
Falx: Synthesis-Powered Visualization Authoring
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Contributes a synthesizer-powered visualization tool that allows users to specify examples of how concrete data values should be rendered, and receive one or more data visualizations that transform data accordingly. Finds that users can effectively adopt Falx to create visualizations.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Matt Davidson, Amy J. Ko, Brett Wortzman (2021)
Investigating Item Bias in a CS1 exam with Differential Item Functioning
ACM Technical Symposium on Computer Science Education (SIGCSE), Research Track
Demonstrates how to conduct a DIF analysis through a case study of a particular large CS1 exam, which revealed some item bias.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Maryam Arab, Jenny Liang, Yang Kyu Yoo, Amy J. Ko, Thomas D. LaToza (2021)
HowToo: A Platform for Sharing, Finding, and Using Programming Strategies
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
Contributes HowToo, a system for encoding strategic programming knowledge as step-by-step procedures for structuring programming tasks. Finds that novice developers found the platform helpful for guiding their work, except when they were rushing toward deadlines.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Qiang Hao, David Smith, Lu Ding, Amy J. Ko, Camille Ottaway, Jack Wilson, Kai Hicks, Alistair Turcan, Timothy Poehlman, Tyler Greer (2021)
Towards Understanding the Effective Design of Automated Formative Feedback for Programming Assignments
Computer Science Education
Finds that feedback addressing the gap between expected and actual outputs is critical to effective learning and that feedback lacking sufficient detail on these gaps leads to gaming behaviors.

▸ cite ⋅ doi ⋅ 🔗

Amy J. Ko, Anne Beitlers, Brett Wortzman, Matt Davidson, Alannah Oleson, Mara Kirdani-Ryan, Stefania Druga, Jayne Everson (2021)
Critically Conscious Computing: Methods for Secondary Education
Online
A book for secondary educators who want to teach CS from critical lens, examining it from technical, sociotechnical, and sociopoitical stance.

▸ cite ⋅ doi ⋅ 🔗

2020– Informatics program chair; promoted to Professor; COVID-19; gender transition 🔗

Amy J. Ko (2020)
Foundations of Information
Online
This book covers information, and all of the things that intersect with it: power, politics, oppression, data, knowledge, encoding, metadata; information technology, management, and policy; and the many domains in which information is crucial, including science, social media, automation, health, democracy, and sustainabiity. It serves as the foundation of our survey course on information, INFO 200 Intellectual Foundations of Informatics.

▸ cite ⋅ doi ⋅ 🔗

Alannah Oleson, Amy J. Ko, Brett Wortzman (2020)
On the Role of Design in K-12 Computing Education
ACM Transactions on Computing Education
Disentangles the role of design in K-12 CS education, finding that design ideas are pervasive in curricula and standards, but conflict program space and problem space design, masking the distinct challenges of problem space design.

▸ cite ⋅ doi ⋅ 🔗

Amy J. Ko, Alannah Oleson, Mara Kirdani-Ryan, Yim Register, Benjamin Xie, Mina Tari, Matt Davidson, Stefania Druga, Dastyni Loksa, Greg Nelson (2020)
It’s Time for More Critical CS Education
Communications of the ACM (CACM)
Argues that CS educators at all levels have the responsibility of the role of computing in injustice.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Benjamin Xie, Greg Nelson, Harshitha Akkaraju, William Kwok, Amy J. Ko (2020)
The Effect of Informing Agency in Self-Directed Online Learning Environments
ACM Learning at Scale (L@S)
Finds that affording more agency in learning environments may increase motivation, but may not improve learning because of the increased burden on decision-making.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Yim Register, Amy J. Ko (2020)
Learning Machine Learning with Personal Data Helps Stakeholders Ground Advocacy Arguments in Model Mechanics
ACM International Computing Education Research Conference (ICER)
Finds that teaching machine learning by tracing an algorithm with learners' personal data increases the extent to which they can identify bias in self-advocacy arguments.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Dastyni Loksa, Benjamin Xie, Harrison Kwik, Amy J. Ko (2020)
Investigating Novices' In Situ Reflections on Their Programming Process
ACM Technical Symposium on Computer Science Education (SIGCSE), Research Track
Presents evidence that self-regulation during programming is highly variable in practice, and that teaching self-regulation skills to improve programming outcomes may require differentiated instruction based on students self-awareness and existing programming practices.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amanda Swearngin, Chenglong Wang, Alannah Oleson, James Fogarty, Amy J. Ko (2020)
Scout: Rapid Exploration of Interface Layout Alternatives through High-Level Design Constraints
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Presents a system to support designers ideation of user interface layouts, allowing designers to provide high-level preferences, obtain a set of layouts that meet those preferences, provide feedback on those preferences, and iteratively converge towards preferred layouts. An evaluation showed that designers who used Scout accelerated ideation and found more diverse designs that designers without it.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Alannah Oleson, Meron Solomon, Amy J. Ko (2020)
Computing Students' Learning Difficulties in HCI Education
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Investigates the challenges that computing students face in learning HCI, finding challenges in understanding design as distinct from engineering, understanding design methods, finding resources to support design, interpreting feedback, scoping design problems, choosing between alternatives, and designing for diversity.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Thomas D. LaToza, Maryam Arab, Dastyni Loksa, Amy J. Ko (2020)
Explicit Programming Strategies
Empirical Software Engineering
Presents the concept of explicit programming strategies, the idea of encoding manual procedures for solving programming programmers to accelerate software development problem solving. Presents a notation for encoding strategies and evidence that developers using explicit strategies, independent of expertise, were more successful at the design and debugging tasks.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Alannah Oleson, Amy J. Ko (2020)
Toward the Development of HCI Pedagogical Content Knowledge
ACM SIGCHI Symposium on HCI Education (EduCHI 2020)
Discusses the need for research on HCI educator's pedagogical content knowledge.

▸ cite ⋅ pdf ⋅ 🔗

2019– Informatics program chair; gender transition 🔗

Brad A Myers, Amy J. Ko, Thomas D. LaToza, YoungSeok Yoon (2019)
Human-Centered Methods to Boost Productivity
Sadowski C., Zimmermann T. (eds) Rethinking Productivity in Software Engineering. Apress, Berkeley, CA
Describes several methods for understanding how to increase developer productivity.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2019)
Individual, Team, Organization, and Market: Four Lenses of Productivity
Sadowski C., Zimmermann T. (Eds) Rethinking Productivity in Software Engineering. Apress, Berkeley, CA
Discusses several levels of abstraction useful for reasoning about software engineering productivity.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2019)
Why Not to Measure Productivity
Sadowski C., Zimmermann T. (eds) Rethinking Productivity in Software Engineering. Apress, Berkeley, CA
Presents several arguments for the dangers of measuring software developer productivity.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2019)
Informatics Teaching Guide
Online
Supports new higher education teachers, including new faculty, guest faculty, doctoral student teachers, and teaching assistants. It uses our undergraduate Informatics program for examples, but is in essence a practical, but also research-informed introduction to teaching in a university.

▸ cite ⋅ doi ⋅ 🔗

Greg Nelson, Anne Drew Hu, Benjamin Xie, Amy J. Ko (2019)
Towards Validity for a Formative Assessment for Language-Specific Program Tracing Skills
ACM Koli Calling International Conference on Computing Education Research
Contributes theoretical perspectives on how to assess program tracing skills, and how to think about assessment validity.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Qiang Hao, David H. Smith IV, Naitra Iriumi, Michaeil Tsikerdekis, Amy J. Ko (2019)
A Systematic Investigation of Replications in Computing Education Research
ACM Transactions on Computing Education
Analyzed the replication rate and trends in computing education, finding that only about 2% of studies were replicated in the field in the last decade.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Saba Kawas, Laura Vonessen, Amy J. Ko (2019)
Teaching Accessibility: A Design Exploration of Faculty Professional Development at Scale
ACM Technical Symposium on Computer Science Education (SIGCSE), Research Track
Describes and evaluates a vision for teaching higher education CS faculty how to integrate accessibility topics into their classes, discovering many personal and organizational barriers to adoption.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Thomas D. LaToza, Stephen Hull, Ellen Ko, William Kwok, Jane Quichocho, Harshitha Akkaraju, Rishin Pandit (2019)
Teaching Explicit Programming Strategies to Adolescents
ACM Technical Symposium on Computer Science Education (SIGCSE), Research Track
Attempts to teach explicit strategies for reuse and debugging, finding that adolescents struggle to follow systematic approaches to problem solving.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Benjamin Xie, Matt Davidson, Min Li, Amy J. Ko (2019)
An Item Response Theory Evaluation of a Language-Independent CS1 Knowledge Assessment
ACM Technical Symposium on Computer Science Education (SIGCSE), Research Track
Evaluates the SCS1 language independent concept inventory, demonstrating how to use IRT to evaluate assessments, and that the SCS1 has several items that may be too hard to be useful.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Benjamin Xie, Dastyni Loksa, Greg Nelson, Matt Davidson, Dongsheng Dong, Harrison Kwik, Alex Tan, Leanne Hwa, Min Li, Amy J. Ko (2019)
A Theory of Instruction for Introductory Programming Skills
Computer Science Education
Proposes a theory that identifies four distinct skills that novices learn incrementally: tracing, writing syntax, comprehending reusable abstractions of programming knowledge (templates). Demonstrates that teaching these skills incrementally can result in improved completion rate on practice exercises and decreased error rate and improved understanding of the post-test.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Lauri Malmi, Ian Utting, Amy J. Ko (2019)
Tools and Environments
Cambridge Handbook on Computing Education Research (Sally Fincher, Anthony Robin, Eds.)
Surveys the role of tools in learning computing, and the genres of tools that researchers have explored, including IDEs, e-books, automated assessment and feedback, and visualization. Identifies a range of open questions.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Andrew Begel, Amy J. Ko (2019)
Learning Outside the Classroom
Cambridge Handbook on Computing Education Research (Sally Fincher, Anthony Robin, Eds.)
Surveys the small but growing literature on informal learning of computing and the contexts, technologies, and skills that support it. Identifies a several open questions about how to support informal learning and integrate it with more formal learning.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Sally Fincher (2019)
A Study Design Design Process
Cambridge Handbook on Computing Education Research (Sally Fincher, Anthony Robin, Eds.)
Describes a process for designing studies in the domain of computing education research, though the process is more broadly useful for empirical study design in general.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Paul Luo Li, Amy J. Ko, Andrew Begel (2019)
What Distinguishes Great Software Engineers?
Empirical Software Engineering
Via a survey of nearly 2,000 developers, finds that great engineers are distinguished by writing good code, adapting to future business value and costs, being great decision-makers, avoiding making others’ jobs harder, and continuously learning.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2018– Informatics program chair 🔗

Amanda Swearngin, Amy J. Ko, James Fogarty (2018)
Scout: Mixed-Initiative Exploration of Design Variations through High-Level Design Constraints
ACM Symposium on User Interface Software and Technology (UIST)
Describes early work on a system to support rapid design exploration of user interface layouts.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Harrison Kwik, Benjamin Xie, Amy J. Ko (2018)
Experiences of Computer Science Transfer Students
ACM International Computing Education Research Conference (ICER)
Finds that while transfer students in one university were more diverse demographically, the support of the university for transfer student orientation mitigated any social effects of transferring, but did not eliminate gaps in academic performance.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Alannah Oleson, Christopher Mendez, Zoe Steine-Hanson, Claudia Hilderbrand, Chris Perdriau, Margaret Burnett, Amy J. Ko (2018)
Pedagogical Content Knowledge for Teaching Inclusive Design
ACM International Computing Education Research Conference (ICER)
Through interviews and observations, identifies 11 components of pedagogical content knowledge required to teach inclusive design in HCI education, including strategies for anticipating and addressing resistance to the topic of inclusion, strategies for modeling and scaffolding perspective taking, and strategies for tailoring instruction to students’ prior beliefs and biases.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper
Greg Nelson, Amy J. Ko (2018)
On Use of Theory in Computing Education Research
ACM International Computing Education Research Conference (ICER)
Argues that our desire to both advance explanatory theory and advance design splits our attention, which prevents us from excelling at both; that our emphasis on applying and refining general theories of learning is done at the expense of domain-specific theories of computer science knowledge; and our use of theory as a critical lens in peer review prevents the publication of designs that may accelerate design progress.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2018)
Mining the Mind, Minding the Mine: Grand Challenges in Comprehension and Mining
ACM International Conference on Mining Software Repositories
Argues that program comprehension and mining software repositories research are mutually relevant in surprising ways.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ honorable mention
Rahul Banerjee, Jason Yip, Amy J. Ko, Caroline Pitt, Kiley R. Sobel, Kung Lee, Leanne Liu, Meng Wang, Zoran Popovic (2018)
Empowering Families Facing English Literacy Challenges to Jointly Engage in Computer Programming
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Investigates how a programming-by-demonstration programming environment with no English text can facilitate joint-media engagement between children and their parents.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amanda Swearngin, Wil Li, Mira Dontcheva, Morgan Dixon, Amy J. Ko (2018)
Rewire: Interface Design Assistance from Examples
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Presents a technique for automatically converting screenshots of imagines into vector representations for use by designers. Demonstrates that designers find the technique accurate enough and superior to replicating user interfaces manually.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Benjamin Xie, Greg Nelson, Amy J. Ko (2018)
An Explicit Strategy to Scaffold Novice Program Tracing
ACM Technical Symposium on Computer Science Education (SIGCSE), Research Track
Describes and evaluates a simple but powerful strategy for scaffolding the tracing of program execution. With less than 30 minutes of practice, novices in a CS1 course had midterm grades 7% higher.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Kristen Shinohara, Saba Kawas, Amy J. Ko, Richard E. Ladner (2018)
Who Teaches Accessibility? A Survey of U.S. Computing Faculty
ACM Technical Symposium on Computer Science Education (SIGCSE), Research Track
Investigates the prevalence of instruction on accessibility among computing and information science faculty. Finds that nearly all HCI faculty teach something about accessibility, that most faculty want to teach it, but don't have the experience, and don't know how it fits into their course content.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Leanne Hwa, Katie Davis, Jason Yip (2018)
Informal Mentoring of Adolescents about Computing: Relationships, Roles, Qualities, and Impact
ACM Technical Symposium on Computer Science Education (SIGCSE), Research Track
Investigates the nature of informal computing mentoring relationships amongst students seeking to be the first in their family to attend college. Demonstrates that mentorship occurs even in low-resource settings with little CS expertise, and most often amongst peers and siblings.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2017– Informatics program chair 🔗

Amy J. Ko (2017)
User Interface Software and Technology
Online
An online, self-published book that surveys the past, present, and future of user interfaces from theoretical, practical, and ethical perspectives.

▸ cite ⋅ doi ⋅ 🔗

Greg Nelson, Benjamin Xie, Amy J. Ko (2017)
Comprehension First: Evaluating a Novel Pedagogy and Tutoring System for Program Tracing in CS1
ACM International Computing Education Research Conference (ICER)
Contributes a new theory of what it means to know a programming language, a novel pedagogy and computer-based tutorial for teaching this knowledge, and evidence that 1) the tutorial promotes significantly higher learning gains over conventional programming language tutorials, and 2) that these gains predict the majority of the variance in CS1 midterm grades.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Katie Davis (2017)
Computing Mentorship in a Software Boomtown: Relationships to Adolescent Interest and Beliefs
ACM International Computing Education Research Conference (ICER)
Finds that interest in computing was strongly related to having a mentoring relationship and not to gender or socioeconomic status, that teens with mentors also engaged in significantly more computing education and had more diverse beliefs about peers who engaged in computing education, and that teens who took a class from an instructor who aimed to become students’ teacher-mentor had significantly greater positive changes in interest in computing than those who already had a mentor.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Kyle Thayer, Amy J. Ko (2017)
Barriers Faced by Coding Bootcamp Students
ACM International Computing Education Research Conference (ICER)
Finds that bootcamps can be an alternate path into the software industry for people who missed earlier computing education opportunities, particularly for women, but students face great personal costs and risks, including significant time, money, and effort before, during, and after bootcamps.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Amanda Swearngin, Amy J. Ko, James Fogarty (2017)
Genie: Input Retargeting on the Web through Command Reverse Engineering
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Presents Genie, a system to reverse engineer and retarget interactive commands on a website, which allows users to engage with the web via arbitrary input modalities, including speech, keyboard, and command-line input.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Ada Kim, Amy J. Ko (2017)
A Pedagogical Analysis of Online Coding Tutorials
ACM Technical Symposium on Computer Science Education (SIGCSE), Research Track
Using a set of pedagogical principles, evaluates the current state of tutorial instruction, finding that only a few tutorials follow best practices.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Annie Yan, Michael J. Lee, Amy J. Ko (2017)
Predicting Abandonment in Online Coding Tutorials
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
Investigates the feasibility of predicting when a learner will abandon a coding tutorial, finding that account activation and help-seeking behavior predict continued learning, while tutorial features indicating struggle predicted abandonment.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2017)
A Three-Year Participant Observation of Software Startup Software Evolution
ACM/IEEE International Conference on Software Engineering (ICSE), Software Engineering in Practice
Describes an analysis my personal diary reflecting on 9,000 hours of software startup work, presenting nine claims about software engineering work.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Paul Luo Li, Amy J. Ko, Andrew Begel (2017)
Cross-Disciplinary Perspectives on Collaborations with Software Engineers
International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE)
Finds that non-software developers who collaborate with software developers described great software engineers as masters of their own technical domain, open-minded to the input of others, proactively informing everyone, and seeing the big picture of how pieces fit together.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Will Jernigan, Amber Horvath, Michael J. Lee, Margaret M. Burnett, Taylor Cuilty, Sandeep Kuttal, Anicia Peters, Irwin Kwan, Faezeh Bahmani, Amy J. Ko, Christopher J. Mendez, Alannah Oleson (2017)
General Principles for a Generalized Idea Garden
Journal of Visual Languages and Computing
Describes the general architecture for the Idea Garden paradigm and a series of studies that suggest it's effectiveness at unblocking learners' problem solving.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Dastyni Loksa, Amy J. Ko (2017)
Modeling Programming Problem Solving Through Interactive Worked Examples
Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU)
Briefly overviews a vision for programming process worked examples that demonstrate an expert working through a programming problem.

▸ cite ⋅ pdf ⋅ 🔗

2016– Informatics program chair 🔗

Amy J. Ko (2016)
Cooperative Software Development
Online
An online, self-published book that covers software engineering fundamentals, but from a human, social, collaborative, and organizational perspective, rather than from a purely technical perspective.

▸ cite ⋅ doi ⋅ 🔗

Amy J. Ko (2016)
What is a Programming Language, Really?
ACM Workshop on Workshop on Evaluation and Usability of Programming Languages and Tools (PLATEAU)
Reflects on the implications of various definitions of programming languages on research and practice.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Richard Ladner (2016)
AccessComputing Promotes Teaching Accessibility
ACM Inroads
Discusses AccessComputing efforts to implement and disseminate curriculum on accessibility into academia and industry.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Dastyni Loksa, Amy J. Ko (2016)
The Role of Self-Regulation in Programming Problem Solving Process and Success
ACM International Computing Education Research Conference (ICER)
Finds that while most novices engage in self-regulation to navigate and inform their problem solving efforts, these self-regulation efforts are only effective when accompanied by programming knowledge adequate to succeed at solving a given problem.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Dastyni Loksa, Amy J. Ko, William Jernigan, Alannah Oleson, Chris Mendez, Margaret M. Burnett (2016)
Programming, Problem Solving, and Self-Awareness: Effects of Explicit Guidance
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Teaching novice programmers the structure of programming problem solving activities and to be aware of which activity they are conducting and whether they are succeeding has a substantial impact on their productivity, self-efficacy, independence, and growth mindset.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Brad A. Myers, Amy J. Ko, Thomas D. LaToza, YoungSeok Yoon (2016)
Programmers are Users Too: Human Centered Methods for Improving Tools for Programming
IEEE Computer
Argues that HCI methods are directly applicable to designing and developing software development tools, and provides an overview of several methods that have been used in research to invent and evaluate new developer tools.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2016)
The World is Your Test Suite
Perspectives on Data Science for Software Engineering, 1st Edition
Discusses the importance of monitoring, bug reporting, and user feedback in verifying requirements.

▸ cite ⋅ doi ⋅ 🔗

Steve Cooper, Jeff Forbes, Armando Fox, Susan Hambrusch, Amy J. Ko, Beth Simon (2016)
The Importance of Computing Education Research
CRA Computing Community Consortium
Argues for the foundational role of computing education research in computer science departments.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2015– Sabbatical 🔗

Amy J. Ko (2015)
Design Methods
Online
An online, self-published book that surveys HCI and Design methods in a simple, accessible, and direct manner.

▸ cite ⋅ doi ⋅ 🔗

BJ Burg, Amy J. Ko, Michael D. Ernst (2015)
Explaining Visual Changes in Web Interfaces
ACM Symposium on User Interface Software and Technology (UIST)
An algorithm and interface that allows web developers to select a before and after state of a web site to see the code that caused the changes.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper
Parmit K. Chilana, Amy J. Ko, Jacob O. Wobbrock (2015)
From User-Centered Design to Adoption-Centered Design: A Case Study of a Research System Becoming a Product
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Research prototypes and evidence of their usability and usefulness are only a tiny part of creating a successful whole product.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

William Jernigan, Amber Horvath, Michael J. Lee, Margaret M. Burnett, Taylor Cuilty, Sandeep Kuttal, Anicia N. Peters, Irwin Kwan, Faezeh Bahmani, Amy J. Ko (2015)
A Principled Evaluation for a Principled Idea Garden
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
In-context help on programming problem solving helps learners with diverse learning strategies work more independently.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Paul Luo Li, Amy J. Ko, Jiamin Zhu (2015)
What Makes a Great Software Engineer?
ACM/IEEE International Conference on Software Engineering (ICSE)
Software engineering expertise is much more about personality, interpersonal skills, and decision-making expertise than about technical knowledge and ability.

▸ cite ⋅ pdf ⋅ doi ⋅ blog ⋅ 🔗

Michael J. Lee, Amy J. Ko (2015)
Comparing the Effectiveness of Online Learning Approaches on CS1 Learning Outcomes
ACM International Computing Education Research Conference (ICER)
Mastery learning approaches to teaching programming produce better skills in program tracing than task-free open-ended creative environments.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2014– Promoted to Associate with tenure; returned from AnswerDash 🔗

Amy J. Ko, Bryan Dosono, Neeraja Duriseti (2014)
Thirty Years of Software Problems in the News
International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE)
Software failures reported in the news describe mostly minor consequences, though reports of death, injury, or loss of access to basic needs is increasing.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Polina Charters, Michael J. Lee, Amy J. Ko, Dastyni Loksa (2014)
Challenging Stereotypes and Changing Attitudes: The Effect of a Brief Programming Encounter on Adults' Attitudes toward Programming
ACM Symposium on Computer Science Education
Adult attitudes toward programming, while generally negative, can be quickly turned positive with a brief exposure to a debugging game.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ most influential paper
Michael J. Lee, Faezeh Bahmani, Irwin Kwan, Jillian LaFerte, Polina Charters, Amber Horvath, Fanny Luor, Jill Cao, Catherine Law, Michael Beswetherick, Sheridan Long, Margaret M. Burnett, Amy J. Ko (2014)
Principles of a Debugging-First Puzzle Game for Computing Education
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
Teaching debugging first, while redirecting blame to the computer instead of the learner, can increase learning efficiency and engage learners of all genders equally.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2013– On leave at AnswerDash 🔗

★ honorable mention
BJ Burg, Richard Bailey, Amy J. Ko, Michael D. Ernst (2013)
Interactive Record/Replay for Web Application Debugging
ACM Symposium on User Interface Software and Technology (UIST)
A system for recording inputs to a web application and replaying them deterministically with minimal time or space overhead.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper
Michael J. Lee, Amy J. Ko, Irwin Kwan (2013)
In-Game Assessments Increase Novice Programmers' Engagement and Learning Efficiency
ACM International Computing Education Research Conference (ICER)
Including brief assessments in a programming game improves engagement and learning efficiency.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Parmit K. Chilana, Amy J. Ko, Jacob O. Wobbrock (2013)
A Multi-Site Field Study of Crowdsourced Contextual Help: Usage and Perspectives of End-Users and Software Teams
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Contextual help is well-liked by both end-users and software teams in real world use.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Thomas D. LaToza, Margaret M. Burnett (2013)
A Practical Guide to Controlled Experiments of Software Engineering Tools with Human Participants
Empirical Software Engineering
Software engineering researchers rarely evaluate new tools with software developers, but there are several best practices that can make such evaluations less risky and difficult to conduct.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Parmit K. Chilana, Amy J. Ko, Jacob O. Wobbrock (2013)
Crowdsourced Q&A-based Contextual Help for Web Applications: Challenges and Opportunities
CSCW Workshop on Social Media Question Asking
Summarizes preliminary work on the LemonAid prototype.

▸ cite ⋅ pdf ⋅ 🔗

2012– Founded AnswerDash 🔗

Parmit K. Chilana, Christina Holsberry, Flavio Oliveira, Amy J. Ko (2012)
Designing for a Billion Users: A Case Study of Facebook
ACM Conference on Human Factors in Computing Systems, Case Studies (CHI)
Efforts to make a universal product can limit innovation, both because of the diversity of user knowledge and the limits of interpreting data science patterns.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2012)
Mining Whining in Support Forums with Frictionary
ACM Conference on Human Factors in Computing Systems (CHI), alt.chi
A technique for extracting aggregate patterns in software problems by analyzing the natural language text of technical support discussion forums.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Travis Kriplean, Michael Toomim, Jonathan Morgan, Alan Borning, Amy J. Ko (2012)
Is This What You Meant? Promoting Listening on the Web with Reflect
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
A technique for demonstrating listening on the web through restatement that engages lurkers in supporting common ground in groups.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Parmit K. Chilana, Amy J. Ko, Jacob O. Wobbrock (2012)
LemonAid: Selection-Based Crowdsourced Contextual Help for Web Applications
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
A technique for extracting queries from selections of user interface elements that provides reliable, relevant help retrieval.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Michael J. Lee, Amy J. Ko (2012)
Investigating the Role of Purposeful Goals on Novices' Engagement in a Programming Game
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
Representing data as animate, vertebrate objects instead of abstract, inanimate objects greatly increases engagement in online coding tutorials.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Michael J. Lee, Amy J. Ko (2012)
Representations of User Feedback in an Agile, Collocated Software Team
International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE)
As issues reported by end users are transformed from emails, to support ticket, to discussion, to bug report, to feature specification, information from the original request is lost, but remains in the distributed memory of the team.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2011 🔗

Todd Kulesza, Simone Stumpf, Weng-Keen Wong, Margaret M. Burnett, Stephen Perona, Amy J. Ko, Ian Oberst (2011)
Why-Oriented End-User Debugging of Naive Bayes Text Classification
ACM Transactions on Interactive Intelligent Systems, 1(1), Article 2
A technique that allows end-users to debug the underlying logic of machine learned classifiers.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Michael J. Lee, Amy J. Ko (2011)
Personifying Programming Tool Feedback Improves Novice Programmers' Learning
ACM International Computing Education Research Conference (ICER)
Giving a computer eyes and having it use personal pronouns that double the amount of time that learners engaging in discretionary computing education by focusing their attention on instructional material.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Paul Luo Li, Ryan Kivett, Zhiyuan Zhan, Sung-eok Jeon, Nachiappan Nagappan, Brendan Murphy, Amy J. Ko (2011)
Characterizing the Differences Between Pre- and Post- release Versions of Software
ACM/IEEE International Conference on Software Engineering (ICSE), Software Engineering in Practice
Beta-testers use more applications, engage in more self-install, come from non-representative countries, and use more advanced hardware, skewing data on crashes, hangs, and kernel panics that is used for bug triage.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Michael J. Lee, Valentina Ferrari, Stephen Ip, Charlie Tran (2011)
A Case Study of Post-Deployment User Feedback Triage
International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE)
User feedback is often ignored in because software teams perceive most feedback as minority opinion that conflicts with prior architectural decisions.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Parmit K. Chilana, Amy J. Ko, Jacob O. Wobbrock, Tovi Grossman, George Fitzmaurice (2011)
Post-Deployment Usability: A Survey of Current Practices
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Technical support and design are rarely linked in software companies despite significant opportunities for user research, usability feedback, and feature ideas.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Xing Zhang (2011)
Feedlack Detects Missing Feedback in Web Applications
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
A static program analysis for finding all inputs in a user interface that do not produce some form of feedback.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Parmit K. Chilana (2011)
Design, Discussion, and Dissent in Open Bug Reports
iConference
Developers in open source can be passionate about making decisions that positively impact users, but often lack the information necessary to do this objectively, instead relying on anecdote, speculation, and hyperbole.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Robin Abraham, Laura Beckwith, Alan Blackwell, Margaret M. Burnett, Martin Erwig, Chris Scaffidi, Joseph Lawrance, Henry Lieberman, Brad A. Myers, Mary Beth Rosson, Gregg Rothermel, Mary Shaw, Susan Wiedenbeck (2011)
The State of the Art in End-User Software Engineering
ACM Computing Surveys, 43(3), Article 21
Defines end-user programming and end-user software engineering, then presents the extensive history of efforts to integrate software engineering activities into programming environments.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Yann Riche (2011)
The Role of Conceptual Knowledge in API Usability
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
Test the feasibility of a feature in an API requires architectural and terminology knowledge of the API, which is often missing from documentation.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Travis Kriplean, Michael M. Toomim, Jonathan T. Morgan, Alan Borning, Amy J. Ko (2011)
Supporting Active Listening and Grounding on the Web through Restatement
ACM Conference on Computer Supported Cooperative Work (CSCW), Horizons Workshop
Shares preliminary work on a vision for an internet that incentivizes listening.

▸ cite ⋅ pdf ⋅ 🔗

Parmit K. Chilana, Amy J. Ko, Jacob O. Wobbrock (2011)
Designing Crowdsourced, Context-Sensitive Help for Web Applications
Workshop on Crowdsourcing and Human Computation
Summarizes progress on the LemonAid help system.

▸ cite ⋅ pdf ⋅ 🔗

2010 🔗

Kayur Patel, Naomi Bancroft, Steven M. Drucker, James Fogarty, Amy J. Ko, James A. Landay (2010)
Gestalt: Integrated Support for Implementation and Analysis in Machine Learning Processes
ACM Symposium on User Interface Software and Technology (UIST)
A development environment that supports the translation of data into machine learned classifier that improves the ability of developers to find and fix defects.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Brad A. Myers (2010)
Extracting and Answering Why and Why Not Questions about Java Program Output
ACM Transactions on Software Engineering and Methodology, 22(2)
An application of static and dynamic program slicing, precise call graphs, reachability analyses, and value provenance that enables developers to localize faults through why and why not questions.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ honorable mention
Amy J. Ko, Parmit K. Chilana (2010)
How Power Users Help and Hinder Open Bug Reporting
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Most bug reports submitted from non-core open source contributors describe non-issues that devolve into technical support, providing little valuable information to the core community.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Parmit K. Chilana, Amy J. Ko, Jacob O. Wobbrock (2010)
Understanding Expressions of Unwanted Behaviors in Open Bug Reporting
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
The more a bug reporter expresses the issue in terms of a violation of community expectations of program behavior rather than individual expectations, the more likely it is to be fixed.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper
Amy J. Ko, Jacob O. Wobbrock (2010)
Cleanroom: Edit-Time Error Detection with the Uniqueness Heuristic
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
Detecting and highlighting identifiers that only appear once in dynamic languages helps developers find and fix challenging defects more quickly.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2010)
Software Engineering Through Qualitative Methods
Making Software: What Really Works, and Why We Believe It
Argues that qualitative methods are essential for understanding the dynamics of software engineering teamwork.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Parmit K. Chilana, Jacob O. Wobbrock, Amy J. Ko (2010)
Understanding Usability Practices in Complex Domains: Implications for Training the Next Generation of Usability Professionals
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Lack of domain expertise can be a significant hurdle for conducting effective user research and usability testing.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Parmit K. Chilana (2010)
How Do Open Source Developers Talk about Users?
CHI 2010 Workshop on The Future of FLOSS in CHI Research and Practice
Shows that many open source developers speak in generalities about users and their needs, leaning on stereotypes and instinct instead of evidence.

▸ cite ⋅ pdf ⋅ 🔗

2009 🔗

Amy J. Ko, Brad A. Myers (2009)
Finding Causes of Program Output with the Java Whyline
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
A user interface for debugging that supports why and why not questions that enables developers to localize faults significantly faster than breakpoint debuggers.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Todd Kuleza, Weng-Keen Wong, Simone Stumpf, Stephen Perona, Rachel White, Margaret M. Burnett, Ian Oberst, Amy J. Ko (2009)
Fixing the Program My Computer Learned: Barriers for End Users, Challenges for the Machine
International Conference on Intelligent User Interfaces (IUI)
Enabling end users to debug machined learned programs by asking why questions about classifications causes end-users to locally optimize one class at the expense of other classes.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2009)
Attitudes and Self-Efficacy in Young Adults' Computing Autobiographies
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
First encounters with code must be accessible, error-tolerant and socially engaging, and must occur repeatedly across a lifetime to lead to deeper investment in computing education.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Parmit K. Chilana, Carole Palmer, Amy J. Ko (2009)
Comparing Bioinformatics Software Development by Computer Scientists and Biologists: An Exploratory Study
Workshop on Software Engineering for Computational Science and Engineering
Colleagues play a significant role in information seeking activities by developers in bioinformatics because of the high demand for domain knowledge.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Parmit K. Chilana, Amy J. Ko, Jacob O. Wobbrock (2009)
Designing Software for Unfamiliar Domains
Workshop on Cooperative and Human Aspects of Software Engineering (CHASE)
A brief abstract summarizing progress on how designers work with domain experts, finding that learning about a domain is the biggest challenge.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Jacob O. Wobbrock, Amy J. Ko, Julie A. Kientz (2009)
New to the Movement: Reflections on the Future of Information Schools from Inspired Junior Faculty
Interactions
Presents a unification of information science and human-computer interaction concerns.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Brad A. Myers, Amy J. Ko (2009)
The Past, Present and Future of Programming in HCI
Human-Computer Interaction Consortium (HCIC)
Summarizes past work on programming interfaces and open challenges in supporting the broader set of programming activities.

▸ cite ⋅ pdf ⋅ 🔗

2008– Job search, finished PhD, started at UW, divorce 🔗

Brad A. Myers, Amy J. Ko, SunYoung Park, Jeffrey Stylos, Thomas D. LaToza, Jack Beaton (2008)
More Natural End-User Software Engineering
International Workshop on End-User Software Engineering
Shares recent progress on the Natural Programming project.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Brad A. Myers (2008)
Source-Level Debugging with the Whyline
Workshop on Cooperative and Human Aspects of Software Engineering
Describes a pilot study of the Whyline for Java, showing that users were twice as fast at fixing a particular defect than without the Whyline.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper + most influential paper
Amy J. Ko, Brad A. Myers (2008)
Debugging Reinvented: Asking and Answering Why and Why Not Questions about Program Behavior
ACM/IEEE International Conference on Software Engineering (ICSE)
Static and dynamic program slicing algorithms for extracting and answering developers questions about program output that substantially decrease fault localization time.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ most influential paper honorable mention
Brad A. Myers, Sun Young Park, Yoko Nakano, Greg Mueller, Amy J. Ko (2008)
How Designers Design and Program Interactive Behaviors
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
Designers report that behavior is much more difficult to define than appearance, that it requires collaborating with developers, and that communicating behavior to developers is of particular difficulty.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2008)
Asking and Answering Questions about the Causes of Software Behavior
Carnegie Mellon University Dissertation
Describes the discoveries of several investigations into debugging and presents the concept, implementation, and evaluation of interrogtative debugging, and the several prototypes that embody it, including the Whyline for Alice, the Whyline for Java, and Crystal.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2007 🔗

Mauro Cherubini, Gina Venolia, Rob DeLine, Amy J. Ko (2007)
Let's Go to the Whiteboard: How and Why Software Developers Draw Code
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Most of the diagrams that software developers make are transient and facilitate interpersonal communication and decision making and not for specifying or documenting program behavior.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Rob DeLine, Gina Venolia (2007)
Information Needs in Collocated Software Development Teams
ACM/IEEE International Conference on Software Engineering (ICSE)
Developers have many distinct information needs to be productive; some are frequent and well-served, such as information about changes to artifacts and coworker progress, but others are infrequent but critical, such as information about requirements, design rationale, and reproduction steps.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2006– MSR intern 🔗

Michael J. Coblenz, Amy J. Ko, Brad A. Myers (2006)
JASPER: An Eclipse Plug-In to Facilitate Software Maintenance Tasks
Proceedings of the 2006 OOPSLA Workshop on Eclipse Technology eXchange
A tool for organizing related code fragments during software maintenance tasks.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2006)
Debugging by Asking Questions About Program Output
ACM/IEEE International Conference on Software Engineering (ICSE)
A doctoral consortium abstract, summarizing progress on the Whyline for Java.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Brad A. Myers, Amy J. Ko, Margaret M. Burnett (2006)
Invited Research Overview: End-User Programming
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
A brief summary of the research area of end-user programming.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Brad A. Myers, David Weitzman, Amy J. Ko, Duen Horng Chau (2006)
Answering Why and Why Not Questions in User Interfaces
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Algorithms and user interfaces that allow end users to ask 'why' questions about unexpected application behavior.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Brad A. Myers (2006)
Barista: An Implementation Framework for Enabling New Tools, Interaction Techniques and Views for Code Editors
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Abstractions that make it easy to build rich multimedia interfaces in a code editor without sacrificing the ability to write code as text.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Brad A. Myers, Duen Horng Chau (2006)
A Linguistic Analysis of How People Describe Software Problems in Bug Reports
Visual Languages and Human-Centric Computing
There is regularity in the grammar of bug report titles, suggesting the potential for applying natural language processing to searching, filtering, mining, and aggregating bug reports.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Chris Scaffidi, Amy J. Ko, Brad A. Myers, Mary Shaw (2006)
Dimensions Characterizing Programming Feature Usage by Information Workers
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
End user programmers use of programming is typically limited to one of three language features: macros for automation, linked structure queries, or imperative features.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ most influential paper
Amy J. Ko, Brad A. Myers, Michael J. Coblenz, Htet Htet Aung (2006)
An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks
IEEE Transactions on Software Engineering
Developers work on software maintenance tasks by searching for relevant code, navigating dependencies, and constructing a mental model of the dependencies of a feature, suggesting the need for environments that explicitly support feature location through dependency navigation.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Brad A. Myers, Michael J. Coblenz, Jeffrey Stylos (2006)
End-User Programming Productivity Tools
2nd Workshop on End-User Software Engineering (WEUSE)
Shares examples of research from the Natural Programming project.

▸ cite ⋅ pdf ⋅ 🔗

Amy J. Ko (2006)
The Role of Science in Supporting Software Development
Workshop on Supporting the Social Side of Large-Scale Software Development, at the 2006 ACM Conference on Computer Supported Cooperative Work
Argues that progress on inventions requires scientific, theoretical explanations of their value.

▸ cite ⋅ pdf ⋅ 🔗

Amy J. Ko (2006)
Asking and Answering Questions about the Causes of Software Behaivors
Carnegie Mellon University, Thesis Proposal
Proposes the Java Whyline.

▸ cite ⋅ pdf ⋅ 🔗

2005 🔗

Amy J. Ko, Brad A. Myers (2005)
Citrus: A Language and Toolkit for Simplifying the Creation of Structured Editors for Code and Data
ACM Symposium on User Interface Software and Technology (UIST)
A programming language and library with language-level support for constraints, restrictions and change notifications on primitive and aggregate data.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Brad A. Myers (2005)
Human Factors Affecting Dependability in End-User Programming
1st Workshop on End-User Software Engineering (WEUSE)
Enumerates many cognitive biases that shape the mistake that people make when engaging in end user programming.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper
Amy J. Ko, Htet Htet Aung, Brad A. Myers (2005)
Eliciting Design Requirements for Maintenance-Oriented IDEs: A Detailed Study of Corrective and Perfective Maintenance Tasks
ACM/IEEE International Conference on Software Engineering (ICSE)
When developers fix a big or enhance a feature, they find a set of task-relevant code fragments and navigate dependencies between these fragments, but spend up to 35% of their time simply operating navigational controls in IDEs.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Htet Htet Aung, Brad A. Myers (2005)
Design Requirements for More Flexible Structured Editors from a Study of Programmers' Text Editing
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
When writing code in a text editor, developers regularly pass through invalid syntactic states, making strict structured editors infeasible.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper
James Fogarty, Amy J. Ko, Htet Htet Aung, Elspeth Golden, Karen P. Tang, Scott E. Hudson (2005)
Examining Task Engagement in Sensor-Based Statistical Models of Human Interruptibility
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Developers' interruptibility can be predicted with high certainty by observing and classifying features of programming activity.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Michael J. Coblenz, Amy J. Ko, Brad A. Myers (2005)
Using Objects of Measurement to Detect Spreadsheet Errors
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
A language feature for specifying the unit of analysis for values in a spreadsheet, allowing the language to identify type errors as unit of analysis mismatch.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Brad A. Myers (2005)
A Framework and Methodology for Studying the Causes of Software Errors in Programming Systems
Journal of Visual Languages and Computing
Software defects arise from latent errors in specifications, developer knowledge, tool support, and program implementation, ultimately producing runtime failures.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Brad A. Myers, Amy J. Ko (2005)
More Natural and Open User Interface Tools
Workshop on the Future of User Interface Design Tools
A 2 page abstract on progress on Brad Myer's Natural Programming project.

▸ cite ⋅ pdf ⋅ 🔗

Amy J. Ko, Brad A. Myers, Mary Shaw (2005)
Identifying Types of End Users: Hints from an Informal Survey
Carnegie Mellon University ISRI Technical Report, no. CMU-HCII-05-101 and Human Computer Interaction Institute Technical Report, no CMU-ISRI-05-110
Presents a clustering of end user programming activities, finding revealing variation in advanced feature usage.

▸ cite ⋅ pdf ⋅ 🔗

2004 🔗

Amy J. Ko, Brad A. Myers (2004)
Designing the Whyline: A Debugging Interface for Asking Questions About Program Failures
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Algorithms and user interfaces for extracting and answering why and why not questions about program behavior.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Brad A. Myers, John F. Pane, Amy J. Ko (2004)
Natural Programming Languages and Environments
Communications of the ACM (CACM)
Describes the Natural Programming project, which sought to make it possible for people to express their ideas in the same way they think about them.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ most influential paper
Amy J. Ko, Brad A. Myers, Htet Htet Aung (2004)
Six Learning Barriers in End-User Programming Systems
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
When people write programs, they face at least six distinct problems: 1) identifying a solution, 2) finding abstractions to implement the solution, 3) coordinating these abstractions, 4) properly configuring these abstractions, 5) identifying possible causes of program failures, and 6) getting information about program execution.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2004)
Designing a Flexible and Supportive Direct-Manipulation Programming Environment
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
Finds that structured editors need some room for passing between unstructured phases to flexibly support editing.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2003 🔗

Amy J. Ko (2003)
A Contextual Inquiry of Expert Programmers in an Event-Based Programming Environment
ACM SIGCHI Conference on Human Factors in Computing Systems (CHI)
Summarizes preliminary insights on debugging, showing that end-user programmers often start from faulty assumptions about runtime behavior.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko, Bob Uttl (2003)
Individual Differences in Program Comprehension Strategies in Unfamiliar Environments
IEEE International Workshop on Program Comprehension (IWPC)
Programming expertise shapes strategy, but domain expertise is a better predictor of bug fixing success than strategy.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2003)
Preserving Non-Programmers Motivation with Error-Prevention and Debugging Support Tools
IEEE Symposia Human-Centric Computing Languages and Environments
Summarizes Amy's early thoughts on her dissertation work.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

★ best paper + most influential paper
Amy J. Ko, Brad A. Myers (2003)
Development and Evaluation of a Model of Programming Errors
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
Software defects are caused by cognitive breakdowns in specification, implementation and debugging activities and analysis of these breakdowns can be used to identify tools that prevent these breakdowns.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2002– Graduated from college; 1st year of PhD 🔗

Amy J. Ko, Margaret M. Burnett, Thomas R.G. Green, Karen J. Rothermel, Curtis R. Cook (2002)
Improving the Design of Visual Programming Language Experiments Using Cognitive Walkthroughs
Journal of Visual Languages and Computing, 13(5)
An adaption of the cognitive walkthrough for finding potential difficulties of novice users in an evaluation of a developer tool.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Amy J. Ko (2002)
Individual Differences in Programming, Testing, and Debugging Strategies in a Statistical End-User Programming Environment
Oregon State University, Honors Thesis
Programming expertise shapes strategy, but domain expertise is a better predictor of bug fixing success than strategy

▸ cite ⋅ pdf ⋅ 🔗

2001– Junior year of college 🔗

Margaret M. Burnett, Bing Ren, Amy J. Ko, Curtis Cook, Gregg Rothermel (2001)
Visually Testing Recursive Programs in Spreadsheet Languages
IEEE Symposia on Human-Centric Computing Languages and Environments
Two technique for testing recursive spreadsheets, neither of which can be understood by spreadsheet users.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

2000– Sophomore year of college 🔗

Thomas R.G. Green, Margaret M. Burnett, Amy J. Ko (2000)
Using the Cognitive Walkthrough to Improve the Design of a Visual Programming Experiment
IEEE Symposium on Visual Languages and Human-Centered Computing (VL/HCC)
An adaptation of the cognitive walkthrough technique that helps identify uncontrolled source of variations in experiments on developers tools.

▸ cite ⋅ pdf ⋅ doi ⋅ 🔗

Last updated 11/16/2024. To the extent possible under law, Amy J. Ko has waived all copyright and related or neighboring rights to the design and implementation of Amy's faculty site. This work is published from the United States. See this site's GitHub repository to view source and provide feedback.

Amy J. Ko, Ph.D.

These are my academic publications.

2024– Informatics program chair; Associate Dean for Academics 🔗

2023– Sabbatical (Winter/Spring/Summer) 🔗

2022– Informatics program chair; gender transition; sabbatical (Summer/Fall) 🔗

2021– Informatics program chair; COVID-19; gender transition 🔗

2020– Informatics program chair; promoted to Professor; COVID-19; gender transition 🔗

2019– Informatics program chair; gender transition 🔗

2018– Informatics program chair 🔗

2017– Informatics program chair 🔗

2016– Informatics program chair 🔗

2015– Sabbatical 🔗

2014– Promoted to Associate with tenure; returned from AnswerDash 🔗

2013– On leave at AnswerDash 🔗

2012– Founded AnswerDash 🔗

2011 🔗

2010 🔗

2009 🔗

2008– Job search, finished PhD, started at UW, divorce 🔗

2007 🔗

2006– MSR intern 🔗

2005 🔗

2004 🔗

2003 🔗

2002– Graduated from college; 1st year of PhD 🔗

2001– Junior year of college 🔗

2000– Sophomore year of college 🔗