An illustration of 1’s and 0’s with a stick figure sitting on one, reading.
User interfaces are where people and technology meet.

User Interface Software and Technology

edited by Amy J. Ko, Jacob O. Wobbrock, Eric Whitmire

1st edition

The HCI community has a long history of innovation in user interface software and technologies, but there are few surveys or review of all of this research. This book attempts to summarize much of this knowledge, providing a living online textbook for UX designers, researchers, design-literate developers to learn from and leverage this ever-expanding body of knowledge.

I hope you enjoy! If not, please file an issue on the the book’s GitHub repository.

Chapters

A photograph of the original computer mouse.Chapter 1

A History of Interfaces

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A very close subpixel image of a mouse cursor.Chapter 2

A Theory of Interfaces

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A top down photograph of three people using one laptop, all with arms reached out to point.Chapter 3

What Interfaces Mediate

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A braille numeric keypadChapter 4

Accessibility

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A close up shot of a code editor showing some Python code.Chapter 5

Programming Interfaces

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A screenshot of the Xerox star user interface and it’s word processing application and desktopChapter 6

Interactive Interfaces

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A modern architecture of an apartment building, figuratively resembling software architecture.Chapter 7

Interface Architecture

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A photograph of a person touching an Apple Watch screenChapter 8

Pointing

by Amy J. Ko, Jacob O. Wobbrock

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A photograph of a kitten sitting on a computer keyboard.Chapter 9

Text Entry

by Amy J. Ko, Jacob O. Wobbrock

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
Two human hands in the sun facing a blue skyChapter 10

Hand-Based Input

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A man and a woman playing XBox Kinect game.Chapter 11

Body-Based Input

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A screenshot of the original Pong video game with two paddles, a ball, and scores.Chapter 12

2D Visual Output

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A screenshot of Microsoft’s Hololens Minecraft experienceChapter 13

3D Visual Output

by Amy J. Ko, Eric Whitmire

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A Makerbot 3D printer printing a nested dodecahedron.Chapter 14

Physical Output

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A stick figure man looking frustrated at a computer screen.Chapter 15

Help

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
Legal paperwork with Apple and Microsoft logosChapter 16

Intellectual Property

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
The front desk of the University of Washington’s CoMotion, which supports technology transfer from research.Chapter 17

Translation

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }
A photograph of a Ring Doorbell mounted on a brick wall.Chapter 18

Interface Ethics

by Amy J. Ko

function($$renderer5) { $$renderer5.push(`<span class="etc svelte-g71yi0">`); Muted($$renderer5, { children: ($$renderer6) => { if (editable) { $$renderer6.push("<!--[0-->"); Toggle($$renderer6, { on: chapter.isForthcoming(), save: (on) => setChapter(edition, chapter, chapter.asForthcoming(on)), children: ($$renderer7) => { $$renderer7.push(`<!---->${escape_html(etc)}`); } }); } else { $$renderer6.push(`<!--[-1-->${escape_html(etc)}`); } $$renderer6.push(`<!--]-->`); } }); $$renderer5.push(`<!----></span>`); }

References

Everything cited

Glossary

Definitions

Index

Common words and where they are

Search

Find where words occur

Media

Images and video in the book

Acknowledgements

Thank you all of the students who’ve provided extensive and insightful feedback on drafts of this book over the years.

License

Creative Commons Attribution-NoDeriviatives 4.0

Print

Want to print this book or generate a PDF? See all chapters on a single page and then print or export. Long books can take some time to render.

E-book

Reading on an e-reader? Build an EPUB of this book. Images are shrunk to fit the size you choose, so pick the one closest to your device. Long books with many images can take a minute.

Citation

Amy J. Ko, Jacob O. Wobbrock, Eric Whitmire (2026). User Interface Software and Technology. …, retrieved 9/12/2026.