A visualization of many interconnected node
Program comprehension is about understanding dependencies
Chapter 10

Comprehension

by Amy J. Ko

Despite all of the activities that we’ve talked about so far—communicating, coordinating, planning, designing, architecting—really, most of a software engineers time is spent reading code 16 16

Walid Maalej, Rebecca Tiarks, Tobias Roehm, and Rainer Koschke (2014). On the comprehension of program comprehension. ACM Transactions on Software Engineering and Methodology (TOSEM).

. Sometimes this is their own code, which makes this reading easier. Most of the time, it is someone else’s code, whether it’s a teammate’s, or part of a library or API you’re using. We call this reading  program comprehension .

Being good at program comprehension is a critical skill. You need to be able to read a function and know what it will do with its inputs; you need to be able to read a class and understand its state and functionality; you also need to be able to comprehend a whole implementation, understanding its architecture. Without these skills, you can’t test well, you can’t debug well, and you can’t fix or enhance the systems you’re building or maintaining. In fact, studies of software engineers’ first year at their first job show that a significant majority of their time is spent trying to simply comprehend the architecture of the system they are building or maintaining and understanding the processes that are being followed to modify and enhance them 6 6

Barthélémy Dagenais, Harold Ossher, Rachel K. E. Bellamy, Martin P. Robillard, and Jacqueline P. de Vries (2010). Moving into a new software project landscape. ACM/IEEE International Conference on Software Engineering.

.

What’s going on when developers comprehend code? Usually, developers are trying to answer questions about code that help them build larger models of how a program works. Because program comprehension is hard, they avoid it when they can, relying on explanations from other developers rather than trying to build precise models of how a program works on their own 19 19

Tobias Roehm, Rebecca Tiarks, Rainer Koschke, and Walid Maalej (2012). How do professional developers comprehend software?. ACM/IEEE International Conference on Software Engineering.

. When they do try to comprehend code, developers are usually trying to answer questions. Several studies have many general questions that developers must be able to answer in order to understand programs 14,22 14

Thomas D. LaToza, Brad A. Myers (2010). Developers ask reachability questions. ACM/IEEE International Conference on Software Engineering.

22

Jonathan Sillito, Gail C. Murphy, and Kris De Volder (2006). Questions programmers ask during software evolution tasks. ACM SIGSOFT Foundations of Software Engineering (FSE).

. Here are nearly forty common questions that developers ask:

  1. Which type represents this domain concept or this UI element or action?
  2. Where in the code is the text in this error message or UI element?
  3. Where is there any code involved in the implementation of this behavior?
  4. Is there an entity named something like this in that unit (for example in a project, package or class)?
  5. What are the parts of this type?
  6. Which types is this type a part of?
  7. Where does this type fit in the type hierarchy?
  8. Does this type have any siblings in the type hierarchy?
  9. Where is this field declared in the type hierarchy?
  10. Who implements this interface or these abstract methods?
  11. Where is this method called or type referenced?
  12. When during the execution is this method called?
  13. Where are instances of this class created?
  14. Where is this variable or data structure being accessed?
  15. What data can we access from this object?
  16. What does the declaration or definition of this look like?
  17. What are the arguments to this function?
  18. What are the values of these arguments at runtime?
  19. What data is being modified in this code?
  20. How are instances of these types created and assembled?
  21. How are these types or objects related?
  22. How is this feature or concern (object ownership, UI control, etc) implemented?
  23. What in this structure distinguishes these cases?
  24. What is the “correct” way to use or access this data structure?
  25. How does this data structure look at runtime?
  26. How can data be passed to (or accessed at) this point in the code?
  27. How is control getting (from here to) here?
  28. Why isn’t control reaching this point in the code?
  29. Which execution path is being taken in this case?
  30. Under what circumstances is this method called or exception thrown?
  31. What parts of this data structure are accessed in this code?
  32. How does the system behavior vary over these types or cases?
  33. What are the differences between these files or types?
  34. What is the difference between these similar parts of the code (e.g., between sets of methods)?
  35. What is the mapping between these UI types and these model types?
  36. How can we know this object has been created and initialized correctly?

If you think about the diversity of questions in this list, you can see why program comprehension requires expertise. You not only need to understand programming languages quite well, but you also need to have strategies for answering all of the questions above (and more) quickly, effectively, and accurately.

So how do developers go about answering these questions? Studies comparing experts and novices show that experts use prior knowledge that they have about architecture, design patterns, and the problem domain a program is built for to know what questions to ask and how to answer them, whereas novices use surface features of code, which leads them to spend considerable time reading code that is irrelevant to a question 13,23 13

Thomas D. LaToza, David Garlan, James D. Herbsleb, and Brad A. Myers (2007). Program comprehension as fact finding. ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE).

23

Anneliese von Mayrhauser A. Marie Vans (1994). Comprehension processes during large scale maintenance. ACM/IEEE International Conference on Software Engineering.

. Reading and comprehending source code is fundamentally different from those of reading and comprehending natural language 4 4

Dave Binkley, Marcia Davis, Dawn Lawrie, Jonathan I. Maletic, Christopher Morrell, Bonita Sharif (2013). The impact of identifier style on effort and comprehension. Empirical Software Engineering.

; what experts are doing is ultimately reasoning about  dependencies  between code 24 24

Mark Weiser (1981). Program slicing. ACM/IEEE International Conference on Software Engineering.

. Dependencies include things like  data dependencies  (where a variable is used to compute something, what modifies a data structure, how data flows through a program, etc.) and  control dependencies  (which components call which functions, which events can trigger a function to be called, how a function is reached, etc.). All of the questions above fundamentally get at different types of data and control dependencies. In fact, theories of how developers navigate code by following these dependencies are highly predictive of what information a developer will seek next 8 8

Scott D. Fleming, Christopher Scaffidi, David Piorkowski, Margaret M. Burnett, Rachel K. E. Bellamy (2013). An information foraging theory perspective on tools for debugging, refactoring, and reuse tasks. ACM Transactions on Software Engineering and Methodology (TOSEM).

, suggesting that expert behavior is highly procedural. This work, and work explicitly investigating the role of identifier names 15 15

Lawrie, D., Morrell, C., Feild, H., & Binkley, D (2006). What's in a name? A study of identifiers. IEEE International Conference on Program Comprehension (ICPC).

, finds that names are actually critical to facilitating higher level comprehension of program behavior.

Of course, program comprehension is not an inherently individual process either. Expert developers are resourceful, and frequently ask others for explanations of program behavior. Some of this might happen between coworkers, where someone seeking insight asks other engineers for summaries of program behavior, to accelerate their learning 11 11

Amy J. Ko, Rob DeLine, and Gina Venolia (2007). Information needs in collocated software development teams. ACM/IEEE International Conference on Software Engineering.

. Others might rely on public forums, such as Stack Overflow, for explanations of API behavior 17 17

Lena Mamykina, Bella Manoim, Manas Mittal, George Hripcsak, and Björn Hartmann (2011). Design lessons from the fastest Q&A site in the west. ACM SIGCHI Conference on Human Factors in Computing (CHI).

. These social help seeking strategies are strongly mediated by a developers’ willingness to express that they need help to more expert teammates. Some research, for example, has found that junior developers are reluctant to ask for help out of fear of looking incompetent, even when everyone on a team is willing to offer help and their manager prefers that the developer prioritize productivity over fear of stigma 2 2

Andy Begel, Beth Simon (2008). Novice software developers, all over again. ICER.

. And then, of course, learning is just hard. For example, one study investigated the challenges that developers face in learning new programming languages, finding that unlearning old habits, shifting to new language paradigms, learning new terminology, and adjusting to new tools all required materials that could bridge from their prior knowledge to the new language, but few such materials existed 21 21

Shrestha, N., Botta, C., Barik, T., & Parnin, C (2020). Here we go again: why is it difficult for developers to learn another programming language?. ACM/IEEE International Conference on Software Engineering.

. These findings suggest the critical importance of teams ensuring that newcomers view them as psychologically safe places, where vulnerable actions like expressing a need for help will not be punished, ridiculed, or shamed, but rather validated, celebrated, and encouraged.

While much of program comprehension is individual and social skill, some aspects of program comprehension are determined by the design of programming languages. For example, some programming languages result in programs that are more comprehensible. One framework called the  Cognitive Dimensions of Notations 9 9

Thomas R. G. Green (1989). Cognitive dimensions of notations. People and computers.

 lays out some of the tradeoffs in programming language design that result in these differences in comprehensibility. For example, one of the dimensions in the framework is  consistency , which refers to how much of a notation can be  guessed  based on an initial understanding of a language. JavaScript has low consistency because of operators like  == , which behave differently depending on what the type of the left and right operands are. Knowing the behavior for Booleans doesn’t tell you the behavior for a Boolean being compared to an integer. In contrast, Java is a high consistency language:  ==  is only ever valid when both operands are of the same type.

These differences in notation can have some impact. Encapsulation through data structures leads to better comprehension that monolithic or purely functional languages 3,25 3

Pamela Bhattacharya and Iulian Neamtiu (2011). Assessing programming language impact on development and maintenance: a study on C and C++. ACM/IEEE International Conference on Software Engineering.

25

Scott N. Woodfield, Hubert E. Dunsmore, and Vincent Y. Shen (1981). The effect of modularization and comments on program comprehension. ACM/IEEE International Conference on Software Engineering.

. Declarative programming paradigms (like CSS or HTML) have greater comprehensibility than imperative languages 20 20

Guido Salvaneschi, Sven Amann, Sebastian Proksch, and Mira Mezini (2014). An empirical study on program comprehension with reactive programming. ACM SIGSOFT Foundations of Software Engineering (FSE).

. Statically typed languages like Java (which require developers to declare the data type of all variables) result in fewer defects 18 18

Baishakhi Ray, Daryl Posnett, Vladimir Filkov, and Premkumar Devanbu (2014). A large scale study of programming languages and code quality in GitHub. ACM SIGSOFT Foundations of Software Engineering (FSE).

, better comprehensibility because of the ability to construct better documentation 7 7

Stefan Endrikat, Stefan Hanenberg, Romain Robbes, and Andreas Stefik (2014). How do API documentation and static typing affect API usability?. ACM/IEEE International Conference on Software Engineering.

, and result in easier debugging 10 10

Stefan Hanenberg, Sebastian Kleinschmager, Romain Robbes, Éric Tanter, Andreas Stefik (2013). An empirical study on the impact of static typing on software maintainability. Empirical Software Engineering.

. In fact, studies of more dynamic languages like JavaScript and Smalltalk 5 5

Oscar Callaú, Romain Robbes, Éric Tanter, David Röthlisberger (2013). How (and why) developers use the dynamic features of programming languages: the case of Smalltalk. Empirical Software Engineering.

 show that the dynamic features of these languages aren’t really used all that much anyway. Despite all of these measurable differences, the impact of notation seems to be modest in practice 18 18

Baishakhi Ray, Daryl Posnett, Vladimir Filkov, and Premkumar Devanbu (2014). A large scale study of programming languages and code quality in GitHub. ACM SIGSOFT Foundations of Software Engineering (FSE).

. All of this evidence suggests that that the more you tell a compiler about what your code means (by declaring types, writing functional specifications, etc.), the more it helps the other developers know what it means too, but that this doesn’t translate into huge differences in defects.

Code editors, development environments, and program comprehension tools can also be helpful. Early evidence showed that simple features like syntax highlighting and careful typographic choices can improve the speed of program comprehension 1 1

Ron Baecker (1988). Enhancing program readability and comprehensibility with tools for program visualization. ACM/IEEE International Conference on Software Engineering.

. I have also worked on several tools to support program comprehension, including the Whyline, which automates many of the more challenging aspects of navigating dependencies in code, and visualizes them 12 12

Amy J. Ko and Brad A. Myers (2009). Finding causes of program output with the Java Whyline. ACM SIGCHI Conference on Human Factors in Computing (CHI).

:

The Whyline for Java, a debugging tool that faciliates dependency navigation

The path from novice to expert in program comprehension is one that involves understanding programming language semantics exceedingly well and reading  a lot  of code, design patterns, and architectures. Anticipate that as you develop these skills, it will take you time to build robust understandings of what a program is doing, slowing down your writing, testing, and debugging.

References

  1. Ron Baecker (1988). Enhancing program readability and comprehensibility with tools for program visualization. ACM/IEEE International Conference on Software Engineering.

  2. Andy Begel, Beth Simon (2008). Novice software developers, all over again. ICER.

  3. Pamela Bhattacharya and Iulian Neamtiu (2011). Assessing programming language impact on development and maintenance: a study on C and C++. ACM/IEEE International Conference on Software Engineering.

  4. Dave Binkley, Marcia Davis, Dawn Lawrie, Jonathan I. Maletic, Christopher Morrell, Bonita Sharif (2013). The impact of identifier style on effort and comprehension. Empirical Software Engineering.

  5. Oscar Callaú, Romain Robbes, Éric Tanter, David Röthlisberger (2013). How (and why) developers use the dynamic features of programming languages: the case of Smalltalk. Empirical Software Engineering.

  6. Barthélémy Dagenais, Harold Ossher, Rachel K. E. Bellamy, Martin P. Robillard, and Jacqueline P. de Vries (2010). Moving into a new software project landscape. ACM/IEEE International Conference on Software Engineering.

  7. Stefan Endrikat, Stefan Hanenberg, Romain Robbes, and Andreas Stefik (2014). How do API documentation and static typing affect API usability?. ACM/IEEE International Conference on Software Engineering.

  8. Scott D. Fleming, Christopher Scaffidi, David Piorkowski, Margaret M. Burnett, Rachel K. E. Bellamy (2013). An information foraging theory perspective on tools for debugging, refactoring, and reuse tasks. ACM Transactions on Software Engineering and Methodology (TOSEM).

  9. Thomas R. G. Green (1989). Cognitive dimensions of notations. People and computers.

  10. Stefan Hanenberg, Sebastian Kleinschmager, Romain Robbes, Éric Tanter, Andreas Stefik (2013). An empirical study on the impact of static typing on software maintainability. Empirical Software Engineering.

  11. Amy J. Ko, Rob DeLine, and Gina Venolia (2007). Information needs in collocated software development teams. ACM/IEEE International Conference on Software Engineering.

  12. Amy J. Ko and Brad A. Myers (2009). Finding causes of program output with the Java Whyline. ACM SIGCHI Conference on Human Factors in Computing (CHI).

  13. Thomas D. LaToza, David Garlan, James D. Herbsleb, and Brad A. Myers (2007). Program comprehension as fact finding. ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE).

  14. Thomas D. LaToza, Brad A. Myers (2010). Developers ask reachability questions. ACM/IEEE International Conference on Software Engineering.

  15. Lawrie, D., Morrell, C., Feild, H., & Binkley, D (2006). What's in a name? A study of identifiers. IEEE International Conference on Program Comprehension (ICPC).

  16. Walid Maalej, Rebecca Tiarks, Tobias Roehm, and Rainer Koschke (2014). On the comprehension of program comprehension. ACM Transactions on Software Engineering and Methodology (TOSEM).

  17. Lena Mamykina, Bella Manoim, Manas Mittal, George Hripcsak, and Björn Hartmann (2011). Design lessons from the fastest Q&A site in the west. ACM SIGCHI Conference on Human Factors in Computing (CHI).

  18. Baishakhi Ray, Daryl Posnett, Vladimir Filkov, and Premkumar Devanbu (2014). A large scale study of programming languages and code quality in GitHub. ACM SIGSOFT Foundations of Software Engineering (FSE).

  19. Tobias Roehm, Rebecca Tiarks, Rainer Koschke, and Walid Maalej (2012). How do professional developers comprehend software?. ACM/IEEE International Conference on Software Engineering.

  20. Guido Salvaneschi, Sven Amann, Sebastian Proksch, and Mira Mezini (2014). An empirical study on program comprehension with reactive programming. ACM SIGSOFT Foundations of Software Engineering (FSE).

  21. Shrestha, N., Botta, C., Barik, T., & Parnin, C (2020). Here we go again: why is it difficult for developers to learn another programming language?. ACM/IEEE International Conference on Software Engineering.

  22. Jonathan Sillito, Gail C. Murphy, and Kris De Volder (2006). Questions programmers ask during software evolution tasks. ACM SIGSOFT Foundations of Software Engineering (FSE).

  23. Anneliese von Mayrhauser A. Marie Vans (1994). Comprehension processes during large scale maintenance. ACM/IEEE International Conference on Software Engineering.

  24. Mark Weiser (1981). Program slicing. ACM/IEEE International Conference on Software Engineering.

  25. Scott N. Woodfield, Hubert E. Dunsmore, and Vincent Y. Shen (1981). The effect of modularization and comments on program comprehension. ACM/IEEE International Conference on Software Engineering.