A software team hard at work
Early days at the author’s software startup in 2012.
Chapter 2

Organizations

by Amy J. Ko

The photo above is a candid shot of some of the software engineers of  AnswerDash , a company I co-founded in 2012, that was later acquired in 2020. There are a few things to notice in the photograph. First, you see one of the employees explaining something, while others are diligently working off to the side. It’s not a huge team; just a few engineers, plus several employees in other parts of the organization in another room. This, as simple as it looks, is pretty much what all software engineering work looks like. Some organizations have one of these teams; others have thousands.

What you  can’t  see is just how much  complexity  underlies this work. You can’t see the organizational structures that exist to manage this complexity. Inside this room and the rooms around it were processes, standards, reviews, workflows, managers, values, culture, decision making, analytics, marketing, sales. And at the center of it were people executing all of these things as well as they could to achieve the organization’s goal.

Organizations are a much bigger topic than I could possibly address here. To deeply understand them, you’d need to learn about  organizational studies organizational behavior information systems , and business in general.

The subset of this knowledge that’s critical to understand about software engineering is limited to a few important concepts. The first and most important concept is that even in software organizations, the point of the company is rarely to make software; it’s to provide  value 8 8

Alexander Osterwalder, Yves Pigneur, Gregory Bernarda, Alan Smith (2015). Value proposition design: how to create products and services customers want. John Wiley & Sons.

. Software is sometimes the central means to providing that value, but more often than not, it’s the  information  flowing through that software that’s the truly valuable piece.  Requirements , which we will discuss in a later chapter, help engineers organize how software will provide value.

The individuals in a software organization take on different roles to achieve that value. These roles are sometimes spread across different people and sometimes bundled up into one person, depending on how the organization is structured, but the roles are always there. Let’s go through each one in detail so you understand how software engineers relate to each role.

  • Marketers  look for opportunities to provide value. In for-profit businesses, this might mean conducting market research, estimating the size of opportunities, identifying audiences, and getting those audiences attention. Non-profits need to do this work as well in order to get their solutions to people, but may be driven more by solving problems than making money.
  • Product  managers decide what value the product will provide, monitoring the marketplace and prioritizing work.
  • Designers  decide  how  software will provide value. This isn’t about code or really even about software; it’s about envisioning solutions to problems that people have.
  • Software engineers  write code with other engineers to implement requirements envisioned by designers. If they fail to meet requirements, the design won’t be implemented correctly, which will prevent the software from providing value.
  • Sales  takes the product that’s been built and try to sell it to the audiences that marketers have identified. They also try to refine an organization’s understanding of what the customer wants and needs, providing feedback to marketing, product, and design, which engineers then address.
  • Support  helps the people using the product to use it successfully and, like sales, provides feedback to product, design, and engineering about the product’s value (or lack thereof) and its defects.

As I noted above, sometimes the roles above get merged into individuals. When I was CTO at AnswerDash, I had software engineering roles, design roles, product roles, sales roles,  and  support roles. This was partly because it was a small company when I was there. As organizations grow, these roles tend to be divided into smaller pieces. This division often means that different parts of the organization don’t share knowledge, even when it would be advantageous 3 3

Parmit K. Chilana, Amy J. Ko, Jacob O. Wobbrock, Tovi Grossman, and George Fitzmaurice (2011). Post-deployment usability: a survey of current practices. ACM SIGCHI Conference on Human Factors in Computing (CHI).

.

Note that in the division of responsibilities above, software engineers really aren’t the designers by default. They don’t decide what product is made or what problems that product solves. They may have opinions—and a great deal of power to enforce their opinions, as the people building the product—but it’s not ultimately their decision.

There are other roles you might be thinking of that I haven’t mentioned:

  • Engineering managers  exist in all roles when teams get to a certain size, helping to move information from between higher and lower parts of an organization. Even  engineering  managers are primarily focused on organizing and prioritizing work, and not doing engineering  5 5

    Eirini Kalliamvakou, Christian Bird, Thomas Zimmermann, Andrew Begel, Robert DeLine, Daniel M. German (2017). What makes a great manager of software engineers?. IEEE Transactions on Software Engineering.

    . Much of their time is also spent ensuring every engineer has what they need to be productive, while also managing coordination and interpersonal conflict between engineers.
  • Data scientists , although a new role, typically  facilitate  decision making on the part of any of the roles above  1 1

    Andy Begel, Thomas Zimmermann (2014). Analyze this! 145 questions for data scientists in software engineering. ACM/IEEE International Conference on Software Engineering.

    . They might help engineers find bugs, marketers analyze data, track sales targets, mine support data, or inform design decisions. They’re experts at using data to accelerate and improve the decisions made by the roles above.
  • Researchers , also called user researchers, also help people in a software organization make decisions, but usually  product  decisions, helping marketers, sales, and product managers decide what products to make and who would want them. In many cases, they can complement the work of data scientists,  providing qualitative work to triangulate quantitative data .
  • Ethics and policy specialists , who might come with backgrounds in law, policy, or social science, might shape terms of service, software licenses, algorithmic bias audits, privacy policy compliance, and processes for engaging with stakeholders affected by the software being engineered. Any company that works with data, especially those that work with data at large scales or in contexts with great potential for harm, hate, and abuse, needs significant expertise to anticipate and prevent harm from engineering and design decisions.

Every decision made in a software team is under uncertainty, and so another important concept in organizations is  risk 2 2

Boehm, B. W (1991). Software risk management: principles and practices. IEEE Software.

. It’s rarely possible to predict the future, and so organizations must take risks. Much of an organization’s function is to mitigate the consequences of risks. Data scientists and researchers mitigate risk by increasing confidence in an organization’s understanding of the market and its consumers. Engineers manage risk by trying to avoid defects. Of course, as many popular outlets on software engineering have begun to discover, when software fails, it usually “did exactly what it was told to do. The reason it failed is that it was told to do the wrong thing. 10 10

James Somers (2017). The coming software apocalypse. The Atlantic Monthly.



Open source communities are organizations too. The core activities of design, engineering, and support still exist in these, but how much a community is engaged in marketing and sales depends entirely on the purpose of the community. Big, established open source projects like  Mozilla  have revenue, buildings, and a CEO, and while they don’t sell anything, they do market. Others like Linux 6 6

Gwendolyn K. Lee, Robert E. Cole (2003). From a firm-based to a community-based model of knowledge creation: The case of the Linux kernel development. Organization science.

 rely heavily on contributions both from volunteers 11 11

Yunwen Ye and Kouichi Kishida (2003). Toward an understanding of the motivation Open Source Software developers. ACM/IEEE International Conference on Software Engineering.

, but also paid employees from companies that depend on Linux, like IBM, Google, and others. In these settings, there are still all of the challenges that come with software engineering, but fewer of the constraints that come from a for-profit or non-profit motive. In fact, recent work empirically uncovered 9 reasons why modern open source projects fail: 1) lost to competition, 2) made obsolete by technology advances, 3) lack of time to volunteer, 4) lack of interest by contributors, 5) outdated technologies, 6) poor maintainability, 7) interpersonal conflicts amongst developers, 8) legal challenges, 9) and acquisition 4 4

Jailton Coelho and Marco Tulio Valente (2017). Why modern open source projects fail. ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE).

. Another study showed that funding open source projects often requires substantial donations from large corporations; most projects don’t ask for donations, and those that do receive very little, unless well-established, and most of those funds go to paying for basic expenses such as engineering salaries 9 9

Cassandra Overney, Jens Meinicke, Christian Kästner, Bogdan Vasilescu (2020). How to not get rich: an empirical study of donations in open source. ACM/IEEE International Conference on Software Engineering.

. Those aren’t too different from traditional software organizations, aside from the added challenges of sustaining a volunteer workforce.

All of the above has some important implications for what it means to be a software engineer:

  • Engineers are not the only important role in a software organization. In fact, they may be less important to an organization’s success than other roles because the decisions they make (how to implement requirements) have smaller impact on the organization’s goals than other decisions (what to make, who to sell it to, etc.).
  • Engineers have to work with  a lot  of people working with different roles. Learning what those roles are and what shapes their success is important to being a good collaborator 7 7

    Paul Luo Li, Amy J. Ko, and Andrew Begel (2017). Cross-disciplinary perspectives on collaborations with software engineers. International Workshop on Cooperative and Human Aspects of Software Engineering.

    .
  • While engineers might have many great ideas for product, if they really want to shape what they’re building, they should be in a product role, not an engineering role.

All that said, without engineers, products wouldn’t exist. They ensure that every detail about a product reflects the best knowledge of the people in their organization, and so attention to detail is paramount. In future chapters, we’ll discuss all of the ways that software engineers manage this detail, mitigating the burden on their memories with tools and processes.

References

  1. Andy Begel, Thomas Zimmermann (2014). Analyze this! 145 questions for data scientists in software engineering. ACM/IEEE International Conference on Software Engineering.

  2. Boehm, B. W (1991). Software risk management: principles and practices. IEEE Software.

  3. Parmit K. Chilana, Amy J. Ko, Jacob O. Wobbrock, Tovi Grossman, and George Fitzmaurice (2011). Post-deployment usability: a survey of current practices. ACM SIGCHI Conference on Human Factors in Computing (CHI).

  4. Jailton Coelho and Marco Tulio Valente (2017). Why modern open source projects fail. ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE).

  5. Eirini Kalliamvakou, Christian Bird, Thomas Zimmermann, Andrew Begel, Robert DeLine, Daniel M. German (2017). What makes a great manager of software engineers?. IEEE Transactions on Software Engineering.

  6. Gwendolyn K. Lee, Robert E. Cole (2003). From a firm-based to a community-based model of knowledge creation: The case of the Linux kernel development. Organization science.

  7. Paul Luo Li, Amy J. Ko, and Andrew Begel (2017). Cross-disciplinary perspectives on collaborations with software engineers. International Workshop on Cooperative and Human Aspects of Software Engineering.

  8. Alexander Osterwalder, Yves Pigneur, Gregory Bernarda, Alan Smith (2015). Value proposition design: how to create products and services customers want. John Wiley & Sons.

  9. Cassandra Overney, Jens Meinicke, Christian Kästner, Bogdan Vasilescu (2020). How to not get rich: an empirical study of donations in open source. ACM/IEEE International Conference on Software Engineering.

  10. James Somers (2017). The coming software apocalypse. The Atlantic Monthly.

  11. Yunwen Ye and Kouichi Kishida (2003). Toward an understanding of the motivation Open Source Software developers. ACM/IEEE International Conference on Software Engineering.