4 Identifying the Key Elements of Software Engineering

The goal of this chapter is to present the key element type of Essence – the alpha – the things we work with when developing software. The reader will attain the ability to

  • identify the alphas that help in delivering value to a customer
  • identify the alphas that deliver value through a solution
  • identify the alphas that help with the endeavor.
  • explain at an abstract level, how each alpha is important for software development

This knowledge helps to layout a model of software engineering with areas of concern, key elements, etc..


4.1 Getting to the Basics

Essence was developed to help people like Smith and companies like TravelEssence who rely heavily on software to run their business. After Smith had been working in the software industry for several years he had his fair share of ups and downs. He wished there were more ups than downs. Without doubt, software development is a creative process, but Smith had come to recognize that there are some fundamental basics – some things to be mindful of to avoid making unnecessary mistakes.

Smith’s colleagues also recognized that, but they had difficulty articulating these fundamentals due to their different backgrounds, experience and consequently, the different terms they used. It seemed that every time a new team was formed, members had to go through a storming and norming process to iron out these terms before starting to deal with the challenges.

If you have been in the software industry for some time, you can empathize with Smith. For new students to software engineering, we want you to appreciate the complexities of a software engineering endeavor as you read this chapter and compare that against the complexities of your class or project assignments that you have worked on.

What the contributors of Essence did was to lay down the foundation of software engineering for folks like Smith and yourself to cut short this time of starting up period, and to ensure health and speed as your software engineering endeavors progress.

Figure 15       The things involved in all software engineering endeavors.

Let’s begin with some commonly used terms we find in software engineering, which we will briefly introduce in italics. Regardless of size and complexity, all software engineering endeavors involve the following things (see Figure 15):

There are customer needs to be met

  • Someone has a problem or opportunity to address
  • There are stakeholders who use and/or benefit from the solution produced and some will fund the endeavor.

There is a solution to be delivered

  • There are certain requirements to be met
  • A software system of one form or another will be developed

There is an endeavor to be undertaken

  • The work must be initiated
  • Form an empowered team of competent people
  • With an appropriate way of working

These terms map out what software engineering is about. When something goes wrong, it normally has to do with one or more of these things. The way we handle these things has a direct impact on the success or failure of the endeavor. We will now look at each of these things in turn. Later in chapter 6 we will once more discuss these and their relationships.

4.2 Software Engineering is about Delivering Value to Customers

Firstly, software engineering is about delivering value to customers. This value can be in improvements to existing capabilities or providing new capabilities that are needed by the customer. In TravelEssence, customers are its users. They can be travelers, or travel agents who make reservations on behalf of actual travelers. Different customers would have different needs. If the endeavor does not deliver value, then it is a waste of time and money. As the saying goes, life is too short to build something that nobody wants!

4.2.1 Opportunity

Every endeavor is an opportunity for success or failure. It is therefore very important to understand what the opportunity is about. Perhaps you have heard of Airbnb. Airbnb started out with two poor guys, Brian Chesky and Joe Gebbia, who were struggling to pay rent. So they came up with the idea of renting out three airbeds on their living-room floor and providing their guests with breakfast. It turned out that during that time, there was an event going on in the city and many participants weren’t able to book accommodations. Brian and Joe thought to themselves, perhaps they were hitting on something. To cut the story short, Airbnb became a 1.3 billion US dollar business in 2016. That was indeed one big spectacular opportunity. But really, it was a series of opportunities, each leading to the next one as Airbnb grew.

However, not all businesses are like that. In fact, far more companies do not make it, and miss many opportunities. In fact, there has been a 90% failure rate for start-ups[1]. Many successful companies become failures due to missed opportunities. Thus, understanding opportunities is critical.

An opportunity is a chance to do something including fixing an existing problem. It refers to a set of circumstances to do something, which in our context involve building or enhancing some software to meet some need. Regardless of what the opportunity is about, it can either succeed or fail. It can deliver real value, or it could be something that nobody wants.

As an example, in our TravelEssence case, it was found that customers like to engage travel staff because the latter can make recommendations to them. The opportunity is that if TravelEssence can provide recommendations online through a software solution, it can provide better service to customers and thereby shorten the time customers need to make a purchase decision. Of course, whether this opportunity is truly viable depends on many factors.

Thus, when working with an opportunity, it is important to continually evaluate the viability of the opportunity as it gets implemented.

  • When the opportunity is first conceived, some due diligence is necessary to determine if it truly addresses a real need or a real new idea that customers are willing to pay money for.
  • It would likely be the case that different solution options are available to address the opportunity, and some difficult choices will have to be made.
  • When the solution goes live, it normally takes some time before the benefits become visible to customers.

4.2.2 Stakeholders

For opportunities to be taken up, there must be some people involved in the decision. We call those individuals, organizations or groups, stakeholders. Stakeholders always have some interest or concern either in the system to be developed or they are interested in the software engineering endeavor itself; the former being external stakeholders and the latter internal stakeholders. They can either affect or be affected by the system or the endeavor. In our TravelEssence case, internal stakeholders include a software development team assembled to develop the new services for travelers, along with key managers in the organization who need to agree to the new venture. Examples of external stakeholders being affected by the system include a manager in our TravelEssence organization who needs to agree to fund the new software effort, or a traveler who might benefit by using the new services.

One of the biggest challenges in a software engineering endeavor is getting stakeholders to agree. Before even agreeing, they must first be involved in some way. The worst thing that could happen is that when all is said and done, someone says, “this is not what we want.” This happens too often.

Thus, it is really important early in the endeavor to:

  • Understand who the stakeholders are, what their concerns are.
  • Ensure that they are adequately represented and involved in the process.
  • Ensure that they are satisfied with the evolving solution.

4.3 Software Engineering Delivers Value through a Solution

What sets a software engineering endeavor apart from other endeavors is that the solution that addresses the opportunity is via a good piece of software. Nobody wants a poor-quality product. Customers’ needs are ever evolving, so the solution needs to evolve as well, and for that to happen it has to be extensible. This applies to both the requirements of the solution, and the software system that realizes the requirements.

In TravelEssence, the requirements for the solution cover different usage scenarios for different kinds of customers (e.g. new travelers, frequent travelers, corporate travelers, agents, etc.). The software system involves a mix of mobile applications, and a cloud-based backend.

4.3.1 Requirements

Requirements provide the stakeholder view of what they expect the software system to provide. They indicate what the software system must do, but do not explicitly express how it must do it. They identify the value of the system in respect to the opportunity and indicate how the opportunity will be pursued via the creation of a new software system.

As such, requirements serve like a medium between the opportunity and the software system. Among the biggest challenges software teams face are changing requirements. Usually, there is more than one stakeholder in an endeavor, and stakeholders will of course have different and even conflicting preferences and opinions. Even if there is only one stakeholder, he/she might have different opinions at different times. Moreover, the software system will evolve together with the requirements. What we see affects what we want. Thus, requirements changes are inevitable because the team’s understanding of what’s needed will evolve. What we want to prevent is unnecessary miscommunication and misunderstanding, which Smith did encounter in TravelEssence when working on a discount program. The team had thought that this enhancement would be very simple. However, the stakeholders have different ideas on, for instance. how long the program should last, which group of users the discount program should focus on, the impact of the discount program on reservation rates. They had wanted to launch the discount program within a month’s time, yet there was very much debate even to the very last hour.

Thus, how a team works with requirements is absolutely crucial:

  • Ensure that requirements are continually anchored to the opportunity.
  • Organize the requirements in a way that facilitates understanding. Resolve conflicting requirements.
  • Ensure that the requirements are testable, i.e. one can verify that the software system does indeed fulfill the requirements without ambiguity.
  • Use the requirements to drive the development of the software system. In fact, code needs to be well structured and easy to relate back to the requirements. Progress is measured in terms of how much of the requirements have been completed.

4.3.2 Software System

The primary outcome of a software endeavor is of course the software system itself. Software systems come in different forms. It could be the app running on your mobile phone; it could be embedded in your air-conditioner; it could help you register for your undergraduate program; it could tally election votes. It could run on a single machine or be distributed across a vast network as in the Internet today.

Whatever the case, there are three important characteristics of software systems necessary before it is of value to users and stakeholders, namely: functionality, quality and extensibility.

The need to have a functionality is obvious. Software systems are designed and built to make the lives of humans easier. They each must serve some function, which are derived from the software system’s requirements.

The need for quality is easy to understand. Nobody likes a software system that is of poor quality. You do not want your word processor to crash when you are finishing your report, especially if you have not saved your work. You want your social media posts to be instantaneous. Thus, quality attributes like reliability and performance are important. Other qualities like ease of use, a rich user experience, are becoming more important as software systems are becoming more ubiquitous. Of course, the extent of quality needed depends on the context itself. This again can be derived from the software system’s requirements.

The third characteristic is that of being extensible. It can be said that this is another aspect of quality, but we want to call this out separately. Software engineering is about changing and evolving the software system, from one version to the next, giving it more and more functionality to serve its users. This evolution occurs over time as a series of increments of more functionality, where every increment is described by more requirements. As mentioned earlier, Smith’s job in TravelEssence is about introducing changes to the existing travel reservation system. Every now and then, TravelEssence would introduce new discount programs, membership subscription incentives, integrate with new accommodation providers, etc.

There are several important aspects of this evolution. Firstly, it is not merely hacking or patching code into the software system. Otherwise, as the software system grows in size, it will be harder to add new functionality. Consequently, teams often organize software systems into interconnecting parts know as components. Each component realizes part of the requirements and has a well-defined scope and interface. As a student, the lessons you will learn about object orientation, etc. are about organizing the software system into manageable components.

Secondly, code needs to be well structured and easy to relate back to the requirements. Just as the requirements will evolve, the software system needs to be extensible to such changes.

Thirdly, the evolution involves transitioning the software system across different environments, from the developers’ machine, to some test environment, to what is known as the production environment, where actual users will be using the software system. It is not unusual to find that software working on the developer’s machine will have defects (or bugs) in the test or production environment. Many senior developers get irritated when they hear novices say: “But it works on my machine.” A developer’s job is not done until it works well in the production environment. A quality software system must:

  • Have a design that is a solution to the problem and agreed to.
  • Have critical interfaces demonstrated.
  • Be usable adding value to stakeholders.
  • Have operational support in place.

4.4 Software Engineering is also about Endeavors

An endeavor is any action that we take with the purpose of achieving an objective, which in our case is delivering value according to the opportunity and satisfying stakeholders. Within software engineering, an endeavor involves a conscious and concerted effort of developing software from the beginning to the end. It is a purposeful activity conducted by a software team that achieves its goals by doing work according to the particular way of working.

4.4.1 Team

Software engineering involves the application of many diverse competencies and skills in a manner similar to a sport team.

Good teamwork is essential for high performance. It creates a synergy, where the combined effect of the team is much greater than the sum of individual efforts.

But getting to a high-performance state does not come naturally; instead it results from a deliberate attempt to succeed.    

A team typically involves several people and team effectiveness have a profound effect upon any software engineering endeavor. Without the team there will be no software. To obtain a high level of performance the team members should reflect on the way they work together and their focus on the team goal.

Teams need to:

  • Be formed with enough people to start the work.’
  • Be composed of personnel possessing the right competencies/skills
  • Work together in a collaborative way.
  • Continuously adapt to changing environment.

When working in TravelEssence, Smith belonged to a development team. Although, members of Smith’s team had slightly different skill set, they collaborated to achieve the team’s goal together. Smith was in particular more focused on backend technologies (i.e. the part of the software system running on the cloud), whereas Grace, fellow colleague focused on front-end JavaScript and React Native technologies. Since these technologies are not the emphasis of this book, you don’t need to know them. Moreover, technologies change rather quickly. What we want to emphasize is that effective teams have to address the Opportunity presented to them to satisfy Stakeholders.

4.4.2 Work

A team comes together to do the work of bringing the opportunity to reality in the software system that they build. This work involves some effort and the purpose of it is to achieve some goal. In general, there is a limited amount of time to reach the goal. The idea is to get things done fast but with high quality. The team members must be able to prepare, coordinate, track, and complete (stop) their work. This has a profound effect upon meeting commitments and delivering value to stakeholders. Thus, the team members must understand how to perform their work and recognize if the work is progressing in a satisfactory manner.

Doing the work involves:

  • Getting prepared
  • Communicating the work to be done
  • Ensure that progress and risk are under control.
  • Getting closure to the work

In TravelEssence, Smith and his team manage their work through a backlog. The backlog is a list of things to do, which originated from requirements. They communicate regularly with their Stakeholders to make sure that their backlog is accurate, up-to-date, representing the Stakeholders’ priorities. In this way, they can focus on getting the right things done.

4.4.3 Way of Working

A team can perform the work in different ways and this may lead to different results. It can be performed in an ad hoc manner meaning that you decide how to work while doing the work. For instance, while cooking a soup, you may not follow any recipes. You decide on the fly which ingredients to use and in what order to mix and cook them together. When following an ad hoc way of working the result may or may not be of high quality. This depends on many factors, among them, the skill of the people involved and the number of people involved in the process.

All of us are acquainted with the saying too many cooks spoil the broth. If too many cooks cook the soup in an ad hoc manner, the soup will taste awful. Translating the analogy to software, if too many people participate in agreeing on how to do the job, the job will probably not be done well. There are many reasons for this. One of them is that each person has his/her own idea of how to conduct the job and often, they do not work in an orchestrated manner. Smith’s team addresses this issue by regularly looking at their prioritized backlog. They made sure that they correctly understood the scope of each item in the backlog, checking with Stakeholders and getting feedback from them. Smith’s team regularly examined their method, or in other words, their way of working. If things did not seem right, they made changes.

It is therefore important for team members to come into some kind of consensus regarding their way of working. Disagreements about the way of working are significant barriers to team performance. You would think that coming to an agreement is easy. The truth of the matter is that it is not.

At the small scale, within a single team, there is still a need for members to agree on the foundations and principles, followed by specific practices and tools. This would of course need to be adapted towards the team’s context, and evolve as the environment and needs change.

The way of working must:

  • Include a foundation of key practices and tools.
  • Be used by all the team members.
  • Be improved by the team when needed.

In an industrial scale, one of the things we hope to achieve through Essence is simplifying the process of reaching a common agreement. We do this at an industrial scale by identifying a common ground or a kernel and to have a way to deal with diversity of approaches. In the subsequent chapters, we will discuss the approach taken by Essence in greater detail. In this chapter, we have introduced the terms: opportunity, stakeholders, requirements, software system, team, work, way of working in a layman’s language. Essence will give these terms greater rigor and provide guidance to put software teams on a stronger foundation to achieve their goals.

What should you now be able to accomplish?

After studying this chapter, you should be able to

  • name the various alphas
  • identify the alpha, their purpose and identify the relationship between related alphas
  • explain the purpose of each alpha at an abstract level
  • explain the purpose of each relationship between related alphas

An understanding of the main core of Essence is gained. This core provides important knowledge to guide a software development effort at an abstract level. Still many details are not discussed yet.