6 The Kernel of Software Engineering

The goal of this chapter is to present the complete Essence kernel with its areas of concern, all its alphas with states and their relationships, its activity spaces, its competencies and to show how the pattern construct can be used for different subjects. Thus, the complete picture is now presented and a map within the whole range of aspects of things to work with, things to do and so on, is given. As a map is always an abstraction from the whole set of details this chapter will provide the bird’s eye view.  Thus, the reader will attain the ability to

  • identify the basic organization of the kernel based on the three areas of concern
  • identify how the various alphas are related to these areas of concerns
  • identify all the alphas of the kernel
  • identify all the activity spaces in the kernel
  • explain at an abstract level, how activities relate to progress with respect to the alpha states
  • identify all the competencies
  • explain why competencies are important in software development
  • explain at an abstract level how patterns can be used

Now once this is mastered the entire picture of software development is given through the lens of Essence.

In the previous chapter, we have introduced the Essence language of software engineering through a simple programming practice. In this chapter, we summarize the elements of software engineering as viewed through the lens of Essence.

6.1 Organizing Essence Kernel

Essence takes a structured approach in organizing the elements of software engineering. Firstly, it organizes elements into three discrete areas of concern, each focusing on a specific dimension of software development as portrayed in Figure 24.

  • Customer This area of concern contains everything to do with the actual use and exploitation of the software system to be produced.
  • Solution - This area of concern contains everything related to the specification and development of the software system.
  • Endeavor - This area of concern contains everything related to the development team and the way that they approach their work.

Figure 24       The three areas of concern

The kernel elements are fundamentally of four kinds:

  1. The essential things to work with – the alphas
  2. The essential things to do – the activity spaces
  3. The essential capabilities needed – the competencies
  4. The essential arrangements of elements – the patterns.

Finding the right elements is crucial. They must be universally acceptable, significant, relevant and guided by the notion that, “You have achieved perfection not when there is nothing left to add, but when there is nothing left to take away.”[1] When identifying kernel elements, we took a conservative approach. We adopted only elements we were sure would always be found in all types of software system endeavors. For instance, the element architecture was discussed as a kernel element. The opinion was that while for many systems it is critical to identify an architecture there are many simpler systems where architecture is not an issue. Thus, since it is not common to all endeavors, architecture is not a concern that every endeavor has to face, it didn’t qualify as a kernel element.

6.2 The Essential Things to Work With: The Alphas

While it is true that Essence attempts to provide a concise list of words commonly found in software engineering, it is not a static dictionary that you read. Essence is dynamic and actionable. By that we mean you can make it come alive. You can use it in real situations to run an endeavor (e.g. a project). It includes the essential elements prevalent in every endeavor such as Requirements, Software System, Team, and Work. These elements have states representing progress and health, so as the endeavor moves forward, these elements progress from state to state. The alphas in Essence are shown in Figure 25.

Figure 25       The Essence alphas and their relationships

In the Customer area of concern the team needs to understand the stakeholder needs and the opportunity to be addressed:

  • Opportunity – The set of circumstances that makes it appropriate to develop or change a software system. The opportunity articulates the reason for the creation of the new, or changed, software system. It represents the team’s shared understanding of the stakeholder’s needs and helps shape the requirements for the new software system by providing justification for its development.
  • Stakeholders – The people, groups, or organizations that affect or are affected by a software system. The stakeholders provide the opportunity and are the source of the requirements and funding for the software system. The team members are also stakeholders. As much stakeholder involvement as possible throughout a software engineering endeavor is important to support the team and ensure that an acceptable software system is produced.

In the Solution area of concern the team needs to establish a shared understanding of the requirements, and implement, build, test, deploy and support a software system that fulfills them:

  • Requirements – What the software system must do to address the opportunity and satisfy the stakeholders. It is important to discover what is needed from the software system, share this understanding among the stakeholders and the team members, and use it to drive the development and testing of the new system.
  • Software System – A system made up of software, hardware, and data that provides its primary value by the execution of the software. The primary product of any software engineering endeavor, a software system can be part of a larger software, hardware, business or social system solution.

In the Endeavor area of concern the team and its way-of-working have to be formed, and the work has to be done:

  • Team – A group of people actively engaged in the development, maintenance, delivery and support of a specific software system. The team plans and performs the work needed to create, update and/or change or retire the software system.
  • Work Activity involving mental or physical effort done in order to achieve a result. In the context of software engineering, work is everything that the team does to meet the goals of producing a software system matching the requirements, and addressing the opportunity, presented by the stakeholders. The work is guided by the practices that make up the teams way-of-working.
  • Way-of-Working The tailored set of practices and tools used by a team to guide and support their work. The team evolves their way of working alongside their understanding of their mission and their working environment. As their work proceeds they continually reflect on their way of working and adapt it as necessary to their current context.

All alphas are vital during software engineering endeavors. Thanks to their states, one may think that they may live their own lives. This is partly true. All alphas are related to one another and they complement each other by considering their own aspects of the software engineering endeavor.

As illustrated in Figure 25, the relationships are shown with arrows and there are many of them. They may be read as follows. Stakeholders provide Opportunity, which then helps to identify Requirements and focuses on the most important ones. These Requirements are then fulfilled by implementing a Software System. The Software System implementation addresses the Opportunity and it is used and consumed by Stakeholders.

  • It is the Team that produces the Software System by doing Work. The Work is set up to address Opportunity and it implies updating and changing the Software System. Work is guided by a Way of Working that is applied by Team while performing its Work. The Team is continuously supported by Stakeholders who provide feedback about the Software System to the team.
  • The OMG standard defines the states for each kernel alpha as shown in Figure 26. The details of each state can be found in the Essence standard, and we will not go deeper into each of them in this chapter. You should later be able to download them from the web site of the book.

Figure 26       Essence kernel alpha states

So, you now are equipped with knowledge of things to work with that will in the following two sections be related to things you do and the competencies that are needed.

6.3 The Essential Things to Do: The Activities

In every software development endeavor, you carry out a number of activities. Examples of activities include: agreeing on a user story with a product owner, demonstrating the system to a customer representative, and estimating work. Essence as such does not define any activities (how your team goes about capturing and communicating the requirements can be very different from team to team). However, Essence defines a number of so-called activity spaces. You can think of activity spaces as generic, that is method-independent, placeholders for specific activities that will be added later, on top of the kernel.  Since the activity spaces are generic they could be standardized and are thus part of the Essence standard. The activity spaces are packages used to group activities, which are related to one another.  This helps to keep them organized which in turn makes them easier for practitioners to find and use when they face common challenges where a practice can help.

The following activity spaces provide guidance to you and your team on the things to do in each of the three areas of concern as the team marches towards achieving the states specified in the alphas. The activity spaces of Essence are shown in Figure 27 where they are denoted by dashed arrowed pentagons and organized into the areas of concern.

Figure 27       Essence activity spaces (from the Essence standard)

Activity space cards have very similar contents as activity cards. Figure 28 shows an example of an activity space card. Going back to the simple programming example in the previous chapter the Write Code activity would be allocated to the Implement the System activity space. The dashed symbol in the upper left-hand corner of the card is the symbol for an activity space.

Figure 28       Implement the System activity space card

In addition to being placeholders for specific activities, the activity spaces represent the essential things that have to be done to develop software. They provide general descriptions of the challenges a team faces when developing, maintaining and supporting software systems, and the kinds of things that the team will do to meet them. Each activity space then can via the practices be extended with concrete activities that progress one or more alphas – either alphas in the kernel or practice specific alphas.

In the top row of Figure 27 there are activity spaces to understand the opportunity, and support and involve the stakeholders:

  • Explore Possibilities – Explore the possibilities presented by the creation of a new or improved software system. This includes the analysis of the opportunity and the identification of the stakeholders.
  • Understand Stakeholder Needs – Engage with the stakeholders to understand their needs and ensure that the right results are produced. This includes identifying and working with the stakeholder representatives to progress the opportunity.
  • Ensure Stakeholder Satisfaction – Share the results of the development work with the stakeholders to gain their acceptance of the system produced and verify that the opportunity has been addressed.
  • Use the System – Observe the use the system in a live environment and how it benefits the stakeholders.

In the middle row of Figure 27 there are activity spaces to develop an appropriate solution to exploit the opportunity and satisfy the stakeholders:

  • Understand the Requirements – Establish a shared understanding of what the system to be produced must do.
  • Shape the system – Shape the system so that it is easy to develop, change and maintain, and can cope with current and expected future demands. This includes the architecting and overall design of the system to be produced.
  • Implement the System – Build a system by implementing, testing and integrating one or more system elements. This includes bug fixing and unit testing.
  • Test the System – Verify that the system produced meets the stakeholders’ requirements.
  • Deploy the System – Take the tested system and make it available for use outside the development team.
  • Operate the System – Support the use of the software system in the live environment.

In the bottom row of Figure 27 there are activity spaces to form a team and to progress the work in line with the agreed way-of-working:

  • Prepare to do the Work – Set up the team and its working environment. Understand and commit to completing the work.
  • Coordinate Activity – Co-ordinate and direct the team’s work. This includes all on-going planning and re-planning of the work, and re-shaping of the team.
  • Support the Team – Help the team members to help themselves, collaborate and improve their way of working.
  • Track Progress – Measure and assess the progress made by the team.
  • Stop the Work – Shut-down the software engineering endeavor and handover of the team’s responsibilities.

To make Figure 27 easy to read the activity spaces are shown in a sequence from left to right in each row. The sequence indicates the order in which things are finished and not necessarily the order in which they are started. For example, you can start shaping the system before you have finished understanding the requirements, but you can’t be sure you have finished shaping the system until you have finished understanding the requirements.

6.4 Competencies

To participate in a software endeavor you need to have competency in different areas. You need competency relevant to the specific tasks you are working on, but also other competencies to understand what your teammates are working on.

Competencies are defined in the kernel and can be thought of as generic containers for specific skills. Specific skills, for example Java programming, are not part of the kernel because this skill is not essential on all software engineering endeavors. But competency is always required, and it will be up to the individual teams to identify the specific skills needed for their particular software endeavor.

A common problem on software endeavors is not being aware of the gap between the competencies needed and the competencies available. The kernel approach will raise the visibility of this gap. Each of the competencies has a competency level, and this is the same across all of the kernel competencies. Thus, the competency levels that we presented in 5.3 are the same for all six competencies in Figure 29.

Figure 29       The Kernel Competencies

In the Customer area of concern the team has to be able to demonstrate a clear understanding of the business and technical aspects of their domain and have the ability to accurately communicate the views of their stakeholders. This requires the following competencies to be available to the team:

  • Stakeholder Representation - This competency encapsulates the ability to gather, communicate, and balance the needs of other stakeholders, and accurately represent their views.

In the Solution area of concern the team has to be able to capture and analyze the requirements and build and operate a software system that fulfills them. This requires the following competencies to be available to the team:

  • Analysis - This competency encapsulates the ability to understand opportunities and their related stakeholder needs, and to transform them into an agreed upon and consistent set of requirements.
  • Development - This competency encapsulates the ability to design, program and code effective and efficient software systems following the standards and norms agreed upon by the team.
  • Testing - This competency encapsulates the ability to test a system, verify that it is usable and that it meets the requirements.

In the Endeavor area of concern the team has to be able to organize itself and manage its workload. This requires the following competencies to be available to the team:

  • Leadership - This competency enables a person to inspire and motivate a group of people to achieve a successful conclusion to their work and to meet their objectives.
  • Management - This competency encapsulates the ability to coordinate, plan and track the work done by a team.

6.5 Patterns

Patterns are generic solutions to typical problems. In our daily life, we see patterns every day. In a classroom, we expect to have the teacher in front, with rows of desks and chairs for students. Such a pattern is designed for the teacher to transmit knowledge as efficiently as possible. Some classrooms, are designed such that students are arranged in circles for greater discussion and discovery. In this case, the arrangement (i.e. pattern) is for the purpose of enhancing collaboration between students. Both are examples of patterns. There are many different kinds of patterns in software engineering. Examples of typical problems include how to assign responsibilities associated with work assignments to team members. This problem can be solved by defining a role pattern, such as the Product Owner role, which defines the responsibilities of a product owner. Other examples include how to set objective checkpoints, how to conduct analysis and design. These are guidelines you may decide to give when describing a practice. Patterns are optional elements (not required element of a practice definition) that may be associated with any other language element. In Figure 30 we see a pattern named Student Pairs with a brief description. This pattern could be used to help two students who are working closely together on the same piece of code.

Figure 30       Student Pair pattern card

6.5.1 Roles

Many pieces of work require more than one competency. For instance, when developing software, you need more than just a Development competency. You also need to know how to find out what to develop, which requires an Analysis competency.  Thus, when assigning an individual this work, he/she needs to have these competencies so a common solution to the problem of assigning work is to define a special kind of pattern, which does that. We call such a pattern a role. A role requires a minimum level of each competencies to do the job effectively. 

When someone is given a role she/he should have the needed competencies to succeed. If not, he/she should be offered additional training and coaching to carry out the role successfully.

Figure 31       Coder role (pattern) card

6.5.2 The Checkpoint Pattern

As another example, a pattern could be used to define a checkpoint. A checkpoint is a set of criteria to be achieved at a specific point in time in a software engineering endeavor. They are frequently used within all types of software engineering endeavors. Checkpoints are key points in the life cycle of a software endeavor where an important decision is to be taken. They are used by organizations as points where they stop and think about whether it makes sense to proceed or not dependent on if there is or isn’t faith in what has been done. A decision could also be made to move forward but to go in a different direction because of a known problem or risk.

A checkpoint is simply expressed by a set of alpha states that must have been achieved in order to pass the checkpoint. This pattern can be reused for other similar endeavors trying to get to the same checkpoint.

In the simplest case, the timeline of a software engineering endeavor can be divided into three phases, pre-development, development, and post-development (see Figure 32).  In the figure, the rounded boxes are states associated with the alphas.

Figure 32       Checkpoint pattern example

Pre-development could be defined as the phase when the team and the stakeholders are determining what they need to do. The end of the pre-development phase needs to include any essential items that need to be achieved before the team can begin developing the software system. One example might be funding approval to proceed with development.

Development is the phase when the team works collaboratively to create a software system that addresses the agreed needs and objectives. The phase ends when the software has reached a point where it can be used by real users.

Post-development is the phase when the software system is delivered to the user community and the software is used with real clients.

In this example, there are two checkpoints, namely: ready for development and development complete. The criteria for these two checkpoints are expressed using alpha states. For example, the Ready for Development checkpoint is defined as Requirements: Bounded (an abbreviation of Requirements alpha: Bounded state), and Software System: Architecture Selected, and the Development Complete checkpoint is defined as Requirements: Addressed, and Software System: Ready (for production).

Note that different kinds of development will define the checkpoints differently. For example, a software engineering endeavor that has life-threatening consequences should it fail will have different checkpoints for exploratory development (for instance developing a new product) than for maintenance (updates after delivery of the first version of the system).

Figure 33 provides a simple example of a checkpoint pattern in our programming practice. In this simple programming assignment, we just need the Software System to be demonstrable.

Figure 33       A checkpoint pattern card

What should you now be able to accomplish?

After studying this chapter, you should be able to

  • explain the kernel and its three areas of concerns
  • explain how the various alphas are related to each other and mapped onto the areas of concern
  • identify the relationship between alphas and alpha states
  • explain in principle how progress is made by performing activities
  • identify the competencies
  • explain how patterns can be used to express frequently occurring combinations of Essence elements (alphas, activities etc.) in software development

Now the basic picture of software development is sketched completely. More detail will come in later chapters of this book.  and the concrete “instantiation” of these elements in a particular software development endeavor is the application of these.


[1] Antoine de Saint-Exupéry