21 Essentializing Practices

The first dimension of scaling as presented in chapter 20 is about zooming into details and providing the necessary explicit guidance to practitioners.  Earlier in part 3, we have demonstrated how teams in TravelEssence took reusable practices, adapted and applied them to the specific situation they faced on their own endeavor. These practices had been presented in their essential form, in terms of alphas, work products, activities, (i.e. using the Essence language to facilitate the understanding and use of the practices by the team). So, where do these practices come from, how do these practices end up in this essentialized form? And how do we arrive at these practices like the ones we illustrated in part 3?  We answer these questions in this chapter.

21.1 Practice Sources

Let’s first discuss where practices come from.  Practices come from many sources. They are just good ways to deal with specific problems in certain contexts which practitioners in the industry have either discovered or invented.

Some of these practices are given names, while others have not. The popular and more successful ones spread across the community, through the numerous software engineering conferences and events. These practices were presented, debated, tried and tested before ending up in papers, books, and adopted by fellow practitioners as practices within their methods.  Examples include scrum, use-cases, extreme programming (XP), and recently, at the time of our writing this book, large-scale development frameworks such as the Scaled Agile Framework (SAFe), Disciplined Agile Delivery (DAD), Large Scale Scrum (LeSS), and Scaled Professional Scrum.  These large-scale agile development methods have been developed based on original small-scale agile development methods such as Scrum and XP. They have added higher layers of coordination between teams to meet business objectives.

Practice sources are not limited to agile methods and include traditional ones (often adapted to agile).  For example, how to deal with systems engineering, object-oriented analysis and design, and use cases. Moreover, each large organization, due to its unique growth path, evolves its unique set of practices. Since these practices are very domain specific, they are not as wide spread as the more generic and more popular ones.

Organizations would typically evolve their own practices as they find better ways of working. Lim, the development VP from our TravelEssence case, emphasized this strongly. He would hold regular meetings with program managers and team leaders to stimulate discussions leading to continual improvement of their way of working. Through this effort, Lim drove the evolution of a number of practices related to the improvement of quality of TravelEssence’s products, the delivery of those products, and how individual teams within his organization could improve collaboration with operations, which was led by Tan, the operations VP. These home-grown practices initially appeared only in their conversations and meetings. They were not labelled with brand names, and were not made explicit through documentation.  This often happens in many organizations.  However, once his organization adopted Essence, Lim understood that he had a solid architectural structure he could rely on to coach his colleagues and sustain these improvements well into the future as his enterprise continued to grow.

21.2 Monolithic Methods and Fragmented Practices

Because of the large number of methods, in the past people used different terminology, or the same word with different meanings (homonyms), and organized their practices in very different ways. This caused these methods to become either too monolithic or too fragmented.  When we use the term “monolithic” in this book we mean that a method is non-modular resulting in that it is difficult to add, modify, replace (substitute or exchange) a practice from outside (e.g. another author’s practices, or another group’s improvements ideas) or to replace an existing practice with another one. A method is fragmented when it is difficult to see or understand how its constituent pieces fit into the whole, and we will again explain what fragmentation is with another analogy.

Methods like Rational Unified Process (RUP), Scalable Agile Framework (SAFe) [23], Disciplined Agile Delivery (DAD) [24], and Large Scale Scrum (LeSS) [25], are monolithic by nature.  RUP is an iterative software development method. SAFe is a lean software and systems method. DAD is a process decision method. LeSS is Scrum applied to many teams working together on one product. These are examples of monolithic methods.  Since this book is for new students to software engineering, we will not dive into these methods. But instead, we will consider the essence of how these methods organize practices.

Monolithic methods are an amalgam of practices where the practices are not clearly distinguishable from one another. Practitioners rarely, if ever, apply any of these methods in their entirety, but only parts of them. The question then is which part? How is a part extracted and clearly defined so team members understand what is expected of them? Another question you might be asking yourself is, how can we make it easier for teams to reuse parts of a method that make sense just for their team?  This brings us to the subject of essentialization, which we will soon introduce. 

Fragmented practices are loose bits of guidance with different concepts and jargon. They are usually some kind of summary to address a certain challenge, such as product design, team collaboration, and so on, with no regards as how they relate to other practices. Professionals who encounter them needs to spend the effort to link concepts and terminology.

Many times, different terms are used to denote the same thing. For example, iterations and sprints are largely similar. Some use the term squads for teams, and tribes for large teams, and chapters for communities. Although it may be at times useful to use different adjectives to highlight certain qualities, using different terms especially when there are so many practices confuses software professionals who already have much to learn. For example, scrum’s use of the term “sprint” metaphorically emphasizes that a team should have everything ready to achieve the iteration goals. A sprinter definitely doesn’t adjust his shoes or detour during the 100 meter race. No, he has absolute focus and will not be distracted. Nevertheless, a sprint is still an iteration, a time-box when things get done. Thus, there are two terms “sprint” and “iteration” to refer to essentially the same thing. With  fragmented practices, there are a lot more terms referring to the same thing. Some terms may have overlapping meanings. They are similar but somewhat different, creating much confusion to professionals.  It is hence not easy if not impossible to understand how practices fit together. You cannot fit a practice from one source (method) with another practice from another source. If you ever want to merge practices, the two practices must be described using the same language, which is what Essence provides.

This now brings to the discussion about essentialization. From a methodology point of view, it is about identifying and scoping a practice from a practitioner’s perspective, describing it using the Essence language such that it is compatible with other practices.  In the larger picture, it is also about introducing such a practice to teams, teaching and training them such that they can learn, apply and adapt the practice to their context.

21.3 Essentializing Practices

You have seen that essentialized practices can come from many sources. Essentialized practices can come from the work of explicitly calling out practices from well-known practice sources (like SAFe, RUP, XP, etc.). By doing so, we make the practice more precise, easier to teach and apply, even within different contexts. A natural consequence of essentialization is that practices become easier for teams to compare and choose from available ones.

The essentialization of a practice is the activity of making a practice explicit by describing it using the Essence kernel and language as a platform. The practices that we presented in part 3 are examples of essentialized practices.

In many cases, practices are implicitly described or described in a non-standard way, and teams find them difficult to understand and apply. Another problem is that people don’t have an answer to the question is “how should we start describing a practice?”  Prior to Essence, we did not have a well-defined, widely accepted way to explicitly define our practices. However, now we have a language – the Essence language - to explicitly describe practices.

The Essence standard tells us what a practice is and what a practice is intended to be used for.  By definition from the standard, “a practice is a repeatable approach to doing something with a specific purpose in mind”.  The Essence language provides a well-defined way to express a practice.   A practice expressed in the Essence language provides explicit guidance to teams by clarifying the things to work with, the things to do, the required competencies and patterns. In particular, alpha state progression emphasizes an end-to-end view, from beginning to the end, E.g. for Requirements, from Conceived to Fulfilled, for Use Cases, from Goal Established to All Stories Fulfilled. This end-to-end view channels teams’ energy to full completion. Other elements of a practice (e.g. activities and patterns) are about providing guidance and approaches to make state progression more effective and hence faster. Appropriate competencies are of course necessary, without them, the team cannot achieve good progress.

The size of a practice is dependent on the number of elements in the practice (i.e. total number of alphas, work products, activities, competencies and patterns). In general, the smaller the number of elements in a practice, the easier it is to understand and apply.  There is no lower limit for how small a practice can be, but there is a practical limit.  This is because, smaller practices result in a larger number of practices, which places a cognitive load for teams who need to select and compose them.

Given the nature of Essence we can describe many practices using Essence (both the kernel and the language) as a vocabulary. Every such practice should be designed to be reusable meaning that they could be applied in different contexts and scope. This will become more apparent as you read this chapter and the next.

21.4 Establishing a Reusable Practice Architecture

Having extracted practices, it is important to have a way to organize and present the practices so that they can be reused by teams selecting their own method. Making a practice reusable means ensuring that the practice selected is a proven practice and that the elements described are essential elements only. This is one reason why essentialization is so important. Moreover, often teams need many practices that are designed to be composed together in what we call a practice architecture. Even if practices are separate elements, they are not independent. Let us take an example composing two practices - use cases and microservices.  When talking about use cases here we talk about the more complete Use-Case 2.0 practice (not the Use-Case Lite practice we talked about in part 3); it has also a realization part describing how a use case is realized by collaborating system elements, in this case microservices. The microservice practice guides people to create small rapidly evolvable and deployable microservices.  These two practices have overlapping elements namely use case, use-case slices and microservice, the effect of which has to be described when composing the two practices. A simple way to think about practice composition is to think about a merge operation. For example, a practice may require that an alpha may have some checklist items and another practice may require the same alpha to have other checklist items. The composition or merge operation will merge these two checklist items together into that alpha. This same idea can be applied to the composition of work products, activities, and so on.

One way to construct a practice architecture is in a layered form with more generic practices at the bottom and more specialized practices at the top.

In our story, the development VP, Lim, understood the need to learn and share practices across teams. It was because of this continual sharing and evolving that he was able to build up a systematic knowledge base allowing the company to become a learning organization with a strong team of developers.

However, it wasn’t like that a few years ago, before Essence was introduced to the company. Back then, TravelEssence was smaller, and team members worked fervently to deliver the software systems. Teams used the practices they deemed to be appropriate.  Now as TravelEssence grew, with new applications, and more members, their old way of working needed to be adapted to fit their current size and organizational structure.   It had to change.

Fortunately, Lim immediately was attracted to the work and the contributions of Essence, including the essentialization of practices. He was able to reuse these practices and the language and terminology across teams following an accepted standard. Several team members were skeptical at first, but became gradually convinced when they saw how speaking the same language of practices reduced unnecessary misunderstanding and helped them focus on what they need to do, (i.e. deliver great quality software).

Smith supported what Lim was trying to achieve, which was why he strongly supported the use of a practice-based approach to run his development team.

A practice architecture was developed by Lim with some help from Smith to be used for TravelEssence and presented in Figure 151. Lim needed practices not only for development teams, but also teams all across the IT organization (see Figure 151).

Figure 151     Practice Architecture in TravelEssence

Lim organized the practice architecture into two layers separated by a dashed line.

21.4.1 Development Practices

At the bottom layer, there are practices for development teams. This includes practices like User Story, Scrum, Use Case and Microservices, the Lite version of which have been presented and discussed in part 3. In addition are several other practices that Lim found important.

  • Product Ownership – This is a practice for deciding and prioritizing the kind of requirements that a team would deliver. In part 3, we have seen that Angela played the role of a Product Owner. This practice provides more explicit guidance to let her do her job better.
  • Team Backlog – This practice provides guidance on how to manage and track items in a team’s backlog.
  • Team Sprint – This practice provides guidance for how teams can work iteratively, from sprint planning to sprint delivery.
  • Team Retrospective – This practice provides guidance for how teams can continually reflect on how they are working and making necessary improvements.

If you think about it, Scrum, which we presented in part 3, does include product ownership, team backlog, team sprint, and retrospective, so why do we have these four practices instead of just Scrum? As we have just shown, Scrum can be represented as a composition of four practices, so that teams can choose to only use some practices in Scrum and not being forced to use all of Scrum. This makes our practice architecture more reusable.

21.4.2 Program Practices

You can think of a “program” as a large “project” involving smaller “projects”. Organizing the work within a program can be complicated because you need to balance priorities from different stakeholders, and manage the dependencies across the teams within the program. Moreover, the work may cut across different departments.

In our story, Lim recognized the importance of having practices that go right from business departments to development to operations. He called these program level practices. Product Management – This practice is about managing the source or requirements and analyzing them before they are placed in the program backlog for development.

  • Program Backlog – Once the requirements are analyzed and agreed for development, their progress are tracked using the program backlog practice. Like the Team Backlog practice, this practice provides guidance on the prioritization and acceptance of the program backlog items.
  • Program Sprint – Programs under Lim applied an iterative cycle to deliver Program Backlog Items. These PBIs are allocated to development teams. In the next chapter, you will see how Jones, a program manager for the Core Hotel Management System (CHMS), work with his subordinate teams.
  • Program Retrospective – Lim’s programs applied regular retrospective to analyze their way of working and seek improvements.
  • DevOps – After new requirements such as use case slices have been implemented and integrated, production-like test environments have to be updated too. Tan manages the automated deployment pipeline that maintains these systems, and is responsible that they run smoothly, taking care of processor and disk capacities. Basically, an automated deployment pipeline is a set of automated scripts and mechanisms that performs checks such as compilation checks, and testing to ensure that the deployed software works not just on the developer’s machine, but everywhere the deployed software needs to run, even in the production environment. DevOps is the practiced set of technologies to achieve this.