February 24, 2020

A Basis for Automation: Predictable, Repeatable, Auditable Workflow

To consider automation, we have to first understand the work. Otherwise, we are just taking bad things and making them go faster.

Matthew D Edwards

This article was previously published on LinkedIn.

I remember a math teacher from long ago telling the class, "Until you know how to do the math with a pencil and eraser, you cannot use a calculator." And I have subscribed to this logic my entire life journey. Learn everything at the most basic level. Understand why not just what. Optimize the information after I understand so I can then learn even more.

This is one of the great experiences that made me want to be a lifelong learner.

Another was a physics teacher sitting beside me, walking me through a book from the Smithsonian Institute on space. His questions to me, as we thumbed through the pages looking at high-resolution pictures of the planet were simple, "Can you imagine what it would take to travel to that planet?" "What do you think it would take to live on that planet?" And I, thereafter, sought to understand what it would take to solve these problems.

This experience taught me, "Think very big and do not be intimidated by large, opaque, complex, high-risk problems."

I also remember, with great sadness, when I realized there was more to learn than I would live long enough to understand and value.

All Big Things Become Small Things

So, I looked for methods of discovering, organizing, and leveraging greater volumes of information in my everyday life. I needed a way to discover and appreciate the depth and breadth of a seemingly infinite set of bodies of knowledge while accepting I only have one lifetime to do it. I needed a way to take big things and break them into small things so I could choose what, when, where, why, and under what circumstances they may directly apply, or cross-apply, to other endeavors in my life.

Common words for organizing big things into little things and so on include paradigms, frameworks, patterns, templates, reference architectures, taxonomy, decomposition, configuration management, deconstruction, classifications and even Table of Contents.

Figure 1. Big things decompose into small things.

To the dismay of all logophiles out there, I'm going to condense this discussion to one word – patterns.

I look for reusable patterns in everything. Patterns to understand things around me. Patterns to organize. Patterns to do work. Patterns to assess risk. Patterns to deliver. Life is full of reusable patterns.

Life is full of reusable patterns.

So, when you're trying to understand something big, break it down into parts and pieces. It will be easier to understand, organize, prioritize, and build back up again at scale because you will discover one or more patterns. Whether something is broken down to the atomic level or to user classes, epics and user story threads depends upon your project.

Work Has A Predictable Pattern

Most people understand the idea of work. I am cold. I put on a coat. I am warm. I have dirty dishes. I wash them. I have clean dishes.

State A. Do something. State B.

And many people want to see the results they want to see – right now.

I want pecan pie right now. I want it to look and taste just like what my Grandmother used to make for me on holidays and birthdays. My Grandmother has passed away, I don't have her recipe and I failed to ask her to teach me how to make the pie. I want it nonetheless. Now. Exactly the way she used to make it. With whipped cream. Do not fail me, maker of pecan pie.

What I want and what I can get is dependent upon an ability to:

  • See the big thing (pecan pie);
  • See the parts and pieces (ingredients); and
  • Understand the method (order and method of operations).

Whether I'm making Grandmother's pecan pie, building a barn or implementing a continuous delivery pipeline in the cloud, they all require the same steps, in the same order, to begin and complete the work.

The basic pattern of work looks like this:

  • a request for work,
  • entry criteria (criteria by which we agree to start work),
  • a method of doing and a method of checking work,
  • exit criteria (criteria defining "done and acceptable"),
  • and a deliverable.
Figure 2. "Do Work" - A simple view of how work happens for one person.

People Do Not Perform Consistently

The unpredictable, non-conforming, variability of people is one of the things that makes life an adventure. People and their culture are rich and unique, and our memories and the stories we tell about them form a vibrant tapestry.

When it comes to work, if we know the pattern of work, then why do so many hard-working teams fail to deliver, let alone deliver well?

Humans are not automatons. Humans are, by nature, variably behaved, expressed and experienced. For 100 people to complete 50 individual tasks 100 times in a predictable, repeatable and auditable manner is a pretty tall order. Which may explain why a grande caramel macchiato retrieved from the same chain store in different cities tastes different so often. There is a recipe, an order of steps and method. Nonetheless, sometimes the coffee tastes like the highly caffeinated liquid weight gain I expected and other times a painful waste of money.

If there is a pattern and the work is human and manual, there will be variable results. Human results are variable. Why does only one team win the annual football bowl game while others do not? More interestingly for our conversation, explain why that same team doesn't win every single year of their existence. Variability.

We know what a predictable, repeatable pattern of work looks like and we know how human variability can impact that pattern in everyday life.

Now let's look at what happens when we have many people.

Work Patterns Get Complicated As They Scale

Assembling around an objective to achieve a result is seen in all of nature. It isn't unique to humans.

Bees. Ants. Animal packs. Sports teams. Military units. Projects.

For Humans, work becomes more complicated as the number of people involved increases. With more people come more units of work, more steps and more latency between steps.

Consider the following behavioral pattern many of us see in organizations, "This team does work, then this team does work, then this team does work." Have you seen this before?

Queue work, start work, do work, hand-off. Repeat.

When we watch ants decompose food, there appears to be a constant flow of activity. When we watch bees collect honey, we see the same characteristics. Flow.

When we watch people on projects, it simply isn't the same.

Imagine being in the passenger seat of an old 1968 F-150 pickup while a teenager is learning to drive a stick-shift. Now imagine every time said teen pops the clutch, taps the accelerator, hits the brakes, or all of them at the same time, your head rocks back and forth between the glass behind you and dash in front of you. There was no padding in the dashboard. Learning to control the clutch is a practiced, learned behavior. Learning to push the accelerator while letting off the clutch is also a learned behavior.

Flow is a learned behavior. Flow must be sought on-purpose.

Now, imagine smacking your face on the dashboard, glass or both every time work moves from person to person on a project in your company. Start. Go fast. Stop. Start. Stop suddenly. Kiss the dashboard.

What if your face was the barometer of your organization's flow? Imagine increasing the number of people, concurrent projects and tasks to span the company and you are the only person who hits glass and dashboard for all projects, all people, all steps, all starts and stops. Need a helmet?

If you think I'm exaggerating to make a point, I am. A little. The stick shift story is real. I feel bad for my dad and all the watermelons we ran over in the field that day. He ended up sitting in the seat sideways with arms against the seat and dash in a self-defense position. If there had been predictable start and stop patterns that day, perhaps he could have navigated the situation more enjoyably. I still remember the look on his face.

Figure 3. "Do Work and Wait" - A simple view of work for multiple people.

When it comes to performing work and delivering results, the ideal experience achieves a smooth flow of work being performed, with little to no wait times in between steps, and smooth transitions.

Wait time and unpredictable transitions likely cost my dad time on this earth realized as dynamic, premature aging. Wait time and unpredictable transitions cost companies time and money.

The "start and stop" method is also known by some as the "throw it over the wall" principle.

"My part is done. Worked for me. Good luck!"

We want flow like ants and bees. We more likely experience starting, stopping and pain like my dad while teaching me to drive a stick-shift.

Manufacturing Controls Flow-Through Batch Sizes

If you and I are on the same page regarding the value of flow versus kissing a dashboard with your face, let's talk about how to get there.

Decades ago, the manufacturing industry began the use of assembly lines (automation) to increase flow, throughput, predictability, quality and manage their scaling economics. They received another boost in productivity and value when they moved from large to small batches along the same assembly line. Wait times decreased and flow increased.

And due to batch sizes, their ability to adapt to change increased.

In the manufacturing world, wait times (inventory in a wait state) are considered unrealized revenue and therefore waste. Manufacturing supply chains, therefore, seek to eliminate waste. They build things to make money. They do not build things to store them in the supply chain. The key? Flow.

Figure 4. Too much in-flight, undelivered work is unrealized revenue.

If warehouses full of product are considered unrealized revenue and therefore waste in the manufacturing industry, how do we then categorize in-flight, incomplete, undelivered or otherwise unfinished software solutions in companies? What do you think that implies with regards to the numbers of in-flight user stories or numbers of in-flight software projects?

What do you think happens when we introduce the idea of rework?

Work Always Has Rework

Ideally, when we run projects, things always go as planned. And when they don't? We end up dealing with two subjects that weren't in the original plan – technical debt and refactoring.

Technical debt is defined by work you know you need to do now, but decide to kick down the road until later. This creates additional work in the backlog. Just like interest rates on debt, the longer it sits there, the more time, complexity and/or cost it will take to address. Technical debt is work.

Refactoring is defined by changing, modifying or otherwise evolving something from a previously acceptable state of existence to a new and improved state of existence for the purposes of delivering desired value. Refactoring is also work.

Figure 5. Rework - "I found a problem. What do I do with it?"

When John finishes his task, the deliverables move down-line for Jane to complete her task. Jane finds a problem with the inherited deliverable and either fixes it, ignores it or sends it back up-line for John's eventual attention.

If Jane fixes it on behalf of John, was it correct and complete? If she ignores it, will it be found and addressed later? By whom? If she sends it back up-line, how will John know? And when will John get to the refactoring work given his existing backlog of prioritized work?

The problem discovered by Jane impacts her ability to complete planned work. And depending upon her decision, it will become work for one or more others.

Now multiply John's and Jane's experience by the numbers of people, teams, projects, stories, and associative decisions to acknowledge, fix, send it back up-line to someone else's queue or ignore it altogether.

This churn contributes to wait times between steps. And if a person doesn't plan for rework, it also contributes to cranky people.

Rework happens. Plan for it. Manage impact to flow by decomposing all work into small, edible pieces. Manage your batch sizes. Seek flow.

How Do We Achieve Flow?

To consider automation, we have to first understand work, batch sizes, and flow. Otherwise, with automation, all we're really doing is taking bad things, making them go faster and calling it digital transformation.

Steps to achieve flow:

1. Manage batch sizes. Break big things down into small things.

2. Minimize and eliminate wait times between steps and people.

3. Plan for, invite, and accept rework. Manage it through batch sizes.

4. Automate.

5. Repeat.

Automation is not the goal. Predictable, repeatable, auditable flow is the goal.

Automation is only the medium.

My math teacher has made sense for a very long time.

Pencil first. TI-88 programmable calculator second.

Read Basis for Automation, Part II, to learn the steps for ensuring a project team addresses requirements and prioritizes backlog before automating a continuous delivery pipeline.

Interested in Examining Flow?

Trility joined Bâton Global for a podcast series on Human-Centric Transformation. In the four-part series, they discuss how leaders and technology can simplify and automate processes for team members, stakeholders, and customers. Before you listen, view our companion infographics highlighting key takeaways as these organizations discuss how to better respond to industry headwinds and pressures.