29,99 €
This practical guide to DevOps uses GitHub as the DevOps platform and shows how you can leverage the power of GitHub for collaboration, lean management, and secure and fast software delivery.
The chapters provide simple solutions to common problems, thereby helping teams that are already on their DevOps journey to further advance into DevOps and speed up their software delivery performance. From finding the right metrics to measure your success to learning from other teams’ success stories without merely copying what they’ve done, this book has it all in one place. As you advance, you’ll find out how you can leverage the power of GitHub to accelerate your value delivery – by making work visible with GitHub Projects, measuring the right metrics with GitHub Insights, using solid and proven engineering practices with GitHub Actions and Advanced Security, and moving to event-based and loosely coupled software architecture.
By the end of this GitHub book, you'll have understood what factors influence software delivery performance and how you can measure your capabilities, thus realizing where you stand in your journey and how you can move forward.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 521
Enhance software delivery performance with GitHub Issues, Projects, Actions, and Advanced Security
Michael Kaufmann
BIRMINGHAM—MUMBAI
Copyright © 2022 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Group Product Manager: Rahul Nair
Publishing Product Manager: Vijin Boricha
Senior Editor: Athikho Sapuni Rishana
Technical Editor: Nithik Cheruvakodan
Copy Editor: Safis Editing
Project Coordinator: Ashwin Kharwa
Proofreader: Safis Editing
Indexer: Tejal Daruwale Soni
Production Designer: Sinhayna Bais
Marketing Coordinator: Nimisha Dua
Senior Marketing Coordinator: Sanjana Gupta
First published: September 2022
Production reference: 2180822
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
978-1-80181-335-8
www.packt.com
To my family, who had to spend many weekends and nights without me. To my colleagues from Xpirit and the DevOps community for giving me feedback, challenging my ideas, and giving me the opportunity to learn.
– Michael Kaufmann
In 2011, entrepreneur Marc Andreessen famously claimed in the Wall Street Journal that “Software is eating the world,” predicting that the rise of software would digitally metamorphose every industry and every sector of the world as we know it. As I sit here writing this a decade later, there is nothing to say except “Marc was right.” Software has transformed our lives, and with that, it has also transformed every company and organization. Look at how it has fundamentally changed entertainment with Netflix, or travel and hospitality with Airbnb, or quite nearly everything purchasable under the sun with Amazon. Grocery stores now have a digital experience, parking meters are being replaced by phone apps, the oldest and most traditional banks have moved to the cloud, and cars get over-the-air updates more frequently than your mobile phone.
Every company is becoming a software company. Millions of lines of code already represent the foundation of the global economy. Software developers are the architects of this new, digital world. No longer can any organization, no matter the size or the industry, compete and thrive without software and developers.
And this trend isn’t showing any sign of slowing down. According to the World Economic Forum, an estimated 70% of new value created in the economy over the next decade will be based on digitally enabled platform business models. To harness this opportunity, every organization will need to tap into the power of open source. Without open source, they won’t stay competitive. Companies will also need to adopt DevOps practices to help refresh and strengthen their internal culture and continuously enhance software delivery performance. Naturally, as GitHub’s CEO, I believe that GitHub is the best place for every organization to achieve this.
When Marc wrote his article in 2011, GitHub was still in its early days, focused on the hosting of git repositories. Today, GitHub has evolved to a full DevOps platform with features that support developers in every step of the developer lifecycle. With more than 83 million developers using our platform, we provide the home for the world’s developers. GitHub is the place where every project - open source, cloud native, mobile, or enterprise - can be created, modernized, and deployed to its destination. It’s the place where the interconnected community of developers builds the world of tomorrow.
I am thrilled that somebody as experienced as Michael has penned Accelerate DevOps with GitHub. Whether you are a professional software developer, computer science student, solutions architect, or site reliability engineer, this book is for you. Accelerate DevOps with GitHub provides clear, concise, and practical ways for you and your organization to harness the power of DevOps and GitHub. I think it will help you in the years to come, preparing you for the golden era of software development.
I am so proud of the long hours of hard work that you spent writing this book, Michael, but beyond this, I’m even more proud of all the meaningful change and progress I know it will help create for others.
Thomas / @ashtom
CEO GitHub Inc.
Michael and I first met at a conference where we were both speaking on DevOps. We bonded over our shared passion for DevOps, and met often on the speaking circuit. It became a tradition for us to take selfies together each time we met. Our friendship and his passion for DevOps are why I was so excited to see that he was going to share his knowledge with the world by writing this book.
As time passes, the tools we use may change. However, the information shared in this book applies universally to organizations embarking on their DevOps transformation.
With COVID driving the world to remote, I really appreciated Michael covering asynchronous work. This has quickly become our new normal and teams must develop this muscle to stay agile and productive on remote and distributed teams.
It was great to read about the use of feature flags, which can be a game changer. Feature flags separate delivery from releasing and enable more advanced deployment strategies. It also reduces the need to rollback and drastically reduces the time to recover from bad code. However, as with anything, there is a cost. Michael does a great job covering the cost of using feature flags and how to mitigate it. This allows the reader to make an informed decision about whether feature flags are right for them.
Many teams I meet assume that going faster means cutting corners, but Michael explains the importance of infusing quality and security into your process. Additionally, he provides practical guidance about how to accomplish this. When DevOps is implemented correctly, you deliver secure, higher-quality code faster.
Often, to harness the true power of DevOps, your application must be refactored. Michael covers software architecture and the impact it has on your process and your team. He also covers the trade-offs of each option to help teams decide which is best.
I am confident that the readers of this book will find it to be an indispensable tool to support their DevOps transformation.
Donovan Brown
Partner Program Manager, Azure Incubations, Azure Office of the CTO
Michael Kaufmann believes that developers and engineers can be happy and productive at work. He loves DevOps, GitHub, Azure, and modern work – not only for developers.
He is the founder and CEO of Xpirit Germany, a consulting company of the Xebia group, and he has been working in the IT sector for more than 20 years. Michael helps his clients to succeed through cloud and DevOps transformation and the implementation of new ways of working.
Microsoft awarded him with the titles Microsoft Regional Director (RD) and Microsoft Most Valuable Professional (MVP) – the latter in the DevOps category and GitHub since 2015.
Michael shares his knowledge through books and training, and is a regular speaker at international conferences.
“I want to thank the people who have been close to me and supported me, especially my wife, Gladys, and my parents.”
Mickey Gousset is a staff DevOps architect at GitHub. He is passionate about DevOps and helping developers achieve their goals. Mickey speaks on DevOps and cloud topics at various user groups, code camps, and conferences around the world, and is also the author of several books on Application Lifecycle Management (ALM) and DevOps.
Stefano Demiliani is a Microsoft MVP and Microsoft Certified Trainer (MCT), a Microsoft Certified DevOps Engineer and Azure Architect, and a long-time expert on Microsoft technologies. He works as a CTO for EID NAVLAB, and his main activities are architecting solutions with Azure and Dynamics 365 ERPs. He’s the author of many IT books for Packt and a speaker at international conferences about Azure and Dynamics 365. You can reach him on Twitter, LinkedIn, or via his personal website.
Unai Huete Beloki has been working as a DevOps expert for the last 5 years. He started in 2017, working as a customer engineer at Microsoft, providing support and education related to DevOps (mainly GitHub and Azure DevOps) and Azure around the EMEA region. In July 2020, he moved on to the Azure technical trainer role at Microsoft, where he provides Azure and DevOps training to customers worldwide and is one of the global leads for the AZ-400: Designing and Implementing Microsoft DevOps Solutions course/exam. He received a BSc in electronic and communications engineering and a master’s in telecommunication engineering, both from the University of Navarra.
We are in the 2020s and research has shown us for more than 10 years that companies with high developer performance not only outperform their competitors in velocity and throughput, they also score higher in quality, innovation, security, employee satisfaction, and most importantly, customer satisfaction.
And yet, besides some unicorn companies, the majority of traditional businesses struggle to transform themselves. Established rigid structures and slow processes, monolithic application architectures, and long release cycles for traditional products make it hard for companies to change.
This, however, is not a new phenomenon. Transformational changes are always hard and take many years to succeed, if the companies do succeed at all. The probability of failure is also very high. This is because transformation has to happen on so many levels – and if these changes are not aligned, the transformation is bound to fail. This book will help you with your transformation - not only by providing the research for high developer performance but also by providing practical examples on how you can accelerate your software delivery.
This book is a practical guide to DevOps. It helps teams that are already on their DevOps journey to further advance into DevOps and speed up their software delivery performance by providing simple solutions to common problems. It will help teams find the right metrics to measure their success and learn from other success stories without just copying what these teams have done themselves. The book uses GitHub as the DevOps platform and shows how you can leverage the power of GitHub for collaboration, lean management, and secure and fast software delivery.
By the end of this book, readers will understand what influences software delivery performance and how they can measure delivery capabilities. They will therefore realize where they stand and how they can move forward in their journey with transparency and simple solutions for cross-team collaboration. Equipped with simple solutions for common problems, they will understand how they can leverage the power of GitHub to accelerate: by making work visible with GitHub Projects, measuring right metrics with GitHub Insights, using solid and proven engineering practices with GitHub Actions and Advanced Security, and moving to an event-based and loosely coupled software architecture.
This book is for developers, solution architects, DevOps engineers, and SREs, as well as for engineering or product managers who want to enhance software delivery performance. They may be new to DevOps or already have experience but struggle to achieve maximum performance. They may already have experience with GitHub Enterprise or come from a platform such as Azure DevOps, Team Foundation Server, GitLab, Bitbucket, Puppet, Chef, or Jenkins.
Chapter 1, Metrics That Matter, explains the theory behind lean management and how you can measure performance and cultural change. It looks into developer productivity and why this is so important to attract talent and achieve outstanding customer satisfaction.
Chapter 2, Plan, Track, and Visualize Your Work, is about work insights: accelerate your software delivery performance by applying lean principles. You’ll learn how to plan, track, and visualize the work across your teams and products using GitHub Issues, Labels, Milestones, and Projects.
Chapter 3, Teamwork and Collaborative Development, explains the importance of collaborative development of software and how GitHub can be used for collaboration across teams and disciplines.
Chapter 4, Asynchronous Work: Collaborate from Anywhere, explains the benefits of asynchronous ways of working and how you can leverage them for improved and shared responsibilities, distributed teams, better quality, and cross-team collaboration. It shows how you can use GitHub Mobile, Microsoft Teams, Slack, and GitHub Pages, Wikis, and Discussions to collaborate from any location and any device.
Chapter 5, Influence of Open and Inner Source on Software Delivery Performance, describes the history of free and open source software and the importance it has gained over the recent years and in the context of cloud computing. It will teach you how to leverage open source to speed up your software delivery. Moreover, it will explain how open source practices applied to inner source will help you transform your organization, and the impact open and inner source can have on your in- and out-sourcing strategy.
Chapter 6, Automation with GitHub Actions, explains the importance of automation for quality and speed. It introduces you to GitHub Actions and how you can use them for any kind of automation – not only continuous delivery.
Chapter 7, Running Your Workflows, explains how you can tackle hybrid-cloud scenarios or hardware-in-the-loop tests using the different hosting options for the GitHub Actions workflow runners. It shows how to set up and manage self-hosted runners.
Chapter 8, Managing Dependencies Using GitHub Packages, describes how you can use GitHub Packages and semantic versioning together with GitHub Actions to manage dependencies between your teams and products.
Chapter 9, Deploy to Any Platform, shows how you can easily deploy to any cloud and platform with simple hands-on examples for Microsoft Azure, AWS Elastic Container Service, and Google Kubernetes Engine. It shows how you can perform staged deployments with GitHub Actions and how to use Infrastructure as Code to automate the provisioning of your resources.
Chapter 10, Feature Flags and the Feature Lifecycle, explains how Feature Flags – or Feature Toggles – can help you to reduce complexity and manage the lifecycle of features and your software.
Chapter 11, Trunk-Based Development, explains the benefits of trunk-based development and introduces you to the best Git workflows to accelerate your software delivery.
Chapter 12, Shift Left Testing for Increased Quality, takes a closer look at the role of quality assurance and testing on developer velocity and shows how you can shift left testing with test automation. The chapter also covers testing in production and chaos engineering.
Chapter 13, Shift Left Security and DevSecOps, takes a broader look at the role of security in software development and how you can bake security into the process and practice DevSecOps, zero-trust, and how you can shift left security. The chapter looks at common attack scenarios and how you can practice security and create awareness using attack simulations and red team | blue team exercises. The chapter also introduces you to GitHub Codespaces as a secure development environment in the cloud.
Chapter 14, Securing Your Code, describes how you can use GitHub Advanced Security to eliminate bugs, security, and compliance issues by performing static code analysis with CodeQL and other tools, successfully manage your software supply chain with Dependabot, and eliminate secrets in your code base using Secret Scanning.
Chapter 15, Securing Your Deployments, shows how you can secure deployments to your environments and how you can automate your complete release pipeline in a secure, compliant way to also meet regulatory requirements. The chapter covers Software Bills of Materials (SBoM), code and commit signing, dynamic application security testing, and security hardening your release pipelines.
Chapter 16, Loosely Coupled Architecture and Microservices, explains the importance of loosely-coupled systems and how you can evolve your software design to achieve this. The chapter covers microservices, evolutionary design, and event-based architectures.
Chapter 17, Empower Your Teams, is about the correlation of the communication structure of your organization and your system architecture (Conway’s law) and how you can use this to improve architecture, organization structure, and software delivery performance. It covers the two-pizza team, the Inverse Conway Maneuver, and a mono- versus multi-repo strategy for your code.
Chapter 18, Lean Product Development and Lean Startup, is about the importance of lean product management at a product and feature level. It shows how you can incorporate customer feedback into your product management, create Minimal Viable Products, and how you can manage your enterprise portfolio.
Chapter 19, Experimentation and A|B-Testing, explains how you can evolve and continuously improve your products by conducting experiments to validate hypotheses through evidence-based DevOps practices like A|B-testing. It also explains how you can leverage OKR to empower your teams to conduct the right experiments and to build the right products.
Chapter 20, GitHub: The Home for All Developers, explains how GitHub can serve as the holistic, open platform for your teams. It explains the different hosting options, pricing, and how you can integrate it in your existing toolchain.
Chapter 21, Migrating to GitHub, will discuss strategies to migrate from different platforms to GitHub and integration points for other systems. It explains how you can find the right migration strategy and how you can use the GitHub Enterprise Importer and Valet to perform the heavy lifting.
Chapter 22, Organize Your Teams, talks about best practices to structure your repositories and teams into organizations and enterprises to foster collaboration and facilitate administration. The chapter covers role-based access, custom roles, and outside collaborators.
Chapter 23, Transform Your Enterprise, puts all the pieces together. This book gives you a lot of tools that you can use to drive a successful transformation and to gain developer velocity. But only if all pieces are put together will the transformation succeed. The chapter will explain why many transformations fail, and what you should do to make your transformation a success.
If you want to follow the hands-on labs to deploy to Azure, AWS, or Google you will need an account for the given cloud environment.
If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.
The examples and hands-on labs of this book are on GitHub at http://github.com/wulfland/AccelerateDevOps and https://github.com/PacktPublishing/Accelerate-DevOps-with-GitHub. If there are updates to the code or labs, the GitHub repository will get updated.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/vzP6B
There are a number of text conventions used throughout this book.
Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “You can customize the dialog to choose the issue template by adding a file config.yml to .github/ISSUE_TEMPLATE.”
A block of code is set as follows:
name: 💡 Custom Issue Formdescription: A custom form with different fieldsbody: - type: input id: contact attributes: label: Contact DetailsWhen we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
blank_issues_enabled: truecontact_links: - name: 👥 Discussions url: https://github.com/wulfland/AccelerateDevOps/discussions/new about: Please use discussions for issues that are not a bug, enhancement or feature requestAny command-line input or output is written as follows:
$ gh secret set secret-name
Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “Open the following repository and create a fork by clicking Fork in the top-right corner of the repository.”
Tips or important notes
Appear like this.
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Once you’ve read Accelerate DevOps with GitHub, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.
Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.
In Part 1, you will learn how to reduce ballast in your development process and move to a lean and collaborative way of working that allows your teams to accelerate their value delivery. You’ll learn how to use GitHub to work together from everywhere effectively and use work insights and the right metrics to optimize your engineering productivity.
This part of the book comprises the following chapters:
Chapter 1, Metrics That MatterChapter 2, Plan, Track, and Visualize Your WorkChapter 3, Teamwork and Collaborative DevelopmentChapter 4, Asynchronous Work – Collaborate from AnywhereChapter 5, Influence of Open and Inner Source on Software Delivery PerformanceThe hardest part when implementing DevOps is a shift in conversations with management. Management is used to asking the following questions:
How much will it cost?How much will we earn from it?From a management perspective, these are reasonable questions. But in a DevOps world, they can be toxic and can lead to a large amount of planning upfront if they are answered at the wrong time and in the wrong way. In this chapter, I'll show you metrics that can shift discussions with management away from efforts toward general engineering velocity and developer productivity.
I'll explain how to measure engineering velocity and developer productivity and how to make your DevOps acceleration measurable.
The following topics will be covered in this chapter:
Why accelerate?Engineering velocityHigh-performance companiesMeasuring metrics that matterThe SPACE) framework for developer productivityObjectives and key resultsThe expected lifespan of companies is decreasing rapidly. According to Richard Foster from the Yale School of Management, the average lifespan of a Standard & Poor's (S&P) 500-listed company 100 years ago was 67 years. Today, it is 15 years. Every 2 weeks, an S&P-listed company goes out of the market, and by 2027, it is expected that 75% of the top 500 companies will be replaced by new companies. Another study from the Santa Fe Institute (The Mortality of Companies) concludes that the average lifespan of a United States (US) company across all industries is about 10 years.
To remain competitive, companies must not only solve a customer problem; they also need to deliver products and services that delight their customers, and they must be able to engage with the market and respond quickly to changing demands. Time to market is the most important driver for business agility.
Software is at the heart of every product and service in every industry, not only because the digital experience has become as important as (or maybe even more important than) the physical experience. Software touches every part of a product life cycle, for example:
Production:Supply chain managementCost optimization/predictive maintenance/roboticsProduct individualization (lot size 1)Sales, after-sales, and service:WebshopCustomer service and supportSocial mediaDigital assistantDigital product:Companion appIntegrationsMobile experienceNew business models (pay-by-use, rent, and so on)These are just examples to illustrate that most interactions your customers have with your company are digital. You do not just buy a car today—you are already aware of the brand from social media and the press. You buy and configure a car on a website or in a store with a salesperson, but also by looking at the screen of a tablet. The price of the car is influenced by the optimization of your assembly line by robotics and artificial intelligence (AI). The first thing you do with the car is to connect your phone. While driving you listen to music, make a phone call, or respond to a text message using your voice. The driving assistant keeps you safe by braking for you if something is in your way and by making sure you stay in your lane; and soon, cars will do most of the driving autonomously. If you have a problem with a car or an app, the chances that you'll use the app or email to contact after-sales are high, especially for the younger generations. A car is mainly a digital product. Not only are there millions of lines of code that run in a car, but there are also millions of lines of code that power cars' apps, websites, and the assembly line, (see Figure 1.1).
Figure 1.1 – Software and data at the heart of the customer experience
The good thing is that software can be changed much faster than hardware can. To accelerate your time to market and your business agility, software is the key driver. It is much more flexible than hardware components and can be changed in days or weeks, not months or years. It also allows a much better connection to your customers. A customer that is using your app is more likely to respond to a survey than one in a physical shop. Also, hardware does not provide you with telemetry of how your products are being used.
To be one of the companies that stay in business for longer than 10 years, your company must leverage the power of software to accelerate its market response and delight customers with a great digital experience.
How does your company measure developer velocity? The most common approach is effort. There used to be some companies that used metrics such as lines of code or code test coverage, but those are obviously bad choices, and I'm not aware of any company today that still does this. If you can solve a problem in one line of code or in 100 lines of code, one line is obviously preferable since every line comes with a maintenance cost. The same goes for code test coverage. The coverage itself says nothing about the quality of the tests, and bad tests also introduce additional maintenance costs.
Note
I try to keep the wording agnostic to the development method. I've seen teams adopt DevOps practices that use Agile, Scrum, Scaled Agile Framework (SAFe), and Kanban, but also Waterfall. But every system has its own terminology, and I try to keep it as neutral as possible. I talk about requirements and not user stories or product backlog items, for example, but most of the examples I use are based upon Scrum.
The most common approach to measure developer velocity is by estimating requirements. You break down your requirements into small items—such as user stories —and the product owner assigns a business value. The development team then estimates the story and assigns a value for its effort. It doesn't matter if you use story points, hours, days, or any other number. It's basically a representation of the effort that is required to deliver the requirement.
Measuring velocity with estimated effort and business value can have side effects if you report the numbers to management. There is some kind of observer effect: people try to improve the numbers. In the case of effort and business value, that's easy—you can just assign bigger numbers to the stories. And this is what normally happens, especially if you compare the numbers across teams: developers will assign bigger numbers to the stories, and product owners will assign bigger business value.
While this is not optimal for measuring developer velocity, it also does no big harm if the estimation is done in the normal conversation between the team and the product owner. But if the estimation is done outside your normal development process, estimates can even be toxic and have very negative side effects.
The search for the answer to the question How much will it cost? for a bigger feature or initiative normally leads to an estimation outside the normal development process and before a decision to implement it. But how do we estimate a complex feature and initiative?
Everything we do in software development is new. If you had done it already, you could use the software instead of writing it anew, so even a complete rewrite of an existing module is still new as it uses a new architecture or new frameworks. Something that has never been done before can only be estimated to a limited certainty. It's guessing, and the larger the complexity, the bigger the cone of uncertainty (see Figure 1.2).
Figure 1.2 – The cone of uncertainty
The cone of uncertainty is used in project management and its premise is that at the beginning of a project, cost estimation has a certain degree of uncertainty that then is reduced due to rolling planning until it is zero at the end of the project. The x axis is normally the time taken, but it can also relate to complexity and abstraction: the more abstract and complex a requirement is, the bigger the uncertainty in estimation.
To better estimate complex features or initiatives, these are broken down into smaller parts that can better be estimated. You also need to come up with a solutions architecture as part of the work breakdown. Since this is done outside the normal development process and in time upfront and outside the context, it has some unwanted side effects, as outlined here:
Normally, the entire team is not present. This leads to less diversity, less communication, and therefore less creativity when it comes to problem-solving.The focus is on finding problems. The more problems you can detect beforehand, the more accurate your estimates probably are. In particular, if you treat estimates later to measure performance, people learn fast that they can buy more time if they find more problems and can therefore add higher estimates to the requirements.If in doubt, the engineers who are assigned with the task of estimation take the more complex solution. If, for example, they are not sure if they can solve a problem with an existing framework, they might consider writing their own solution to be on the safe side.If these numbers were only used by management to decide upon the implementation of a feature, it would not do that much harm. But normally, the requirements—including the estimates and the solution architecture—are not thrown away and later are used to implement features. In this case, there is also a less creative solution visible that is optimized for problems and not for solutions. This inevitably leads to less creativity and outside-the-box thinking when implementing features.
#NoEstimates
Estimates are not bad. They can be valuable if they take place at the right time. If the development team and the product owner discuss the next stories, estimates can help to drive the conversation. If the team plays, for example, planning poker to estimate user stories and the estimates differ, this is an indication that people have different ideas on how to implement it. This can lead to valuable discussion and may be more productive, as you can skip some stories with a common understanding. This is also true for the business value. If the team does not understand why the product owner assigns a very high or very low number, this can also lead to important discussions. Maybe the team already knows a way how to achieve a successful outcome, or there are discrepancies in the perception of different personas.
But many teams feel more comfortable without estimating the requirements at all. This is often referred to under the hashtag #noestimates. Especially in highly experimental environments, estimation is often considered a waste of time. Remote and distributed teams also often prefer not to estimate. They often take discussions from in-person meetings to discussions on issues and pull requests (PRs). This also helps when documenting the discussions and helps teams to work in a more asynchronous way, which can help to bridge different time zones.
With developer velocity off the table, teams should be allowed to decide on their own if they want to estimate or not. This also might change over time. Some teams gain value from this, while some do not. Let teams decide what works for them and what doesn't work.
So, what is the best way to estimate more complex features or initiatives so that the product owner can decide if these are worth implementing? Get the entire team together and ask the following question: Can this be delivered in days, weeks, or months? Another option is to use an analogy estimation and compare the initiative to something that has already been delivered. The question is, then: Is this initiative smaller, equal, or more complex than the previous one delivered?
The most important thing is not to break the requirements down or to already lay out a solution architecture—what is important is just the gut feeling of all engineers. Then, have everyone assign a minimum and a maximum number for the unit. For the analogy estimation, use percentages relative to the original initiative and calculate the results using historical data.
The easiest way to report this would look like this:
Given the current team,if we prioritize the initiative <initiative name>,the team is confident to deliver the feature in between <smallest minimum> and <highest maximum>Taking the smallest minimum and the highest maximum value is the safest way, but it can also lead to distorted numbers if the pessimistic and optimistic estimates are far apart. In this case, the average might be the better number to take, as illustrated here:
Given the current team,if we prioritize the initiative <initiative name>,the team is confident to deliver the feature in between <average minimum> and <average maximum>But taking the average (the arithmetic mean; in Excel, =AVERAGE() is used for this) means having a higher or lower deviation, depending on the distribution of the single estimates. The higher the deviation, the less confident you really can be that you can deliver that feature in that period. To get an idea of how your estimates are distributed, you can calculate the standard deviation (=STDEV.P() in Excel). You can look at the deviation for the minimum and the maximum, but also the estimate of each member. The smaller the deviation, the closer the values are to the average. Since standard deviations are absolute values, they cannot be compared with other estimations. To have a relative number, you can use the coefficient of variation (CV): the standard deviation divided by the average, typically represented as a percentage (=STDEV.P() / AVERAGE() in Excel). The higher the value, the more distributed the values from the average; the lower the value, the more confident each team member is with their estimates or the entire team is with regard to minimum and maximum. See the example in the following table:
Table 1.1 – Example for calculating estimations
To express uncertainty in the deviation of the values, you can add a confidence level to the estimation. This can be text (such as low, medium, or high) or a percentage level, as illustrated here:
Given the current team,if we prioritize the initiative <initiative name>,the team is <confident level> confident to deliver the feature in <arithmetic mean>I don't use a fixed formula here because this would involve knowing the team. If you look at the data in the example (Table 1.1), you can see that the average of the minimum (2,7) and the maximum (6,3) are not so far away. If you look at the individual team members, you can see that there are more pessimistic and optimistic members. If past estimations confirm this, it gives you very high confidence that the average is realistic, even if the minimum and maximum values have a pretty high CV. Your estimate could look like this:
Given the current team,if we prioritize the initiative fancy-new-thing,the team is 85% confident to deliver the feature in 4.5 months"This kind of estimation is not rocket science. It has nothing to do with complex estimation and forecasting systems such as the three-point estimation technique (https://en.wikipedia.org/wiki/Three-point_estimation), PERT distribution (https://en.wikipedia.org/wiki/PERT_distribution), or the Monte Carlo simulation method (https://en.wikipedia.org/wiki/Monte_Carlo_method), and they all depend upon a detailed breakdown of the requirements and an estimation on a task (work) level. The idea is to avoid planning upfront and breaking down the requirements and relying more on the gut feeling of your engineering team. The technique here is just to give you some insights into the data points you collect across your team. It's still just guessing.
Effort is not a good metric for measuring developer velocity, especially if it is based upon estimates, and in cross-functional teams, velocity does not only depend upon developers. So, how do you shift from a developer velocity to an engineering velocity?
Organizations with a high engineering velocity outperform their competitors and disrupt markets. But what exactly are high-performance companies?
In April 2020, McKinsey published their research about the Developer Velocity Index (DVI) (Srivastava S., Trehan K., Wagle D. & Wang J. (2020)). This is a study taken among 440 large organizations from 12 industries that considers 46 drivers across 13 capabilities. The drivers are not only engineering capabilities—they also contain working practices and organizational enablement such as the company culture. The study shows that the companies in the top quartile of the DVI outperform other companies in their market by four to five times, and not only on overall business performance. Companies in the top quartile score between 40 and 60% higher in the following areas:
InnovationCustomer satisfactionBrand perceptionTalent managementThe study conducted interviews with more than 100 senior engineering leaders at 440 large organizations across 12 industries. The interview contained 46 drivers across 13 capabilities in 3 categories, outlined as follows:
Technology: Architecture; infrastructure and cloud adoption; testing; toolsWorking practices: Engineering practices; security and compliance; open source adoption, agile team practicesOrganizational enablement: Team characteristics; product management; organizational agility; culture; talent managementThe DVI, therefore, goes way beyond pure developer velocity. It analyzes the engineering velocity and all the factors that influence it and relates them to business outcomes such as revenue, shareholder returns, operating margin, and nonfinancial performance indicators such as innovation, customer satisfaction, and brand perception.
The findings align with the results from the DevOps Research and Assessment (DORA) State of DevOps report (https://www.devops-research.com/research.html#reports) but take them one step further by adding the business outcomes. The DevOps Report 2019 states how elite performers compare against low performers (Forsgren N., Smith D., Humble J. & Frazelle J. (2019)), as outlined here:
Faster value delivery: They have a 106-times faster lead time (LT) from commit to deploy.Advanced stability and quality: They recover 2,604 times faster from incidents and have a 7-times lower change failure rate (CFR).Higher throughput: They do 208 times more frequent code deployments.High-performance companies not only excel in throughput and stability but are also more innovative, have higher customer satisfaction, and greater business performance, (see Figure 1.3).
Figure 1.3 – High-performance companies
Focusing on the measures that highlight the capabilities that set apart high-performance companies from medium and low performers, you can make your transformation visible and provide management with metrics that hopefully matter more to them than lines of code or estimation-based velocity.
"The key to successful change is measuring and understanding the right things with a focus on capabilities."
– Forsgren. N., Humble, J. & Kim, G. (2018) p. 38
To measure where you are on your transformation journey, it's best to focus on the four metrics that are used in DORA—two for performance and two for stability, as follows:
Delivery performance metrics:Delivery lead timeDeployment frequencyStability metrics:Mean time to restoreChange fail rateThe delivery lead time (DLT) is the time from when your engineers start working on a feature until the feature is available to the end users. You could say from code commit to production—but you normally start the clock when the team starts to work on a requirement and changes the state of it to doing or something similar.
It is not easy to get this metric automated from the system. I will show you in Chapter 7, Running Your Workflows, how you can use GitHub Actions and Projects together to automate the metric. If you don't get the metric out of the system, you can set up a survey with the following options:
Less than 1 hourLess than 1 dayLess than 1 weekLess than 1 monthLess than 6 monthsMore than 6 monthsDepending on where you are on the scale, you conduct the survey more or less often. Of course, system-generated values would be preferable, but if you are on the upper steps of that scale (months), it doesn't matter. It gets more interesting if you measure hours or days.
Why not lead time?
From a Lean management perspective, the LT would be the better metric: how long does a learning from customer feedback flow through the entire system? But requirements in software engineering are difficult. Normally, a lot of steps are involved before the actual engineering work begins. The outcome could vary a lot and the metric is hard to guess if you must rely on survey data. Some requirements could stay for months in the queue—some, only a few hours. From an engineering perspective, it's much better to focus on DLT. You will learn more about LT in Chapter 18, Lean Product Development and Lean Startup.
The deployment frequency focuses on speed. How long does it take to deliver your changes? A metric that focuses more on throughput is the DF. How often do you deploy your changes to production? The DF indicates your batch size. In Lean manufacturing, it is desirable to reduce the batch size. A higher DF would indicate a smaller batch size.
At first glance, it looks easy to measure DF in your system. But at a closer look, how many of your deployments really make it to production? In Chapter 7, Running Your Workflows, I will explain how you can capture the metric using GitHub Actions.
If you can't measure the metric yet, you can also use a survey. Use the following options:
On-demand (multiple times per day)Between once per hour and once per dayBetween once per day and once per weekBetween once per week and once per monthBetween once per month and once every 6 months Less than every 6 monthsA good measure for stability is the mean time to restore (MTTR). This measures how long it takes to restore your product or service if you have an outage. If you measure your uptime, it is basically the time span in which your service is not available. To measure your uptime, you can use a smoke test—for example, in Application Insights (see https://docs.microsoft.com/en-us/azure/azure-monitor/app/monitor-web-app-availability). If your application is installed on client machines and not accessible, it's more complicated. Often, you can fall back on the time for a specific ticket type in your helpdesk system.
If you can't measure it at all, you can still fall back to a survey with the following options:
Less than 1 hourLess than 1 dayLess than 1 weekLess than 1 monthLess than 6 monthsMore than 6 monthsBut this should only be the last resort. The MTTR should be a metric you should easily get out of your systems.
As with DLT for performance, MTTR is the metric for time when it comes to stability. The pendant of DF that focuses on throughput is the change fail rate (CFR). For the question How many of your deployments cause a failure in production?, the CFR is specified as a percentage. To decide which of your deployments count toward this metric, you should use the same definition as for the DF.
These four metrics based upon the DORA research are a great way to measure where you are on your DevOps journey. They are a good starting point to change your conversations with management. Put them on a dashboard and be proud of them. And don't worry if you're not yet an elite performer—the important thing is to be on the journey and to improve continuously.
It's very simple to start with survey-based values. But if you want to use automatically generated system data you can use the Four Keys Project to display the data in a nice dashboard, (see Figure 1.4).
Figure 1.4 – The Four Keys dashboard
The project is open source and based upon Google Cloud (see https://github.com/GoogleCloudPlatform/fourkeys), but it depends on webhooks to get the data from your tools. You will learn in Chapter 7, Running Your Workflows, how to use webhooks to send your data to the dashboard.
It is important that these metrics are not used to compare teams with each other. You can aggregate them to get an organizational overview, but don't compare individual teams! Every team has different circumstances. It's only important that the metrics evolve in the right direction.
Also, the metrics should not become the goal. It is not desirable to just get better metrics. The focus should always be on the capabilities that lead to these metrics and that we discuss in this book. Focus on these capabilities and the metrics will follow.
The DORA metrics are a perfect starting point. They are easy to implement and there is lots of data to compare. If you want to take it one step further and add more metrics, you can use the SPACE framework for developer productivity (Forsgren N., Storey M.A., Maddila C., Zimmermann T., Houck B. & Butler J. (2021)).
Developer productivity is the key ingredient to achieving a high engineering velocity and a high DVI. Developer productivity is highly correlated to the overall well-being and satisfaction of developers and is, therefore, one of the most important ingredients to thrive in the war of talents and attract good engineers.
But developer productivity is not just about activity. The opposite is often the case: in times of firefighting and meeting deadlines when activity is normally high, productivity decreases through frequent task switching and less creativity. That's why metrics that measure developer productivity should never be used in isolation, and never to penalize or reward developers.
Also, developer productivity is not solely about individual performance. As in team sports, individual performance is important, but only the team as a whole wins. Balancing measures of individual and team performance is crucial.
SPACE is a multidimensional framework that categorizes metrics for developer productivity into the following dimensions:
Satisfaction and well-beingPerformanceActivityCommunication and collaborationEfficiency and flowAll the dimensions work for individuals, teams, and the system as a whole.
Satisfaction and well-being are about how happy and fulfilled we are. Physical and mental health also fall into this dimension. Some example metrics are given here:
Developer satisfaction Net promoter score (NPS) for a team (how likely it is that someone would recommend their team to others) RetentionSatisfaction with the engineering systemPerformance is the outcome of the system or process. The performance of individual developers is hard to measure. But for a team or system level, we could use measures such as LT, DLT, or MTTR. Other examples could be uptime or service health. Other good metrics are customer satisfaction or an NPS for the product (how likely it is that someone would recommend the product to others).
Activity can provide valuable insights into productivity, but it is hard to measure it correctly. A good measure for individual activity would be focus time: how much time is a developer not spending on meetings and communication? Other examples for metrics are the number of completed work items, issues, PRs, commits, or bugs.
Communication and collaboration are key ingredients to developer productivity. Measuring them is hard, but looking at PRs and issues gives you a good impression of how the communication is going. Metrics in this dimension should focus on PR engagement, the quality of meetings, and knowledge sharing. Also, code reviews across the team level (cross-team or X-team) are a good measure to see what boundaries there are between teams.
Efficiency and flow measure how many handoffs and delays increase your overall LT. Good metrics are the number of handoffs, blocked work items, and interruptions. For work items, you can measure total time, value-added time, and wait time.
"One way to see indirectly what is important in an organization is to see what is measured, because that often communicates what is valued and influences the way people behave and react."
– Forsgren N., Storey M.A., Maddila C., Zimmermann T., Houck B. & Butler J. (2021) p. 18
All the dimensions are valid for individuals, teams, groups, and on a system level, (see Figure 1.5).
Figure 1.5 – Examples for SPACE metrics
It is important to not only look at the dimension but also at the scope. Some metrics are valid in multiple dimensions.
It is also very important to select carefully which metrics are being measured. Metrics shape behavior and certain metrics can have side effects you did not consider in the first place. The goal is to use only a few metrics but with the maximum positive impact.
You should select at least three metrics from three dimensions. You can mix the metrics for individual, team, and system scope. Be cautious with the individual metrics—they can have the most side effects that are hard to foresee.