Are Agile and DevOps basically the same? How are they related? Can they coexist in an organization? If so, how can you make the most of combining Agile and DevOps?
Over the past decade, there have been more and more articles, interviews, and talks, that try to answer these questions. At EXIN, we see Agile and DevOps as being highly connected. Interest in this topic only seems to be increasing. So, we invited Jon Kern, one of the co-authors of the Agile Manifesto for Software Development, for an interview. We asked him for his thoughts regarding Agile software development, the challenges, and where he sees DevOps fitting.
1. What is the current status of software development?
Interestingly, my first reaction is “not much different than it always was.” Because, despite knowing better, there are hordes of organizations and teams toiling away in desperate “feature factories” riddled with poor working conditions and soul-crushing command-and-control environments. Many of them would say they are practicing agile development – probably because they have implemented some version of Scrum. Sad to say.
But on the bright side, I will say that the ability to produce positive outcomes through software (and especially the Internet) has never, ever been easier.
There is no excuse for teams to not be creating highly professional software on the cutting edge of DevOps “literacy,” so to speak.
By “literacy,” I mean understanding the principles behind DevOps concepts. For example, knowing why containerization can be useful or understanding that a developer’s job doesn’t end when code is committed. Or being aware that simplifying deployment and releases is only the beginning – you also need to consider monitoring and self-healing. And so on.
In addition, it has never been easier to learn how to build some things with software. It has also never been easier to actually deploy an application to “The Web” or even build an app for your Smart Phone.
Like any engineering discipline, there is a lot more than “meets the eye” to building quality commercial software.
2. It’s been almost twenty years since the Manifesto for Agile Software Development was published. What are the top challenges still being faced today?
The biggest challenge is what I call “agile in name only.” That is the usage of a process or a framework, but not really understanding that a tool or a process alone, while it may be necessary, may not be sufficient.
In 2000, we battled the marketing arms of the likes of IBM and Rational, touting heavyweight processes like RUP.
Today we battle the marketing arm of Scrum and SAFe™. Scrum is great stuff. A wonderful way to get started on your journey. But like anything, just doing ceremonies in a dogmatic fashion for six months in a row, well, that might be “agile in name only.” You failed to understand the essence of agile. To learn. To grow. To be curious. Agile is a mindset, not a set of ceremonies and process steps and roles and documentation templates.
With all of the teams I work with, I stress knowing the business purpose. Don’t just be an “order taker” and build a set of features. But rather understand the context in which the user needs the feature – what outcome is the user expecting.
And, since software development is basically like gambling, I help teams learn how to break features down into smaller bits of value – essentially placing smaller “bets” in the game of software poker.
The real key is being able to get rapid feedback – be it at the code level from TDD (test-driven development) or BDD (behavior-driven development), or at the user level from working with a running system. The faster (and more reliably) a team can deploy code, the better.
3. Where does DevOps fit in the context of software development?
So in 1995, my small boutique consulting firm was architecting and building out a new manufacturing execution system for IBM. One of the first things we got working – even as we were finishing some of the first-pass, high-level requirements gathering – was a server, a build and deploy script, and a build “button” that even I could push to deploy the latest code. And I could check nightly build failures, test results, lint checker results, etc. 1995.
Welcome to DevOps. Only back then, we did not have a name for the practice. We just called it being smart. Oh, wait. No, our mantra (thanks to my work with Peter Coad back in the 90s and beyond) was “frequent, tangible, working results.”
I have seen the anguish in shops that are too busy fighting fires to stop and try to get it right. So many teams and organizations tackle the symptoms. When the root cause can often be traced back to not having a reliable development and deployment process. Some organizations deploy and then have a day of firefighting when they see what is really happening in production. Shockingly, it is like changing a flat tire on a car that is slowly rolling away. Teams are literally paralyzed for fear of breaking production because they cannot have a reliable test environment or automated tests to trust.
4. Which of the Agile challenges mentioned above can be solved by applying practices from DevOps? And what are the main benefits in your opinion?
The biggest reason to implement a DevOps strategy is really to combine the best of development and operations knowledge to simplify how you can deploy, run, maintain, and monitor an application.
From virtually Day One, you can get started with attempting to set up techniques and processes and automation. There are even Platform As a Service (PaaS) offerings (e.g., Heroku) that make it easy to set up a deployment pipeline and automate Ops stuff based on a Dev action.
This solves the problem of getting early and frequent feedback.
5. Do you see any challenges in putting together Agile and DevOps? How do you think they can be solved?
The primary challenges are not for new teams starting out on their journey. They should understand how to set up a lean DevOps process and structure.
But for some enterprise organizations that have numerous departments, it is easy for the Dev + Ops to get lost.
The best way to solve that is to slow down and solve the problem. Or set aside a special cross-functional team charged with coming up with a CI/CD automated pipeline solution.
6. Have you come across any misconceptions with respect to making Agile and DevOps work together? How would you debunk them?
I have seen one enterprise organization treat DevOps like it is some other group… In other words, it is the DevOps team’s responsibility to take the developer artifacts and make it real and in front of users.
Well, it will work a lot better if the two “sides” actually collaborate as there is much to be learned when you have to do it all yourself. You learn better ways to code and architect that improve operations. And you learn better ways to orchestrate deployments and servers when you understand the code.
Work together. It is the easiest path to success.
7. How can organizations get started with Agile and DevOps? And why should they?
It is pretty easy. In theory anyway. We often kickstart a team on a new project with a 2-4 week jump start package that combines to guide a team through getting a project started, and includes a rudimentary process and deployment automation
If you blend Dev + Ops from the outset and have the mindset about delivering business value, you will reap the rewards of a pleasant and lean process and team. Start small, when that’s all you need, and grow with your needs.
The key is not to let silos form between “departments” and imagine the simplest process possible for development and DevOps. And then fight hard to keep things simple. And push hard for automation.
For organizations stuck working in slow motion, we can often help diagnose the issues with their value streams (processes) to get them back on the right track with agile and DevOps.
8. Do you see any emerging trends in software development?
I hope the trend is one of continuing to wake people up that breaking down the silos of development and operations is a key to improving on the “people” side.
Also, developers that understand more of the “big picture” – and not just some shiny new tech – will be more valuable to organizations. Higher-skilled developers are “worth their weight in gold” as the saying goes.
In addition, the ease with which containerization can be achieved at the development team level through to deployment tools is spectacular. This should help reduce the friction experienced by many traditional non-containerized technology stacks in terms of everything from onboarding new developers, to deploying, to offering resilient and A/B deployment options.
Hopefully, the trend will continue where it is easier to “snap” into place systems that are not as core to your business, but rather are contextually needed – be it for security or monitoring, or other purposes.
More about Jon Kern
Currently working at Adaptavist.com as an Agile consultant. Jon, a co-author of the Agile Manifesto and aero engineer by training, is passionate about helping organizations succeed in delivering business value through software. He works with teams to articulate, design, architect, and deliver software that solve challenging business problems.
His varied career has spanned jet engine R&D, fighter plane flight simulators, to being an object-oriented and lightweight process evangelist through the 90s, authoring books with Peter Coad, helping to form TogetherSoft, and his work co-authoring the Agile Manifesto for Software Development in 2001. Jon has worked with startups and smaller orgs, with mid-sized companies, and with enterprises.
Jon seeks better ways for teams to accomplish their goals from the perspectives of people, process, and technology. Jon likes to help teams build an autonomous environment that enables effective practices, solid architecture, agile development, quality-by-design (not accident), and laser-like focus on delivering business value through strategic use of s/w development.
Specialties: beer, agile development, agile coach, domain modeling, architectural solutions that meet business needs, leading distributed teams, highly varied domain expertise, leadership by example.