This post was written as part of the Supporting Agile Adoption program, an Agile Alliance initiative dedicated to supporting organizations and their people to become more Agile.
The evolution of product development
Product development has changed significantly over the past decades. Waterfall worked well for smaller projects focusing on smaller (software) products. When products became larger over time, waterfall didn’t work so well anymore because the number of people involved in the creation of value became very large.
In 2001, the Agile Manifesto was created as a counter-movement to this linear and rigid development (waterfall). Not much later — driven by the spread and the speed of the internet — products became connected, and they became larger and larger. With this product development, complexity amplified the dependencies between components and between people. Centrally steered teams became a bottleneck. That problem was addressed by the approaches defined by the Agile Manifesto.
Increasing complexity
Yet, this wasn’t the end of the story because then software products became even more complex: with the emergence of open source and cloud computing technology, software and hardware became de-coupled, and software solutions made use of many components that were not even developed by a single company. That created even more complex software ecosystems with many interdependencies.
On top of that, with the demand for digitization, software spread not only across industries, but globalization made it possible for software to also spread quickly around the globe. While software products had only a few users in the past, now they could have billions of users. And that had a huge impact on society.
The need for responsibility and sustainability
Thus, how products are created has changed, and how and by whom they are consumed has changed. The consequence of both is that societal responsibility for product development has changed. Stakeholders should be aware of the societal impact of their efforts.
To summarize, due to the growing entanglement of different players (both creators and consumers), software development and its impact increased in complexity, and therefore, development teams need to be highly aware of their responsibility and sustainability. Waterfall, Agile, XP, and Lean have not been designed with this challenge in mind.
The future of product development
Where is Agile going next? Or maybe more relevant, where is product development going?
To answer this question, we first need to better understand the context of product development by exploring three core aspects:
- How value is consumed? (a question of scale and responsibility)
- How value is created? (a question of experience)
- How is consumption affecting value creation? (a question of collaboration)
How is value consumed?
The increasing impact of products
We have seen that the result of product development is having a big impact on mankind, society, and the planet. For example, fifteen years ago, the iPhone was launched. Before that, there was no mobile internet. Ten years ago, Facebook was launched. Before that, there was no omnipresent social media. Also, ten years ago, the “Internet of Things” was in its early days. Today, we have billions of devices connected to the internet — from light switches over thermostats to cameras and personal speech assistants.
The rise of societal concerns
As a consequence, today, we see topics like security, privacy, ecology, and ethics getting more and more important. This is new. On top of that, in the past, software products had to deal with a few users only. Now, they have to serve billions of users. The scale of the user base alone has led software to have a much bigger impact on society than before.
The societal impact of social media and AI
Social media has a real impact on society. For example, with Elon Musk having taken over Twitter, he decided to ignore the idea of filtering content. The software was changed on X, and now all content comes unfiltered, which has a real impact on societies and causes a societal response (for example, people leaving X and investigations in the EU). Furthermore, in today’s world, where artificial intelligence is in its infant learning phase, large language models are being fed by the available data that is sourced from the World Wide Web, like the unfiltered X content. This means the AI might show some of the bias that the X community shows as well. This is a reason for concern.
The need for sustainable and responsible product development
Similar to sustainable and responsible supply chains, where a manufacturer tries to guarantee that the many distributed sources they are using are ethical and environmentally friendly, also in today’s software industry, we have the challenge of knowing where things come from and what they include.
Today, product development has to integrate sustainability concerns, shareholder views, customer-centricity, and others. This integrative perspective is new as well. Products today have a wider and deeper impact on mankind, society, and the planet. While software is (or has been) eating the world, now product development is “eating the world.” This systemic impact of products requires a systemic view.
How is value created?
Value streams across companies and industries
Value streams are not within one company anymore but stretch across companies and industries. Consequently, in today’s world, product development needs to be addressed more holistically and systematically, both from a development point of view and a usage point of view. The challenge and invitation alike is that developing products now means reusing components (e.g., via open source, microservices, etc.), integrating multi-vendor solutions, and co-developing with other partners. However, this creates many dependencies with the result that software is not only large from the development point of view but also regarding the necessary interactions of co-developers. While software development in the 1980s was complicated, today, it is complex because of the entanglement of the software ecosystems.
The challenge of end-to-end customer experiences
Additionally, for developing products that are beneficial to our customers, we often examine customer journeys. That is an exploration of how customers will use the product and support them in the best possible way. Yet today, customers are not using a sole product but rather a series or combination of products for their journey. Thus, integrating these products into an end-to-end experience is a challenge. And let’s not forget about the complexity of the related maintenance and lifecycle management because all these individual products that form the solution have their own release and update cycle.
The disconnect between developer perspectives and customer experiences
The whole experience is so complex that developers create a perspective for themselves to keep it manageable. This perspective has a close connection to customers, but the developers collaborate with them only on their specific product. This means the developers ignore the end-to-end customer experience, how their specific product is embedded in the full customer journey that might cross not only products but also companies. Therefore, the customer journey is no longer fully owned by and in full control of a specific company. Consequently, the connection between the various perspectives is either lost or not there from the beginning. The individual product companies seem unable to manage the complexity of aligning and integrating on a solution level, and thus, the customers need to integrate all the products themselves.
The need for cross-company collaboration
The change that has been going on for some time now is that it is not any more “only” customer collaboration as promoted by the Agile Manifesto, but also collaboration with the other stakeholders/providers from different companies. The first value described in the Manifesto, “Individuals and interactions over processes and tools,” now becomes even more important: “Individuals and interactions” have been mostly interpreted as “within the same company” – yet in the past 20 years, the need to have “individuals and interactions” collaborate across companies has surged. Thus, the promise of freedom in Agile ways of working doesn’t give people the freedom to define their boundaries anymore. Instead, we need to answer the question, “Who is taking care of the end-to-end perspective?” Everyone? Noone?
Sharing responsibility for value creation
A valid approach is sharing the responsibility! This means sharing needs to happen across departments, companies, and economic ecosystems (e.g., open source). This need has made value creation increasingly non-linear and complex.
How is consumption affecting value creation?
The limits of Agile in a complex value chain
Things are becoming more complex and interwoven. Has Agile maybe reached its limits of what it can achieve now that complexity includes a value chain with many interdependencies? What preconditions need to be fulfilled for such a value chain to operate in an Agile and sustainable way?
The non-linear nature of customer journeys
As customer journeys aren’t linear anymore, it’s more likely that the customer jumps from one thing to the other, and the journey turns into a web of actions and options. Customers often have an explorative approach to satisfy their needs — and that means each customer might have a different journey. For value creators, there is an increasing motivation to nevertheless make these complex and unique journeys enjoyable. For, if they fail with that, customers will change their journey and remove these non-value-adding parts from their explorative value chain.
The need for a cross-company, customer-centric perspective
To take the customers’ web of options into account, value delivery now requires a chain perspective, focused on a customer need, that goes across companies and ecosystems.
The collaboration challenge
The challenge for value creators is that the necessity to integrate their contribution with many different neighboring offerings generates a pressing need for collaboration. That entanglement drives complexity to a level where traditional (Agile) approaches come to a limit because of the collaboration capability limit of humans.
Reducing entanglement through disentanglement
So, instead of increasing or trying to optimize collaboration, we need to decrease entanglement which will reduce the need for collaboration. Collaboration should happen where it is unavoidable or absolutely needed. However, we need to learn to identify where and when collaboration is avoidable and to find strategies to replace collaboration with structures, practices, and processes that enable disentanglement.
Example: Microservices
An example is the use of microservices: due to a very well-defined API and guaranteed backward compatibility, microservice developers can work independently of the users of the microservice. This includes lifecycle management independence. And, application developers can work independently of microservice and other component developers.
The benefits of disentanglement
By disentangling, we enable effective collaboration and seamless flow across complex value chains. This way, we limit the collaboration effort and improve collaboration quality. On top of that, we can parallelize value creation — working simultaneously and independently of each other — reaching new levels of efficiency.
Conclusion
The changing landscape of value creation and consumption
Since the introduction of the Agile Manifesto, the creation, consumption, and how consumption affects the creation have changed. Created value is affecting societies and ecosystems. It is having a planetary impact that we need to be aware of. The consumption of value has changed and now requires us to have a systemic perspective that includes the customers, the shareholders, and the ecosystem.
Expanding the scope of value creation
Thus, it is not enough to create value for the customers’ advantage (as described in the second principle of the Agile Manifesto) but also for the planet’s and society’s advantage.
Shared responsibility for outcomes
We must create a deeper awareness of how value is consumed. Everyone who is working in product development has a responsibility for the outcomes in terms of societal, ecological, ethical, and economic impact. Awareness of our impact must lead to sharing the responsibility, as we are all affected by the outcome.
Organizing the value creation network
When taking responsibility and collaborating across the wider value chain, we need to learn to organize this huge network. At a minimum, we need to disentangle it. This means we need, where possible, to work in parallel both locally and globally. That requires a common sense of direction — a “global” value creation alignment that empowers local action and decision-making. That alignment needs to be based on the above-mentioned awareness of our responsibility regarding the outcomes we create jointly.
Recommendation: Think global, act local
Our recommendation, therefore, is to address all three perspectives to revitalize your value creation flow.
In summary, be aware of the ecosystem impacts you have when creating value. Working together means we have to collaborate across very large networks. To do that effectively, we need to have a shared sense of direction and drive disentanglement that helps us work in parallel. Simply put, “Think global, act local.”
