As organizations increasingly strengthen their innovation capabilities, the practice of experimentation is gaining wider recognition and awareness. Owen Gardner, our Head of Experimentation, uses the definition: “the process of performing a scientific procedure, especially in a laboratory, to determine something” and “the action or process of trying out new ideas, methods of activities.”

Ron Kohavi and Stefan Thomke, in their recent in-depth HBR article, demonstrated the value the large digital players are gaining from experimentation, citing a potential 12% revenue increase (more than $100m annually) from one single experiment. The value of Experimentation includes the ability to drive rapid innovation, validate strategic ideas and business model changes, and mitigate risk of failure in the long run. Our own Salesforce teams are enabling organizations to become customer-centric though rapid experimentation (see my colleague Stephen Baines' blog on driving success from experimentation).

Pitfall

In the rush to capitalize on the benefits of experimentation, organizations need to be careful of not falling into “magic bullet” thinking and consider it the cure-all for their transformation delivery. Searching Google already yields 180,000 search hits for both “Magic Bullet” and Experimentation together. The application of experimentation's disciplined and structured approaches to validate ideas in rapid, short-sharp focused experiments provides faster insight and improves return on investment. However, a pitfall comes from confusing experimentation as the new delivery lifecycle to execute and deliver innovation. An approach which mistakenly centers everything on experimentation and leads to a tendency to rush out a hypothesis and quickly start an experiment.

That approach misses out on additional benefits and opportunities from key changes in the industry which can be combined with experimentation. Key changes which includes the convergence of Digital, Agile, DevOps and Cloud to create responsive customer centric platforms, and changes in “way of working,” driven by Design Thinking, Lean Startup and Agile, which are challenging traditional practices and driving essential cultural transformation. The following sections consider these changes and the benefits of combining them to create an overall delivery lifecycle. A lifecycle within which experimentation adds value at many stages.

Individual Capabilities

Let's unpack those industry changes further:

Design Thinking:

Design Thinking, popularized by IDEO, is today at the forefront of innovation approaches. Read Tim Brown's own words for a clear definition and value statement. Focusing on generating great customer experiences is a pivotal part of the journey to becoming a customer-centric organization. The structured observation of human behaviors, and developing conclusions on what people need and want, ensures innovation ideas are grounded in simplified and humanized needs. The prototyping of ideas and acceptance of failure naturally merges with experimentation practices.

Lean Start-up:

Lean Start-up, proposed by Eric Ries, has become essential reading for entrepreneurs. “Minimal Viable Product” and “Pivoting” are now established vocabulary, and the focus on customer experiments and gathering continuous feedback to drive design iterations is becoming essential practice for innovation teams. See one of Eric's mentors, Steve Blank, discuss why Lean Start-up approaches are relevant to every kind of organization, and critical for responding to disruptive forces. The Build-Measure-Learn loop is also a natural point to merge in Design Thinking's Ideate outputs and prototyping steps. Similarly the focus on rapid development of a product incrementally nicely incorporates Agile practices.

Enterprise Agile (with DevOps) :

Agile well established in many ways, yet many organizations are “doing” Agile rather than “being” Agile. Agile is still seen narrowly as a software development process, rather than an organizational culture shift. Which is key to achieving enterprise-level scaling of Agile. The real value comes from the cultural acceptance, top to bottom, of having an Agile mindset. Enabling the empowerment of teams to make rapid decisions, creating multi-disciplinary teams focused on product-centric delivery and the application of lean practices to continually optimize flow. Scrum's sprint demos and retrospectives can readily foster the feedback loops and evidence gathering of “experiments”. Likewise DevOps, an integral part of an Enterprise Agile environment, enables further lean practices to look at end-to-end flow with targeted automation, breaking down more team silos but also embracing measured feedback loops to continuously improve the operating service.

Platforms:

One key ingredient that is often overlooked is the Digital Experience platforms the leading organizations have created to engage their consumers. These organizations continue to gain advantage by iteratively optimizing consumer engagements through A/B testing of experiences with thousands of adjustments. Airbnb's engineers describe running 500 concurrent experiments. Pinterest 1000. And Ron and Stefan's article referenced earlier talks of 10,000+ experiments annually with up to 2.7 million users. Using a platform which gathers analytics and supports the rapid and controlled deployment of change is critical to extract value from experiments. Without this, then just setting up an innovation lab, calling all lab work experiments and missing the insights from researching and building awareness of the customer perspective will limit the transformation results.

Combined Capabilities

At a Gartner conference, I attended earlier in the year, analysts acknowledged the need to prioritize velocity over cost savings (realistically both), and that the demand for application development capabilities will far outstrip available resources many times over. However, HBR also estimates that digital transformation failures range from 66% to 84%.

The pressure is on to increase speed and also achieve transformation successfully. Each of the individual capabilities outlined can drive real change and improvement. But combining these approaches, creating your unique delivery lifecycle, will deliver far greater success and faster value. And at the same time, this can reduce blockers that often impede successful transformation:

• With Design Thinking's Empathize and Define stages to ensure an investment in deeply understanding your consumer. Bring “outside in” perspectives to help challenge the “way we have always done it” thinking, with evidence. But be careful of not rushing to experiment with an idea - value the research time to be human-centered. Experimentation can be effective for progressing ideas that challenge traditional thinking. See Scott Cook, co-founder of Intuit, talk on innovation fuelled by experimentation; how experimentation leads to the best outcomes based on evidence rather than constraining ideas to maintain status quo.
• With Lean Start-up's focus on gathering customer feedback and pivoting when needed. This can help address one of the major transformation blockers in large enterprises: a culture that stigmatizes failure. When a scientist completes an experiment, there are results and learning; we don't say they have failed. With the right framing of experimentation by well-defined hypothesis and metrics defining outcomes, this can give teams the sanction (with executive sponsorship) to try innovative approaches at a lower cost and without personal risk. Stephen Baines highlights this in his blog post on experimentation being all about embracing failure.
• With Enterprise Agile's focus on production-ready software, teams that are motivated to remove flow inefficiencies and multi-disciplinary teams that break down process silos. It also promotes iterative sprint cycles to prioritize and add features that can scale to meet enterprise level usage.
• All underpinned by a Platform that focuses on business functionality, rather than technology plumbing, and enables the rapid delivery of capabilities to engage customers using the insights gathered.

In conclusion, embrace the value of experimentation with all the benefits of scientific rigor but integrate it within a broader delivery cycle with enhanced ways to working to maximize your delivery transformation. Defining the right combination of each of these areas is a challenge, and evangelists of each will describe their capability as all- conquering. There are also further considerations such as incorporating Behavior Driven Development (BDD) or even Hypothesis Driven Development. More interesting still is the use of machine-learning systems to generate hypothesis for experiments (see MIT's R&D, meet Experiment & Scale).

Understandably, everyone would like a pre-packaged, ready-made answer to a new “Innovation led, Customer Centric” framework that combines all the best elements. The answer? There are practical experiences that can accelerate definition and adoption (see my colleague Dr. Natalie Petouhoff's guidance). But there is also a need to come back to the underlying principles of Agile ways of working. The right answer depends on where your teams are in the evolution of their capabilities, and their ability to incorporate new approaches to improve flow. Put the team in the driver's seat, give them access to new approaches, then experiment, and evolve through feedback.

About the Author

Narinder Sahota is part of Salesforce’s Expeditions (formerly ITC) team, a Success Cloud offering that helps our customers achieve their innovation and transformation outcomes. As the EMEA Expeditions CTO he operates across multiple engagements and works with teams and customer executives to challenge the inertia from status-quo thinking, collaboratively creates innovative solutions and provides delivery leadership to propel organizations forward.

See here to learn more about how the Expeditions team guide businesses through digital and business transformations to enhance and grow revenue,