Experiment in the age of data
Relying on the current capabilities of data processing, many companies have placed continuous experimentation at the heart of their innovation strategy. How can you develop the rigor required for such an approach?
Carried by the technological progress of the latter years, an increasing number of companies rely on continuous experimentation to improve their offerings and operating modes. It is nowadays easy to collect masses of data to identify the highest performing variants in offerings, communication tools, tariffs, production processes, etc. We can also rely on inexpensive solutions to develop prototypes or virtual simulations of various user-experience modes.
This is how Expedia, Netflix or Google have built their business model. These companies conduct hundreds or even thousands of tests every day, with a genuine scientific rigor. When doing this, they are not looking for the next disruptive innovation: they bet on hundreds of small improvements to constantly perfect their product. At Microsoft Bing for example, daily improvements brought to the search engine have enabled an increase from 10% to 25% of the annual revenues per search for several years! This approach does not only concern online services: companies such as Wal-Mart, FedEx or H&M also use continuous experimentation to optimize their offering and develop their revenues.
These successes seem appealing, but they hide a trap: it is easy to extrapolate wrong conclusions from tests that are meant to be scientific. Biases are many and, of course, unconscious. This is how Petco pet shop chain tested its ideas in its best 30 outlets for years… before realizing that decisions were made on the basis of data that were not really representative of what was happening in other shops. Nowadays, the company makes sure it chooses its testing stores randomly, verifying that they are representative of the whole network in terms of size, demographics and localization.
Experimentation has its full place in a strategy based on continuous optimization. But it demands a great vigilance on how experiences are built and their results interpreted. Here are a few pieces of advice to this effect.
the synopse (8 p.)
VisitorI want to buy
this synopsis (8 p.)
Innovate through experimentation
Experimentation is a powerful innovation accelerator which, thanks to the digital revolution, is much more accessible than before. Yet it remains under-utilized. How can you prompt people to acquire the reflex to experiment with their ideas?
Experimentation WorksStefan H. Thomke
Massively experimenting in a rigorous manner to develop your innovation capacity.
How Not to Be WrongJordan Ellenberg
Mathematics are part of our daily life. How can we get the right tools to avoid misinterpreting figures?
La démocratie des crédulesGérald Bronner
[The democracy of the gullible] An analysis of the intimate functioning of our brain, which contributes to easily make us dupes.
Building a Culture of ExperimentationStefan Thomke
How you can encourage the state of mind required for an experimentation culture in the company.
A Step-by-Step Guide to Smart Business ExperimentsEric T. Anderson, Duncan Simester
Concrete advice to conduct experimentations that are rapid, effective and rich in learnings.