How to Use Experiments to Build an Innovation Culture

By Tim Kastelle

When you were learning how to ride a bike, did you work out the theory of how to do it, jump on the bike, and start riding flawlessly? No. You started with training wheels and worked it out through trial and error.

When Thomas Edison invented his version of the light bulb (the 24th to be patented!) did he discover that tungsten was the best thing to use as a filament through logic?  No. He famously found 9,999 things that didn’t work before settling on Tungsten – through trial and error.

When Caterina Fake and Stewart Butterfield launched Flickr, did they work out the perfect business model in advance, then launch the site? No. They started out building massively multiplayer online game – Game Neverending, which became relatively successful, just not successful enough to keep the company afloat. So the team built a site around a photo-sharing protocol that they developed for the game, and that became Flickr. It was business model development through trial and error.

Experimenting is a critical innovation skill. None of us think twice about learning to ride a bike through trial and error, so why is it so rare in business?

There are two main reasons. One is risk aversion. In a work setting, people don’t like to make mistakes, and they don’t like to look foolish. Trial-and-error can cause both of these things to happen when things don’t work out as expected. The second reason is that our organizational cultures are often not amenable to experimenting. In larger organizations, we are often trying to improve efficiency. Doing this means that we must reduce variation. But innovation and experimentation increase variation. There is a tension between efficiency and innovation.

But the benefits of experimenting outweigh these issues. The problem that experimenting solves is this: it’s nearly impossible to know in advance which ideas will work and which won’t.  If we experiment, instead of guessing which ideas will work, we can test them.  This helps us get better making decisions based on data.

Steve Blank recently wrote Why the Lean Start-up Changes Everything and he talks about this as the fallacy of the perfect business plan. The approach is built around identifying your assumptions about how your idea will work, and then gathering data from customers to validate or disprove these hypotheses.

buildmeasure_250Eric Reis has outlined a methodology for Lean Start-up – Build-Measure-Learn.

This is the core method for experimenting. This version works for tech-based startups, but the general principles apply more widely.  We need to use this build-measure-learn loop more in larger organizations as well.

There are many advantages to experimenting. The first is that it keeps failures small. The key to this is building prototypes and testing them.

As Diego Rodriguez says:

“Anything can be prototyped. Prototypes aren’t just for physical products. I routinely see people prototyping services, complex experiences, business models, and even ventures.

You can prototype with anything. You want to get an answer to your big question using the bare minimum of energy and expense possibly, but not at the expense of the fidelity of the results.”

Prototyping is the Build-Code steps in the Lean Start-up loop, and it leads to better results than predicting does.

The second advantage to experimenting is that it gets us out of the trap of over-thinking. The further away we are from our customers, the easier it is to believe that we can just figure out what they need most through logic. But we can’t. We have to test the ideas out – trial and error helps us find what will really work.

Finally, experimenting is a great way to get rid of air sandwiches. That is what Nilofer Merchant calls the gap between management aspirations and the reality down at the coalface.  This is a frequent problem with innovation. It is rare to find a CEO that doesn’t want their organization to be more innovative – but few have tangible plans for how to achieve this vision. Experimenting can help.

If we build a culture of experimentation, people will be more comfortable with examining and testing their assumptions. This is a great way to build the skills that you need to innovate, and it is also useful for finding the best way to make new ideas real.

Here is a method that anyone can apply:

  1. Idea: Think about how much you can get away with – if you manage a budget, how much discretion to you have? If you don’t have a budget, what are the parts of your job that you control?
  2. Build: Make a list of 10 things that you can do within the current scope of your work that will make things better for the people with whom you interact – customers, co-workers, bosses, whoever.
  3. Prototype: Do those things.
  4. Measure: Figure out which ones worked, and do those more.
  5. Data: Figure out which ones didn’t work, learn why not, then forget about them.  Do more of the ones that did work.
  6. Learn: Apply what you learned to the next set of ideas.

If you’re a manager, encourage those on your team to do this as well. As people get better at, things will start to improve. And you’ll start to build a culture of experimentation.

It’s a lot like learning to ride a bike.

Note: I’d like to thank Sarah Green from Harvard Business Review Blogs for help with editing this post.

About Tim Kastelle

kastelle_photoStudent and teacher of innovation – University of Queensland Business School – links to academic papers, twitter, and so on can be found here.


Editor’s Note:  This article originally appeared on Tim Kastelle’s “The Discipline of Innovation and can be viewed here.

Join us at the PARC Innovation Lab, March 13 – 14, 2014. Register today!

Read PARC’s new white paperThe Camera Doesn’t Lie:  Rapid Observation to Create Better Customer Experiences



Additional information

Focus Areas

Our work is centered around a series of Focus Areas that we believe are the future of science and technology.

Licensing & Commercialization Opportunities

We’re continually developing new technologies, many of which are available for Commercialization.


PARC scientists and staffers are active members and contributors to the science and technology communities.