From pilot projects to A/B testing, executives love applying scientific methods to decisions. But companies don’t have years to spend on data analysis: they have to define clear objectives and make strategic decisions now.

That’s why business experiments often become stage performances with scripted endings, according to a recent study. Test designers may consciously or unconsciously advocate an agenda rather than generating truly unbiased information. And that may not be so bad.

Decision makers can and should view the results of experiments critically.

Rebecca Karp, an assistant professor at Harvard Business School who focuses on how companies formulate and execute strategies, studied how experiments can play different roles in business. They can not only be used to collect and learn from data, but also to persuade the public on key decisions.

“The information leaders receive is still very useful for making decisions,” says Karp, co-author of the paper “Business Experiments as Persuasion” with Orie Shelef and Robert Wuebker, professors at the University of Utah’s Eccles School of Business. “Information can also be a useful tool for decision makers when they have other audiences they need to persuade.”

The research comes at a time of consumer skepticism. However, the audience for such experiments tends to be other business leaders and they do not expect them to be comprehensive. After all, companies cannot boast the scientific rigor of other disciplines that rely on double-blind clinical trials or field research over several years.

“Decision makers can and should view the results of experiments critically,” explains Karp.

What the ‘smark’ knows

In recent years, several studies have evaluated the merit of using scientific approaches to business decision making. There is a growing consensus that testing the validity of ideas can help a company decide how to reshape its business model, guide product innovation and investments, and guide training and hiring decisions. These include randomized controlled trials, iterative hypothesis testing, focus groups, and crowdfunded product development.

But some studies indicate that those who design and conduct experiments do not always objectively set out to gather as much information as possible to help managers make decisions. Instead, experimental designers often try to influence managers to reach certain conclusions to secure a strategic advantage, whether it be more resources or better access to a new technology.

Before his recent study, Karp says he personally observed the early stages of 13 pilot programs and experiments related to the adoption of new technologies or other innovations in companies. In each case, he says, the business experiments were intended to produce results that reflected the opinions of the people overseeing them.

Karp relies on the language of professional wrestling to explain the phenomenon. For example, “kayfabe” (a party being organized) could be a presentation to a group of investors. The decision maker influences a “smark” (or an intelligent mark) who is aware that the experiment is rigged as part of the staged performance. However, the smark still benefits from access to information.

When bias is ‘a feature, not a bug’

Karp devised a model to examine how a founder could use an experiment to convince a venture capitalist to invest. The founder could structure the experiment to favor a particular investment, a situation the investor would expect in a pitch.

The better you are at influencing (the VC knows this); They are looking for people who know how to play.

Karp compared the bias inherent in entrepreneurial experiments to that of venture capitalists listening to entrepreneurs’ funding pitches. The public understands that persuasion is ultimately “a feature, not a bug” of the process of receiving information and making critical decisions, as one venture capitalist explained to him.

Persuasion has some plusses

The researchers are not trying to discourage executives from conducting experiments before making key business decisions, Karp says, highlighting how critical it is for companies to carefully research and analyze options when faced with challenges. Researchers also do not encourage decision makers to judge those whose experiments almost inevitably deviate from objective analysis to subjective persuasion.

Instead, decision makers must accept that those who design and conduct experiments are human and that no experiment is perfect or free of some form of bias. Experiments designed to persuade their audiences are still worthwhile, Karp says, as long as decision makers understand their limits.

In fact, storytelling has become a powerful communication skill for today’s leaders. The right experiment could lead to a compelling narrative that helps a startup land its next investment or a company use the right technology provider, the research indicates.