private equity Archives

Using Independent Research to Avoid the Bullseye of Bias

Posted on January 28, 2019January 28, 2019 by jfhannon

Over the past ten years the Justkul research team has worked on hundreds of customer surveys for investment deals and strategic initiatives. Many of our clients could execute research themselves, but they have come to value the quality of our work, our expertise in knowing what questions to ask, our skill in avoiding common mistakes, and the way we free up their team’s time to concentrate on other aspects of a deal or strategic initiative. However, there is perhaps one factor more important than all of these: the independence of our research. Unlike most other parties in a particular deal or initiative, Justkul never has significant vested interest in any particular outcome or conclusion: Justkul is paid the same amount whether a deal or strategic initiative goes forward or whether it does not. We believe this type of independence is essential for minimizing bias.

Bias, or an inherent prejudice for or against a given outcome, can have a significant impact on the conclusions of a research study. It is often underestimated how easy it is for bias to enter a research project. Bias can creep into a survey at almost any stage in the process:

In the decisions of which questions to include,
In who is invited to take part in the survey (digital redlining, etc.),
In the phrasing of particular questions and the order and phrasing of answer choices,
In adopting a narrative flow that favors certain answers over others,
In decisions as to which data points are included or removed from the final data set,
In how the data is interpreted or summarized.

A lack of independence at any of these stages can produce misleading results, which lead to poorer and more risky decisions.

Hence, in order to make sound decisions, it is important to understand the potential causes of bias so that conscious decisions can be made about how to minimize the impact of these causes.

The Bullseye of Bias

The causes of bias are various, but one way to organize these causes is as the bullseye at the top of this article with intentionally malicious causes of bias in the center, intentionally non-malicious causes of bias in the next concentric circle, and unintentional causes of bias in the outer circle.

In this schema the most nefarious type of bias is in the center, but as its relative area suggests, in our experience it is fortunately also the rarest type. Far more commonly, biases arise from the causes in the outer circles. Because of the importance of avoiding this bias, it is worth looking at each of these types of causes in more detail.

Intentional and Malicious Causes of Bias

Intentionally-malicious causes of bias occur when decision-makers make a conscious decision to either lie or omit important details that would be likely to change the overall outcome of a project or decision. “Intentionality” is a complicated concept (See Anscombe, Intention), but in this case, we only mean that it is the result of explicit choices.

Justkul team members have encountered this situation on more than one project in the past. For instance, in one project we worked on the buy-side of a private equity deal and were asked to gather customer data on a high-end consumer product company. The potential investment looked great from its own presentations: its products were attractive, prior survey work showed that clients liked the products when they initially bought them, the management team seemed experienced, and the short-term financial data looked solid. Nevertheless, at the time the potential investment had a dark secret it was actively seeking to repress: its products were prone to breaking down. In an apparently deliberate effort to conceal this fact, none of the prior survey work or reports mentioned issues about repair and maintenance, and the company deliberately went out of its way to delay sharing customer contacts with the Justkul research team. The malice of this strategy was made explicit when a leader at the company asked if they could just wait a few more days to share customer contacts, which would mean that we would only get the contacts after the deal closed. This raised red flags and after we communicated it to our private equity client, the deal was eventually called off.

In this case, the explicit request from the executive made it fairly easy to identify the bias in the information the company shared. However, in our experience such obviously malicious cases are fortunately rare. More commonly, biases are introduced for non-malicious reasons, either intentionally or unintentionally.

Intentional and Non-Malicious Causes of Bias

In intentional and non-malicious causes of bias, a conscious decision is made to provide incomplete or potentially misleading data, but unlike the malicious case, decision-makers have grounds to justify their decision. Decision-makers might even convince themselves that they are making the right decision under the circumstances, such as when a company gives into the temptation to ignore survey results that do not fit in with their overall narrative. For instance, a company might nearly always receive a very customer satisfaction score in their internal survey results, but one independently-conducted survey might show a far lower score. Perhaps the company could provide a fairly good argument that because 95% of surveys showed a different result, this result is merely an outlier and can be ignored.

However, it might also be the case that there is something wrong with the other 95% of surveys. This suggestion may seem radical, but it’s actually not unusual: perhaps the company’s survey tool was inherently biased to begin with by making respondents only focus on positive experiences, or perhaps there were factors in the recruiting process that encouraged only respondents who were more enthusiastic about the company to complete the survey, or perhaps survey respondents were not assured the anonymity of their results. The importance of anonymity has been underscored for us many times on telephone interviews — especially in B2B industries where customer-sales rep relationships are key. In such interviews, a respondent will often ask for reassurance that the interview is truly anonymous before proceeding to give critical feedback. This is the reason Justkul generally insists that our survey respondents be assured anonymity in addition to confidentiality.

Unintentional and Non-Malicious Causes of Bias

Finally, the most common case is non-malicious and unintentional causes of bias. Examples are numerous. An investor who has an interest in having a deal go forward might overlook some key hypotheses that would suggest it should not go forward. A consultant who thinks a company should focus on countering the strategies of a specific competitor may overlook the challenges posed by other competitors. A clothing company that believes the quality of its fabrics are obviously better than competitors might be less inclined to put that hypothesis to the test, even though customers may actually perceive the fabric as lower in quality. All of these assumptions can have a significant impact on the outcome of research, and thus produce data that is inferior to research that is conducted independently.

Finally, even if data is conducted well—using surveys that are designed not to favor any particular outcome and recruiting participants in an un-biased manner so that the resulting data is truly representative of the underlying population—there is still considerable possibility for bias in the interpretation of those results. Negative data points can be ignored, other positive data points may be artificially promoted, and almost any survey result is potentially manipulated through spin or by ignoring important qualifications. The solution to all of this is to take steps to ensure the interpretation of the research is also independent.

Conclusion

Companies often have to rely on self-collected data to make routine business decisions, and this is good and entirely appropriate, provided the company take steps to limit bias in the data set. However, if an unusually important or difficult strategic decision needs to be made, for the reasons outlined here, we believe there is no substitute for data from an independent third-party research partner.

Why Would A Private Equity Firm Hire An Outside Customer Research Company?

Posted on June 25, 2015September 17, 2026 by John Hannon

One could argue that no component is more important for an investment's long-term success than understanding the relationship between a company and its current and potential customers.

As the CEO of a customer research firm that works primarily with private equity, I am often asked what value customer research can bring to a private equity deal. People outside the private equity industry sometimes find it surprising that customer research can have a role in transactions that are so often portrayed as driven solely by deal-making and hard numbers. Even some people within the industry undervalue customer research, thinking that a few telephone calls or general market analysis is all that is required. However, the private equity industry has changed significantly over the last few decades. Whereas a simple LBO model with minimal customer research might have been sufficient for good returns two decades ago, high valuations and increased competition among firms is leading the industry to more sophisticated and strategic thinking. This is reflected in who is involved in a deal: whereas in the early days an investment might be largely driven by bankers or financial officers, now strategic consultants – whether in-house or hired from external strategy houses – are playing an increasingly important role in investment decisions. This rise in strategic thinking coincides with an increased focus on the “softer” sides of a deal, such as customer research. Despite often being classified as a “softer” component, one could argue that no component is more important for an investment’s long-term success than understanding the relationship between a company and its current and potential customers. In the end, the success of every business depends on this crucial relationship: it is where growth comes from, and it is what gives a company an advantage over its competition. Why would a private equity firm want to conduct customer research? In my experience there are at least five reasons why private equity firms conduct detailed customer research before a deal closes:

Good customer research can help investors identify unexpected strengths or weaknesses of an investment. This is perhaps the most traditional reason for conducting due diligence research. Knowing what risks investors may incur and learning how the investment’s products and services compare to its competitors can be important not only for adjusting valuations, but also for deciding whether the deal is even worth pursuing to begin with.
Good customer research can help investors develop and validate future growth strategies. Often an investment hinges on a number of very specific strategic growth hypotheses. By testing each of these hypotheses in detail with the current and potential client base, a private equity firm can get a sense of whether their assumptions are correct and what strategic changes will have the biggest impact on a deal’s success.
Good customer research can reduce uncertainty. Good research can also help answer a multitude of smaller questions and thereby reduce the uncertainty inherent in a deal. What effect will relocating a company headquarters have on sales? Will a potential acquisition be an asset to the brand? Will customers be open to purchasing an adjacent product or service? All these questions can be answered through good customer research.
Providing an acquisition with the results of good research can help set future expectations. Companies often initially succeed with a great idea, a superior product, and funding from angel and VC investors. At these early stages, a company may perform A/B testing and benchmark performance against its competition, but detailed market research is seen as an expensive distraction. However, when a firm reaches sufficient growth to enter the middle market and become a potential private equity investment, the role of research can change dramatically. Now, not only will research be important for setting wider brand strategies, but it is also necessary to fulfill board reporting requirements and for ensuring a firm is sufficiently transparent with investors. Conducting good customer research before a deal closes can help set expectations for what these future reports should look like.
Good research can create a useful dialogue between a company’s management and investors. Good dialogue between investors and a company’s leadership is essential for the success of an investment. Research helps a private equity firm gauge their ability to work with a company’s leaders. A management team that engages with unexpected insights rather than seeking to merely dismiss them is a good first sign that a successful partnership based on mutual trust will develop. I know of deals that were called off because such dialogue failed to be productive or management was unusually defensive. More than this, by talking through a research study’s findings and developing strategies together, both investors and the company will begin to see a deal as not just a transaction, but as the first step in a long-term partnership together.

By talking through a research study's findings & developing strategies together, both investors & the company will begin to see a deal as not just a transaction, but as the first step in a long-term partnership together.

As these five reasons suggest, customer research can be an important tool in the due diligence process. It not only provides investors with better information at the time of an investment decision, but it also can be useful for helping set future expectations and strategy. I emphasize above that “good” customer research does all these things. However, research that is performed without sufficient care and attention can lead investors seriously astray. Examples of these types of errors include insufficient care in phrasing or in ordering questions, paying insufficient attention to the UX of survey participants, not appropriately stratifying sample, not sufficiently pilot-testing tools to avoid technical errors, or not sufficiently exploring unexpected findings. These kinds of mistakes can often provide misleading data that makes bad investments look good or conversely can lead an investment team to pass on a very good investment. Despite the ubiquity of fast and inexpensive survey tools, getting research right can often require considerable thought, time and expertise. Why would a private equity firm want to work with a third-party research firm? Some private equity firms may have research expertise in house, and others may obtain the required expertise through strategy consulting firms. Even in these cases, though, considerable advantages can be obtained by partnering with a specialty customer research firm. A third-party research firm can provide four additional advantages to the process:

A third-party research firm can provide experience and expertise. Good business research requires skills that can be very different from those that make for a good investor or management consultant. A company with a culture that values and promotes employees who have these skills will often produce more solid results. In addition, third-party research firms often conduct research in a variety of industries and geographies, some of which may be outside of a private equity firm’s or consultant’s expertise. A third-party firm may therefore better know how to properly adjust research methodologies for these different markets and recognize unexpected questions that have cross-market application. For instance, a question that produces fruitful results in consumer packaged goods may also have applications in B2B contexts, and a research approach that makes sense in the U.S. may not make sense in Singapore.
A third-party research firm can help provide an additional level of discretion. Deals are complicated transactions that can have significant impact on different stakeholders. It is often useful for a board and potential investors to investigate a potential deal or partnership without causing unnecessary disruption if the parties involved in the deal decide to go in different directions. A third-party research firm can serve as a firewall between the research and the larger public, thereby preventing a potential deal from garnering unnecessary attention until it reaches a stage at which it makes sense to do so.
A third-party research firm can help reduce bias. There sometimes can be intentional biases in a deal: a company seeking to be acquired may only report positive data or fail to report potential risks adequately. In this case, independent research can ensure important questions are asked before a deal closes. However, more often biases enter unintentionally: a company that has a strong culture may be blind to its own weaknesses, and a company that only relies on A/B testing of current customers may have unrealistic expectations of the potential in the larger market. In addition, even research conducted internally by a private equity firm may carry some bias: team members who are personally invested in the success of a deal may be unable to step back sufficiently to recognize risks. Research from a third party that has no stake in whether a deal closes or not can significantly reduce this bias.
A third-party research firm can be a valuable resource for an acquisition after a deal closes. As I mentioned earlier, many companies that PE firms invest in are in the process of making a transition between a startup/VC-fueled business model and a business model that is focused on expansion and growth. An independent research firm can serve as a bridge, providing research guidance until those capabilities can be built up in-house. Furthermore, even after a company develops extensive internal research capabilities, the company’s executives may nevertheless find it useful to bring in the external company that conducted the due diligence research in order to verify assumptions before making important strategic decisions.

As company multiples steadily increase, and as it becomes consequently less likely that a private equity firm will find an amazing company at a great price, good customer research is becoming an increasingly important component of successful private equity deals. Given that good customer research adds little to the overall cost of a deal, and given the significant benefits and insight independent research can provide, we think good customer research should be an important part of any private equity deal. By @jfhannon, CEO at Justkul Inc., a research firm focused on the needs of strategy and private equity.