Improve your research with the mixed-method approach
As product managers, we often find ourselves neck-deep in research, of which a lot is dedicated to finding, validating and understanding the problems we are solving.
What do our customers care about? What are they doing with our product? Why are they behaving this way? How is our last release performing? Why is it performing like that? Is there an appetite for this idea on the market?
We rely on data to answer these questions. Quantitative data; expressed in numbers, producing objective and empirical evidence. And then there’s qualitative data, which is descriptive and allows us to explore and understand opinions, attitudes and experiences.
These data types are then used to converge towards evidence that supports or invalidates our theories and assumptions. However, people often focus their research on a single method only. This limits the story the data is telling us.
To illustrate the importance of using mixed methods, let’s consider an example involving a mobile app that sees a decline in user engagement. If you would take a qualitative-only approach, you might gather rich insights into your users’ pain points by conducting a series of interviews. However, you don’t know if these issues reflect on your entire user base, as these individual experiences don’t paint a clear picture of the full scale of your problem or the demographics most affected.
On the other hand, if you only rely on qualitative data, you would gain a broad view of user engagement and understand exactly where it has dropped. This data, however, does not explain the underlying reasons that caused this decline.
By combining both research types, you can pinpoint the specific areas of your app and the affected user profiles, which will help you find the right users to interview and gather the nuanced reasons behind the decline in engagement.
Mixed methods open a deeper perspective on your research
Mixed method research is a practice originating from the social and behavioural sciences that blends qualitative and quantitative research methods to answer a single research question. It helps you to gain a more comprehensive perspective and improve the credibility of your findings by leveraging the strengths of both data types. Additionally, this method allows you to address more complex research questions where a single qualitative or quantitative study would fall short.
Let’s take a look at some examples of mixed-method research questions:
- How do changes in the user interface design of our mobile app (quantitative) impact user perceptions of usability and satisfaction (qualitative)?
- How do different pricing strategies (quantitative) impact customers' perceived value and willingness to pay (qualitative) for our product?
- How do user perceptions of the onboarding process (qualitative) correspond to drop-off rates at different stages of onboarding (quantitative)?
Choosing between mixed methods and single-method
There are several variables at play to determine if a single qualitative or quantitative study will not be sufficient.
Are your findings generalizable? Is the size of your sample large enough to be statistically significant and is the sample representative enough for your population?
Is your research credible? As you want your findings to be valid and confirmed, a mixed-method approach provides an opportunity for triangulation. This involves using multiple data sources and methods to confirm if your findings from both methods align.
Do your research findings need more context? Adding qualitative context to your quantitative analysis will add richer detail to your conclusion.
If your research question can be answered in a credible manner by only using a qualitative or quantitative approach, mixed methods might not be the right fit.
Designing your mixed methods study
The design strategy you choose within your study depends on how you approach the collection and analysis of your data. There are three main types of mixed-method design approaches to conducting your research:
In parallel convergent design, you collect both quantitative and qualitative data simultaneously and then compare results. This is great when you want to triangulate data to strengthen the credibility of your findings.
Exploratory sequential design starts with qualitative data collection and analysis. Quantitative research is then used to further generalize your findings. This approach would be ideal if we conduct user interviews, and then use these insights to develop a survey for wider validation.
Explanatory sequential design begins with quantitative research. Qualitative data collection is used later to understand these quantitative results. The goal here is to first discover patterns and trends, and then explain the ‘why’ behind them.
The challenges of using mixed methods
While the mixed-methods approach offers many benefits, there are also challenges to factor in when you decide on your approach.
The analytical complexity of mixed-methods research requires more effort and resources than single-method studies. As data volumes are larger, data collection and its integration to provide meaningful insights become more difficult. Conducting and analysing your research requires more time, which isn’t always available. Researchers need a diverse skill set to work with both methods, avoid bias, and ensure reliability.
Additionally, your results can be difficult to interpret if you get conflicting results from both data types, creating uncertainty on how to proceed.
