CX Research Strategy and Quarterly Survey Design for Online Casinos

Preface: Before proceeding further, it’s important to emphasise that the choice between exploratory and evaluative research depends on the iGaming enterprise’s research maturity and its current game offerings. Furthermore, the survey design proposed in this blog is subject to amendments and entirely up to your research needs. I recommend conducting pilot tests and iterating on the survey design to effectively capture customer metrics.

Research teams understand that successful online casinos place significant emphasis on understanding their customers’ perspectives. They also recognize that business metrics such as revenue and churn only tell part of the story.

This is where my CX (Customer Experience) research initiative comes in. In this blog, I’ll be discussing best practices for CX research strategy and quarterly survey design specifically tailored for online casinos. The implementation of this research program is based on the concept of collecting customer feedback, both qualitative and quantitative, with the overarching goal of benchmarking performance over time and cultivating trust in the long term through customer research and iterative game development.

CX Research Strategy

Stakeholder engagement

Laying out the research strategy and securing research buy-in run in tandem with understanding your stakeholders’ needs. It is advisable to engage not only with game developers but also to arrange one-on-one discussions with members of the broader product team, including individuals from UX, marketing, product management, and other cross-functional teams.

In terms of setting up the research roadmap, there are various approaches to consider. You might undertake exploratory studies to initiate new projects and explore emerging opportunities. Alternatively, conducting evaluative studies might be required to assess the usability and effectiveness of a game offering following a design overhaul, among other scenarios.

Assessing the current state of research, customer feedback and the market

After engaging with your stakeholders, 2 questions must be asked:

“Are there any gaps in the literature and research in general?”

“What’s changed or is out-of-date?”

A research team must allocate sufficient time to gain a comprehensive understanding of their existing knowledge about the game offerings, users, and market positioning. This involves:

1. Identifying published research: It’s highly likely that research materials pertaining to the CX of online casinos are available, particularly in sources like Elsevier, which hosts open-access journals.

2. Replication and Validation: You don’t need to reinvent the wheel here. Instead, you can adopt the same research model and metrics utilised in prior studies, (with slight modifications, such as the inclusion of one or two variables aligned with your research objectives) and reach out to previous researchers from other subsidiaries who have worked on similar or related topics.

3. Identifying low-hanging fruit: I highly recommend text mining live chat and recorded call data. This will provide insights into prevalent topics and customer sentiment through methods like word frequency analysis, topic modeling, and sentiment analysis.

4. Collecting all existing artefacts: Assess what is already known and what remains unknown by deriving meaningful insights from personas, customer journey maps and need statements.

5. Exploring ways to expedite return on investment (ROI) through competitor analysis: Many online casinos offer the same popular games and rely on factors like brand equity, trust, and customer support to retain players.

Therefore, incorporating the voice of the player into your ‘fact base’ is imperative for the long-term success of your product.

Identifying key CX research goals

Building upon the previous research stages outlined earlier, the next steps involve:

1. Compiling a list of the most crucial questions you have regarding your customer base.
2. Considering the tools and instruments required for successful research execution.

To facilitate data-driven insights and collaborative efforts, the following tools are recommended:

• HubSpot: HubSpot can serve as a centralised platform for CX metrics, capturing vital KPIs from NPS (Net Promoter Score), CES (Customer Effort Score), and CSAT (Customer Satisfaction) surveys.
• Jupyter Notebook and NVIVO: These tools are useful for structuring extensive datasets, encompassing both quantitative and qualitative data.
• Miro: Miro aids in the collaborative creation of customer journey maps and other artefacts.
• Insights repository (e.g. Dovetail or EnjoyHQ): A centralised data storage serving as a historical reference and feedback loop for decision-making across cross-functional teams.

During this stage, the research team should also establish higher-level goals by distinguishing between tactical and strategic studies:

Tactical Studies:

• Closely aligned with the research roadmap and typically completed within a relatively short time frame.
• E.g. Quarterly CX surveys.

Strategic Studies:

• Not directly tied to the existing roadmap and focuses on achieving long-term goals, such as understanding the entire user journey or creating customer personas.
• E.g. Interviews, focus groups, and diary studies.

Best practice at this stage would be to collaborate with your colleagues to formulate a Gantt chart, outlining project milestones to be achieved in that quarter. Alternatively, you can distribute the studies across all four quarters. This approach provides a comprehensive overview of the timing of all studies, both tactical and strategic, depending on budget constraints.

Quarterly CX Survey

Now it’s time to delve into the core of this research project.

The primary tactical study will be the Quarterly CX Survey. This survey’s development will be guided by the research questions and business objectives, which have been formulated in collaboration with your stakeholders to align with their needs.

Distribution of the survey will target current players. Marketing will play a crucial role in identifying:

• The most effective channel for survey delivery.
• The optimal timing for survey distribution.
• The preferred language of customers.

The foundational metrics from this survey should include the CSAT score and NPS. These metrics offer rapid and dependable insights into customer satisfaction levels and their loyalty to the online casino.

Assess the timing of the survey and its ease of comprehension by conducting pilot testing on a sample of 10 to 12 individuals before rollout.

Sampling

A probability sampling approach involves obtaining a random sample from the overall player population. Specifically, a stratified sample framework can be utilised to ensure that the sample is as representative as possible concerning key profile and behavioural characteristics, including:

• Demographics
• Player seniority (or ranking)
• Game preferences
• Frequency of gameplay
• Duration of gameplay

To define your sampling method, it’s important to establish inclusion and exclusion criteria:

• Lapsed players (those who haven’t been active within the past 30 days) should be excluded.
• Similarly, players who have not made any deposits should also be excluded.

Sample size calculation can be performed using Python or another sample size calculator. The exact sample size can be estimated based on the sample size formula, considering a:

• Desired confidence level of 95%
• Standard deviation of 0.5
• Margin of error of 5%

Touchpoint Survey: Customer Effort Score (CES)

CES Question

In addition to the Quarterly CX survey, touchpoint surveys can be dispatched to evaluate the level of effort customers must exert when interacting with the online casino in real-time. The CES provides a straightforward method for measuring this real-time metric. These brief surveys can be distributed during customer interactions with your support team.

Quality assurance and risk management after rollout

How do quality assurance and sample monitoring factor into this?

Following the rollout of the survey, responses should be subject to ongoing monitoring to ensure respondent consistency, and any outliers should be identified during the data collection process.

Non-response bias and low response rate are among the primary concerns in this study. To mitigate bias effectively, I recommend the following strategies:

• Reminder outreach: For those who haven’t responded to your survey, send reminders through alternative channels, with email targeting considered as the primary method.
• Scripting issue identification: Identify and address scripting issues, such as excessive use of text fields for missing categories.
• Addressing low response rates near the deadline: As the deadline approaches and response rates remain low, send reminder emails and up the incentive for completing the survey, such as offering additional bonuses.
• Initiating conversations: A proactive approach can be taken by initiating organic conversations. This involves reaching out to non-responders with a message like, “We were hoping to learn more from you and understand the feedback and the sentiment of our partnership. We recently sent out a survey, and we noticed that you didn’t respond. Is there anything that you’d be willing to share with me today?”
• Creating a welcoming and empathetic environment: Recognizing that non-responders may be less engaged or have specific reasons for not providing feedback, an environment will be fostered that is welcoming and empathetic. This approach aims to encourage these individuals to share their feedback in a safe and comfortable space.

These potential respondents could potentially contribute to kickstarting a strategic study in the next quarter, which may involve one-on-one interviews.

Data Analysis, Report Writing and Debriefing Session

Once you’ve compiled all your data, it’s time to start the iterative process of writing your report. It’s important to note that iteration isn’t solely reserved for product design but also applies to generating efficient and optimal deliverables, such as artefacts, for senior management and other involved stakeholders. These artefacts may include customer journey maps, empathy maps and customer quotes.

Prior to the final report, an insights session should be held. During this session, the research team should explore methods to visualise and present the results of text mining from open-ended responses. The use of word frequency analysis, topic modeling, and sentiment analysis allows the team to examine observations and situations from different angles. Converging findings from these analyses can offer new insights into various situations or phenomena, exemplifying the essence of CX research.

In the debriefing session, reflect on the lessons learned throughout the project, discuss avenues for future research, and explore ways to promote research throughout the company. This presents an excellent opportunity to scale up your research efforts in the next quarter by proposing the following methods:

• Churn research
• Streamlining the research process
• Additional machine learning capabilities
• Formation of a more centralised team or a governance task team to prioritise opportunities and facilitate cross-functional collaboration
• Integration of CX considerations into all process and design stages

Quarterly CX Survey Design

This last section will delve into my proposed 11-question survey for quarterly CX research.

Start by crafting a clear headline and presenting a compelling survey introduction that encompasses the following key elements:

• Unique Selling Proposition: Highlight what makes your research stand out from the rest of the competition and the research’s relevance to your customers.
• Estimated completion time and assuring respondents of their anonymity.
• Incentive for survey completion.
• Contact details for any additional inquiries.

Q1: Age; Q2: First-time Registration; Q3: Frequency of Play; Q4: Device/Operating Systems

The first four questions concern age, frequency of usage, first-time registration, and preferred devices/operating systems. These screening questions will serve to validate profile characteristics and facilitate the development of user personas.

I included “I’m not sure” and “Prefer not to say” options mainly because we have to give your respondents a way out in case of uncertainty and take the pressure off. It could be that:

• Respondents have conflicting feelings about the topic and are not sure about where they stand.
• Respondents understand the question and are willing to be forthcoming, but honestly do not know the answer.

Q5: Game Preference

The list of games in Q5 shouldn’t be exhaustive, which is why I included an ‘Other’ section.

Q6: Multiple-choice grid for rating gaming aspects

Q6 facilitates the quantification of online casinos’ gaming and usability aspects, streamlining analysis. I’ve included the following aspects:

· Variety of existing games

· Variety of new games

· Registration

· Identity verification

· Welcome bonus

· Loyalty rewards and promotions

· Guaranteed winnings

· Tournaments

· Depositing process

· Withdrawal process

· Security

· Fair play

· Customer support

· Mobile Casino

However, it’s worth noting that this section is subject to amendments. Therefore, I recommend considering other gaming aspects, such as:

• Payment methods
• Bonus offerings as standalone gaming aspects (in addition to the welcome bonus)
• Responsible gaming policies
• Different types of customer support, including email and 24/7 live chat, among others.

Within this multiple-choice grid format, we can easily identify outliers who rush through the survey, such as ‘straight-liners’ (e.g. consistently selecting ‘Very Satisfied’) and respondents who zigzag between two response options. One potential approach to mitigate this is to reconsider the use of grid or matrix questions, as they can inadvertently encourage such behaviours. However, this decision should be discussed with your team, as grids can help reduce respondent fatigue.

The use of the 7-point Likert Scale for this grid is another point of discussion. While the 7-point scale provides slightly better discrimination compared to the 5-point scale, its impact may not be significant in a multiple-choice grid like this, which presents 14 gaming aspects.

To address potential order effects, I recommend shuffling the order of the rows, which can help reduce bias caused by the presentation order.

Q7: Describing additional aspects not mentioned in Q6

Q7 will provide insights into additional gaming aspects that were not covered in the preceding question. Respondents may wish to specify a particular game, game category, or comment on the overall usability of the website, among other aspects.

Q8: Competitor Comparison

Q8 focuses on competitor comparison. In this section, respondents will choose other online casinos with which they engage the most and provide additional choices in the free-form text field below. Best practice here would be to conduct a SWOT analysis to derive the list of competitors.

Q9: CSAT Question

Q10: CSAT Follow-up

Q11: NPS Question

The final three questions constitute the core components of any CX research: CSAT question, CSAT follow-up (“What could be done better?”) and NPS question. These standardised research instruments also serve as benchmarks, essential for comparing results across various game offerings.

To mitigate potential biases, the questions are intentionally neutral, addressing ‘overall satisfaction’ and encompassing both ‘likely’ and ‘unlikely’ responses.

Conclusion

There are a total of 11 questions, and it should take around 5-10 minutes to complete. The survey is structured to minimise cognitive load, starting with straightforward questions (e.g. demographics and frequency of usage), progressing to more intricate ones (multiple-choice grid), and concluding with lighter questions (CSAT and NPS).

I anticipate that respondent fatigue will not pose a significant burden. I am confident that this survey not only captures essential CX metrics but also provides valuable insights for other cross-functional teams.

Thanks for reading :)