As AI technologies continue to develop faster than policy guardrails, working with publics to surface their hopes, fears and expectations around these tools is essential. And if we want this evidence to rebalance the power inequities inherent in AI developments and applications, we must bring to the fore the views of those people that might be most affected by them.

In this blog we explore the value and challenges of producing quantitative public attitudes research about AI that is both legible evidence for policy decision-making and designed to be inclusive and equitable.

In particular, we address a critical problem with real-world impact: that quantitative attitudes research is legible to policymakers, who tend to privilege evidence found through robust social science methods as an objective representation of people’s attitudes, and it has representation issues. This means there is a concrete risk that the views, expectations, hopes and concerns of people most affected – usually minoritised people – are not represented in decisions about these technologies.

Drawing on participatory concepts articulated in democratic theory, we make the case for enhancing the inclusion of marginalised voices to help mitigate participatory inequalities. We present a specific approach to quantitative research that, by deepening the analysis of small data samples and minoritised perspectives, enables us to see how epistemological norms are shaping the policy evidence landscape. Lastly, we point to an intersectional analytic approach that may be more resilient to common challenges in survey data, such as small sample sizes.

With an eye to the next nationally representative survey on UK public attitudes towards AI – a UKRI-funded, biennial tracker survey, conducted in collaboration between the Ada Lovelace Institute and the Alan Turing Institute in 2023, 2025 and programmed for two further rounds in 2027 and 2029 – we use frameworks from Quantitative Criticalism (or QuantCrit) to examine our survey practices. Specifically, we consider how referencing systems of power and oppression can improve inclusive survey work – a currently under-theorised component of participatory quantitative research.

Quant’s representation deficit

Quantitative research is an important component of the work of involving people in AI development and deployment, not just engineers, professional users or AI researchers, but a range of people from different backgrounds, experiences and expertise, who may be positively or negatively affected by AI technologies. It may seem unusual to talk about participatory methods in relation to surveys, but we see this work as critical now for three interrelated reasons.

Firstly, policymakers tend to pay more attention to large-scale, nationally representative attitudinal evidence than smaller qualitative studies, even when these may provide rich descriptions of people’s experiences and motivations.

Secondly, we know from working on our own quantitative research that it’s difficult to represent the perspectives of minoritised groups in this type of evidence due to how surveys are funded, how survey companies compose their panels, and constraints that limit the use of critical survey design and analysis paradigms.

Thirdly, people’s attitudes need to be understood individually in the context of legal rights and protections, but also collectively, so that deeply held views of groups and communities in society can shape the direction of development and application of current and future technologies.

Our biennial attitudes survey shows that people have varied views about AI applications which differ depending on a person’s age, ethnicity, income level and digital confidence. A baseline question in a recent poll on attitudes towards AI regulation asked whether people feel they have a say in what the UK government does. Most people (60%) disagreed. This feeling of disempowerment is higher:

• if you’re older
• if you’re a woman
• if you’re less digitally confident
• if you have a lower level of overall education
• if you live outside the capital city.

However, we can’t say anything about differences in perspectives across minoritised ethnic groups. This is because, by nature, national surveys are poorly set up to represent people from minority groups: the data we do have is not sufficient to support detailed analysis.

Even in large-scale national surveys, design decisions can create these same representation issues. The first wave of our national tracker survey had a sample of over 4,000 people, recruited to be representative of the British public and its demographic distribution. This sample was split in half to increase the number of applications of AI we could ask about – from autonomous vehicles to the use of AI in cancer detection. While a sample of 2,000 people may seem large, it isn’t enough.

We wanted to explore differences in attitudes to AI by people’s race, acknowledging the bias in data capture and representation underpinning many AI systems. However, we found that, for most AI use cases analysed, we were left with a sample of around 50 Black British participants to do this. For context, to detect medium-sized statistical differences between groups, we would have needed at least 64 people per group, which increases to 394 if the differences between groups are more subtle. Knowing the views of 50 people was not enough to reliably detect group differences.[1]

For the second round of our attitudes tracker, in 2025, we used sample boosting as a way of improving the representation of minoritised groups. Sample boosting means that certain groups of people are deliberately overrepresented to enable more robust statistical analysis. It’s a technique often used in UK household studies (e.g. Understanding Society), but not in attitudinal surveys on AI. Based on our understanding of the literature on who is most impacted by AI, we chose to oversample on the basis of ethnicity, income and digital confidence. However, in the process we found that the UK ecosystem of probability samples is not well set up for this.

Probability sampling is often seen as the gold standard of survey design, where participants are randomly offered the opportunity to sign up, meaning that in theory everyone has an equal chance of being represented. However, even the strongest providers of these panels acknowledge that they have a representation issue. For instance, Black British participants tend to be poorly represented here too.

There are a limited number of groups one can oversample on before the budget runs thin. And even when researchers do have categories to oversample on, the categories risk being arbitrary or overly simplistic. For example, we had to reduce all Asian participants into an Asian subgroup, even though we know that the experiences of Bangladeshi people in the UK differs from those of Indian people, Chinese people, Pakistani people and so on. There are inequalities in how people experience AI tools and headline figures, though compelling for policymakers, can often obscure many of these inequalities.

So, while sitting low in the Spectrum of Public Participation, survey approaches are important to critically examine to ensure we use inclusive and equitable methods. They offer a view on landscape insights at scale with scope for generalisability, and opportunities to track beliefs over time, which is increasingly useful in an environment of rapid technological change. They also occupy a space of authority. Survey evidence is often seen as an objective representation of attitudes – a way of detangling human feelings and beliefs – in uncritical ways.

A sharper bird’s-eye view: QuantCrit and multilevel analysis

In this context, Quantitative Criticalism – or QuantCrit – offers a framework for reflexivity. With roots in Critical Race Theory, QuantCrit has five key principles:

1. The centrality of racism in society – even in the absence of statistical significance.
2. That numbers are not neutral and therefore are not free from researcher positionality.
3. That categories are neither natural nor given – the way we define people and society is grounded in histories that are often contested.
4. Data cannot speak for itself – it needs careful interpretation and communication
5. That statistics play a role in social justice struggles – they are leveraged in societal debates and carry impact.

These principles speak to the ways that quantitative data can further embed existing power relations, but also to how to resist this and mobilise change.

If we accept the representation limitations of our nationally representative survey, we risk furthering the idea that data can speak for itself, when in fact many researcher decisions and wider systemic factors affected the type of data we could collect.

Indeed, QuantCrit principles were key to how we communicated our 2025 tracker findings. We intentionally compared attitudes from subgroups against the general population, rather than a reference category like ‘White, high income, digitally literate’, to decentre the experiences of privileged groups. For instance, we found that among those with fewer digital skills, 41% felt LLMs were beneficial to society compared to 63% of the general population.

Even so, we published headline findings that focus mostly on majority views, as these tend to be the most compelling and easy to interpret, for instance, our finding that there is majority support for independent regulation and majority concern around overreliance on technology.

When analysing results, we were able to look at identity markers only in isolation rather than in combination. For the most part, we reported on the additive effects of different demographic factors, such as income level, age, gender and so on, which does not consider the multiple identities people have, or how various systems of oppression and privilege intersect to create inequalities in societies. It is here that quantitative methods often fall short. Traditional approaches – like multiple regressions – struggle to handle the complexity of intersecting identities and come up against limitations of national survey data more broadly, such as small sample sizes. However, analytic innovations offer new opportunities: as we approach the third iteration of our survey series on UK public attitudes we are trying to bake QuantCrit principles into our research from the start.

Multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) is a novel analytic approach that allows us to robustly apply an intersectional framework. In particular, MAIHDA can deal with issues of small sample sizes and handle large combinations of identity markers. This approach has been used to study inequalities in healthcare outcomes, for instance intersectional differences in mental health trajectories, body mass index and blood pressure, but as far as we are aware, it has not been applied to attitudinal data.

In our most recent attitudes tracker we focused on facial recognition in policing. Data on this application of AI was difficult to interpret. On the one hand, our existing qualitative evidence suggested quite a high level of concern around these tools. On the other hand, our 2025 quantitative evidence painted a more optimistic picture, with many people seeing this tool as beneficial to society, and only a few, around 2 in 5, being concerned about it.

Through sample boosting, we were able to explore some subgroup differences in more depth. We found some indications of inequalities of experience – people from minoritised ethnic groups have higher levels of concern about this technology than the general public, as well as those from lower income backgrounds. Men are also more concerned than women (43% vs 36%), as are those from younger age groups than older age groups (47% of 18-34 year olds vs 32% of 65+ year olds).

Digging further into the data after the 2025 survey results were published, we conducted some preliminary MAIHDA on our dataset. We looked at people’s responses in relation to gender, age, ethnicity, income and digital confidence in combination with one another. We found that approximately 9% of variation in whether someone is concerned by the use of facial recognition technologies in policing can be attributed to the effects of the combination – or intersection – of these identity markers, rather than the individual contribution of each characteristic.

Mapping out these findings shows a story of complexity. The following graph displays the predicted percentage of people concerned by facial recognition in policing for each combination of identity markers (which can be combined in over 150 ways). We find that historically privileged groups have lower levels of concern around this technology than others – they are marked in red, and at the lower end of the graph.

More specifically, we find that Black men and women under the age of 35 are the most concerned by facial recognition in policing, irrespective of income level, while women with their ethnicity recorded as ‘other’ and without digital skills were least concerned. We see here how the technology may relate to systems of oppression that over-surveil some communities.

When applied, QuantCrit’s principles can improve the representation of the experiences of diverse publics in debates around AI. Working through the challenges of doing this at funding, researcher and policy levels will improve the quality of the evidence we have on attitudes to AI, and therefore the recommendations for change we can make to policymakers. And while it is true that we need to include more communities and collect more data, the application of QuantCrit principles shows that sometimes what we need is more careful coverage and nuanced presentations of the data.

The integration of quantitative critique into established norms of survey production is still an emerging practice that brings together disparate strands of research practice to think holistically about more equitable and inclusive ways of doing quantitative public attitudes surveys.

For survey evidence to make a difference, however, we need participation to be ‘ecologised’. This requires that governments consistently use appropriate participatory methods and evidence, and close the loop on evidence informing policy by meaningfully involving people in AI governance. Meaningful involvement means people’s opinions are sought, heard and acted on in ways that make a difference to how technologies are developed, deployed and regulated.

Quantitative research is often seen as the ‘gold standard’ of evidence, but if social science norms and survey provider limitations are accepted uncritically, this evidence risks reproducing specific systemic biases that reinforce societal inequalities by excluding people from the evidence base. We know that there isn’t one ‘public’ and that demographics and experience make a difference to people’s attitudes towards AI. Drawing from social and democratic theory and practice, we can apply critical concepts of participation to survey methodology – throughout survey workflows, from funding to design, analysis and dissemination – to produce more inclusive and equitable quantitative research and mitigate participatory inequalities.

Addressing the representation issues that mean minoritised perspectives are left out of the evidence more legible to policymakers is essential if we are to move towards an equitable AI ecosystem in which technologies are developed and deployed in ways that respect people’s preferences.

 

[1] Based on an a priori calculation of sample sizes for t tests looking at differences between two independent means.