Archive blog from September 2021
Rose Bonner and Kat Quatermass take a look at the ethics of user research - and outline four areas where Neontribe is trying to do better.
Like most digital agencies, we think a lot about the ethics of the user research we do at Neontribe.
We think about it during projects, and discuss it with our clients and partners. But recently we were challenged to think about whether we are doing enough.
Together with CAST, we were mentoring charities on Comic Relief and Paul Hamlyn Foundation’s Tech for Good Build programme. One of the questions several participants asked was:
“Isn’t user research extractive? Isn't it exploiting people?”
Accepting that user research can be problematic
The question felt important to us because we know that user research can seem one sided. But the last thing we want is to exploit people. Our gut reaction to the idea that we might be bad guys was pretty uncomfortable.
We decided that we need to acknowledge that user research itself IS a very transactional activity. We use observational techniques and specific methods of asking questions about what people have done or experienced and we don’t make a lot of space for people to tell us what they want.
We do this because we’ve learned that if you ask people what they want or what they will do, what they say won’t reflect what they actually do later. It’s an awkward truth of all human psychology that we are very poor predictors of our own behaviour. We know that exploring needs through user research techniques instead of asking people to make predictions allows us to build services that work. It is a crucial step to avoiding waste of money and failed projects.
But we want to be wary of this turning into an “ends justify the means” argument. And done badly, user research could easily be dehumanising, tokenistic or worse. The reliance on these methods could be experienced by participants as an “experts know best” paternalistic approach. This could be even more magnified for marginalised groups.
Four areas where we are trying to do better
We know that we don’t want to change the research and testing activities themselves. But we’ve tried to draw together some of the things we were already doing on projects, to make the research as ethical as possible, and use that to think about what we can do better.
The areas are:
- Giving something back to participants
- Being clear about what participants can expect
- Putting the right activities in front of the right person
- Fitting the research activities into a wider engagement conversation
Giving something back - aiming for everyone to benefit.
We don’t want participants to come away from our research feeling like they’ve been used. We want people to feel like they’ve gotten something out of the process and hopefully enjoyed it.
Like most agencies, we encourage the charities we work with to set aside a budget to make thank you payments (in money or vouchers). We consider expenses that they might have (e.g. travel) and the fact that we need to value their time. We get paid to be there, why shouldn’t our participants?
But we also look at how we can design the experience of the user research session to have elements that are beneficial in other ways.
Two examples:
As part of our work with infrastructure organisation NCVO, we’ve been doing a lot of content design research with particular topic focus. We often ask people to demonstrate how they look for information they need online as part of that research. In the last 15 minutes of their call we’ll stop researching and switch to providing expert advice tailored to their situation. (This is sometimes from the NCVO staff and sometimes from one of our user researchers, who used to be a fundraising specialist!)
A few years ago we began our first project looking at digital support for people experiencing domestic abuse, with The Haven, Wolverhampton. We designed a group session, with coffee and biscuits and a chance for women in similar situations to simply chat with each other, supported by a domestic abuse worker, with some group research interviews, and a space to the side where we did testing and interview questions one on one. These participants were heartfelt as they thanked The Haven for the session, and the chance they had to share their experiences meaningfully with each other and be heard by us.
To continue to provide successful mutual benefit we need to apply service design thinking to each piece of research we do.
Being clear about what participants can expect
We’ve discovered this is harder and more important than we first thought.
We all know about the importance of getting informed consent. And like most agencies, we know from UX design how hard it is to get people to read and understand what they are being asked to consent to.
That doesn’t get any easier for user research.
We’ve found it is easiest when we work with a partner who has frontline workers. Our task is to make sure that the frontline workers really understand the project - and make sure there is time in the planning schedule for them to prepare people for the research sessions.
With projects without a frontline connection, where recruitment is digital, it gets harder.
We use variants on the user research templates that NCVO provide - but have recently discovered that people with certain learning difficulties and some neurodiverse individuals feel overwhelmed by the amount of information supplied.
We’re determined to do more - but we’re not quite sure what yet. We’d like to start fewer sessions with people looking confused, or sounding a bit cagey, and do more to reassure them in advance.
NCVO advice is based on work by many agencies involved with Catalyst, and includes a downloadable research template. Review it here.
Putting the right activities in front of the right person
One of the fears that comes out often when talking to charities is that their participants feel “over consulted”. We think there are two particular ways that we can help stop user research contributing to that problem.
- “You don’t need to research the things you already have good evidence about”
We spend a lot of time at the beginning of projects asking for access to previous research and evaluations, to conversation and email logs (where privacy statements allow it, or summaries where they do not), and we always try to do interviews with frontline staff to uncover institutional assumptions before we work with service users. It’s really important to make sure we’re not using our user research sessions to uncover information the client already knew!
We’d also like to see more sharing of research between charities and between agencies as a way to reduce this problem even further.
- “You don’t always need to focus the research on the most painful aspects of a marginalised group’s experience”
Doing this can lead to people feeling over consulted - because everyone they talk to is asking them the same questions. Instead, it might be more useful to a project to focus on specific digital activity.
For example we’ve recently been involved with quite a few organisations who want to help marginalised groups of people find events/services that will be safe and fun for them. We always want the research in those projects to focus on “how do you find events at the moment?” or “how do you mostly use the internet at the moment?” not “what is the experience of being young/trans/disabled/from a Black, Asian or other ethnic background in general?”.
Of course, we always need to be ready to make space when people do want to talk about their broader lived experience - and have pathways available if they need specialist support.
The team at Chayn have been working on ethical user research in relation to people who have experienced gender based violence. See their blog.
Fitting the research activities into a wider package of engagement
We believe that means charities and their digital partners must make user research part of a genuine package of engagement, not a standalone activity. And we would love to say Neontribe is great at making sure there is joined up thinking and that participants are involved as much or as little as they want to with our projects, but we would be lying.
Our instinct as a digital agency is to lean on our charity clients to help with this. Because we think that the conversation is exactly the same one that charities have been having about consultation and engagement for the last 20+ years.
But at the same time, we know that charities are always pushed for time, and so maybe it needs to be our responsibility to ask these follow up questions and make sure our user research is sitting within a wider project.
- What happens for the participant after the user research interview?
- What follow up support do you have the capacity to offer? If you haven’t got it within your organisation, what helpline should we be referring to?
- How are you going to share what you learned, and what you intend to do with that learning, with the participants? (What documents do you need from us to help with that?)
- Do you have an additional engagement activity related to the project for one or two participants that might want that to get more involved? (We’ll happily have participants OR some of your experts by experience group in insight mapping sessions for example)
One of our aims for the end of this year is to start building time for this work into project proposals. Another is to find a project where we train experts by experience to deliver the user research sessions with us (we last did this at least five years ago, probably longer, with young people).
What else are other agencies doing to help support charities with the engagement that needs to sit alongside user research?
Join a conversation about this on Agencies for Good Slack