Sadhvi Dar, School of Business and Management, QMUL
‘Capturing Evaluation Creatively – A ‘Clumsier’ Way of Evaluating Qualitative Engagement’
Two years ago, I met Rosie Hunter, the Executive Director for People’s Palace Projects, at a QM seminar about emotions. We had a few drinks at the reception where we also spoke about arts-based social projects and the dominance of quantitative indicators in arts-projects evaluation processes. We questioned the rationale of using a positivist methodology to measure the arts and pondered:
‘How can you measure the value of an art intervention with a numerical indicator?’
‘What is taken out of the evaluation when we reduce the aesthetic experience to an excel spreadsheet, a ratio or a percentage?’
These questions of evaluating qualitative projects are ones that come up commonly in public engagement as we ask ourselves how can we capture these in ways that accurately reflect the quality and benefit of interactions.
By the end of the evening, and after thinking about these questions with her, I felt I had made a connection with a kindred-spirit. And, this was confirmed when Rosie told me to get in touch with a group of academics and practitioners called The Clumsy Evaluators.
The very name of the group made me smile broadly – the idea of being clumsy, off-kilter, dizzy, off-centered while claiming the identity of an evaluator that personifies structure, balance and objectivity seemed to encapsulate the oxymoron that arts-evaluation is. Art has the power to unlock imagination and possibility in a way that disrupts (dizzies) our ideas of objectivity and neutrality yet when it is applied as a social policy (for community-building, public engagement, empowerment…) its value is often translated into a number, impact factor, or point on a Likert-scale that is reported to a higher authority.
This quantification tends to flatten our accounts of what an arts intervention feels like for both the participants and the organizers. Art’s power seems to lie in its propensity to create relations, communicate things beyond the confines of language and make us feel re-connected to ourselves and others. A feedback questionnaire doesn’t allow us to reflect on this aesthetic experience in a meaningful way – it doesn’t question what kind of community we just created, who felt included in that experience and who was unable to speak or express themselves.
Arts practioners understand the limits of a quantified formal report, how can arts-practioners work against the pressures to report without undermining the need to offer a formal account of a project’s value? A Clumsy Evaluator captured the conflictual role of an arts-evaluator wonderfully and I met with the group shortly after.
The Clumsies were working through the problems of evaluating arts-interventions. Being practitioners themselves, they were dealing with the conflicted reality of providing an account of how their interventions or projects were valuable alongside creating arts interventions that imagine new relations and new communities.
It was through these conversations with the Clumsies that I met Maggie, a lecturer at the QMUL School of English and Drama who was developing a performance-based tool to evaluate university services offered to young people in care. I had worked on developing radical democratic evaluation tools with marginalised communities and part of my past practice involved using theatre-methods to ‘report’ to beneficiaries.
Maggie’s idea was to create a community-tool that is performative and inclusive. She told me about The Verbatim Formula and how she uses testimonies of young people in care as sound-scripts that are then performed by another young person. This sounds complicated – but it is actually quite simple. Interviews with young people are recorded on an ipod, a young person uses a pair of headphones to listen to an interview and repeats what they hear. That re-speaking becomes the performance – and also a method for evaluation, a verbatim report that communicates the affective dimension of being in care, without asking the young person to reduce that dimension to a number or digit. The verbatim is performed to relevant stakeholders who need to listen to young people’s voices as a way to tailor services to user-needs. Since our initial meeting, Maggie and I have worked with QMUL Artist Fellow Sylvan Baker on three collaborative projects using verbatim theatre to create verbatim performances that are – in this case – performed by young people in care to a range of stakeholders including university departments, the Greater London authority and social workers.
“The verbatim performance … has made a great impact on our delegates, has inspired them to change their working practices, and has elicited very positive feedback.”
Rebecca Palmer, Education and Youth Team, Greater London Authority
The theatrical form breaks away from conventional reporting formats. And it is this difference in form, rather than content, that seems critical to creating new possibilities for listening, community-building and empowering. We need more performed-evaluation and we need to communicate art’s ineffable value through the power of art. I look forward to nurturing our collaboration and to continue learning together about alternative forms of arts-evaluation.