We keep getting more data, it's not helping

My name’s Abby and I secretly love making dashboards (“Hello, Abby,” I hear you all chorus). 

I love the formulae, I love the logic. I love plugging in a big data set and seeing what pops up in the front sheet. I love it when they’ve got graphs that go up and down and pie charts and numbers that change colour depending on how big they are. I even love when they don’t work and I get to ferret around in the formulae and figure out why (because there is always a logical answer and there is something incredibly satisfying about that). 

Nonetheless, last week I l ran a session at Oxford’s Marmalade Festival, together with Andy and our friend Marcus Jenal, called “We keep getting more data but it’s not helping”: why current learning approaches trap us in the status quo.

We felt so seen in that session, like we were in a room full of people who were totally with us on this idea that there is more to growing our understanding, knowledge and wisdom than gathering and analysing data. We heard the wry laughs of recognition when we recounted our friend saying she was a “…recovering evaluator working in the change space; ‘recovering’ because no-one ever paid any attention to data that did not confirm their biases.” 

Sausage Numbers

The genesis for that session, for me and Andy, goes way back. We met around eight years ago, heading up the sales operations team for an education charity. We bonded over a shared love of big questions, being awkward and a lot of “is it just me or are all of these dashboards, matrices and spreadsheets a lot of work for very little return?” 

The trouble was that in previous jobs people would say “Abby could you pull me the data on how many of *this* type of flange we sold last week?”, I would generate the numbers, they would say “hmm, that’s interesting”, and then absolutely nothing would happen. We just weren’t learning anything meaningful from it. 

The closer to the frontline of the work I was, the more I understood that there was a lot more going on than the numbers could possibly say – I knew what that stuff was, and it wasn’t something I could convey easily through any kind of metric. The further away from the frontline that I got, the more that I knew I didn’t know - and the more meaningless the numbers seemed. Thanks to Marcus, I now have a term for this:  ‘sausage numbers.’ They’re very tasty but you don’t really know what’s inside them.  

I remember once presenting some dashboard numbers to a funder and really hoping they didn’t ask a particular question about a particular number, because I knew it could, just possibly, result in them pulling some funding. The numbers were entirely accurate, they just told a different story depending on how much you knew about the situation they were trying to represent. (Should I feel guilty that my colleagues’ jobs were more important to me than the funder’s priorities?)  

I was thankful that they didn’t know what was inside those sausage numbers, thankful they didn’t see that the numbers they thought were meaningful would turn out to be meaningless if they dug a little deeper (which they didn’t, to my relief). I kept producing numbers, though, because I didn’t really know how else to show a nuanced and valuable picture of what was going on. It was just the done thing. They had numbers they wanted to monitor and I provided that data. 

Multiple Sauces

However, something that we said a few times at Marmalade last week was this: Please don’t think that we don’t think that data is important for learning. It is.  

What we want to communicate is that data is not everything. It is not the sole source of insight. There is also insight that we hold in our hearts, like our feelings, our passions – and insight that we hold in our guts, like that uneasiness when you have a sense that something isn’t quite right, but you can’t put your finger on what that is exactly. (Andy wrote a lovely blog about this back in February.) 

For us to learn together, to make good judgements about what is actually happening, what has happened, or what opportunities or pitfalls lie before us, we need to bring in these multiple sources, and we need to combine our insights with those of others. In our learning partnership work at Collective Impact Agency we start with relationships, with connection, with uncovering people’s real experiences and then take it from there. Data might still come into it, but it's there to inform the work, not provide a full understanding of it. 

That was what I was reaching for those years ago, when I held this uneasy scepticism about the numbers that formed the basis of my work. I knew it wasn’t enough, that I couldn’t make this way of thinking make sense because it doesn’t make sense. It’s wonderful to know that other people feel the same.  

So, in the spirit of connecting over our stories, I would love to hear your stories about learning. What’s your relationship with learning looked like over the years? What would you like to explore around approaches to learning?  

(Folks who came to Marmalade and had questions that we didn’t have time to answer, send them below! If we get a good bunch of questions, we’ll create an online space to talk about them!) 

As always, we love it when you pick up conversations with us. No thought or experience is too small, no feeling or instinct is insignificant – we love to hear them all!