Data is King; Long live the King…or should we behead him?

Your answers please, and results next blog-time!

Not enough information to answer? – read on, but respond now!!!

 

Data is everything

We are informed by the good and the great that data is everything… and we are steadily becoming awash with it. We can tell how many tackles Paul Pogba made (or didn’t make) in any match, how many passes he completed, how many kilometres he ran. We can then interpret these data and work out if he was worth the crazy money Man United paid for him – “No”, by the way in my opinion.

The point is that data is everywhere and thus we can find facts and figures to look at all aspects of life.

 

So how does this work in the heritage sector?

Well, of course, market research, visitor surveys and so on have been going on for years. And we all know that the data they give us can be invaluable in forming plans, strategies and designs for enhancing a visit.

 

Surveys

We use surveys to categorise visitor segments, find out what might sell in the shop, what special events might be most popular, and so on and so forth. Many use data supplied by TripAdvisor results to tell visitors how wonderful the place is and prove what a good time they will have… and to satisfy demanding trustees!!!

But we have to be very careful. Facts and figures can be manipulated and interpreted to provide varying answers.

There is a very good book by Sir David Spiegelhalter entitled, “The Art of Statistics – Learning from Data”

It’s a comprehensive and very readable volume on how data can be collected and used. What’s great about it (although Sir David is probably the pre-eminent statistician in the UK), is that he shows how data can mislead and misinform decisions. Pitfall after pitfall.

You might now be thinking I’m a bit of a geek. Well, it’s a pretty accessible read in fact and although it delves into some of the mathematics, (it’s reminded me what Standard Deviation is, which I haven’t really touched on since Stats during A Level Maths!!!), it’s more about the competent use of the data we gather.

Reading it reminded me not only of those Stats lessons but also of one of the worst pieces of visitor surveying I have ever see:

No names, no pack-drill and all that, but an historic property (that should have known better!) carried out some visitor research a while ago and the following question formed part of a wide-ranging questionnaire with little structure or background provided – in much the same vein as the question at the start of this text about beheading King Charles 1^st, which gives you no information on which to base your life or death decision.

This particular question was also set in isolation of any other similar enquiries about audio visual media. The question was:

“Do you think your visit would be enhanced by the use of sound?”

 

On face value an innocuous question that could help inform a design refresh, but if you analyse it just a little you find it is completely and utterly flawed on many levels.

• First: How do you define “sound”? What does “sound” mean?Is it music, sound-effects, oral histories, the voice of a curator, or an historian as a narrator, or recorded pieces of scripted dialogue between historical characters, or what?

• Secondly: What is the purpose of the “sound”? Is it to convey historical detail? Or is it to create mood or atmosphere? Or is it to deliver wayfinding instructions?

• Thirdly: How and where will the “sound” be delivered? Throughout the property or in a single room? Via loud speakers, or on a personal Audio Guide? Downloaded as a PodCast or an App? Is it to work in isolation or be part of a mix of interpretative media?
• And fourthly: Given this lack of information, what will inform the visitors’ answers? It will be their own imagination and assumptions; their own interpretation of what “sound” means, all of which may well be defined by any previous experience they have had at a location where “sound” has been replayed. And if that was bad sound, either in its creation or delivery or both, they are not going to say yes, ever!
• Finally: The survey was scored by a scale of 0-10 where zero was “Don’t like”, to ten which was “Definitely do like”. Now, what if the scores had all been five, or at least an average of five? – an answer which presumably would mean, sound might help but not sure.Do you use sound or not? Do you invest half-heartedly in some average sound that doesn’t cost too much to see if it works? And lo-and-behold then find that if you do that and re-issue the survey, you will probably end up with more negative responses because the job was not well enough produced, created or delivered.

So now you are no better off than before regarding whether “sound” will enhance a visit or not, and in fact you have wasted money. I use this example only because it stuck in my mind and made me angry!!

Consider the effectiveness of surveys?

Of course, there are some very good market research companies out there that will ensure such errors are not made, but often surveys are developed “in-house” with little or no understanding of how to word questions, or gather and interpret data to provide really useful and usable information. And, of course, that is often down to the cost of professional surveys, which can be preclusive.

So, for our real world, all I am saying, all I’m urging you to do, is to think very carefully about how you gather data and then how you interpret it.

There’s no doubt data is growing in importance; we must therefore understand it better and make certain that the statistics are worthy and properly scrutinised.

Actually, I think I am saying, read Sir David’s book!! (BTW – I do not know him, never met him, so am not on any commission at all, just in case you were wondering.)

As a quick extension to all this, experience has taught me that having informal conversations with random visitors provides very interesting anecdotal comment, and while it is obviously difficult to measure this and create the metrics, it should not be ignored as a practical way to provide some of your feedback.