Intelligence. Beautifully engineered. We are a data science agency.

Not all data is created equally

Grrrr, Excel. It’s that time of year when the corporate planning cycle kicks in and proper work is kicked into touch as next years plans are written, spun, chopped, pulled apart and horse-traded over.

Apart from the obviously painful process perhaps what’s even more worrying is that many Excel models are so poorly constructed. Not because the numbers don’t add up, they do, (well most of the time). No, the real threat is the lack of transparency on the quality of assumptions that underpin the decisions being taken. Indeed we’ve seen many decision models that give no indication of whether a number is a historical provable fact, an estimate from a trusted source or just a heroic guess – that had to be used as there was nothing else to hand!

And given that in business most decisions are forward facing, which by definition means you can’t absolutely know what will happen, this is a bit of an issue. You could call it the curse of the Black Swan, made famous in Nassim Nicholas Taleb’s excellent critique of the financial services industry and treatise on unpredicted events.

Be it planning next year’s budget, launching a new product, defining a business strategy or even inventing a new exotic financial derivative what is required is transparency – a clear indication of what’s a fact and what’s a guess. And it’s this principle of risk management that decision makers should adopt i.e. it’s OK to take a risk as long as you understand what the risk is. It’s when that risk is opaque that things tend to go awry.
This is not to dismiss the value of guesswork, despite what the “garbage in, garbage out” protagonists say. Guesses are valuable. Consider, for instance, planning a business in a market that doesn’t properly exist yet then garbage is all you have – so you’ve just to deal with it.

And in a world of Twitter, blogs and Facebook running in parallel to traditional content providers this challenge is amplified especially as these diverse sources of information are aggregated, analysed and visualised in VAIMs. Consequently I think it’s important to consider what I call ‘hard’ and ‘soft’ data; hard data being factual, proven news, stock prices histories, known dates of new regulation etc whereas soft data is opinion, gossip, rumour, photos etc. Its this ‘soft’ imperfect information that joins the dots. How many of the salespeople only respond to historical known facts? No, instead they’re also mining the rumour, gossip and opinion to help them fully understand the situation and react faster, with better market intelligence.

I’d suggest that humanistic predictions and decision-making quite naturally merge these two elements, whereas traditional Business Intelligence tools tend to cope well with hard data only. It’s something that the emerging field of Predictive Analytics must explore – knitting together the analytics bit with the softer human decision-making.

Interestingly the Met Office uses a data index to help structure it’s weather forecasting algorithms. This approach is extremely useful and one that we have used regularly to develop index of ‘quality indicators’ that can flag confidence, and when appropriate even treat the range of data from hard to soft differently. For instance you may want to range the variance on a prediction based upon gossip more than a published forecast – or not if you have an impeccable informal source, for example opinion sourced from Breaking Views or Lex could actually have very high quality.

And of course these quality indicators can evolve over time as sources improve or degrade.

This reminds me of an interesting book Certain to Win by Chet Richards that explores military strategy, and highlights how context is constantly changing therefore the key approach must be to Observe, Orientate, Decide and Act – what he calls a OODA loop – and that these OODA loops must be fast and often. Professor Don Sull at London Business School has called this ‘strategy agility’.

Someone, somewhere said “life’s a beta”. So as we’re sitting developing scenarios and plans for next year (and if we’re really brave the year after) I wonder whether in the same way GIS systems use xyz coordinates to pin together otherwise unrelated data, we can use timelines in the same way, with ‘plan’, ‘actual’ and ‘forecast’ being three quite distinct timelines that are always in place, make use of hard and soft data flagged with intuitive quality indicators and assume a healthy dose of garbage will actually make those Excel models more informed.

You get what you pay for

So the FT meets iTunes, just as Rupert Murdoch announces that News Corp intends to charge for all its news websites the FT is considering launching a ‘pay-per-article’ service styled upon the iTunes model of simple downloads and even simpler pricing.

This is interesting on a number of levels, firstly it highlights a perception that maybe, just maybe, not all information ‘wants to be free’ as consumers value other attributes such as simplicity, design and convenience alongside price.

The FT has 117,000 subscribers (out of 1.3m registered users) paying just over £150 ($250,  175) for one year’s online access. I have to admit I subscribe to the pulp version, delivered to my door before 7am every morning, read on the way into the office, enjoying a small moment of peace on the busy commuter route courtesy of South West Trains. I use the online version sporadically, typically for more specific articles that I am interested in. This may be a small but I think it’s a critical distinction – and is one that the FT seems to have taken on board.

Admittedly music is more of a passion than stock prices but Apple’s iTunes makes it far to easy to feed my music vice with one-click purchases. It also increases my spending as I am recommended new artists based upon previous purchases – can the FT or other news providers do the same? If you liked this (possibly slightly caustic) article on management fads by Lucy Kellaway perhaps you’d also be interested in Luke Johnson’s back-to-basic manifesto for entrepreneurs.

The trick is that it’s all about the experience. For me the magic of Apple’s iTunes is not the price, you can certainly buy cheaper elsewhere, if not download for free. Nor is it choice, I really don’t care about being limited by Apple’s DRM – though I know a few people that get extremely vexed by this (whisper it, but life is too short). Instead it’s the simplicity, finding what I want, click, download, sync and listen with absolutely no effort at all.

At 5bn downloads to date this seamless services has changed perceptions – proving that consumers are prepared to pay for something that they can get for free elsewhere, provided that there was some form of new value, in this case simplicity, convenience and a typically ‘superior Apple experience’. To my mind John Ridding, the chief executive of the Financial Times, is absolutely right when he says “It needs to be frictionless, people don’t have a lot of time and don’t want to go through a laborious transaction process for one article [but] I think there’s potential there for monetising a whole new layer of traffic and readers.”

Secondly, it highlights the fact that advertising can’t fund everything. Murdoch says that “quality journalism is not cheap and an industry gives away its content is simply cannibalising its ability to produce good reporting”. Putting aside the idea of The Sun and good reporting, it’s clear that the worst advertising slump in memory has brought this sharply into focus.

And it’s not just news publishers that are wrestling with this issue, look for instance at the ad-funded music services such as Spotify that pay royalties on the music played irrespective of the ad revenue generated – and who are now aggressively looking for new revenue streams with premium services ranging from ad-free subscriptions to mobile apps.

Clearly having exclusive or specific niche content (particularly in high value sectors such as Financial Services) makes charging a premium much easier. However, there may be an argument that business professionals are increasingly comfortable sourcing their general news – and even their ‘market intelligence’ – in bite size pieces from wide range of sources. Perhaps a key element is whether you can put it on an expense claim (obviously I’m talking about us mere mortals here, not MPs). Once news publishers can demonstrate how news informs decisions (and make it easy to have itemised receipts) it’ll be easier to charge through. Tapping into business professionals that act as individuals and purchase as consumers (I’ve heard them called prosumers but I hate that word) opens up a significant new market opportunities as you can dodge the corporate procurement department and drive consumer scale within a business context.

In terms of changing consumer behaviour this may be helped by the convergence of news ‘apps’ with mobile devices (BBC, Thomson Reuters and Bloomberg all have iPhone apps, although these are currently free) as consumers are used to mobile operators who traditionally have used micro-payments for a whole range of services such as sending a text.

Thirdly, it opens up new opportunities. As the news providers put in place the ability to charge for news and consumers get used to sourcing their information in bite-size ‘pay-per-article’ chunks from a range of providers there will be new players that can wrap new value around this core. Look for instance at the fabulous Guardian Chalkboards as an example of adding new insight and building community around a basic news service.

Over the past 3 years Steven Kimbrough, a Professor at the Wharton Business School, University of Pennsylvania, has developed the idea of “Value Added Information Mashups” VAIMs (originally coined by Ellen Miller at the Sunlight Foundation) to describe the combining small pieces news and information from disparate sources to help you join the dots and understand the full picture.

There seems to be a potential symbiotic relationship between news providers and VAIM services such as Twine, AllTop and Flipboard that can act as aggregators of disparate sources, editors as consumers themselves curate their own areas of interest, and even offer analytics to help interact with and interpret the inflow of information.

Convincing consumers to pay for news will not be easy, especially when in competition with public bodies such as the BBC but overall I think that this sort of experimentation is welcome.

I would suggest there is tremendous latent value in interacting with the news in new forms, such as tracking the timeline of a story or viewing heatmaps of opinion. My instinct is that if the news providers don’t purely focus on charging for the news content but think through the wider opportunities and work with the new players offering open up new sources of value there’s an exciting new market opening up.

Be it exclusive information, great design, or trusted brand I guess the old adage of ‘you get what you pay for’ still holds.