In praise of imperfection, for analysts to achieve success

Amongst their many positive qualities, analysts can often be perfectionists, unwilling to settle for imperfection.

A quest for excellence and an attention to detail can be admirable qualities. Together with a restless curiosity, such attitudes can characterise a natural analyst.

But, as all perfectionists will discover, seeking perfection in your work can more often hinder than help.

So, given all the hype on social media about technological perfection, let me offer an antidote.

In this post, I’m going to speak up in favour of imperfection. Sharing 4 examples of where accepting imperfect solutions is key to making progress.

In praise of imperfection in analytics

Now, conquering a tendency to perfectionism is not just relevant for analysts. Many a leadership guru has written about the need for driven personality types to avoid perfectionism. This post from Michael Hyatt is a good example of such sound advice.

But, that can sound like it is only relevant for senior leaders or generalists. With much analytics work feeling like it is searching for the ‘right answer‘, or will be judged on accuracy, is there room for imperfection?

In this post I will argue, yes! In fact, it can be essential for analysts to overcome their perfectionist tendency, if they are to make a difference in their business. The alternative can be an experience other business leaders can all too easily term ‘analysis paralysis‘.

I’m not suggesting accuracy doesn’t matter. If someone is building a machine learning solution to automate surgery, they better be sure it’s as accurate as possible. But, few business applications are actually life and death. Much more value is normally lost by lack of progress, rather than by implementing a ‘minimum viable product‘ mindset.

With that caveat, when might it be helpful to accept less perfect solutions, in order to progress sooner? Here are five specifics.

Imperfection helps you prepare for GDPR

Data protection regulation might sound like a strange place to start. Surely, GDPR non-compliance is high risk? It certainly matters & as I have advised before, that you need to ensure you prepare for GDPR.

But, listening to analysts and data leaders attending GDPR events, I am struck by the need for pragmatism. I would go as far as to suggest that hardly any UK business will be completely compliant by May 2018. Most are still a long way from perfection.

Discussing what they need to consider and the progress made so far, it strikes me that their bigger risk is delay due to indecision. It is clear the ICO intends to be pragmatic and proportionate in how it handles compliance. Like most principle-based regulators, it is looking not just for compliance but also intention. Firms should  evidence identification of potential issues and progress against a clear plan.

What is delaying some firms is debate over legal options & how complete a technical solution they can afford. For some, such debate, seeking a perfect decision, is delaying any visible progress.

This is a classic example of analysis paralysis. I would recommend getting on with it. Audit your existing data & processes. Priorities all potential gaps. Set-up Privacy by Design processes (including Data Protection Impact Assessments) into your projects. Educate your staff & communicate clearly with customers.

You won’t get everything perfect by May 2018, but recognise that progress is better than perfection.

Imperfection helps you handle Data Quality issues

Whilst chairing a panel at #CXC2017 event, I was struck by one of the answers from a Data Science leader. When asked what he most wanted improved, he declined to choose data quality.

Rather, he stressed that his machine learning algorithms could cope with imperfect data quality. The bigger problem was any delays to data access or readiness to deploy models.

Once again, the greater commercial value was realised by deploying sooner. Otherwise, the risk is that data quality can be a never-ending journey.

To balance that advice, all the old truisms of data quality management still stand. Where possible, tracing data quality issues back to source and improving the quality of capture makes sense. Once again, what is needed is pragmatism. Once data quality is sufficient, for analytics to yield robust insights or models, consider stopping. Delaying the work of analysts further may cost you more than further data quality improvements will realise as gains.

Imperfection helps you deliver a Single Customer View

The term Single Customer View (or SCV), has gone in and out of fashion over the decades. When I was building data warehouses in the 1980s it was reaching the height of its popularity. Within a few years it seemed all organisations aspired to have one.

A few decades later and the term is much less popular. The focus have moved onto more options for how you store & connect your data, including use of Hadoop & NoSQL. Yet, every analytics team I’ve worked with still needs a way to create at least a virtual SCV.

One of the reasons for skepticism and reluctance appears to be the money-pits such projects have proven to be, in the past. Many a CIO has had their fingers burnt with SCV projects that ran over time & well over budget. At one stage it seemed that no-one had actually achieved the ‘complete SCV’ nirvana.

From my experience, the culprit (again) is perfectionism. It just costs too much & takes too long to get all your data connected into a complete up-to-date SCV. On the flip-side, that does not mean you should give up. GDPR is only the latest ‘burning platform’ to mean you need a complete de-duplicated view of customers’ data.

The solution is accepting imperfection. As we have advised before, whether using a data  lake or a ‘playpen’, create a space to experiment. There analyse & test one-off extracts. Prove which data items are most important. Those you can act upon or are needed for customer experience. Build the most important data items first & start using your very valuable incomplete SCV.

Imperfection helps you drive action on insights

My recent survey of over 100 data insight leaders, showed that most have progressed to the stage where they needed to invest in execution. By this, I mean the technology to deploy models, triggers, segments or other analytic output.

For some the barrier is the complexity of integrating with existing technology. For others it is resistance from channel owners. Either way, a stalemate can be reached if analytics leaders are lobbying for a perfect deployment.

When sharing on stakeholder management, we raised the need to see things from the others’ perspective. Most CX leaders or channel owners have demanding service levels or sales performance targets. Relatively unproven analytics, can feel too great a risk for the disruption required.

So, it can help to not initially seek such a ‘bells & whistles‘ solution. Piloting can be a powerful ally, as can the ability to deploy with ‘chewing gum & string‘ technology. I still remember proving the huge financial potential of retention analytics, using just MS Access.

Whatever technology solution enables you to rapidly test with real customers, it is worth considering. Yes, it will be imperfect. Yes, it may be an unrepresentative sample of customers with different operational conditions. But, will it convince the senior stakeholders you need to provide the budget for a ‘proper’ deployment?

I’ve seen too many analytics leaders delayed in proving any measurable ROI, because they needed a full IT project. I’d counsel finding ways around that dependency, at least initially. Accepting imperfect solutions can be key to seeing those options.

How has imperfection helped you

All too often in business we hail success stories & impressive technical solutions. I hope this post has encouraged you to willingly embrace imperfection. Often there is more lost by not acting, than taking action with an imperfect solution.

Do you agree? If so, I’d love to hear your examples of pragmatism.

Let’s celebrate our human imperfections and the progress that can be achieved by pragmatism. Please do share your examples, either using comments link below or on social media.

Don’t wait to be perfect, you can add value already!