Getting the most out of your Data Science team, pitfalls to avoid
During our month focussed on Data Science programming languages, my thoughts have turned to getting the most out of your Data Science team.
This topic has arisen, because of what I’ve observed in a number of organisations seeking to implement Data Science.
As I’ve worked with more clients and talked to other leaders at Data Science events, it has struck me how many fall into common pitfalls. These limit the impact made by their Data Science teams and so may limit the lifespan of business willingness to invest.
Like many such business problems, the challenge is not with the technology or innovation itself, but rather how it is managed. Once again, it’s what you do with it that counts.
So, in this shorter post, let me share a few common pitfalls I would encourage leaders to avoid.
Don’t keep your Data Science team in a backroom
Far too often in business, technical specialisms are treated as ‘backroom functions‘, to be kept apart from frontline customer service or the main operation of a business. Whether through concern to nurture this new discipline or allow specialists to focus on what they do best (and are often paid highly for), managers think it best to keep them apart.
I have seen Data Science teams tucked away in practically unvisited parts of buildings or within existing specialist teams who are close to being ivory towers (like R&D or Actuarial).
As I shared, when writing on the importance of nurturing Domain knowledge in analysts, no useful analytical work can be conducted in isolation. All analysis & modelling include a degree of subjective judgement, which should be informed by an understanding of the real world being modelled. Even tremendously clever Data Scientists cannot make the difference they could without knowing how data was captured or the reality of how models will be deployed. The real world meaning of each data item should inform its use and representation.
So, if you have set-up a Data Science pilot in a cupboard in the basement, I encourage you to let them out! Arrange opportunities for your Data Scientists to visit data capture points, like customer service teams, or to talk with those executing marketing campaigns or developing new propositions. Improved domain knowledge will enable improved data usage.
If you only have a hammer, everything begins to look like a nail
That saying is so true in a many a business and takes different forms at different stages of data/analytics maturity. Those businesses still only having MI or BI capabilities are likely to see all analytical questions as a request for a new report or dashboard. Those with only traditional statistical capabilities may see all needs to predictive models as regression problems.
But the same is also true of newer teams or approaches. Like every discipline that becomes fashionable, especially one with as broad a potential scope as “Data Science”, certain algorithms or approaches will get more PR at different times.
Right now, I hear a lot of Data Science teams talk a lot about using the Random Forest algorithm. To be sure this is a very flexible supervised learning method, that can be usefully applied to a range of regression or classification problems. However, it is not a panacea. Like more traditional decision tree algorithms (like CHAID or CART) they can be quick to build but slow when in use (especially for more complex relationships). They also include assumptions and limitations to the training data, so like many predictive methods, can leave the analyst with no more understanding of “what’s really going on” than when they started.
So, it’s worth ensuring that your Data Science team considers a wide range of potential approaches, including more traditional statistical methods and the need for descriptive analytics. Indeed many of the most effective Data Science teams regularly report that they still use logistic regression and k-means clustering methods, alongside machine learning algorithms. Plus, a good way to stay abreast of new developments in Data Science techniques is to build a link with your local universities. The Data Lab in Edinburgh proves what a mutually beneficial relationship that can be.
Don’t neglect developing your Data Scientists
A few Data Science teams I know are beginning to struggle with what is a perennial problem in analytics teams, that is retaining them. As salaries rise and more attractive brands seek to grow their data science capabilities, it is easy for ambitious Data Scientists to be lured away.
Talking to a few analysts and Data Scientist it is interesting to hear how consistent their motivations are. A few leave because it was a mistake hiring them in the first place (for instance insufficient data or data in too much of a mess, for them to make a difference).
However, beyond that hygiene factor, the chief motivations that keep Data Scientists at their existing firm appear to be:
- Interesting problems and an opportunity to make a difference – see their work used
- Continual development, opportunities to learn new skills – develop themselves
Some are more concerned about keeping up with technical skills (new machine learning algorithms), but most also value developing a wider skill-set. Indeed, I’ve found a number want to also develop the kind of softer skills that help all analysts make a difference in their businesses.
In my experience, it can be a mistake to assume that the only career path for data scientists is to become increasingly technical. Recognition of the different competencies needed within a fully effective Data Science team should be used to develop options for career development. Some will want to focus on a Data Engineer/Architect skill-set, others Data Visualisation/Artist, others business consultancy, others Modelling etc. Developing a competency framework to support career development discussions should be a leadership priority.
What do you see as the tips to better manage your Data Science teams?
Well, that’s what I have learnt and heard from tailing to Data Science leaders (as well as the obvious hygiene factors or sensible roles & paying a market rate).
What about your experience? If you lead a Data Science team or have started a pilot, what has worked for you? Any lessons learnt the hard way (by making mistakes)?
Please do share your wisdom in our comment boxes below & I may even ask you for an interview for a future post.
Have a great week leading your Data Science team & helping them make a difference.