Moving from bioinformatics to data science
Almost exactly a year after contemplating my last day as a bioinformatician, I am now contemplating my last day as a data scientist in the Department for Work and Pensions, part of the UK civil service.
As I’ve blogged about previously, bioinformatics experience is highly relevant in the data science job market, and I’ve also discovered how important the ‘science’ bit of data science is. Being able to bring experience in experimental design and interpretation of results to an environment where people have more diverse and often non-scientific backgrounds has been valuable, and I’ve learnt a lot in return about things I have never come across before.
The main conclusion from this year: moving into data science was both smoother and more straightforward than I ever hoped for!
Reasons to stay
So why am I leaving? There are things that I’ve thoroughly enjoyed about working in the civil service. By and large civil servants are a diverse and friendly bunch, committed to making a difference to society. I was lucky to work with some really awesome colleagues. Being a huge organisation, it was relatively easy to find ‘like minds’, especially on the Government Data Science Slack forum which was fantastic for asking for help and advice, and sharing knowledge.
There is also a real drive towards using data for public good, so it’s an exciting place to be. The scale of the problems is vast too, affection potentially millions of people and budgets into the billions. The opportunities are exciting and enticing!
Reasons to go - organisational
Unfortunately, whilst there is a vision at the top of the organisation and willing at the bottom, somewhere in the middle things get snarled up. Government is rightly concerned about the security of its citizens data, and controlling costs. Unfortunately this frequently made it difficult to make progress due to the number of different beaurocratic layers to be negotiated, and the lack of empowerment and trust the organisation has in individual civil servants.
It often felt that we were battling against ‘the system’ to get our jobs done, and had little or no ability to influence or change the things that were important to us. A really key example of this was the difficulty we had in establishing a good data science platform to work on, not just for data scientists, but also the wider analytical community (who were using SAS). Even though there was support at the highest levels for using open source tools such as Python and R, getting a good solution in place proved elusive for reasons that seemed more cultural than technical.
Reasons to go - career progression
Another significant reason, and one that I believe contributes to the problems around delivering digital projects in general, is that there isn’t really a clear career progression for data scientists on the technical side. Once you get to a certain level the only way up is to become a manager, it’s not possible to stay close to the technical work.
This contrasts with my previous experience in the pharmaceutical industry where a scientific leadership career track is available for people who want to continue to focus on science rather than people management. Top tech companies such as Spotify also offer technical career tracks.
This has two unfortunate effects. Firstly, it deprives the organisation not just of crucial technical knowledge but also of leaders with deep technical expertise. Secondly, it deprives senior data scientists looking to make their next career step of mentors and role models who they can learn from.
Technical leadership is not just technical knowledge, it is also about how to develop technical expertise in an organisation and how to integrate technical strategy into business strategy.
Reasons to go - personal
Finally, as with any job move decision, I had to consider what I wanted from a job and how that lined up with what was on offer. I felt strongly that I wanted to continue to develop my technical skills in both the software and mathematical side of data science. It seemed that staying put would pull me further away from that and towards a role helping to manage a group, something that I want to do ultimately, but that I’m not sure I want just yet (and perhaps not somewere with the bureaucratic and political overhead of the civil service).
My ideal role would have been driving and championing the adoption of data science approaches and techniques within DWP, but I didn’t feel my skillset was quite there yet, and that role didn’t exist.
Advice if you’re considering a civil service career
Would I recommend the civil service to budding data scientists? The answer is a definite YES! It’s certainly a great way into data science from academia for example. But equally I would advocate not staying more than a few years at a time because you will learn a different way in the private sector and have more opportunity to work with the latest technology. In my opinion it’s also important for people working in government to also have experience outside of government.
Even if you only want to work in government, I think you’ll become a better civil servant for having some experience of industry, and you will also be able to bring back into the civil service new ways of doing things. In addition, you may find your time in the civil service limited because of the lack of career progression opportunities for data scientists at present, although there are hopeful signs that this is changing!
Looking forward
My next role is a really exciting opportunity with a small, dynamic software company where I will be working alongside experts in cutting edge cloud technologies. I’m looking forward to the new challenge of working in a different environment again, and being the technical dunce with lots to learn from all those smart software and DevOps engineers!! But equally I’m confident that I can bring a different way of looking at things, not just from my scientific career but also from my experience working in government. Bring it on!