I was curious the other day, what is the most demanded devops skills out there on the market?
Not that I didn’t have any clue, as for someone who has been in the industry for a while it is kinda obvious, but sometimes you simply curious or just want to get some sort of stats. So after couple googling attempts which didn’t give any reasonable results apart from boring marketing ads and stupid suggestions like soft skill (who cares!), I decided that best approach would be DIY!
So here is what I did, step by step
1) Went to the web site many have probably used to find a job and put some search criteria
Then switched to Classic View and changed summary to 200 jobs per page, which is the max. Now all I needed is to find all occurrences of some keyword on the resulting page. (this manual part would benefit from selenium/phantomjs if run regularly, I probably will add it later)
2) Obviously I didn’t want to count manually, so I decided hey let’s do it with curl and then scan the output with some predefined keywords. Initially the keywords file was too big, then I skipped some stuff as it appeared to be not that popular(1 or 2 occurrences). But in general the file needs to be maintained as overtime some new kids in the block will pop out. So here the list in the words.txt file:
➜ trendystuff cat words.txt aws azure terraform ansible puppet chef docker kubernetes mesos jenkins ci/cd elasticsearch kibana logstash elk prometheus openstack zabbix vault linux scripting unix bash python groovy ruby git maven ➜ trendystuff
3) Now let’s write some super simple dummy bash script to get though the output and make some stats:
➜ scrips cat jobstats #!/bin/bash if [[ $@ != **-c** ]]; then curl -s $1 > output.txt fi if [[ $@ == **-2** ]]; then sort_arg=" -k 2 -r" fi rm result.txt for word in `cat words.txt`; do echo "$word `grep -io $word output.txt \ | wc -l`" \ | xargs >> result.txt done; cat result.txt | sort -n $sort_arg% ➜ scrips
4) Finally let’s run it, we have to copy the url from the website, which will be generated once you put you search criteria and press search, and pass it as argument to the script:
➜ trendystuff jobstats 'https://www.jobserve.com/gb/en/JobListing.aspx?shid=B141368A285A3CF3D6' ansible 72 aws 163 azure 25 bash 24 chef 12 ci/cd 28 docker 100 elasticsearch 3 elk 2 git 28 groovy 11 jenkins 58 kibana 4 kubernetes 91 linux 81 logstash 4 maven 6 mesos 1 openstack 8 prometheus 2 puppet 30 python 38 ruby 4 scripting 32 terraform 66 unix 22 vault 14 zabbix 2
By default it will sort by name, once you got you result back from the site, you can run script in cache mode and also sort by numbers rather than name, so let’s add cache argument ‘-c’ and sort by second column(numbers) argument ‘-2’:
➜ trendystuff jobstats -c -2 aws 163 docker 100 kubernetes 91 linux 81 ansible 72 terraform 66 jenkins 58 python 38 scripting 32 puppet 30 git 28 ci/cd 28 azure 25 bash 24 unix 22 vault 14 chef 12 groovy 11 openstack 8 maven 6 ruby 4 logstash 4 kibana 4 elasticsearch 3 zabbix 2 prometheus 2 elk 2 mesos 1 ➜ trendystuff
5) My thoughts about results.
Does the result made any sense to me? I would say absolutely!
AWS is absolute winner, because where else would you be running you infrastructure? I did some no_cloud or crap_bare_VM kinda projects and know the pain, even though throughout the process you can get your hands very dirty with low level sysadmin skills like NFS, backup, DNS, disk volume management, etc etc, but if you already have this sort of experience then it will be just slowing everything down as you wont’ have much time to implement actual CI/CD and deliver any viable stuff. That’s said, you still will need someone to focus on low level stuff, but it is so much easier and better to do in the cloud environment.
On the other hand it is going to take some time before rivals catch up with AWS.
Docker. If you have been working with dockerized Software delivery, you know how hard to get back to managing actual jars or whatever packaging you used and to deal with all sort of application/application server configuration. So docker is without any doubt sort of technology everybody eventually will be using anyway. It is just a matter of time if you still not suing it.
K8s. Back in 2015 I worked on project where docker containers were deployed with ansible playbooks, we simply adopted the docker a bit earlier than others and no one knew how painful it is to work without any orchestration tool, then we attended KubeCon 2016 London and everything became very clear – K8s is the future and apparently even it’s rivals like Cattle and Swarm has even admitted that after short battle.
Linux, well everything runs on Linux, even AWS so…
Ansible, comparing to it’s rivals like Chef and Puppet(lets forget about existence of CF Engine kinda old scary monsters) is a winner at least because of it’s simplicity to install and use, add on top tons of integrations and ability to easily add a new module in Python(which is according to our stats took first place in scripting languages) and you won’t question why it is a winner any more.
Terraform – even though early AWS adopters may have probably mostly used CloudFormation, it’s verbosity is a pain. Terraform, just like Ansible, is very easy to use and reminds a DSL like programming language, very powerful and less verbose.
Finally Jenkins! Although it may sound funny as why this decade old tool could still be most popular, on the other hand, with Jenkins 2, JobDSL and shiny pipeline as groovy code feature, plus enormous integration with everything humanity has even built, plus flexibility to set it up and configure it’s plugins using Java API and groovy, it is still remaining the favourite tool among devops engineers. Having used a modern and alternative tool like GOCD, I still would use Jenkins even though I have to admin it is obviously missing some CD features, and it’s pipeline architecture and stages are not truly pipelines but still a one big ‘job’, it is nevertheless a best CI tool.
The rest of the stats show that even though there is a demand for it, but it is not as high as the ones I mentioned above.
If you can’t open jobserve because you are scared of your manager, the output etc is on github.