Skip to content

Month: February 2018

The most demanded DevOps skills stats, DIY approach.

The most demanded DevOps skills stats, DIY approach.

Published February 18, 2018

I was curious the other day, what is the most demanded devops skills out there on the market?
Not that I didn’t have any clue, as for someone who has been in the industry for a while it is kinda obvious, but sometimes you simply curious or just want to get some sort of stats. So after couple googling attempts which didn’t give any reasonable results apart from boring marketing ads and stupid suggestions like soft skill (who cares!), I decided that best approach would be DIY!

So here is what I did, step by step

1) Went to the web site many have probably used to find a job and put some search criteria

Then switched to Classic View and changed summary to 200 jobs per page, which is the max. Now all I needed is to find all occurrences of some keyword on the resulting page. (this manual part would benefit from selenium/phantomjs if run regularly, I probably will add it later)

2) Obviously I didn’t want to count manually, so I decided hey let’s do it with curl and then scan the output with some predefined keywords. Initially the keywords file was too big, then I skipped some stuff as it appeared to be not that popular(1 or 2 occurrences). But in general the file needs to be maintained as overtime some new kids in the block will pop out. So here the list in the words.txt file:

➜  trendystuff cat words.txt 
aws
azure

terraform

ansible
puppet
chef

docker
kubernetes
mesos

jenkins
ci/cd

elasticsearch
kibana 
logstash
elk

prometheus
openstack
zabbix
vault

linux
scripting
unix
bash
python
groovy
ruby

git
maven
➜  trendystuff

3) Now let’s write some super simple dummy bash script to get though the output and make some stats:

➜  scrips cat jobstats 
#!/bin/bash

if [[ $@ != **-c** ]]; then 
	curl -s $1 > output.txt  
fi	

if [[ $@ == **-2** ]]; then 
	sort_arg=" -k 2 -r"
fi	

rm result.txt
for word in `cat words.txt`; do
	echo "$word `grep -io  $word output.txt \
	| wc -l`" \
	| xargs  >> result.txt
done; cat result.txt | sort -n $sort_arg%  
➜  scrips

4) Finally let’s run it, we have to copy the url from the website, which will be generated once you put you search criteria and press search, and pass it as argument to the script:

Continue readingThe most demanded DevOps skills stats, DIY approach.
Comments closed
Docker volume monitoring with Ruby, Sensu and Uchiwa.

Docker volume monitoring with Ruby, Sensu and Uchiwa.

Published February 16, 2018

In this post I am going to demonstrate how to monitor your docker volumes with Sensu.
I came across a problem when our Jenkins instances were running out of space and no jobs could be scheduled because of this,
so before it was too late, it would be very useful to have something in place that will show if some container is too greedy, eating all the space on the volume.

In general we are going to look at the next things today:
1. Some Ruby scripting
2. Identifying disk and docker volume usage commands
3. Configuring Sensu server and client for monitoring
4. Making script run as a root
5. Running a simple Uchiwa dashboard

1. Some Ruby scripting
So first we are going to write the script which will check the volume and report if usage is higher than we configured.
Following best Sensu practices we will write it in Ruby, we probably could also use bash, but it really gets messy once we add more logic and lines.

#!/usr/bin/env /opt/sensu/embedded/bin/ruby

max_size = ARGV[0].to_i
container_name_filter = ARGV[1]
message = ""

procs=`du -sk  /var/lib/docker/volumes/* | sort -rn`
procs.each_line do | process|
  result = process.split(" ")
  vol_usage = result[0].to_i/1024
  vol_name = result[1].gsub "/var/lib/docker/volumes/", ''

  if vol_usage > max_size
    cont_name = `docker ps --filter=volume=#{vol_name} --filter=name=#{container_name_filter} --format {{.Names}}`
    if !cont_name.empty?
      message = message + "container: #{cont_name.delete!("\n")} volume exceeds max disk usage(#{max_size}MB): #{vol_usage}MB; \n"
    end
  end
end

unless message.empty?
  puts message
  exit 1
end

2. Identifying disk and docker volume usage commands

Continue readingDocker volume monitoring with Ruby, Sensu and Uchiwa.
Comments closed
Automating gmail check in your shell

Automating gmail check in your shell

Published February 5, 2018

It you are just like me, who likes doing almost everything through the shell scripts rather than fancy UI apps, then here is a nice and easy way of checking a new emails in your gmail account:


function gmail(){
	emails=$(curl -s -u $1:$2 "https://mail.google.com/mail/feed/atom"\
	 | egrep -o '<fullcount>[0-9]*' | cut -c 12-)
	if [ "$emails" -gt 0 ] ; then echo "You have ${emails} emails in your $1 account"; fi
}		
gmail daenerys.targaryen $(vault read -field=value secret/GPASSWORD)
gmail simply.dany $(vault read -field=value secret/G2PASSWORD)

Simply add this to your .zshrc/.bashrc and you are done, next time you open a new tab you might get this:


Last login: Mon Feb  5 21:27:39 on ttys006
You have 1 emails in your daenerys.targaryen account
➜  ~
Continue readingAutomating gmail check in your shell
Comments closed