DevOps Principles & Best Practices by Kayan Azimov
I came across this amazing way of testing if I could reach a port on the host, when literally nothing I tried was available:
vagrant@ ~ () $ echo hi | nc -l -p 8089 & [1] 13651 vagrant@ ~ () $ cat < /dev/tcp/127.0.0.1/8089 hi [1]+ Done echo hi | nc -l -p 8089 vagrant@ ~ () $ vagrant@ ~ () $ cat < /dev/tcp/127.0.0.1/8089 -bash: connect: Connection refused -bash: /dev/tcp/127.0.0.1/8089: Connection refusedComments closed
Today I am going to show how you can fully automate the advanced process of setting up the highly available k8s cluster in the cloud. We will go through a set of terraform and bash scripts which should be sufficient enough for you to literally just run terraform plan/apply to get your HA etcd and k8s cluster up and running without any hassle around.
Part 0 – Intro.
If you do a short research on how to setup k8s cluster you may find quite a lot of ways this could be achieved.
But in general, all this ways could be grouped into 3 types:
1) No setup
2) Easy Set up
3) Advanced Set up
4) Hard way
By No setup I simply mean something like EKS, it is a managed service, you don’t need to maintain or care about details while AWS will do all for you. Never used it can’t say much on that one.
Easy setup, tools like kops and alike make it quite easy – couple commands run kinda setup:
kops ~]$ kops create cluster \ --name=k8s.ifritltd.net --state=s3://kayan-kops-state \ --zones="eu-west-2a" --node-count=2 --node-size=t2.micro --master-size=t2.micro --dns-zone=k8s.ifritltd.net --cloud aws
All you need is setup s3 bucket and dns records and run the command above which I described two years ago in this article
The downside is first of all it is mainly only for AWS, and generates all AWS resources as it wants, so lets say it would generate security groups, asg, etc in it’s own way which means
if you already have terraform managed infra with your own rules, strategies and framework, it won’t feet into that model but just added as some kind of alien infra. Long story short if you want fine grained control over how your infra should be managed from single centralised terraform, it isn’t best solution, yet still easy and balanced tool.
Before I start explaining how to use Advanced Set up, I am just going to mention that 4th, The Hard way is probably only good if you want to learn how k8s works, how all components interact with each other, and as it doesn’t use any external tool to set up components, you do everything manually, you literally know all the guts of the system. Obviously it could become a nightmare to support such system in production unless all members of the ops team are k8s experts or there are some requirements not supported by other bootstrapping tools.
Finally the Advanced Set up.
Comments closedI was curious the other day, what is the most demanded devops skills out there on the market?
Not that I didn’t have any clue, as for someone who has been in the industry for a while it is kinda obvious, but sometimes you simply curious or just want to get some sort of stats. So after couple googling attempts which didn’t give any reasonable results apart from boring marketing ads and stupid suggestions like soft skill (who cares!), I decided that best approach would be DIY!
So here is what I did, step by step
1) Went to the web site many have probably used to find a job and put some search criteria
Then switched to Classic View and changed summary to 200 jobs per page, which is the max. Now all I needed is to find all occurrences of some keyword on the resulting page. (this manual part would benefit from selenium/phantomjs if run regularly, I probably will add it later)
2) Obviously I didn’t want to count manually, so I decided hey let’s do it with curl and then scan the output with some predefined keywords. Initially the keywords file was too big, then I skipped some stuff as it appeared to be not that popular(1 or 2 occurrences). But in general the file needs to be maintained as overtime some new kids in the block will pop out. So here the list in the words.txt file:
➜ trendystuff cat words.txt aws azure terraform ansible puppet chef docker kubernetes mesos jenkins ci/cd elasticsearch kibana logstash elk prometheus openstack zabbix vault linux scripting unix bash python groovy ruby git maven ➜ trendystuff
3) Now let’s write some super simple dummy bash script to get though the output and make some stats:
➜ scrips cat jobstats #!/bin/bash if [[ $@ != **-c** ]]; then curl -s $1 > output.txt fi if [[ $@ == **-2** ]]; then sort_arg=" -k 2 -r" fi rm result.txt for word in `cat words.txt`; do echo "$word `grep -io $word output.txt \ | wc -l`" \ | xargs >> result.txt done; cat result.txt | sort -n $sort_arg% ➜ scrips
4) Finally let’s run it, we have to copy the url from the website, which will be generated once you put you search criteria and press search, and pass it as argument to the script:
Comments closed
