Skip to content

Month: August 2017

Dockerizing Jenkins build logs with ELK stack (Filebeat, Elasticsearch, Logstash and Kibana)

Dockerizing Jenkins build logs with ELK stack (Filebeat, Elasticsearch, Logstash and Kibana)

Published August 22, 2017

This is 4th part of Dockerizing Jenkins series, you can find more about previous parts here:

Dockerizing Jenkins, Part 1: Declarative Build Pipeline With SonarQube Analysis
Dockerizing Jenkins, part 2: Deployment with maven and JFrog Artifactory
Dockerizing Jenkins, part 3: Securing password with docker-compose, docker-secret and jenkins credentials plugin

Today we are going to look at managing the Jenkins build logs in a dockerized environment.

Normally, in order to view the build logs in Jenkins, all you have to do is to go to particular job and check the logs. Depending on a log rotation configuration, the logs could be saved for N number of builds, days, etc, meaning the old jobs logs will be lost.

Our aim in this article will be persisting the logs in a centralised fashion, just like any other application logs, so it could be searched, viewed and monitored from single location.

We also will be running Jenkins in Docker, meaning if container is dropped and no other means are in place, like mounting the volume for logs from a host and taking the backup the logs will be lost.

As you may have already heard, one of the best solutions when it comes to logging is called ELK stack.

The Idea with ELK stack is you collect logs with Filebeat(or any other *beat), parse, filter logs with longstash and then send them to elasticsearch for persistence, and then view them in kibana.

On top of that, because logstash is heavyweight jruby app on JVM , you either skip it at all or use a way smaller application called Filebeat, which is a logstash log forwarder, all it does, collects the logs and sends to longstash for further processing.

In fact, if you don’t have any filtering and parsing requirements you can skip the logstash at all and use Filebeat’s elastic output for sending the logs directly to elasticsearch.

In our example we will try to use all of them, plus, we won’t be running Filebeat in a separate container, but instead, will install it right inside of our Jenkins image, because Filebeat is small enough. I also wanted to demonstrate how we can install anything on our Jenkins image, so it is more interesting.

So the summary of what we are going to look at today is:

  1. Prepare our dockerized dev environment with Jenkins, Sonarqube and JFrog artifactory running the declarative pipeline
  2. Download and install Filebeat on our Jenkins image
  3. Configure Filebeat so it knows how and where collect the Jenkins logs and how to send them further to logstash
  4. Configure and run logstash in a docker container
  5. Configure and run elasticsearch in a docker container
  6. Configure and run kibana in a docker container

1. Prepare our dockerized dev environment with Jenkins, Sonarqube and JFrog artifactory running the declarative pipeline

In this example we will use Jenkins image we created earlier in the part 3 of these series. First thing first, let’s checkout the project:

git clone https://github.com/kenych/dockerizing-jenkins && \
   cd dockerizing-jenkins && \
   git checkout dockerizing_jenkins_part_3_docker_compose_docker_secret_credentials_plugin && \
   ./runall.sh

Let’s see what runall.sh does:

Continue readingDockerizing Jenkins build logs with ELK stack (Filebeat, Elasticsearch, Logstash and Kibana)
Comments closed

Dockerizing Jenkins, part 3: Securing password with docker-compose, docker-secret and jenkins credentials plugin

Published August 12, 2017
Dockerizing Jenkins 2, part 3: Securing password with docker-compose, docker-secret and jenkins credentials plugin

This is 3rd part of Dockerizing Jenkins series, you can find more about previous parts here:
Dockerizing Jenkins 2, Part 1: Declarative Build Pipeline With SonarQube Analysis
Dockerizing Jenkins 2, part 2: Deployment with maven and JFrog Artifactory

In this part we will look at:

  1. How to use docker-compose to run containers
  2. How to use passwords in docker environment with docker-secrets
  3. How to hide sensitive information in Jenkins with credentials plugin

In the part 1 we created basic jenkins docker image in order to run java maven pipeline with test and sonarqube analysis and then in the part 2 we looked at how to perform deployment using maven settings file. As you remember we saved the password in the file without any encryption, which is not you would obviously ever do, of course.
All code for this and previous parts is in my GitHub repo https://github.com/kenych/dockerizing-jenkins and I decided to create a branch for every part, as master branch will change with every part and older article would refer to wrong code base, for this part the code will be in the branch “dockerizing_jenkins_part_3_docker_compose_docker_secret_credentials_plugin” and you can run the below command to check it out:

git clone https://github.com/kenych/dockerizing-jenkins && \
	cd dockerizing-jenkins && \
	git checkout dockerizing_jenkins_part_3_docker_compose_docker_secret_credentials_plugin

In this part we will remove password from the source code and let credentials​ plugin apply credentials to Config File Provider Plugin. But before changing any code, we will need to switch to using docker-compose instead of using docker run command. This will give us a chance to leverage docker secrets feature along with many other features which you will love.

I updated runall.sh script which we used in two parts before and replaced with docker-compose and download.sh script which will just download the minimum stuff we will need in advance. I also removed java 7 and java 8 installation in favour to use embedded java 8 from jenkins container as otherwise our download script takes too long and java comes for free in the image anyway, you can check it later once our jenkins container running.

If you were following part one and two you should know how to pick up specific java version anyway using maven tool mechanism and if you want to play with that just uncomment these lines in download script, java.groovy and in the pipeline as well. Now let’s run download to make sure we have everything we need:

➜  ./download.sh
2.60.1: Pulling from library/jenkins
Digest: sha256:fa62fcebeab220e7545d1791e6eea6759b4c3bdba246dd839289f2b28b653e72
Status: Image is up to date for jenkins:2.60.1
6.3.1: Pulling from library/sonarqube
Digest: sha256:d5f7bb8aecaa46da054bf28d111e5a27f1378188b427db64cc9fb392e1a8d80a
Status: Image is up to date for sonarqube:6.3.1
5.4.4: Pulling from jfrog/artifactory-oss
Digest: sha256:404a3f0bfdfa0108159575ef74ffd4afaff349b856966ddc49f6401cd2f20d7d
Status: Image is up to date for docker.bintray.io/jfrog/artifactory-oss:5.4.4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8334k  100 8334k    0     0   445k      0  0:00:18  0:00:18 --:--:--  444k

Please note if you haven’t ever downloaded the images, it will take some time. Now, while it is downloading the stuff we need, let’s look at docker-compose.ym:

version: "3.1"

services:
  myjenkins:
    build:
      context: .
    image: myjenkins
    ports:
     - "8080:8080"
    depends_on:
     - mysonar
     - artifactory
    links:
      - mysonar
      - artifactory
    volumes:
     - "./jobs:/var/jenkins_home/jobs/"
     - "./m2deps:/var/jenkins_home/.m2/repository/"
     - "./downloads:/var/jenkins_home/downloads"
    secrets:
     - artifactoryPassword
  mysonar:
    image: sonarqube:6.3.1
    ports:
     - "9000"
  artifactory:
    image: docker.bintray.io/jfrog/artifactory-oss:5.4.4
    ports:
     - "8081"

secrets:
  artifactoryPassword:
    file: ./secrets/artifactoryPassword

If you were curious, you would ask why did I call the file docker-compose.yml?

Continue readingDockerizing Jenkins, part 3: Securing password with docker-compose, docker-secret and jenkins credentials plugin
Comments closed