Friday, August 17, 2018

My first experience with JenkinsX


As a DevOps engineer, I thought JenkinsX is a CI/CD tool for containers to build in K8s environments. But it's wrong. JenkinsX is an attempt to automate the whole development process end to end for containerized applications based on Docker and Kubernetes. JenkinsX is an Open Source project and is not a fork of Jenkins. JenkinsX reuses Jenkins Core and it has set of additional tools to achieve its goal. It is easy to customize JenkinsX as we can edit or replace any of its tool sets.
JenkinsX address the following problems:
  1. Frequent deployments
  2. Low Mean Time to Recover
  3. CI/CD
  4. Configuration as Code
  5. Automated Release Management
Once JenkinsX installed, it setup and configure the following for you:
  1. Create a Git repo for a new application with development, staging and production environment.
  2. Create a pipeline configuration in Jenkins for a new application and connect it with a Git repo
  3. Automate the DevOps processes (like builds, artifacts and containers creation and deployments) based on Git operations (branching, commits, PR creating, PR merging)
Building Blocks of JenkinsX:
Strengths of Jenkins X:
  1. It address the pain points and streamlining implementation of DevOps/GitOps principles. It saves a lot of time for new project implentations.
  2. Concept of JenkisX is very strong.
  3. Good toolset, which is already configured and works (k8s, Jenkins, Docker registry, Chartmuseum, Monokular, Nexus)
  4. "JX Quickstarts" make a creation of new apps an easy ride
  5. Ability to customise the pipelines and their templates
  6. It provides a preview environment, which helps in decision making for pull requests.
Not that good points of Jenkins X:
  1. Jenkins X is another framework to learn.
  2. Still its a baby, a lot of things to implement and improve
  3. Documentation lacks comprehensive, it has only basic information
  4. Migration of existing CI/CD pipelines into Jenkins X is difficult
  5. For each team, we need to deploy each JenkinsX instance
JenkinsX Flow:

How to install Mysql8 on OEL7.5

How to install Mysql8 on OEL7.5

Login to the server:

wget https://dev.mysql.com/get/mysql80-community-release-el7-1.noarch.rpm
rpm -ivh <filename>
yum install mysql-community-server -y
systemctl enable mysqld.service
systemctl start mysqld
grep 'temporary password' /var/log/mysqld.log (for getting the temporary password)
/usr/bin/mysql_secure_installation

Wednesday, August 15, 2018

Errors occurred deleting machine: Error deleting host: minikube: Error loading host from store: The system cannot find the file specified.

Error:
When I delete or start my minikube on windows10 machine, I am getting the following

C:\Users\shvijai>minikube delete
Deleting local Kubernetes cluster...
Errors occurred deleting machine:  Error deleting host: minikube: Error loading host from store: open C:\Users\shivin\.minikube\machines\minikube\config.json: The system cannot find the file specified.

Solution for this issue:

Remove the folder .minikube from C:\Users\shivin\
Start minikube again (minikube start)
Then you can see the VM getting downloaded.

Tuesday, August 14, 2018

How to create an insecure registry in OEL7+

I won't encourage you to create an insecure registry. But for me I need to setup this for a demo purpose. I was using an OEL7.4 OS.

Login to the server, update the file with your local registry details:

vi /etc/docker/daemon.json
{
  "storage-driver": "overlay2",
  "ip-masq": false,
  "insecure-registries": ["10.96.202.190:5000"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "20m",
    "max-file": "10"
  }

}

Here in my case, 10.96.202.190:5000 is the local registry. You need to change with yours.

systemctl daemon-reload
systemctl restart docker

Click the link How to test your registry?

How to test docker registry

This is just a sample test, whether your docker registry is working fine or not. We will pull an image and then tag and push it to the registry. Here I have a local registry "192.168.0.2:5000"

# sudo docker pull busybox
# sudo docker images 
# sudo docker tag <image-id> 192.168.0.2:5000/busybox
# sudo docker push 192.168.0.2:5000/busybox

If you didn't get an error, your docker registry is working fine. Else you may face some "certificate error" or "http server gave http response to https client"

Thursday, July 12, 2018

Jenkins Pipeline Parameter

Little background : I am running an inspec test for couple of servers. I accept server ip's as a jenkins parameter. Each IP the inspec test for the mysql profile should execute.

Parameter "IP_MYSQL" holds server ips

node{
    cleanWs()
stage('Checking MySQL'){
    echo 'Inspec test for mysql'
    withCredentials([file(credentialsId: 'mysql-prod', variable: 'SSH_KEY')]) {
    sh '''
sudo git clone https://github.com/dev-sec/mysql-baseline
cd mysql-baseline
    echo "${IP_MYSQL}" | sed -n 1'p' | tr ',' '\n' | while read IP; do
            sudo cp ../inspec.yml inspec.yml
            sudo sed -i -e "s/mysql-baseline/mysql-baseline-$IP/g" inspec.yml
            cd ..
            sudo inspec exec mysql-baseline -t ssh://clouduser@$IP -i $SSH_KEY --reporter junit:Report_$IP.xml || true
        done
'''
junit '*.xml'
    }
  }
}
}

Wednesday, June 13, 2018

failed to link /usr/share/man/man1/java.1 -> /etc/alternatives/java.1: No such file or directory

Error : failed to link /usr/share/man/man1/java.1 -> /etc/alternatives/java.1: No such file or directory

I am trying to install a java rpm in a docker container via dockerfile. I got the above error while installing rpm.

The reason is its a small container that cause the error. A workaround for getting out of this error is to create a directory before the rpm installation in your dockerfile.

mkdir -p /usr/share/man/man1

This works for me :)

Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

Getting the following error in docker:

Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

Issue fix:

1. cd /etc/systemd/system/docker.service.d  (If not present, create this directory structure)
2. Create a file http-proxy.conf in that path and update the following
Environment="HTTP_PROXY=http://yourproxy.com:80/"
Environment="NO_PROXY=localhost,127.0.0.0/8,docker-registry.somecorporation.com"
3. systemctl daemon-reload
4. systemctl show --property Environment docker
Environment=HTTP_PROXY=http://yourproxy.com:80/
5. systemctl restart docker

This fixed my issue. I am using OEL7.5

Friday, April 13, 2018

How can I enable root ssh access in EC2

My requirement is, I need to disable the key authentication and I need to enable root ssh access to my AWS EC2 server.

How can I do that?

Login to your EC2 server with key authentication first
vim /etc/ssh/sshd_config
Disable - PermitRootLogin forced-commands-only and update the same with PermitRootLogin yes
Disable - #PermitEmptyPasswords no and update the same with PasswordAuthentication yes

/etc/init.d/sshd restart

Give root a good password.

Try to login with SSH and given root password.

Wednesday, April 11, 2018

Jenkins Error:ERROR: Error cloning remote repo 'origin'

In my Jenkins job, I am getting the following error

ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Could not init /home/ec2-user/local-jenkins/workspace/vpc_terraform
 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$5.execute(CliGitAPIImpl.java:772)
 at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:564)
 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:153)
 at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:146)
 at hudson.remoting.UserRequest.perform(UserRequest.java:212)
 at hudson.remoting.UserRequest.perform(UserRequest.java:54)
 at hudson.remoting.Request$2.run(Request.java:369)
 at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
 
 
I was trying to checkout/download a repository from gitlab. I have given the repo name, credentials and branch.

Solution:

Go to the node where it is going to run (Slave node configuration)
Under Check Tool Locations under Node Properties 
Select Git and input the location of git (which git)
Save and reconnect the slave again.

 

Monday, April 2, 2018

mysql daemon failed to start in amazon linux

mysql daemon failed to start in amazon linux.

I could see that all the configurations and everything is fine and it was working fine.

Solution:

Issue was with there is not free swap memory on the server. You need to add manually a swap file to fix the issue.

Login to the server

# dd if=/dev/zero of=/swapfile1 bs=1024 count=524288
# chown root:root /swapfile1
# chmod 0600 /swapfile1
# mkswap /swapfile1
# swapon /swapfile1
# vi /etc/fstab
Add the following at the last
/swapfile1 none swap sw 0 0
Save the file
$ free -m


Now try to start mysqld service.

Saturday, March 31, 2018

No valid instance types found

When I try to launch an instance from a private AMI, I am getting the error

"No valid instance types found" 

The image was created from a snapshot.

Solution : 

Virtualization type by default is Paravirtual, you need to change that type to Hardware-assisted virtualization while creating image from EBS. This will help to fix that error.
 

authorization failed in subversion

Error : authorization failed in subversion, when checkout the repo.

Solution:
Login to svn server
Open svnserve.conf
Uncomment the line "authz-db=authz"

Now check, it should work.

Monday, February 26, 2018

Error: Package: php-gd-5.4.45-13.el6.remi.x86_64 (remi)

Error : Package: php-gd-5.4.45-13.el6.remi.x86_64 (remi)

The error happen after I upgraded PHP from 5.3 to 5.6. Then when i tried to install php-gd I am seeing this error.

Solution:

yum --enablerepo=remi,remi-php56 install php-gd

The above end up in another error:

Error: Package: gd-last-2.2.5-1.el6.remi.x86_64 (remi)
Request: libwebp.so.5 () (64bit)



Solution:

It seems that you are requesting libwebp, so just install libwebp from epel

If you have not installed epel yet,


yum install epel-release -y
yum update

Install libwebp


yum install libwebp --enablerepo=epel -y

Install php-gd


yum install php-gd  --enableablepo=remi 

It will works!


 

How to Upgrade PHP 5.3 to PHP 5.6 on CentOS

Here we are upgrading PHP 5.3 to PHP 5.6 on CentOS.

 1. Confirm the current version: php -v

PHP 5.3.3 (cli) (built: Jul 9 2015 17:39:00)
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies


 2. Install the Remi and EPEL RPM repositories

If you haven’t already done so, install the Remi and EPEL repositories

wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm && rpm -Uvh epel-release-latest-6.noarch.rpm

wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm && rpm -Uvh remi-release-6*.rpm

3. Enable the REMI repository globally:


vi /etc/yum.repos.d/remi.repo

Under the section that looks like [remi] and [remi-php56] change the following

enabled=0 to enabled=1

4. Upgrade PHP 5.3 to PHP 5.6

yum -y upgrade php*

let’s verify that you have PHP 5.6 installed:

 php -v

PHP 5.6.14 (cli) (built: Sep 30 2015 14:07:43)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies

Thursday, February 22, 2018

Error when starting Docker from Jenkins

Error:

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.27/images/json: dial unix /var/run/docker.sock: connect: permission denied

I am running jenkins as a user 'myuser'

Solution :

usermod -a -G root myuser
service jenkins restart

 

Tuesday, February 20, 2018

Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

Docker Error : Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

[root@sv23 ~]# docker build -t my-apache2 .
Sending build context to Docker daemon 64.16 MB
Step 1/1 : FROM httpd:2.4
Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
[root@sv23 ~]# 


When I build a dockerfile, I am getting the above error.

How we can fix this?

1. Login to host.
2. mkdir -p /etc/systemd/system/docker.service.d
3. /etc/systemd/system/docker.service.d/http-proxy.conf
4. Add the following line
[Service]
Environment="HTTP_PROXY=http://proxy.example.com:80/"
[Service]
Environment="HTTPS_PROXY=https://proxy.example.com:443/"
5. systemctl daemon-reload
6. systemctl restart docker
7. systemctl show --property=Environment docker

Try the build now, it will works!

[root@sv23 ~]#  docker build -t my-apache2 .
Sending build context to Docker daemon 64.16 MB
Step 1/1 : FROM httpd:2.4
2.4: Pulling from library/httpd
4176fe04cefe: Pull complete
d6c01cf91b98: Pull complete
b7066921647a: Pull complete
643378aaba88: Pull complete
3c51f6dc6a3b: Pull complete
4f25e420c4cc: Pull complete
ccdbe37da15c: Pull complete
Digest: sha256:6e61d60e4142ea44e8e69b22f1e739d89e1dc8a2764182d7eecc83a5bb31181e
Status: Downloaded newer image for httpd:2.4
 ---> 01154c38b473
Successfully built 01154c38b473
[root@sv23 ~]#


 

Tuesday, January 2, 2018

Starting frontend GLOBAL: cannot bind UNIX socket [/var/run/haproxy/admin.sock]

Haproxy Error: 
 [WARNING] 000/231555 (73783) : Can't open server state file '/var/state/haproxy/global': No such file or directory
Starting frontend GLOBAL: cannot bind UNIX socket [/var/run/haproxy/admin.sock]

How to fix this:

Login to the server
mkdir -p  /var/run/haproxy/
Start haproxy service

It works.  

Haproxy needs to write to /var/run/haproxy/admin.sock but it wont create the directory for you. Create the directory /var/run/haproxy