Continuous Deployment to Kubernetes with Dynamic Jenkins Slave and Docker

Fasten your safety belt, and departing …..

1. Create a pipeline job.
2. Setup your SCM and build from Jenkinsfile (sample provided below).

////////////////////
//Sample Jenkinsfile 
////////////////////

def AGENT_LABEL = "slave-${UUID.randomUUID().toString()}"
def K8S_DEPLOYMENT_FILE = "k8s-service.yml"

// This is where the dynamic slave magic happen, 
// it launch the agent using randomly generated number as k8s pod 
pipeline {
    agent {
        kubernetes {
            label "${AGENT_LABEL}"
            defaultContainer 'jenkins-jnlp-slave-docker'
            yamlFile 'k8s-jnlp-slave.yml'
        }
    }

    stages {
        stage('Build artifacts') {
            steps {
                // maven3 need to pre-configure in Jenkins
                // mavenSettingsConfig is optional when you have customize settings.xml file using Config File Management plugin
                withMaven(maven: 'maven3', mavenSettingsConfig: 'maven-settings') {                    
                    sh "mvn clean install -Dmaven.test.skip=true"
                }
            }
        }

        stage('Build Docker Image') {
            steps {
                script {
                    withEnv(["WORKSPACE=${pwd()}"]) {                                            
                        //docker-credential is credential set in Jenkins to authenticate against your docker registry
                        docker.withRegistry("YOUR_NEXUS_REGISTRY", 'docker-credential') {
                            def image = docker.build("YOUR_IMAGE_NAME", "PROJECT_DIR/Dockerfile .")   
                            image.push()                            
                        }
                    }
                }
            }
        }       

        stage('Run kubernetes deployment') {
            steps {                
                // kubeconfig is your kube config file which need to pre-configure in Jenkins as secret file
                // check step 5 below
                withKubeConfig([credentialsId: 'kubeconfig']) {                    
                    sh "kubectl create -f ${K8S_DEPLOYMENT_FILE}"                    
                }
            }
        }
    }
}
##########################
#Sample k8s-jnlp-slave.yml
##########################

apiVersion: v1
kind: Pod
metadata:
  labels:
    name: jenkins-jnlp-slave-docker
spec:
  containers:
  - name: jenkins-jnlp-slave-docker
    image: d1ck50n/jenkins-jnlp-slave-docker:latest
    command: ['cat']
    tty: true
    imagePullPolicy: Always    
    volumeMounts:
    - name: dockersock
      mountPath: "/var/run/docker.sock"
  imagePullSecrets:
  - name: tntcred
  volumes:
  - name: dockersock
    hostPath:
      path: /var/run/docker.sock

Jenkins slave is docker, and we will use it to build our docker images, which means we will need to have a docker engine inside Jenkins slave docker (a.k.a docker in docker). Since I’m not able to make use of ready made docker image available @ docker hub as my jenkins slave (I need jdk, maven, kubectl… too as my jenkins slave) , so I created my own based on jenkins/jnlp-slave image.
To achieve DoD, we map the docker.sock from the host to our container (‘dockersock’ in k8s-jnlp-slave.yml).

3. Install plugins as below:
4.Create jenkins kubernetes plugin by adding new entry.

5. Fill in configuration according to your kubernetes cluster or use your kube config file by adding credential as highlighted below:

6. Regardless you configure manually or using your customize kube config file, you need to test the connection.
7. Build your job and you should see a new node created and start building.
Source available @ github.

Credits to authors and websites below:

 

Run Jenkins JLNP Slave In Kubernetes

  1.  
  2.  
  3. Take note on text highlighted in red as they are parameters use to spawn container in kubernetes later.
  4. Then go to your terminal and run your kubectl command as below (assume you have exported your kubeconfig).
    Note: Replace all environment variables we captured just now as images above.

    kubectl run --image=jenkins/jnlp-slave jnlp-slave \
    --env="JENKINS_URL=https://YOUR_JENKINS_URL" \
    --env="JENKINS_SECRET=005097a22eadbf30b205ac9ecb4c18a1721c60fb2e182fd6104e03670f821184" \
    --env="JENKINS_AGENT_NAME=k8s-slave-1" \
    --env="JENKINS_AGENT_WORKDIR=/home/jenkins"
  5. Refresh your slave node page and you will see it’s now up and running.

Example demonstrate above use Docker container from https://hub.docker.com/r/jenkins/jnlp-slave/

 

Deploy Keycloak With Custom Context Path in Kubernetes With Ingress

New to this and spend a day to figured out, perhaps this will help some of you.

1. Prepare your k8s template yaml.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /<CUSTOM_CONTEXT_PATH>/auth/
  name: keycloak-ingress
spec:
  rules:
  - host: myhost.com
    http:
      paths:
      - path: /<CUSTOM_CONTEXT_PATH>/auth/
        backend:
          serviceName: keycloak-service
          servicePort: 9000
  tls:
  - hosts:
    - myhost.com
    secretName: keycl-secret

2. Update web-context in <KEYCLOAK_HOME>/standalone/configuration/standalone.xml, standalone-ha.xml, or domain.xml depending on your operating mode.

<subsystem xmlns="urn:jboss:domain:keycloak-server:1.1">
    ...
    <web-context><CUSTOM_CONTEXT_PATH>/auth/</web-context>
       <providers>
         <provider>classpath:${jboss.home.dir}/providers/*</provider>
       </providers>
    ...
</subsystem

3. Add proxy-address-forwarding under http-listerner in <KEYCLOAK_HOME>/standalone/configuration/standalone.xml, standalone-ha.xml, or domain.xml depending on your operating mode.

<subsystem xmlns="urn:jboss:domain:undertow:6.0">
    ...
    <http-listener name="default" socket-binding="http"
        proxy-address-forwarding="true"/>
    ...
</subsystem>

4. Update index.html in <KEYCLOAK_HOME>welcome-content/index.html

<head>
    <meta http-equiv="refresh" content="0; url=/<CUSTOM_CONTEXT_PATH>/auth/" />
    <meta name="robots" content="noindex, nofollow">
    <script type="text/javascript">
        window.location.href = "/<CUSTOM_CONTEXT_PATH>/auth/"
    </script>
</head>
<body>
    If you are not redirected automatically, follow this <a href='<CUSTOM_CONTEXT_PATH>/auth'>link</a>.
</body>
</html>

source: https://www.keycloak.org/docs/latest/server_installation/index.html#_setting-up-a-load-balancer-or-proxy

 

Keycloak Invalid parameter: redirect_uri

In case you also facing the same issue, may try 2 approaches as below:

1. If you are using Ingress in Kubernetes, you may need to add proxy-address-forwarding as below in standalone.xml, standalone-ha.xml, or domain.xml depending on your operating mode.

2. Set ‘*” for Valid Redirect URL (this is temporary hack, still figuring out why)

Credit to
https://www.keycloak.org/docs/latest/server_installation/index.html#_setting-up-a-load-balancer-or-proxy
https://stackoverflow.com/questions/45352880/keycloak-invalid-parameter-redirect-uri?rq=1