by dicksonkho on software road

Skeptical To Use Plugin In Future

Migrating my blog from a hosting to another, following some online tutorial steps and manage to port it to new hosting.

BUT… Oooops… page doesn’t display as expected ;-(

Took me sometime to figure out it actually caused by my favourite Syntax Highlighter Plugin.

Now all my posts with this plugin no longer works as it used to be (leave me a comment to share your way of solving this if you stumble upon this post, and know how to… thanks).

Nevertheless, appreciate the great work from creator (aramk) of this plugin.

 
by dicksonkho on software road

Continuous Deployment to Kubernetes with Dynamic Jenkins Slave and Docker

Fasten your safety belt, and departing …..

1. Create a pipeline job.
2. Setup your SCM and build from Jenkinsfile (sample provided below).

////////////////////
//Sample Jenkinsfile 
////////////////////

def AGENT_LABEL = "slave-${UUID.randomUUID().toString()}"
def K8S_DEPLOYMENT_FILE = "k8s-service.yml"

// This is where the dynamic slave magic happen, 
// it launch the agent using randomly generated number as k8s pod 
pipeline {
    agent {
        kubernetes {
            label "${AGENT_LABEL}"
            defaultContainer 'jenkins-jnlp-slave-docker'
            yamlFile 'k8s-jnlp-slave.yml'
        }
    }

    stages {
        stage('Build artifacts') {
            steps {
                // maven3 need to pre-configure in Jenkins
                // mavenSettingsConfig is optional when you have customize settings.xml file using Config File Management plugin
                withMaven(maven: 'maven3', mavenSettingsConfig: 'maven-settings') {                    
                    sh "mvn clean install -Dmaven.test.skip=true"
                }
            }
        }

        stage('Build Docker Image') {
            steps {
                script {
                    withEnv(["WORKSPACE=${pwd()}"]) {                                            
                        //docker-credential is credential set in Jenkins to authenticate against your docker registry
                        docker.withRegistry("YOUR_NEXUS_REGISTRY", 'docker-credential') {
                            def image = docker.build("YOUR_IMAGE_NAME", "PROJECT_DIR/Dockerfile .")   
                            image.push()                            
                        }
                    }
                }
            }
        }       

        stage('Run kubernetes deployment') {
            steps {                
                // kubeconfig is your kube config file which need to pre-configure in Jenkins as secret file
                // check step 5 below
                withKubeConfig([credentialsId: 'kubeconfig']) {                    
                    sh "kubectl create -f ${K8S_DEPLOYMENT_FILE}"                    
                }
            }
        }
    }
}
##########################
#Sample k8s-jnlp-slave.yml
##########################

apiVersion: v1
kind: Pod
metadata:
  labels:
    name: jenkins-jnlp-slave-docker
spec:
  containers:
  - name: jenkins-jnlp-slave-docker
    image: d1ck50n/jenkins-jnlp-slave-docker:latest
    command: ['cat']
    tty: true
    imagePullPolicy: Always    
    volumeMounts:
    - name: dockersock
      mountPath: "/var/run/docker.sock"
  imagePullSecrets:
  - name: tntcred
  volumes:
  - name: dockersock
    hostPath:
      path: /var/run/docker.sock

Jenkins slave is docker, and we will use it to build our docker images, which means we will need to have a docker engine inside Jenkins slave docker (a.k.a docker in docker). Since I’m not able to make use of ready made docker image available @ docker hub as my jenkins slave (I need jdk, maven, kubectl… too as my jenkins slave) , so I created my own based on jenkins/jnlp-slave image.
To achieve DoD, we map the docker.sock from the host to our container (‘dockersock’ in k8s-jnlp-slave.yml).

3. Install plugins as below:
4.Create jenkins kubernetes plugin by adding new entry.

5. Fill in configuration according to your kubernetes cluster or use your kube config file by adding credential as highlighted below:

6. Regardless you configure manually or using your customize kube config file, you need to test the connection.
7. Build your job and you should see a new node created and start building.
Source available @ github.

Credits to authors and websites below:

 
by dicksonkho on software road

Run Jenkins JLNP Slave In Kubernetes

  2.  
  4.  
  5. Take note on text highlighted in red as they are parameters use to spawn container in kubernetes later.
  6. Then go to your terminal and run your kubectl command as below (assume you have exported your kubeconfig).
    Note: Replace all environment variables we captured just now as images above.

    kubectl run --image=jenkins/jnlp-slave jnlp-slave \
--env="JENKINS_URL=https://YOUR_JENKINS_URL" \
--env="JENKINS_SECRET=005097a22eadbf30b205ac9ecb4c18a1721c60fb2e182fd6104e03670f821184" \
--env="JENKINS_AGENT_NAME=k8s-slave-1" \
--env="JENKINS_AGENT_WORKDIR=/home/jenkins"
  7. Refresh your slave node page and you will see it’s now up and running.

Example demonstrate above use Docker container from https://hub.docker.com/r/jenkins/jnlp-slave/

 
by dicksonkho on software road

Backup & Restore Keycloak Easy Way

Finding solution this morning and figure out this can be pretty handy.

  1. Run command below: 
    ./bin/standalone.sh \
-Dkeycloak.migration.action=export \
-Dkeycloak.migration.provider=singleFile \
-Dkeycloak.migration.file=keycloak-backup.json

    In my case, i override few parameter as my keycloak is running.

    ./bin/standalone.sh -P ./standalone/configuration/standalone.properties \
-Djboss.http.port=8090 \
-Djboss.https.port=8098 \
-Djboss.management.http.port=8990 \
-Djboss.management.https.port=8993 \
-Dkeycloak.migration.action=export \
-Dkeycloak.migration.provider=singleFile \
-Dkeycloak.migration.file=keycloak-backup.json

     

  2. Once you get the backup file, you can selectively import from your admin console.

source from: https://www.keycloak.org/docs/latest/server_admin/index.html#_export_import

 
by dicksonkho on software road

Deploy Keycloak With Custom Context Path in Kubernetes With Ingress

New to this and spend a day to figured out, perhaps this will help some of you.

1. Prepare your k8s template yaml.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /<CUSTOM_CONTEXT_PATH>/auth/
  name: keycloak-ingress
spec:
  rules:
  - host: myhost.com
    http:
      paths:
      - path: /<CUSTOM_CONTEXT_PATH>/auth/
        backend:
          serviceName: keycloak-service
          servicePort: 9000
  tls:
  - hosts:
    - myhost.com
    secretName: keycl-secret

2. Update web-context in <KEYCLOAK_HOME>/standalone/configuration/standalone.xml, standalone-ha.xml, or domain.xml depending on your operating mode.

<subsystem xmlns="urn:jboss:domain:keycloak-server:1.1">
    ...
    <web-context><CUSTOM_CONTEXT_PATH>/auth/</web-context>
       <providers>
         <provider>classpath:${jboss.home.dir}/providers/*</provider>
       </providers>
    ...
</subsystem

3. Add proxy-address-forwarding under http-listerner in <KEYCLOAK_HOME>/standalone/configuration/standalone.xml, standalone-ha.xml, or domain.xml depending on your operating mode.

<subsystem xmlns="urn:jboss:domain:undertow:6.0">
    ...
    <http-listener name="default" socket-binding="http"
        proxy-address-forwarding="true"/>
    ...
</subsystem>

4. Update index.html in <KEYCLOAK_HOME>welcome-content/index.html

<head>
    <meta http-equiv="refresh" content="0; url=/<CUSTOM_CONTEXT_PATH>/auth/" />
    <meta name="robots" content="noindex, nofollow">
    <script type="text/javascript">
        window.location.href = "/<CUSTOM_CONTEXT_PATH>/auth/"
    </script>
</head>
<body>
    If you are not redirected automatically, follow this <a href='<CUSTOM_CONTEXT_PATH>/auth'>link</a>.
</body>
</html>

source: https://www.keycloak.org/docs/latest/server_installation/index.html#_setting-up-a-load-balancer-or-proxy

 
by dicksonkho on software road

Keycloak Invalid parameter: redirect_uri

In case you also facing the same issue, may try 2 approaches as below:

1. If you are using Ingress in Kubernetes, you may need to add proxy-address-forwarding as below in standalone.xml, standalone-ha.xml, or domain.xml depending on your operating mode.

2. Set ‘*” for Valid Redirect URL (this is temporary hack, still figuring out why)

Credit to
https://www.keycloak.org/docs/latest/server_installation/index.html#_setting-up-a-load-balancer-or-proxy
https://stackoverflow.com/questions/45352880/keycloak-invalid-parameter-redirect-uri?rq=1

 
by dicksonkho on simple talk

Simple Yet Descriptive

Youtube pitch deck from Alexander Jarvis

AirBnB Pitch Deck from Malcolm Lewis
 
by dicksonkho on simple talk

Miniature

  Photographer: Lay

 
by dicksonkho on software road

AWS Glacier CLI Command

Archived some of my files in Glacier for cold storage, jot down some note which I think useful.

To Download

aws glacier initiate-job --account-id <ACCOUNT_ID> --vault-name <VAULT_NAME> \
--job-parameters '{"Type": "inventory-retrieval"}'

Response as below, “InProgress” indicate it’s currently retrieving your vault information.

{
    "JobList": [
        {
            "InventoryRetrievalParameters": {
                "Format": "JSON"
            },
            "VaultARN": "arn:aws:glacier:us-east-1:<ACCOUNT_ID>:vaults/<VAULT_NAME>",
            "Completed": false,
            "JobId": "1IDvzqYMBTlQ22HZGpr3KmOfeyHV2BIAq3LipToKr3MiN86bTtjb4UiHCVV5Pi4q8fxNhl",
            "Action": "InventoryRetrieval",
            "CreationDate": "2018-09-29T22:19:34.973Z",
            "StatusCode": "InProgress"
        }
    ]
}

OK, tried few hours later and it showed “Succeeded”, go on…

{
    "JobList": [
        {
            "CompletionDate": "2018-09-30T02:07:27.572Z",
            "VaultARN": "arn:aws:glacier:us-east-1:<ACCOUNT_ID>:vaults/<VAULT_NAME>",
            "InventoryRetrievalParameters": {
                "Format": "JSON"
            },
            "Completed": true,
            "InventorySizeInBytes": 4783,
            "JobId": "1IDvzqYMBTlQ22HZGpr3KmOfeyHV2BIAq3Lip",
            "Action": "InventoryRetrieval",
            "CreationDate": "2018-09-29T22:19:34.973Z",
            "StatusMessage": "Succeeded",
            "StatusCode": "Succeeded"
        }
    ]
}

Get archive list with command below which will lead to output written in archive.json.

aws glacier get-job-output --account-id <ACCOUNT_ID> --vault-name <VAULT_NAME> \
--job-id 1IDvzqYMBTlQ22HZGpr3KmOfeyHV2BIAq3Lip archive.json
{
    "VaultARN": "arn:aws:glacier:us-east-1:<ACCOUNT_ID>:vaults/<VAULT_NAME>",
    "InventoryDate": "2018-08-26T21:40:40Z",
    "ArchiveList": [{
            "ArchiveId": "y3QIkLkxRIXLiL1j_lII1yvW13jscAk-wEuifLkoX_SJVCLF6T",
            "ArchiveDescription": "{\"Path\":\"CBB+20180822T070618Z\"}",
            "CreationDate": "2018-08-22T07:07:13Z",
            "Size": 5981,
            "SHA256TreeHash": "072603eaf03ba5ae4eb10083be3285a3ef577f4c"
        },
        {
            "ArchiveId": "IevypBp2RazhieImIi6-y3QIkLkxRIXLiL1j_lII1yvW13jscAk",
            "ArchiveDescription": "{\"Path\":\",\"UTCDateModified\":\"20180822T070623Z\"}",
            "CreationDate": "2018-08-22T07:07:14Z",
            "Size": 1663,
            "SHA256TreeHash": "cede08ebde7ef2fdf36f4610fc4620df56bc64"
        }
    ]
}

Then create a json file with sample content below indicate which archive to download (I use download.json in this case)

{
    "Type": "archive-retrieval",
    "ArchiveId": "dXVi00Z5j1og1REd-zEN1vmP8VB4delaX_mDdGr5IxA",
    "Description": "Test retrieval"
}

Then execute command below:

aws glacier initiate-job  --account-id <ACCOUNT_ID> --vault-name <VAULT_NAME> \
--job-parameters file://download.json

You get response as:

{
    "location": "/<ACCOUNT_ID>/vaults/<VAULT_NAME>/jobs/NiejXgaVsBVWXmNAm6A_ZYhJpgxSiZ8v2aKmGFscLMhXmUg1V",
    "jobId": "NiejXgaVsBVWXmNAm6A_ZYhJpgxSiZ8v2aKmGFscLMhXmUg1V"
}

Check the status job status:

aws glacier list-jobs --account-id <ACCOUNT_ID> --vault-name <VAULT_NAME>
{
    "JobList": [{
            "VaultARN": "arn:aws:glacier:us-east-1:<ACCOUNT_ID>:vaults/<VAULT_NAME>",
            "RetrievalByteRange": "0-6479072",
            "Tier": "Standard",
            "SHA256TreeHash": "a125bef84b6a47bf0e291c6f2f87cfb2b7ffc53955ba88fe164fd3",
            "Completed": false,
            "JobId": "Gx9PNiejXgaV7A6yDre1UZVq4WgBlN3wRuzAm6A_ZYhJpgxSiZ8v2aKmGFscLMhXmUg1V",
            "ArchiveId": "dXVi00Z5MI-P-ItzAo06hc9yE8Ca9nGSzyU5gIj1og1REd-zEN1vmP8VB4delaX_mDdGr5IxA",
            "JobDescription": "Test retrieval",
            "ArchiveSizeInBytes": 6479073,
            "Action": "ArchiveRetrieval",
            "ArchiveSHA256TreeHash": "a125bef84b6a47bf0e291c6f2f87cfb2b7ffc53955ba88fe164fd3",
            "CreationDate": "2018-09-30T12:06:03.400Z",
            "StatusCode": "InProgress"
        }
    ]
}

Come back few hours/minutes, check the status and it’s now turn to “Succeeded”

{
    "JobList": [
        {
            "CompletionDate": "2018-09-30T15:51:05.858Z",
            "VaultARN": "arn:aws:glacier:us-east-1:<ACCOUNT_ID>:vaults/<VAULT_NAME>",
            "RetrievalByteRange": "0-6479072",
            "Tier": "Standard",
            "SHA256TreeHash": "a125bef84b6a47bf0e291c6f2f87cfb2b7ffc53955ba88fe164fd3",
            "Completed": true,
            "JobId": "Gx9PNiejXgaV7A6yDre1UZVq4WgBlN3wRuzAm6A_ZYhJpgxSiZ8v2aKmGFscLMhXmUg1V",
            "ArchiveId": "dXVi00Z5MI-P-ItzAo06hc9yE8Ca9nGSzyU5gIj1og1REd-zEN1vmP8VB4delaX_mDdGr5IxA",
            "JobDescription": "Test retrieval",
            "ArchiveSizeInBytes": 6479073,
            "Action": "ArchiveRetrieval",
            "ArchiveSHA256TreeHash": "a125bef84b6a47bf0e291c6f2f87cfb2b7ffc53955ba88fe164fd3",
            "CreationDate": "2018-09-30T12:06:03.400Z",
            "StatusMessage": "Succeeded",
            "StatusCode": "Succeeded"
        }
    ]
}

Finally you can download your file with command below:

aws glacier get-job-output \
  --account-id <ACCOUNT_ID> \
  --vault-name <VAULT_NAME> \
  --job-id "Gx9PNiejXgaV7A6yDre1UZVq4WgBlN3wRuzAm6A_ZYhJpgxSiZ8v2aKmGFscLMhXmUg1V" \
  movie.zip

To Upload

aws glacier upload-archive --account-id - --vault-name <VAULT_NAME> --archive-description "YOUR_NOTE" --body <FILE_NAME>

To Delete

aws glacier delete-archive --account-id <ACCOUNT_ID> --vault-name <VAULT_NAME> --archive-id "<ARCHIVE_ID>"

 

Credit to: softwaredevelopmentstuff.com & madboa.com

 
by dicksonkho on software road

Docker: exec user process caused “no such file or directory”

Have no idea why Docker showing this (un-friendly) error message.
In fact, it caused by the bash script file created in Windows having different line ending and cannot run properly in Linux container.

standard_init_linux.go:190: exec user process caused "no such file or directory"

Here is the fix using Notepad++.
First show the linefeed symbol.

You will see this.

Convert the linefeed from Windows to Unix.

You will get this, and save it 🙂