by dicksonkho on software road

AWS Glacier CLI Command

Archived some of my files in Glacier for cold storage, jot down some note which I think useful.

To Download

aws glacier initiate-job --account-id <ACCOUNT_ID> --vault-name <VAULT_NAME> \
--job-parameters '{"Type": "inventory-retrieval"}'

Response as below, “InProgress” indicate it’s currently retrieving your vault information.

{
    "JobList": [
        {
            "InventoryRetrievalParameters": {
                "Format": "JSON"
            },
            "VaultARN": "arn:aws:glacier:us-east-1:<ACCOUNT_ID>:vaults/<VAULT_NAME>",
            "Completed": false,
            "JobId": "1IDvzqYMBTlQ22HZGpr3KmOfeyHV2BIAq3LipToKr3MiN86bTtjb4UiHCVV5Pi4q8fxNhl",
            "Action": "InventoryRetrieval",
            "CreationDate": "2018-09-29T22:19:34.973Z",
            "StatusCode": "InProgress"
        }
    ]
}

OK, tried few hours later and it showed “Succeeded”, go on…

{
    "JobList": [
        {
            "CompletionDate": "2018-09-30T02:07:27.572Z",
            "VaultARN": "arn:aws:glacier:us-east-1:<ACCOUNT_ID>:vaults/<VAULT_NAME>",
            "InventoryRetrievalParameters": {
                "Format": "JSON"
            },
            "Completed": true,
            "InventorySizeInBytes": 4783,
            "JobId": "1IDvzqYMBTlQ22HZGpr3KmOfeyHV2BIAq3Lip",
            "Action": "InventoryRetrieval",
            "CreationDate": "2018-09-29T22:19:34.973Z",
            "StatusMessage": "Succeeded",
            "StatusCode": "Succeeded"
        }
    ]
}

Get archive list with command below which will lead to output written in archive.json.

aws glacier get-job-output --account-id <ACCOUNT_ID> --vault-name <VAULT_NAME> \
--job-id 1IDvzqYMBTlQ22HZGpr3KmOfeyHV2BIAq3Lip archive.json
{
    "VaultARN": "arn:aws:glacier:us-east-1:<ACCOUNT_ID>:vaults/<VAULT_NAME>",
    "InventoryDate": "2018-08-26T21:40:40Z",
    "ArchiveList": [{
            "ArchiveId": "y3QIkLkxRIXLiL1j_lII1yvW13jscAk-wEuifLkoX_SJVCLF6T",
            "ArchiveDescription": "{\"Path\":\"CBB+20180822T070618Z\"}",
            "CreationDate": "2018-08-22T07:07:13Z",
            "Size": 5981,
            "SHA256TreeHash": "072603eaf03ba5ae4eb10083be3285a3ef577f4c"
        },
        {
            "ArchiveId": "IevypBp2RazhieImIi6-y3QIkLkxRIXLiL1j_lII1yvW13jscAk",
            "ArchiveDescription": "{\"Path\":\",\"UTCDateModified\":\"20180822T070623Z\"}",
            "CreationDate": "2018-08-22T07:07:14Z",
            "Size": 1663,
            "SHA256TreeHash": "cede08ebde7ef2fdf36f4610fc4620df56bc64"
        }
    ]
}

Then create a json file with sample content below indicate which archive to download (I use download.json in this case)

{
    "Type": "archive-retrieval",
    "ArchiveId": "dXVi00Z5j1og1REd-zEN1vmP8VB4delaX_mDdGr5IxA",
    "Description": "Test retrieval"
}

Then execute command below:

aws glacier initiate-job  --account-id <ACCOUNT_ID> --vault-name <VAULT_NAME> \
--job-parameters file://download.json

You get response as:

{
    "location": "/<ACCOUNT_ID>/vaults/<VAULT_NAME>/jobs/NiejXgaVsBVWXmNAm6A_ZYhJpgxSiZ8v2aKmGFscLMhXmUg1V",
    "jobId": "NiejXgaVsBVWXmNAm6A_ZYhJpgxSiZ8v2aKmGFscLMhXmUg1V"
}

Check the status job status:

aws glacier list-jobs --account-id <ACCOUNT_ID> --vault-name <VAULT_NAME>
{
    "JobList": [{
            "VaultARN": "arn:aws:glacier:us-east-1:<ACCOUNT_ID>:vaults/<VAULT_NAME>",
            "RetrievalByteRange": "0-6479072",
            "Tier": "Standard",
            "SHA256TreeHash": "a125bef84b6a47bf0e291c6f2f87cfb2b7ffc53955ba88fe164fd3",
            "Completed": false,
            "JobId": "Gx9PNiejXgaV7A6yDre1UZVq4WgBlN3wRuzAm6A_ZYhJpgxSiZ8v2aKmGFscLMhXmUg1V",
            "ArchiveId": "dXVi00Z5MI-P-ItzAo06hc9yE8Ca9nGSzyU5gIj1og1REd-zEN1vmP8VB4delaX_mDdGr5IxA",
            "JobDescription": "Test retrieval",
            "ArchiveSizeInBytes": 6479073,
            "Action": "ArchiveRetrieval",
            "ArchiveSHA256TreeHash": "a125bef84b6a47bf0e291c6f2f87cfb2b7ffc53955ba88fe164fd3",
            "CreationDate": "2018-09-30T12:06:03.400Z",
            "StatusCode": "InProgress"
        }
    ]
}

Come back few hours/minutes, check the status and it’s now turn to “Succeeded”

{
    "JobList": [
        {
            "CompletionDate": "2018-09-30T15:51:05.858Z",
            "VaultARN": "arn:aws:glacier:us-east-1:<ACCOUNT_ID>:vaults/<VAULT_NAME>",
            "RetrievalByteRange": "0-6479072",
            "Tier": "Standard",
            "SHA256TreeHash": "a125bef84b6a47bf0e291c6f2f87cfb2b7ffc53955ba88fe164fd3",
            "Completed": true,
            "JobId": "Gx9PNiejXgaV7A6yDre1UZVq4WgBlN3wRuzAm6A_ZYhJpgxSiZ8v2aKmGFscLMhXmUg1V",
            "ArchiveId": "dXVi00Z5MI-P-ItzAo06hc9yE8Ca9nGSzyU5gIj1og1REd-zEN1vmP8VB4delaX_mDdGr5IxA",
            "JobDescription": "Test retrieval",
            "ArchiveSizeInBytes": 6479073,
            "Action": "ArchiveRetrieval",
            "ArchiveSHA256TreeHash": "a125bef84b6a47bf0e291c6f2f87cfb2b7ffc53955ba88fe164fd3",
            "CreationDate": "2018-09-30T12:06:03.400Z",
            "StatusMessage": "Succeeded",
            "StatusCode": "Succeeded"
        }
    ]
}

Finally you can download your file with command below:

aws glacier get-job-output \
  --account-id <ACCOUNT_ID> \
  --vault-name <VAULT_NAME> \
  --job-id "Gx9PNiejXgaV7A6yDre1UZVq4WgBlN3wRuzAm6A_ZYhJpgxSiZ8v2aKmGFscLMhXmUg1V" \
  movie.zip

To Upload

aws glacier upload-archive --account-id - --vault-name <VAULT_NAME> --archive-description "YOUR_NOTE" --body <FILE_NAME>

To Delete

aws glacier delete-archive --account-id <ACCOUNT_ID> --vault-name <VAULT_NAME> --archive-id "<ARCHIVE_ID>"

 

Credit to: softwaredevelopmentstuff.com & madboa.com

 
by dicksonkho on software road

Docker: exec user process caused “no such file or directory”

Have no idea why Docker showing this (un-friendly) error message.
In fact, it caused by the bash script file created in Windows having different line ending and cannot run properly in Linux container.

standard_init_linux.go:190: exec user process caused "no such file or directory"

Here is the fix using Notepad++.
First show the linefeed symbol.

You will see this.

Convert the linefeed from Windows to Unix.

You will get this, and save it 🙂

 
by dicksonkho on software road

Weird Spring Boot Behavior

Encountered a weird behavior when running my Spring Boot + JSF (PrimeFaces) application in Docker container (using java -jar xxx.jar as ENTRYPOINT). It keep complaining “xhtml Not Found in ExternalContext as a Resource” when I access the page, in fact that runs well either using java -jar command or mvn spring:run in host machine.

After some hard time,  manage to figure out why…….. phew….
… it was caused by wrong jar packaging structure, and correct structure should look as below:

Credit to BalusC @ stackoverflow

 
by dicksonkho on simple talk

Extraordinary Merdeka 2018

Nearly didn’t remember when was the last time I celebrate Merdeka (at least hoisting our national flag in front of house). But I feel a little special this year as many Merdeka celebration mtv shared around social media 🙂
~Happy Merdeka Malaysia~

 
by dicksonkho on software road

MSSQL Configuration Tips

Happened to me hence jot down here for future reference, perhaps helpful to you also 🙂

Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host localhost, port 1433 has failed. 
Error: "Connection refused: connect. Verify the connection properties. 
Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. 
Make sure that TCP connections to the port are not blocked by a firewall.

Try to configure the TCP port as below from Sql Server Configuration Manager:

Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user 'xxxx'. 
ClientConnectionId:4ff4b2b5-ffda-47fb-9a03-ca8cfc6cca6b

Try to configure as below from Microsoft SQL Server Management Studio:

 
by dicksonkho on software road

Alfred Alternative For Windows

Have been using Alfred primarily for “any” search in macOS, and found Listary for Windows.

 
by dicksonkho on simple talk

Yet, Another RSS Feed Reader RIP

I use it almost everyday and it shuts down after Google reader and Paper…. Phew…
Whats next ? let’s rock Feedly (finger crossed).

 
by dicksonkho on simple talk

The iPhone ‘iPhone’ could not be restored. An unknown error occurred

To restore iPhone with IOS firmware (IPSW) in recovery mode, but facing error as per title, may try to comment “gs.apple.com” in hosts file as image below:

 

 

 

Resource from: https://support.apple.com/en-my/HT201442

 
by dicksonkho on software road

Java 8 File to String in One Line

public static void main(String[] args) throws IOException {
    String result = new String(Files.readAllBytes(Paths.get("/tmp/result.txt")));
}

 

 
by dicksonkho on software road

Pragmatic Java Application Test

Has been a while, I was thinking to blog this out but “lazy” bugs keep biting me 🙁 so I didn’t manage to do it. Until yesterday, I was looking for some related info but I hardly recall what I did previously. Phew… so decided to jot down here.

Typically software application should at least perform 2 types of automate tests which are unit test and integration test. For me… the distinction between these two are pretty straight forward:

Unit Test Integration Test
Test on smallest unit of application such as method / function Test on combination of few units in application to prove they run nicely with external resource (if there is any)
Cheap to run, fast to run (normally less than a minute) Slower compared with unit test (normally few minutes or more)
NOT depends on external resources such as database, third party services, disk access Normally involve database, disk access, other services
Mock the input of unit test if necessary Use the real service / resource
Maven Plugin Usage
Class name normally suffix with “Test”, for eg: TaxCalculatorTest (Maven Surefire plugin automatically include these wildcard patterns) Class name normally suffix with “IT”, for eg: TaxCalculationServiceIT (Maven Failsafe plugin automatically include these wildcard patterns)
Using Maven Surefire Plugin Using Maven Failsafe Plugin
Run during “test” phase in default Maven lifecycle Run during “integration-test” phase in default Maven lifecycle
Minimal sample Maven configuration:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-surefire-plugin</artifactId>
	<version>${maven.surefire.version}</version>
	<executions>
		<execution>
			<id>default-test</id>
			<goals>
				<goal>test</goal>
			</goals>
		</execution>
	</executions>
</plugin>
 Minimal sample Maven configuration:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-failsafe-plugin</artifactId>
	<version>${maven.failsafe.version}</version>
	<executions>
		<execution>
			<id>default-integration-test</id>
			<goals>
				<goal>integration-test</goal>
			</goals>
		</execution>                
	</executions>
</plugin>
Command to execute:

mvn test
 Command to execute:

mvn integration-test
Report generated in <PROJECT>/target/surefire-reports Report generated in <PROJECT>/target/failsafe-reports

By viewing the status report generated by surefire & failsafe plugin doesn’t create much insight, but we can make use of Jacoco plugin to create code coverage report as below.
Here are minimal sample Maven configuration for Jacoco:

<plugin>
	<groupId>org.jacoco</groupId>
	<artifactId>jacoco-maven-plugin</artifactId>
	<version>${jacoco.version}</version>
	<executions>
		<execution>
			<id>default-prepare-agent</id>
			<goals>
				<goal>prepare-agent</goal>
			</goals>
		</execution>
		<execution>
			<id>default-prepare-agent-integration</id>
			<goals>
				<goal>prepare-agent-integration</goal>
			</goals>
		</execution>
		<execution>
			<id>default-report</id>
			<goals>
				<goal>report</goal>
			</goals>
		</execution>
		<execution>
			<id>default-report-integration</id>
			<goals>
				<goal>report-integration</goal>
			</goals>
		</execution>
	</executions>
</plugin>

Jacoco coverage reports generated at <PROJECT>/target/site/* when we run mvn verify (bind to maven verify phase). To view the coverage reports, simply open <PROJECT>/target/site/jacoco/index.html with browser.

Yay ! we have now cover first part of test (automate unit + integration test), and if you notice, we done all steps manually (run maven command, open report from target folder) just to get the report display.

Moreover, we just able to view the coverage report locally, how if we want to share with other team members or to make it public accessible.
And…….. How can we automate the whole process ?

Da.. Dang !! We have Jenkins (Continuous Integration) + SonarQube (Continuous Code Quality Inspection) to help us 🙂

Setting up Jenkins is pretty straight forward, just follow steps from Jenkins Guided Tour or Installing Jenkins. We should able to see something as below if Jenkins setup successfully and  launch in browser.
Let’s continue with SonarQube setup by following “Get Started in Two Minutes” guide from it’s official site. And if nothing goes wrong, we should able to bring up SonarQube console via browser as below:
Until this stage, you might feel a little tired or sleepy, it’s okie to refresh yourself with a cup of coffee 🙂

To allow Jenkins to work with SonarQube, we might need a little setup as below.

  1. Install SonarQube Scanner for Jenkins.
  2. Generate authentication token from SonarQube and store it somewhere as steps below for subsequent Jenkins configuration use.
  3. Goto Manage Jenkins –> Configure System, There should have SonarQube configuration module available.
    Just configure it by click on Add SonarQube, and fill in necessary information (sample below), then Save it.
  4. After setting up Jenkins – SonarQube, the next step is to add a Post Build Steps (SonarQube Scanner) for your Jenkins job as below:
    Note: Sample analysis parameter added above are very minimal, refer Analysis Parameter or Code Coverage by Unit Tests for Java Project for more parameter available.
  5. Rerun your Jenkins job after configuration, and you will see the analysis report generated in SonarQube as below.
  6. Yay !!!! We can now have beautiful test coverage report generate upon each code commit (of course, we need to configure Jenkins job to trigger build on each code commit to make these “magic” happen).