Go Lang, an AWS S3 SDK quick and simple tutorial

And here we go again. We will continue this tour of AWS SDKs. This is the turn of Go, the Go programming language.

Now, first let’s consider a few links in which you will find some useful information about the language and the SDK. Take note of the followings: homepage for the Go language , here you will find how to install it. Then we have the reference API for the AWS sdk.

The Go programming language

Here is how to check if the installation went all right:

C:\>go version

The next thing you should do is to install an IDE for our code to be easy to write, like having syntax highlighting and so on: I advice you to use Visual Studio Code. The only thing you’re gonna have to do is to create a new Go file and it will automatically ask you for the installation of the relative plugin.

The next thing you should do is to install an IDE for our code to be easy to write, like having syntax highlighting and so on: I advice you to use Visual Studio Code. The only thing you’re gonna have to do is to create a new Go file and it will automatically ask you for the installation of the relative plugin.

Again, let’s install our SDK and its dependecies through the command line of Go itself, go to your shell and type the following:

C:\>go get -u github.com/aws/aws-sdk-go/...

Now it’s time to code. Our first file is the following, it will be named create_buckets.go:

package main

import (
	"bufio"
	"fmt"
	"os"
	"strings"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
)

func main() {

	reader := bufio.NewReader(os.Stdin)
	fmt.Print("Enter a name for yor Bucket: ")
	bucket, _ := reader.ReadString('\n')

	for _, delimiter := range []string{"\n", "\r"} {
		bucket = strings.TrimSuffix(bucket, delimiter)
	}

	sess, err := session.NewSession(&aws.Config{
		Region: aws.String("eu-central-1")},
	)

	svc := s3.New(sess)

	_, err = svc.CreateBucket(&s3.CreateBucketInput{
		Bucket: aws.String(bucket),
	})
	if err != nil {
		exitErrorf("Unable to create bucket %q, %v", bucket, err)
	}

	fmt.Printf("Waiting for bucket %q to be created...\n", bucket)

	err = svc.WaitUntilBucketExists(&s3.HeadBucketInput{
		Bucket: aws.String(bucket),
	})
	if err != nil {
		exitErrorf("Error occurred while waiting for bucket to be created, %v", bucket)
	}

	fmt.Printf("Bucket %q successfully created\n", bucket)
}

func exitErrorf(msg string, args ...interface{}) {
	fmt.Fprintf(os.Stderr, msg+"\n", args...)
	os.Exit(1)
}

It is importing all the AWS S3 SDK necessary packages and some other useful stuff for dealing with the shell’s IO.

Please note the underscore placeholder variable: since functions in Go have the possibility of returning more results, this placeholder simply discard one of them. At this point we’re gonna list our Buckets, let’s create a list_bucktes.go file:

package main

import (
	"fmt"
	"os"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
)

func main() {

	sess, err := session.NewSession(&aws.Config{
		Region: aws.String("eu-central-1")},
	)

	svc := s3.New(sess)

	result, err := svc.ListBuckets(nil)
	if err != nil {
		exitErrorf("Unable to list buckets, %v", err)
	}

	if len(result.Buckets) > 0 {
		fmt.Println("Here are your Buckets:")
		showBuckets(svc, result)
	} else {
		fmt.Println("You have no Buckets yet.")
	}
}

func exitErrorf(msg string, args ...interface{}) {
	fmt.Fprintf(os.Stderr, msg+"\n", args...)
	os.Exit(1)
}

func showBuckets(svc *s3.S3, result *s3.ListBucketsOutput) {
	for _, b := range result.Buckets {

		fmt.Printf("* %s created on %s\n",
			aws.StringValue(b.Name), aws.TimeValue(b.CreationDate))

		resp, err := svc.ListObjects(&s3.ListObjectsInput{Bucket: aws.String(*b.Name)})
		if err != nil {
			exitErrorf("Unable to list items in bucket %q, %v", *b.Name, err)
		}

		for _, item := range resp.Contents {
			fmt.Println("Name:         ", *item.Key)
			fmt.Println("Last modified:", *item.LastModified)
			fmt.Println("Size:         ", *item.Size)
			fmt.Println("Storage class:", *item.StorageClass)
			fmt.Println("")
		}

		fmt.Println("Found", len(resp.Contents), "items in bucket", *b.Name)
		fmt.Println("")

	}
}

Our Bucket is now empty, let’s create the possibility of putting some files in it, through a put_files.go script:

package main

import (
	"bufio"
	"fmt"
	"os"
	"strings"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3/s3manager"
)

func main() {

	reader := bufio.NewReader(os.Stdin)
	fmt.Print("Enter the name of your Bucket: ")
	bucket, _ := reader.ReadString('\n')

	fmt.Print("Enter the name of the file to upload: ")
	filename, _ := reader.ReadString('\n')

	for _, delimiter := range []string{"\n", "\r"} {
		bucket = strings.TrimSuffix(bucket, delimiter)
		filename = strings.TrimSuffix(filename, delimiter)
	}

	file, err := os.Open(filename)
	if err != nil {
		exitErrorf("Unable to open file %q, %v", err)
	}

	defer file.Close()

	sess, err := session.NewSession(&aws.Config{
		Region: aws.String("eu-central-1")},
	)

	uploader := s3manager.NewUploader(sess)

	_, err = uploader.Upload(&s3manager.UploadInput{
		Bucket: aws.String(bucket),
		Key:    aws.String(filename),
		Body:   file,
	})
	if err != nil {
		exitErrorf("Unable to upload %q to %q, %v", filename, bucket, err)
	}

	fmt.Printf("Successfully uploaded %q to %q\n", filename, bucket)
}

func exitErrorf(msg string, args ...interface{}) {
	fmt.Fprintf(os.Stderr, msg+"\n", args...)
	os.Exit(1)
}

Why don’t we delete some files we put inside our Bucket? We’re ready to create our delete_files.go:

package main

import (
	"bufio"
	"fmt"
	"os"
	"strings"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
)

func main() {

	reader := bufio.NewReader(os.Stdin)
	fmt.Print("Enter the name of your Bucket: ")
	bucket, _ := reader.ReadString('\n')

	fmt.Print("Enter the name of the file to delete: ")
	filename, _ := reader.ReadString('\n')

	for _, delimiter := range []string{"\n", "\r"} {
		bucket = strings.TrimSuffix(bucket, delimiter)
		filename = strings.TrimSuffix(filename, delimiter)
	}

	sess, err := session.NewSession(&aws.Config{
		Region: aws.String("eu-central-1")},
	)

	svc := s3.New(sess)

	_, err = svc.DeleteObject(&s3.DeleteObjectInput{Bucket: aws.String(bucket), Key: aws.String(filename)})
	if err != nil {
		exitErrorf("Unable to delete object %q from bucket %q, %v", filename, bucket, err)
	}

	err = svc.WaitUntilObjectNotExists(&s3.HeadObjectInput{
		Bucket: aws.String(bucket),
		Key:    aws.String(filename),
	})
	if err != nil {
		exitErrorf("Error occurred while waiting for object %q to be deleted, %v", filename)
	}

	fmt.Printf("Object %q successfully deleted\n", filename)
}

func exitErrorf(msg string, args ...interface{}) {
	fmt.Fprintf(os.Stderr, msg+"\n", args...)
	os.Exit(1)
}

The last thing to remember is to delete our Bucket, the script will delete our files first and then the Bucket itself, here is delete_buckets.go:

package main

import (
	"bufio"
	"fmt"
	"os"
	"strings"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
	"github.com/aws/aws-sdk-go/service/s3/s3manager"
)

func main() {

	reader := bufio.NewReader(os.Stdin)
	fmt.Print("Enter the name of the Bucket to delete: ")
	bucket, _ := reader.ReadString('\n')

	for _, delimiter := range []string{"\n", "\r"} {
		bucket = strings.TrimSuffix(bucket, delimiter)
	}

	sess, err := session.NewSession(&aws.Config{
		Region: aws.String("eu-central-1")},
	)

	svc := s3.New(sess)

	deleteAllFiles(svc, bucket)

	_, err = svc.DeleteBucket(&s3.DeleteBucketInput{
		Bucket: aws.String(bucket),
	})
	if err != nil {
		exitErrorf("Unable to delete bucket %q, %v", bucket, err)
	}

	fmt.Printf("Waiting for bucket %q to be deleted...\n", bucket)

	err = svc.WaitUntilBucketNotExists(&s3.HeadBucketInput{
		Bucket: aws.String(bucket),
	})
	if err != nil {
		exitErrorf("Error occurred while waiting for bucket to be deleted, %v", bucket)
	}

Make the more experiments as you can with those scripts, create a Bucket, put two or three files in it, list your Bucket and you’ll see all the files in it. Then delete one file and list your Bucket again, it will be missing.

Again, delete your Bucket, even if it’s not empty, the script will delete your files first. This tutorial is made on the Windows OS, but others will be compatible too with some minor changes or no changes at all.

To run the scripts above simply issue a command like the following at your shell:

C:\>go run cretate_buckets.go

This tutorial is just the beginning, the code is simple, no further explanation should be needed, but if you have any questions don’t forget to contact me and I’ll do the best I can to help you.

I’ve chosen Go Lang for this tutorial in that it is an ever more growing and emerging language for the cloud, it has been adopted by many big players of this sector.

For the full code of this tutorial checkout this GitHub repo, in a while I’ll be back with another tutorial, so stay tuned and I’ll see you later.

Did you like this post? Please comment here below and share it on your preferred social networks, thank you!

2 thoughts on “Go Lang, an AWS S3 SDK quick and simple tutorial

  1. Very good post Mirko
    interesting how you can master so many languages
    Mario

Leave a Reply

Give me your opinion, I will be grateful.