Testing Tekton to build and push images for my K3S ARM Oracle cluster

Introduction

In this article, we will explore how to deploy and configure tekton to build and push images to your registry to be consumed from your cluster, we will also see how these are deployed in another article. In this one I want to show you how to get the images ready to use, and also a handy solution for a CI system without having to rely on external factors, in my case, I was having issues with docker building cross-architecture images and after setting up tekton everything was faster and simpler, cross-architecture is slow by default but can also not work a 100% as you would expect, by using this approach we can just forget about the architecture and just build where we run thing, it is definitely faster and even some of your nodes will already have the images available meaning less bandwidth consumption
as well in the long run.

The source code and/or documentation of the projects that we will be testing are listed here:

• tr, go ahead and check it out, my new blog runs there: https://tr.techsquad.rocks you can check the manifests used here in the manifests folder.

The source code and/or documentation of the projects that we will be testing are listed here:

• tekton
• tekton-triggers
• kaniko

Installing tekton-pipelines and tekton-triggers

Why do we need tekton-pipelines or tekton-triggers again? pipelines allows you to run multiple tasks in order and pass things around (this is basic to tekton and to any CI/CD system), then we need to do something when we push for example to our git repository, that’s when tekton-triggers gets handy and let us react to changes and trigger a build or some process, interceptors are a part of tekton-triggers and let’s say it gives you flexibility using events.

Then we need to install tkn locally and configure some packages from the hub

In my deployment I used fixed versions which is recommended for any kind of “production” deployment, you can see the readme here.

Let’s get to business

tekton-pipelines

Okay, so we have tekton and friends installed, ready for business, but what now? well, it’s a bit tricky and require a few manifests to get going, so I will try to explain what is happening with each file and why do we need them.

You can see this file in github as well 01-pipeline.yaml, basically we need to define a pipeline which defines the steps and what it will happen, here we are cloning the repository, then building it with kaniko and then pushing it to the docker registry, note that the script is hardcoded there that could be dynamic but not really necessary for my use case.

You can see this file in github as well 02-pipeline-run.yaml, This is basically to run our defined pipeline with specific values, we will use something very similar from the trigger to run automatically when we push commits to our repo, the docker secret is a regular dockercfg secret mounted so we can push to that registry.

With all that we have a basic pipeline but we need to trigger it or run it manually, let’s add the necessary manifests for it to react to changes in our github repository

tekton-triggers

You can see this file in github as well 01-rbac.yaml, let’s give tekton-triggers some permissions

You can see this file on github as well 02-eventlistener.yaml, This is where things get a bit tricky, in theory you don’t need a secret to read your repo if it is public, but it was private when I started testing this, then it was made public, if you are interested in the format of the secret check below this yaml, however this only “listens” to events in our repo and triggers an event using our pipeline, we still need an ingress for the webhook and other configs as we will see in the next steps.

The secret would be something like the one depicted below, replace secretToken with your generated token this will be used for the webhook configuration so save it somewhere safe until it is configured there.

You can see this file on github as well 04-triggerbinding.yaml, When we receive the webhook we can get some information from it, basically we are interested in the repo URL and the commit SHA.

You can see this file in github as well 05-triggertemplate.yaml, This would be the equivalent of the manually run pipelinerun that we have, but this uses the trigger and the template to automatically trigger, hence the similarities.

You can see this file on github as well 06-ingress.yaml, And last but not least the ingress configuration, without this it won’t work because we need to receive a request from github, to configure that just go to settings on the repository, hit webhooks and create a new one with the secret token that you generated and put your URL as https://subdomain.domain/hooks, then mark TLS on, only push and active.

WHEW! that was a lot of work but trust me it’s worth it, now you can build, push and run your images from your cluster, with no external or weird CI/CD system and everything following a GitOps model since everything can be committed and applied from your repository, in my case I’m using ArgoCD and Kustomize to apply everything but that is for another chapter.

We have the event listener ready:

We have the pipeline, notice that it says failed this is because there is an issue with ARM that it is still not solved but everything actually works as expected:

We can see the pipelinerun being triggered, same issue as described before, see the notes for the github issues:

We can also see some of the other resources created for tekton:

You can also see the pods created or logs using either kubectl or tkn:

I hope this is useful for someone and if you are having issues with your CI/CD system give tekton a go, you will love it, in my particular case I was having many issues with ARM and building for it, it was slow, had a ton of weird errors and all that went away by building the images where I run things, it’s faster and it also utilizes the idle computing power.

Some of the sources and known issues

This post was heavily inspired by these articles, and it was configured and tested following these examples:

triggers/README.md at main · tektoncd/triggers

Build and push an image with Tekton

Tutorial: Tekton Triggers with GitHub integration

There are some issues running on ARM, on other architectures it just works, see more:

fork/exec /busybox/mkdir: exec format error · Issue #4247 · tektoncd/pipeline

docker-build task fails with exec format error · Issue #5233 · tektoncd/pipeline

But everything should just work tm.

Errata

If you spot any error or have any suggestion, please send me a message so it gets fixed.

Also, you can check the source code and changes in the generated code and the sources here.

Originally published at https://techsquad.rocks on October 25, 2022.