Run your GitHub action pipelines on Kubernetes as Self-Hosted agents.

To get a basic understanding of running self-hosted runners on the Kubernetes cluster, this blog is perfect for you.

Why run a self-hosted GitHub actions runner?

Self-hosted runners are ideal for use-cases where you need to run workflows in a highly customizable environment with more granular control over hardware requirements, security, operating system, and software tools than GitHub-hosted runners provides.

Prerequisites

  1. Kubernetes cluster .
  2. Install Helm
  3. ArgoCD
  4. GitHub repository

In our case , we will be using a managed kubernetes cluster on Azure.

Steps for completing this tutorial:

1- Create a GitHub repository

Create a private repository on GitHub , example “YOUR GITHUB ACCOUNT/YOUR REPOSITRY”

2- Install cert-manager on a Kubernetes cluster

  • By default, actions-runner-controller uses cert-manager for certificate management of admission webhook, so we have to make sure cert-manager is installed on Kubernetes before we install actions-runner-controller.
  • Run the below helm commands to install cert-manager on AKS.
  • Verify installation using “kubectl get all -n cert-manager”. If everything is okay, you will see an output as below

3- Setting Up Authentication for Hosted Runners‍

There are two ways for actions-runner-controller to authenticate with the GitHub API (only 1 can be configured at a time, however):

  1. Using a GitHub App (not supported for enterprise-level runners due to lack of support from GitHub.)
  2. Using a PAT (personal access token)

To keep this blog simple, we are going with PAT.

To authenticate an action-runner-controller with the GitHub API, we can use a PAT with the action-runner-controller registers a self-hosted runner.

  • Go to account > Settings > Developers settings > Personal access token. Click on “Generate new token”. Under scopes, select “Full control of private repositories”.
  • Click on the “Generate token” button.
  • Copy the generated token and run the below commands to create a Kubernetes secret, which will be used by action-runner-controller deployment.
export GITHUB_TOKEN=XXXxxxXXXxxxxXYAVNa


kubectl create ns actions-runner-system

Then we have to create secret ( For now to keep the article simple , we will use a normal secret but in an entreprise project , you have to store that secret safely using Vault , SealedSecret …etc )

kubectl create secret generic controller-manager -n actions-runner-system - from-literal=github_token=${GITHUB_TOKEN}

4- Install action runner controller on the Kubernetes cluster

Manual installation

  • Run the below helm commands
helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller


helm repo updatehelm upgrade - install - namespace actions-runner-system - create-namespace - wait actions-runner-controller actions-runner-controller/actions-runner-controller - set syncPeriod=1m

Verify that the action-runner-controller is installed properly using the below commad.

kubectl --namespace actions-runner-system get all

GitOps approach installation using ArgoCD

Create an ArgoCD application manifest (Follow the below folder structure)

Homelabs
|----->tooling
        |----->automatic
                    |----->Templates
                            |------>github-runners

Then create the project on ArgoCD

Click inside the highlighted application and you will redirected to the controllers objects.

Now that the controller is Up and running , we will need to created the hosted runner.

5- Create a Repository Runner

  • Create a RunnerDeployment Kubernetes object, which will create a self-hosted runner named k8s-action-runner for the GitHub repository action-runner
  • Please Update Repo name from to “<Your-repo-name>”

  • To create the RunnerDeployment object, create the file runner.yaml as follows followed by the HorizontalRunnerAutoscaler:

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
 name: k8s-action-runner
 namespace: actions-runner-system
spec:
 replicas: 2
 template:
   spec:
     repository: # ADD YOUR REPO NAME
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: HorizontalRunnerAutoscaler
metadata:
  name: hr-runners-deployment-autoscaler
  namespace: actions-runner-system
spec:
  scaleTargetRef:
    name: k8s-action-runner
  minReplicas: 5
  maxReplicas: 20
  scaleUpTriggers:
  - duration: 5m0s
    githubEvent:
      workflowJob: {}

Then we create the project in ArgoCD , once done we can see the runners are up as pods.

  • If everything goes well, you should see two action runners on the Kubernetes, and the same are registered on Github.
  • Check under Settings > Actions > Runner of your repository.

Check the pods with:

kubectl get pod -n actions-runner-system

Now it is show time , let’s create a simple pipeline with GitHub action.

name: Self-Hosted Agent Example
on:
  push:
    branches: [ main ]
jobs:
  build:
    runs-on: self-hosted # Make sure you keep it as self-hosted 
    steps:
    - name: Checkout code
      uses: actions/checkout@v2
    - name: Run script on self-hosted agent
      run: |
        echo "Hello from self-hosted agent!"

Once we run this pipeline, it should pick one of the available pods (hosted agent pods), so we check the pipeline logs.

And here we can see that is working , it is using one of our deployed runners pod to run the pipeline.

In conclusion, self-hosted agents are a powerful tool for customizing your GitHub Actions workflows and taking full control of your build and deployment environments. They allow you to run your workflows on your own infrastructure, providing greater flexibility and control over your development process.

With self-hosted agents, you can use your own hardware and software configurations, run your workflows behind firewalls or in air-gapped environments, and leverage your existing tools and processes to automate your workflows.

GitHub provides an open-source implementation of a self-hosted agent, which can be easily installed and configured on a variety of platforms. Additionally, third-party tools and services can be used to manage and scale your self-hosted agents to meet the needs of your workflows.

However, setting up and managing self-hosted agents does require some technical expertise and maintenance. You will need to ensure that your agents are properly secured, monitored, and updated to ensure the reliability and security of your workflows.

Conclusion

Overall, self-hosted agents are a valuable addition to the GitHub Actions ecosystem, providing developers with the flexibility and control needed to build and deploy their applications with confidence.