Blogbanner_2

Keycloak deployment on Kubernetes with Helm charts using an external PostgreSQL database

by Devops

Prerequisites:

  1. Kubernetes cluster set up and configured.
  2. Helm installed on your Kubernetes cluster.
  3. Basic understanding of Kubernetes concepts like Pods, Deployments, and Services.
  4. Familiarity with Helm charts and templating.

Introduction:

Deploying Keycloak on Kubernetes with an external PostgreSQL database can be challenging, especially when using Helm charts. One common issue is that keycloak deploys with  a default database service when we use Helm chart, making it difficult to integrate with an external database. 

In this article, we’ll explore the problem we encountered while deploying Keycloak on Kubernetes using Helm charts and describe the solution we implemented to seamlessly use an external PostgreSQL database.

Problem:

The primary issue we faced during the deployment of Keycloak on Kubernetes using Helm was the automatic deployment of a default database service. This default service conflicted with our requirement to use an external PostgreSQL database for Keycloak. The Helm chart, by default, would deploy an internal database, making it challenging to configure Keycloak to connect to an external database.

Problem Analysis

  1. Default Database Deployment: The Helm chart for Keycloak automatically deploys an internal PostgreSQL database. This default setup is convenient for simple deployments but problematic when an external database is required.
  2. Configuration Complexity: Customizing the Helm chart to disable the internal database and correctly configure Keycloak to use an external PostgreSQL database requires careful adjustments to the values.yaml file.
  3. Integration Challenges: Ensuring seamless integration with an external PostgreSQL database involves specifying the correct database connection parameters and making sure that these settings are correctly propagated to the Keycloak deployment.
  4. Persistence and Storage: The internal database deployed by default may not meet the persistence and storage requirements for production environments, where an external managed PostgreSQL service is preferred for reliability and scalability.

To address these issues, the following step-by-step guide provides detailed instructions on customizing the Keycloak Helm chart to disable the default database and configure it to use an external PostgreSQL database.

Overview Diagram:

Step-by-Step Guide

Step 1: Setting Up Helm Repository

If you haven’t already added the official Helm charts repository for Keycloak, you can add it using the following command:

helm repo add codecentric https://codecentric.github.io/helm-charts

helm repo update

By adding the official Helm charts repository for Keycloak, you ensure that you have access to the latest charts maintained by the developers. Updating the repository ensures you have the most recent versions of the charts.

Step 2: Customizing Helm Values

Objective: Customize the Keycloak Helm chart to avoid deploying the default database and configure it to use an external PostgreSQL database.

Configure Keycloak for development mode

Create a values.yaml File

  1. Create a new file named values.yaml.
  2. Add the following content to the file 

image:

  # The Keycloak image repository

  repository: quay.io/keycloak/keycloak

  # Overrides the Keycloak image tag whose default is the chart appVersion

  tag: 24.0.3

  # The Keycloak image pull policy

  pullPolicy: IfNotPresent


resources:

  requests:

    cpu: “500m”

    memory: “1024Mi”

  limits:

    cpu: “500m”

    memory: “1024Mi”


args:

  – start-dev

  – –hostname=<url>

  – –hostname-url=<url>

  – –verbose


autoscaling:

  # If `true`, a autoscaling/v2beta2 HorizontalPodAutoscaler resource is created (requires Kubernetes 1.18 or above)

  # Autoscaling seems to be most reliable when using KUBE_PING service discovery (see README for details)

  # This disables the `replicas` field in the StatefulSet

  enabled: false

  # The minimum and maximum number of replicas for the Keycloak StatefulSet

  minReplicas: 1

  maxReplicas: 2


ingress:

  enabled: true

  #hosts:

  #  – <url>

  ssl:

    letsencrypt: true

    cert_secret: <url>

  annotations:

    kubernetes.io/ingress.class: “nginx”

    kubernetes.io/tls-acme: “true”

    cert-manager.io/cluster-issuer: letsencrypt

    cert-manager.io/acme-challenge-type: dns01

    cert-manager.io/acme-dns01-provider: route53

    nginx.ingress.kubernetes.io/proxy-buffer-size: “128k”

    nginx.ingress.kubernetes.io/affinity: “cookie”

    nginx.ingress.kubernetes.io/session-cookie-name: “sticky-cookie”

    nginx.ingress.kubernetes.io/session-cookie-expires: “172800”

    nginx.ingress.kubernetes.io/session-cookie-max-age: “172800”

    nginx.ingress.kubernetes.io/affinity-mode: persistent

    nginx.ingress.kubernetes.io/session-cookie-hash: sha1

  labels: {}

  rules:

    –

      # Ingress host

      host: ‘<url>’

      # Paths for the host

      paths:

        – path: /

          pathType: Prefix

  tls:

    – hosts:

        – <url>

      secretName: “<url>”

      

extraEnv: |

 – name: PROXY_ADDRESS_FORWARDING

   value: “true”

 – name: QUARKUS_THREAD_POOL_MAX_THREADS

   value: “500”

 – name: QUARKUS_THREAD_POOL_QUEUE_SIZE

   value: “500”

 

This configuration file customizes the Keycloak Helm chart to set specific resource requests and limits, ingress settings, and additional environment variables. By setting the args to start Keycloak in development mode, you allow for easier initial setup and testing.

Configuring for Production Mode

  1. Add or replace the following content in values.yaml for production mode:

args:

  – start

  – –hostname=<url>

  – –hostname-url=<url>

  – –verbose

  – –optimized

  – -Dquarkus.http.host=0.0.0.0

  – -Dquarkus.http.port=8080

Note: The production configuration includes optimizations and ensures that Keycloak runs in a stable environment suitable for production workloads. The –optimized flag is added for performance improvements.

Configuring for External Database

  1. Add the following content to values.yaml to use an external PostgreSQL database:

args:

  – start

  – –hostname-url=<url>

  – –verbose

  – –db=postgres

  – –db-url=<jdbc-url>

  – –db-password=${DB_PASSWORD}

  – –db-username=${DB_USER}

 

postgresql:

  enabled: false

This configuration disables the default PostgreSQL deployment by setting postgresql.enabled to false. The database connection arguments are provided to connect Keycloak to an external PostgreSQL database.

Step 3: Deploying Keycloak with PostgreSQL and Custom Themes Using Helm

Objective: Add custom themes to Keycloak and deploy it using Helm.

  1. Add the following to values.yaml to include custom themes:

extraInitContainers: |

  – name: theme-provider

    image: <docker-hub-registry-url>

    imagePullPolicy: IfNotPresent

    command:

      – sh

    args:

      – -c

      – |

        echo “Copying custom theme…”

        cp -R /custom-themes/* /eha-clinic

    volumeMounts:

      – name: custom-theme

        mountPath: /eha-clinic

extraVolumeMounts: |

  – name: custom-theme

    mountPath: /opt/jboss/keycloak/themes/

extraVolumes: |

  – name: custom-theme

    emptyDir: {}

This configuration uses an init container to copy custom themes into the Keycloak container. The themes are mounted at the appropriate location within the Keycloak container, ensuring they are available when Keycloak starts.

Step 4: Configuring Keycloak

Objective: Log in to the Keycloak admin console and configure realms, users, roles, client applications, and other settings.

Access the Keycloak admin console using the URL provided by your ingress configuration.

  1. Log in with the default credentials (admin/admin).
  2. Configure the following according to your application requirements:
  • Realms
  • Users
  • Roles
  • Client applications

The Keycloak admin console allows for comprehensive configuration of all aspects of authentication and authorization, tailored to the needs of your applications.

Step 5: Configuring Custom Themes

Objective: Apply and configure custom themes within the Keycloak admin console.

  1. Log in to the Keycloak admin console using the default credentials (admin/admin).
  2. Navigate to the realm settings and select the “Themes” tab.
  3. Select and configure your custom themes for:
  • Login pages
  • Account pages
  • Email pages

Custom themes enhance the user experience by providing a personalized and branded interface. This step ensures that the authentication experience aligns with your organization’s branding and user interface guidelines.

Conclusion

By following the steps outlined in this article, you can deploy Keycloak with PostgreSQL on Kubernetes using Helm, while also incorporating custom themes to personalize the authentication experience. Leveraging Helm charts simplifies the deployment process, while Keycloak and PostgreSQL offer robust features for authentication and data storage. Integrating custom themes allows you to tailor the authentication pages according to your branding and user interface requirements, ultimately enhancing the user experience and security of your applications.

Frequently Asked Questions (FAQs)

Keycloak is a modern applications and services open-source identity and access management solution. It offers various functionalities such as single sign-on, identity brokering, and social login which help in making user identity management easy as well as application security

Deploying Keycloak on Kubernetes means that you can have an elastic application that can be extended according to the number of users, resistant to failure on case of external services unavailability or internal server crashes; it is also easy to manage, has numerous protocols for authentication mechanisms and connects with different types of external databases.

Helm charts are pre-configured Kubernetes resource packages that simplify the efficient management and deployment of applications on Kubernetes.

To disable the default PostgreSQL database, set postgresql.enabled to false in the values.yaml file.

Provide the necessary database connection parameters in the values.yaml file, including --db-url, --db-password, and --db-username.

You can add custom themes by configuring init containers in the values.yaml file to copy the themes into the Keycloak container and mounting them at the appropriate location.

Asset 2 (1)

Integrating Apache Jmeter with Jenkins

by Devops

In the world of software development, ensuring the performance and reliability of applications is paramount. One of the most popular tools for performance testing is Apache JMeter, known for its flexibility and scalability. Meanwhile, Jenkins has become the go-to choice for continuous integration and continuous delivery (CI/CD). Combining the power of JMeter with the automation capabilities of Jenkins can significantly enhance the efficiency of performance testing within the development pipeline. In this article, we’ll explore the integration of JMeter with Jenkins and how it can streamline the performance testing process.

Apache Jmeter

Apache JMeter is a powerful open-source tool designed for load testing, performance testing, and functional testing of applications. It provides a user-friendly GUI that allows testers to create and execute several types of tests, including HTTP, FTP, JDBC, LDAP, and more. JMeter supports simulating heavy loads on servers, analyzing overall performance metrics, and finding performance bottlenecks.

With its scripting and parameterization capabilities, JMeter offers flexibility and scalability for testing web applications, APIs, databases, and other software systems. Its extensive reporting features help teams assess application performance under different conditions, making it an essential tool for ensuring the reliability and scalability of software applications. More information available here Apache JMeter.

Jenkins

Jenkins is one of the most popular open-source automation servers widely used for continuous integration (CI) and continuous delivery (CD) processes in software development for several years now. It allows developers to automate the building, testing, and deployment of applications, thereby streamlining the development lifecycle. Jenkins supports integration with various version control systems like Git, SVN, and Mercurial, enabling automatic triggers for builds whenever code changes are pushed.

Its extensive plugin ecosystem provides flexibility to integrate with a wide range of tools and technologies, making it a versatile solution for managing complex CI/CD pipelines. Jenkins’ intuitive web interface, extensive plugin library, and robust scalability make it a popular choice for teams aiming to achieve efficient and automated software delivery processes. The Jenkins docs has a page to help with the Jenkins installation process.

Why Integrate JMeter with Jenkins?

Traditionally, performance testing has been a manual and time-consuming process, often conducted as a separate part of the development lifecycle by test teams. The results had to then be shared with the rest of the team as there was no automated execution or capturing of the test results as part of the CICD pipeline. However, in today’s fast-paced software development environment, there’s a growing need to automate the complete testing processes to include execution of Performance tests as part of the CICD pipeline. By integrating JMeter with Jenkins, organizations can achieve the following benefits:

Automation: Jenkins allows you to automate the execution of JMeter tests as part of your CI/CD pipeline, enabling frequent and consistent performance testing with minimal manual intervention.

Continuous Feedback: Incorporating performance tests into Jenkins pipelines provides immediate feedback on the impact of code changes on application performance, allowing developers to find and address performance issues early in the development cycle.

Reporting: Jenkins provides robust reporting and visualization capabilities, allowing teams to analyze test results and track performance trends over time, helping data-driven decision-making.

Our Proposed Approach & its advantages

We’ve adopted a new approach in addition to using the existing JMeter plugin for Jenkins wherein we enhance the Jenkins Pipeline to include detailed notifications and better result organization.

The key steps of our approach are as follows.

  1. We can install JMeter directly on the agent base OS. This ensures we have access to the latest features and updates.
  2. We use the powerful Blaze Meter plugin to generate our JMeter scripts.
  3. We’ve written a dedicated Jenkins pipeline to automate the execution of these Jmeter scripts.
  4. We have also defined the steps as part of the Jenkins script to distribute the Execution Status and log by email to chosen users.
  5. We also store the results in a configurable Path for future reference.

All of this ensures better automation, flexibility or control of the execution, notification and efficient performance testing as part of the CICD Pipeline.

Setting Up Blaze Meter & Capturing the Test Scripts

To automate the process of script writing we should use the Blaze Meter tool. Navigate to chrome extensions and search for Blaze Meter. Click on add to chrome. Access Blaze Meter official website and create an account there for further process.

Open chrome and now you can find the Blaze Meter extension on the top right corner.

Click on the Blaze Meter chrome extension. A toolbox will be visible. Open the application where you want to record the scripts for Jmeter. Click on the record button to start.

Navigate through the application and perform necessary operations as end users of the application would. Click on the stop button to stop the recording.

Now Blaze Meter has recorded the scripts. To save it in. jmx format, click on save, check the Jmeter only box and click on save as shown in the screenshot below.

For more information on how to record a Jmeter script using Blaze Meter, follow the link

Modifying the Test Scripts in JMeter

The recorded script can then be opened in JMeter, and necessary changes can be made as per the different Load and Performance Scenarios to be assessed for the application.

Select the generated .jmx file and click on open

In addition to these you can add listeners to the thread groups, for better visibility of each sample results.

Setting up a Jenkins pipeline to execute JMeter tests

Install Jenkins: If you haven’t already, install Jenkins on your server following the official documentation.

Create a New Pipeline Job: A new pipeline job should be created to orchestrate the performance testing process. Click on new item to create a pipeline job and select pipeline option.

Post creating a new pipeline, navigate to configure and mention the scheduled time in the below format.

(‘H 0 */3 * *’)

Define Pipeline Script: Configure the pipeline script to execute the JMeter test at regular intervals using a cron expression.

 pipeline {

    agent {

        label ‘{agent name}’

    }

This part of the Jenkins pipeline script specifies the agent (or node) where the pipeline should run. The label ‘{agent name}’ should be replaced with the label of the specific agent you want to use. This ensures that the pipeline will execute on a machine that matches the provided label.

stages {

      stage(‘Running JMeter Scripts’) {

          steps {

              script {

 sh ”’

    output_directory=”{path}/$(date +’%Y-%m-%d’)”

    mkdir -p “$output_directory”

    cd {Jmeter Path}/bin

    sh jmeter -n -t {Jmeter Script Path} -l “$output_directory/{Result file name}” -e -o “$output_directory”

    cp “$output_directory”/{Result file name} $WORKSPACE

    cp -r $output_directory $WORKSPACE

    ”’

       }

            }      

}

This stage named ‘Running JMeter Scripts’ has steps to execute a shell script. The script does the following:
1. Creates an output directory with the current date.
2. Navigates to the JMeter binary directory.
3. Runs the JMeter test script specified by {Jmeter Script Path}, storing the results in the created directory
4. Copies the result file and output directory to the Jenkins workspace for archiving.

post {

        always {

            script {

                currentBuild.result = currentBuild.result

                def date = sh(script: ‘date +\’%Y-%m-%d\”, returnStdout: true).trim()

                def subject = “${currentBuild.result}: Job ‘${env.JOB_NAME}'”

                def buildLog = currentBuild.rawBuild.getLog(1000)

                emailext(

                    subject: subject,

                    body: “”” Hi Team , Jmeter build was successful , please contact team for the results “””,

                    mimeType: ‘text/html’,

                    to: ‘{Receiver Email}’,

                    from: ‘{Sender Email}’,

                    attachLog: true ,

                )

            }

     }

This post block runs after the pipeline completes. It retrieves the last 1000 lines of the build log and sends an email notification with the build result, a message, and the build log attached to specified recipients.

View the generated reports.

In Linux instance navigate to the path where the .html files are stored i.e. the output report of the jmeter scripts

Before you open the HTML file, move the complete folder to your local device. Once the folder is moved open the .html file and you’ll be able to analyze the reports

Conclusion

By following the steps described and the approach suggested, we have shown how Integrating JMeter with Jenkins enables teams to automate performance testing and incorporate it seamlessly into the CI/CD pipeline. By scheduling periodic tests, storing results, and sending out email notifications, organizations can ensure the reliability and scalability of their applications with minimal manual intervention. Embrace the power of automation and elevate your performance testing efforts with Jenkins and JMeter integration. For any assistance in automation of your Performance tests please get in touch with us at [email protected] or leave us a note at form and we will get in touch with you.

Frequently Asked Questions (FAQs)

You can download the latest version from the Apache JMeter homepage, after download extract the downloaded files to a directory on the agent machine and Make sure to add the JMeter bin directory in the system's PATH variable so that all JMeter commands can be run from the command line.

Its a Chrome extension that helps users to record and create JMeter scripts easily. To use it, install the BlazeMeter extension from the Chrome Web Store and record the desired scenarios on your web application, and export the recorded script in .jmx format. This script can then be modified in JMeter and used in your Jenkins pipeline for automated performance testing.

Create a new pipeline job in Jenkins and define the pipeline script to include stages for running JMeter tests. The script should include steps to execute JMeter commands, store the results, and send notifications. Here's a example script for you 

pipeline {

    agent { label 'your-agent-label'

 }

    stages {

        stage('Run JMeter Tests') {

            steps {

                script {

                    sh '''

                    output_directory="/path/to/output/$(date +'%Y-%m-%d')"

                    mkdir -p "$output_directory"

                    cd /path/to/jmeter/bin

                    sh jmeter -n -t /path/to/test/script.jmx -l "$output_directory/results.jtl" -e -o "$output_directory"

                    cp "$output_directory/results.jtl" $WORKSPACE

                    cp -r "$output_directory" $WORKSPACE

                    '''

                }

            }

        }

    }

    post {

        always {

            emailext (

                subject: "JMeter Test Results",

                body: "The JMeter tests have completed. Please find the results attached.",

                recipientProviders: [[$class: 'DevelopersRecipientProvider']],

                attachLog: true

            )

        }

    }

}

You can schedule the JMeter test execution using the cron syntax in the Jenkins pipeline configuration. For example, to run the tests every three hours, you can use:<br><br>

H 0 */3 * * *

<br><br>

This will trigger the pipeline at the top of the hour every three hours.

After the JMeter tests are executed, the results are typically stored in a specified directory on the Jenkins agent. You can navigate to this directory, download the results to your local machine, and open the HTML reports generated by JMeter for detailed analysis.