Cloud Experts Documentation

ROSA

Deploying ROSA in STS mode

Tip The official documentation for installing a ROSA cluster in STS mode can be found here .

Quick Introduction by Ryan Niksch (AWS) and Shaozen Ding (Red Hat) on YouTubeexternal link (opens in new tab)

STS allows us to deploy ROSA without needing a ROSA admin account, instead it uses roles and policies with Amazon STS (secure token service) to gain access to the AWS resources needed to install and operate the cluster.

OpenShift Network Calculator

Overview

The OpenShift Network Calculator helps you determine the network requirements for your OpenShift cluster, specifically designed for clusters using the OVN-Kubernetes Container Network Interface (CNI). This tool calculates the number of pods, services, and nodes your network configuration can support, and identifies potential network conflicts with OVN-Kubernetes reserved networks.

Proper network planning is critical when deploying OpenShift clusters, as incorrect CIDR ranges can lead to IP address exhaustion, network conflicts, or connectivity issues. This calculator is particularly useful when planning deployments for:

Ingress to ROSA Virt VMs with Certificate-Based Site-to-Site (S2S) IPsec VPN and Libreswan

Introduction

This solution uses a Site-to-Site (S2S) VPNexternal link (opens in new tab) as a mechanism in OpenShift Virtualization on ROSA to establish an IP route between the virtual overlay network that VMs are attached to, and the VPC outside your clusterexternal link (opens in new tab) without the need for NAT or load balancers. OpenShift Virtualization provides several built-in features to plug VMs directly into outside networks when deployed on-premises , but these depend upon mechanisms that are not exposed in cloud provider networks. This solution should be considered a stop-gap until there is a native OpenShift Virtualization feature to plug VMs into cloud provider networks.

Deploy ROSA + Nvidia GPU + RHOAI with Automation

Getting Red Hat OpenShift AI up and running with NVIDIA GPUs on a Red Hat OpenShift Service on AWS (ROSA) cluster can involve a series of detailed steps, from installing various operators to managing dependencies. While manageable, this process can be time-consuming when you’re eager to start leveraging OpenShift AI for your projects.

This guide and its accompanying Git repository are designed to streamline your setup significantly. We focus on getting you productive faster by using Terraform to deploy a ROSA cluster with GPUs from the start. From there, Ansible scripts take over, automating the deployment and configuration of all necessary operators for both NVIDIA GPUs and Red Hat OpenShift AI. This means less manual configuration for you and more time spent on what matters: innovating with AI.

Using a Private IngressController with CloudFront on a ROSA Cluster

AWS CloudFront is a great choice for a Content Delivery Network in front of your ROSA cluster. In many situations, it may be useful to ensure that traffic to Routes within your cluster must come from your CloudFront Distribution rather than being able to bypass it (for example, if a WAF is attached to the CloudFront Distribution).

This guide shows how to use the CloudFront VPC Origin feature in combination with a Network Load Balancer (NLB), and a private secondary IngressController type NLB, providing a path from the Internet to your applications that only goes through CloudFront.

Accessing the ROSA HCP API Server from a Different AWS Account

Introduction

You can create a ROSA HCP cluster in one AWS account and configure it to allow access from a different AWS account using the oc command. This guide walks you through the actual AWS setup.

pic1 Note: AWS environments vary, so consider this as one possible setup.

Prerequisites

A ROSA HCP cluster has been already deployed in AWS Account-A, and the following AWS resources are available. ROSA HCP 4.19.0 was used for this guide. pic2

Deploy ROSA with Red Hat Advanced Cluster Management for Kubernetes

In the dynamic world of cloud-native development, efficiently managing Kubernetes clusters across diverse environments is paramount. This blog post dives into a powerful combination: deploying Red Hat OpenShift Service on AWS (ROSA) Hosted Control Planes (HCP) clusters, orchestrated and governed by Red Hat Advanced Cluster Management for Kubernetes (RHACM). This approach offers a compelling suite of benefits, including significant cost reductions by offloading control plane management to Red Hat, accelerated cluster provisioning times, and enhanced operational efficiency through a centralized management plane. By leveraging ROSA HCP with RHACM, organizations can achieve a more streamlined, secure, and scalable Kubernetes footprint on AWS, allowing teams to focus more on innovation and less on infrastructure overhead.

Building LLM Cost and Performance Dashboard with Red Hat OpenShift AI on ROSA and Amazon Bedrock

1. Introduction

As the LLM’s usage increases in the enterprise, not many realize that every LLM API call has two hidden costs: time and money. So while data scientists might argue about data accuracy, infrastructure engineers on the other hand, would need to know if that 2-second response time will scale, and if those $0.015 per thousand tokens cost will blow their quarterly budget, among others. In this guide, we will build a simple cost and performance dashboard for Amazon Bedrockexternal link (opens in new tab) models using Red Hat OpenShift AI (RHOAI) , which is our platform for managing AI/ML projects lifecycle, running on a Red Hat OpenShift Service on AWS (ROSA) cluster.

Creating Agentic AI to deploy ARO cluster using Terraform with Red Hat OpenShift AI on ROSA and Amazon Bedrock

1. Introduction

Agentic AI can be defined as systems that are capable of interpreting natural language instructions, in this case users’ prompts, making decisions based on those prompts, and then autonomously executing tasks on behalf of users. In this guide, we will create one that is intelligent enough that not only that it can understand/parse users’ prompts, but it can also take action upon it by deploying (and destroying) Azure Red Hat OpenShift (ARO) cluster using Terraform.

Integrating Service Mesh into a ROSA Cluster

This is a simple guide to integrate Red Hat OpenShift Service Mesh into your ROSA cluster. In this scenario, we will install Service Mesh using a custom domain (optional) and expose an app to test it. The first half of the guide will be integrating Service Mesh 2.x and second half will be integrating Service Mesh 3.x .

Prerequisites

Set up environment

Install the necessary operators, i.e. Elasticsearch (optional), Jaeger (distributed tracing platform), Kiali, and Service Mesh (2 or 3 depends on your use case) from OpenShift console.

Adding a Private Ingress Controller and a Public ALB to a ROSA Cluster

Starting with OpenShift 4.14, Red Hat OpenShift Service on AWS (ROSA) supports adding additional Ingress Controllers which can be used to configure a custom domain on a ROSA cluster. This guide shows how to leverage this feature to create a complete routing solution with both a private Ingress Controller (which creates an Network Load Balancer (NLB)) and a public Application Load Balancer (ALB) in front of it, providing a path from the internet to your applications.

ROSA - Federating Metrics to AWS Prometheus

Federating Metrics from ROSA is a bit tricky as the cluster metrics require pulling from its /federated endpoint while the user workload metrics require using the prometheus remoteWrite configuration.

This guide will walk you through using the MOBB Helm Chart to deploy the necessary agents to federate the metrics into AWS Prometheus and then use Grafana to visualize those metrics.

As a bonus it will set up a CloudWatch datasource to view any metrics or logs you have in Cloud Watch.

Using local-zones in ROSA Classic

This guide walks through setting up a local-zone in an existing ROSA Classic cluster. Use this approach when you have latency requirements that can be reduced when using a local zone. Since you are not using the default ingress, you will not be able to use the router strategy the cluster has.

Prerequisites

ROSA Classic Cluster with AWS Load Balancer Operator already installed:

Local Zone validations and configuration in your AWS Account:

Setting up Cross-Cluster PostgreSQL Replication with Skupper on ROSA and ARO

This guide demonstrates how to set up a highly available PostgreSQL database with cross-cluster replication between Red Hat OpenShift Service on AWS (ROSA) and Azure Red Hat OpenShift (ARO) using Skupper. This architecture enables disaster recovery capabilities and geographical distribution of your database workloads.

Note: You can create a ROSA cluster using the ROSA with STS deployment guide or an ARO cluster with the ARO quickstart guide . While this tutorial focuses on ROSA and ARO, the same principles can be applied to any two OpenShift clusters, regardless of their hosting environment.

Optimizing Costs with ROSA - Scheduled Cluster Scaling

One of the key benefits of Red Hat OpenShift Service on AWS (ROSA) is its ability to scale efficiently, ensuring you only pay for the resources you actually need. While ROSA includes autoscaling features that adjust cluster size based on demand, you can further optimize costs by scheduling scaling during off-peak hours when the cluster isn’t heavily used. This helps reduce expenses without impacting performance.

In this guide, we’ll show you how to schedule scaling in ROSA, allowing your cluster to automatically adjust its size based on a predefined schedule. You’ll learn how to schedule scale-downs during periods of low activity and scale-ups when additional resources are required, ensuring both cost efficiency and optimal performance.

Configuring Microsoft Entra ID as an external authentication provider

You can set up Red Hat OpenShift Service on AWS (ROSA) with hosted control planes (HCP) to use an external OpenID Connect (OIDC) identity provider for authentication instead of the built-in OpenShift OAuth server. While the built-in OAuth server supports various identity providers, its capabilities are limited. By integrating external OIDC identity providers directly with ROSA with HCP, you can enable machine-to-machine workflows, like CLI access, and gain features unavailable with the built-in OAuth server.

Securely exposing an application on a private ROSA cluser with an AWS Network Load Balancer

Overview

Red Hat strongly recommends creating a private ROSA cluster with no inbound Internet connectivity, isolating both the cluster API and hosted applications from external access. This configuration is a key part of a multi-layered security strategy to protect clusters and workloads from external threats.

However, some applications may require Internet access to support external users or partners. Even with a private cluster, you can securely expose these applications through various methods. For example, a TCP-based application like Kafka may need connectivity to allow partners on the Internet to send or receive data. This document provides guidance on one specific method to securely expose a TCP-based application while maintaining the security of a private cluster.

Securely exposing an application on a private ROSA cluser with an AWS Network Load Balancer

Continuation of Securely exposing an application on a private ROSA cluser with an AWS Network Load Balancer

These instructions go through setting up an additional VPC as part of the overall blog. If you already have a VPC that you would like to use, you can skip these instructions.

Looking at the overall architecture, this section will setup everything in the blue box.

Create a public VPC

The VPC we will be creating will have a VPC with a private subnet, a public subnet where the Network Load Balancer will sit, an Internet Gateway and a Nat Gateway.

Securely exposing an application on a private ROSA cluser with an AWS Network Load Balancer - Jump Host

Continuation of Securely exposing an application on a private ROSA cluser with an AWS Network Load Balancer

These instructions go through setting up a jump host to connect to the private rosa cluster.

Note: the guide assumes you have set envirionment variables as described in the parent guide.

Create a jumphost instance using the AWS CLI

Create an additional Security Group for the jumphost

Grab the Security Group Id generated in the previous step

Configuring Microsoft Entra ID to emit group names

In this guide, we will configure an existing Microsoft Entra ID (formerly Azure Active Directory) identity provider to emit the group name instead of the group ID for optional group claims. This will allow you to reference group names in your role bindings instead of the group ID.

The ability to emit group names instead of group IDsexternal link (opens in new tab) is a preview feature made available by Microsoft and is subject to their terms and conditions around preview features of their services.

Deploying and Running Ollama and Open WebUI in a ROSA Cluster with GPUs

Red Hat OpenShift Service on AWS (ROSA) provides a managed OpenShift environment that can leverage AWS GPU instances. This guide will walk you through deploying Ollama and OpenWebUI on ROSA using instances with GPU for inferences.

Prerequisites

  • A Red Hat OpenShift on AWS (ROSA classic or HCP) 4.14+ cluster
  • OC CLI (Admin access to cluster)
  • ROSA CLI

Set up GPU-enabled Machine Pool

First we need to check availability of our instance type used here (g4dn.xlarge), it should be in same region of the cluster. Note you can use also Graviton based instance (ARM64) like g5g* but only on HCP 4.16+ cluster.

Configuring ROSA with HCP Private Cluster API Access

With ROSA with HCP private clusters, the AWS PrivateLink endpoint exposed in the customer’s VPC has a default security group. This security group has access to the PrivateLink endpoint that is limited to only those resources that exist within the VPC or resources that are present with an IP address associated with the VPC CIDR range. In order to grant access to any entities outside of the VPC, through VPC peering and transit gateway, you must create and attach another security group to the PrivateLink endpoint to grant the necessary access.

Creating Images using Stable Diffusion on Red Hat OpenShift AI on ROSA cluster with GPU enabled

1. Introduction

Stable Diffusionexternal link (opens in new tab) is an AI model to generate images from text description. It uses a diffusion process to iteratively denoise random Gaussian noise into coherent images. This is a simple tutorial to create images using Stable Diffusion model using Red Hat OpenShift AI (RHOAI) , formerly called Red Hat OpenShift Data Science (RHODS), which is our OpenShift platform for AI/ML projects lifecycle management, running on a Red Hat OpenShift Services on AWS (ROSA) cluster, which is our managed service OpenShift platform on AWS, with NVIDIA GPU enabled.

Maximo Application Suite on ROSA ( Red Hat OpenShift on AWS )

IBM Maximo Application Suite (MAS) is a set of applications for asset monitoring, management, predictive maintenance and reliability planning. When combined with Red Hat OpenShift on AWS ( ROSA ), this frees up your Maximo and operations team to focus on what is important to them ( Maximo ) rather than having to worry about managing and building clusters.

This document outlines how to get quickly get started with ROSA and installing Maximo all through automation.

Configure Network Policies and Egress Firewalls for a ROSA Cluster

It’s common to want to restrict network access between namespaces, as well as restricting where traffic can go outside of the cluster. OpenShift achieves this with the Network Policy and Egress Firewall resources.

It’s common to use these methods to restrict network traffic alongside Egress IP and other OpenShift and OVN-Kubernetes resources.

Prerequisites

Project Template

The first thing to do is create a Project Template that containes Network Policys and Egress Firewalls with default deny rules

Migrating EC2 Instances to OpenShift Virtualization

Red Hat OpenShift Service on AWS (ROSA) provides a managed OpenShift environment that can run virtualized workloads using OpenShift Virtualization. This guide will walk you through migrating an existing EC2 instance to OpenShift Virtualization by exporting it to S3, syncing to EFS, and importing as a VM.

Prerequisites

  • A Red Hat OpenShift on AWS (ROSA) 4.19+ cluster
  • AWS CLI configured with appropriate permissions
  • SSH public key at ~/.ssh/id_rsa.pub (REQUIRED - for key-based authentication)
  • Terraform installed
  • OC CLI (Admin access to cluster)
  • virtctl CLI tool

Clone the Repository

First, clone the repository and navigate to the project directory:

Creating a ROSA cluster in AWS GovCloud

This guide outlines the procedure for creating a ROSA cluster in AWS GovCloud. There are some key differences between the ROSA offerings in AWS GovCloud and AWS Commercial. They’re outlined in detail in the AWS documentation hereexternal link (opens in new tab) , but a few requirements in GovCloud that are worth highlighting:

  • Only ROSA Classic is supported (not Hosted Control Plane)
  • STS mode is required
  • PrivateLink is required
  • FIPS mode is required

Prerequisites

Create VPC and Subnets

In this guide, we’ll use Terraform to create a VPC to house our cluster, and we’ll opt for a Single-AZ configuration for simplicity. We’ll also create an EC2 jumphost to aid in accessing our cluster once it comes up. Before running it, you’ll need to ensure your AWS CLI is authenticated to a government region in AWS (us-gov-west-1 or us-gov-east-1).

Running and Deploying LLMs using Red Hat OpenShift AI on ROSA cluster and Storing the Model in Amazon S3 Bucket

1. Introduction

Large Language Models (LLMs)external link (opens in new tab) are a specific type of generative AI focused on processing and generating human language. They can understand, generate, and manipulate human language in response to various tasks and prompts.

This guide is a simple example on how to run and deploy LLMs on a Red Hat OpenShift Services on AWS (ROSA) cluster, which is our managed service OpenShift platform on AWS, using Red Hat OpenShift AI (RHOAI) , which is formerly called Red Hat OpenShift Data Science (RHODS) and is our OpenShift platform for managing the entire lifecycle of AI/ML projects. And we will utilize Amazon S3external link (opens in new tab) bucket to store the model output. In essence, here we will first install RHOAI operator and Jupyter notebook, create the S3 bucket, and then run the model.

Running and Deploying LLMs using Red Hat OpenShift AI on ROSA cluster and Storing the Model in Amazon S3 Bucket

1. Introduction

Large Language Models (LLMs)external link (opens in new tab) are a specific type of generative AI focused on processing and generating human language. They can understand, generate, and manipulate human language in response to various tasks and prompts.

This guide is a simple example on how to run and deploy LLMs on a Red Hat OpenShift Services on AWS (ROSA) cluster, which is our managed service OpenShift platform on AWS, using Red Hat OpenShift AI (RHOAI) , which is formerly called Red Hat OpenShift Data Science (RHODS) and is our OpenShift platform for managing the entire lifecycle of AI/ML projects. And we will utilize Amazon S3external link (opens in new tab) bucket to store the model output. In essence, here we will first install RHOAI operator and Jupyter notebook, create the S3 bucket, and then run the model.

Deploying a ROSA Classic cluster with Terraform

This guide will walk you through deploying a ROSA cluster using Terraform. This is a great way to get started with ROSA and to automate the deployment of your clusters.

Pre-requisites

  • You need the git binary installed on your machine. You can download it from the git websiteexternal link (opens in new tab) .

  • You need to have the terraform binary installed on your machine. You can download it from the Terraform websiteexternal link (opens in new tab) .

    Deploying a ROSA HCP cluster with Terraform

    This guide will walk you through deploying a ROSA HCP cluster using Terraform. This is a great way to get started with ROSA and to automate the deployment of your clusters.

    Pre-requisites

    • You need the git binary installed on your machine. You can download it from the git websiteexternal link (opens in new tab) .

    • You need to have the terraform binary installed on your machine. You can download it from the Terraform websiteexternal link (opens in new tab) .

      Deploying and Managing Virtual Machines on ROSA with OpenShift GitOps

      One of the great things about OpenShift Virtualization is that it brings new capabilities to run virtual machines alongside your containers AND using DevOps processes to manage them.

      This tutorial will show how to configure OpenShift GitOps ( based on ArgoCD ) to deploy and managed virtual machines.

      Pre-requisites

      • A ROSA Cluster with OpenShift Virtualization (see Deploying OpenShift Virtualization on ROSA ) If you follow the guide above, you can skip the Create a Virtual Machine section as we will be using OpenShift GitOps to deploy the cluster.

      Note: If you get a 404 error when you depoy “HyperConverged”, make sure your metal nodes are ready and have enough capacity.

      Deploying Openshift Virtualization on ROSA with NetApp FSx storage.

      OpenShift Virtualization is a feature of OpenShift that allows you to run virtual machines alongside your containers. This is useful for running legacy applications that can’t be containerized, or for running applications that require special hardware or software that isn’t available in a container.

      In this tutorial, I’ll show you how to deploy OpenShift Virtualization on Red Hat OpenShift on AWS (ROSA) using the AWS NetApp FSx service (specifically NFS, not ISCSI or SAN) to provide resilience and live migration. I’ll show you how to create a ROSA cluster, deploy the OpenShift Virtualization operator, deploy the NetApp Trident Operator and create a virtual machine.

      Deploying OpenShift Virtualization on ROSA (CLI)

      OpenShift Virtualization is a feature of OpenShift that allows you to run virtual machines alongside your containers. This is useful for running legacy applications that can’t be containerized, or for running applications that require special hardware or software that isn’t available in a container.

      In this tutorial, I’ll show you how to deploy OpenShift Virtualization on Red Hat OpenShift on AWS (ROSA). I’ll show you how to create a ROSA cluster, deploy the OpenShift Virtualization operator, and create a virtual machine.

      Deploying OpenShift Virtualization on ROSA (GUI)

      OpenShift Virtualization is a feature of OpenShift that allows you to run virtual machines alongside your containers. This is useful for running legacy applications that can’t be containerized, or for running applications that require special hardware or software that isn’t available in a container.

      In this tutorial, I’ll show you how to deploy OpenShift Virtualization on Red Hat OpenShift on AWS (ROSA) using the OpenShift Console. I’ll show you how to deploy the OpenShift Virtualization operator, and create a virtual machine all from inside the Red Hat Cluster Manager and OpenShift Console

      Install Portworx on Red Hat OpenShift Service on AWS (ROSA) with hosted control planes (HCP)

      Portworx storage is a built-for-Kubernetes service that offers flexible and scalable persistent storage for applications in production. In this tutorial we will look at installing Portworx Enterprise on ROSA-HCP.

      Prerequisites

      You must have a Red Hat OpenShift Service on AWS (ROSA) with hosted control plane cluster

      Set environment variable adjusting for ROSA_HCP_CLUSTER_NAME and REGION as necessary

      Open ports for worker nodes via Web console (Note to use cli skip this step)

      Perform the following to add the inbound rules so that the AWS EC2 instance uses your specified security groups to control the incoming traffic.

      Migrating ROSA Ingress Controllers from a CLB to NLB

      This guide will show you how to migrate the default Red Hat OpenShift Service on AWS (ROSA) IngressController from an AWS Classic Load Balancer to an AWS Network Load Balancer.

      In version 4.14 of ROSA, Red Hat introduced changes to IngressControllers to give customers more control over their workloads and configuration. The operation below requires a cluster running version 4.14 or higher. To request early access to this additional functionality in version 4.13, please contact Red Hat support and open a case to request access .

      Configuring AWS CLB Access Logging

      This guide will show you how to enable access logging on the default Classic Load Balancer ingress controller used in Red Hat OpenShift Service on AWS (ROSA) version 4.13 and earlier.

      Prerequisites

      • A ROSA Cluster (Version 4.13 or earlier)
      • A logged in oc CLI
      • A logged in aws CLI

      S3 Bucket Creation

      1. Run the following command, making sure to update the name of the S3 bucket you wish to create and the account number of the Elastic Load Balancing root account (this is not your AWS account):

        Setting custom domains for apps created via OpenShift Dev Spaces

        Red Hat OpenShift Dev Spaces (formally CodeReady Workspaces) is an Operator available for OpenShift that allows users to create dynamic IDEs for developing and publishing code. When using OpenShift Dev Spaces, users can test their code and have the service automatically create a route for users to see their code in real time. By default, this route will use the default Ingress Controller, but it is possible to configure Dev Spaces to use a custom domain instead.

        Add an Ingress Controller to a ROSA Cluster and optionally with a custom domain.

        Starting with OpenShift 4.14, ROSA supports adding additional Ingress Controllers which can use used to configure a custom domain on a ROSA cluster without having to use the now deprecated Custom Domain Operator. This guide shows how to add an additional Ingress Controller ( public or private ) to a ROSA cluster and optionally also configuring a custom domain.

        Prerequisites

        • A Red Hat OpenShift on AWS (ROSA) cluster
        • The oc CLI #logged in.
        • The aws CLI #logged in.
        • The rosa CLI #logged in.
        • (optional) A Public Route53 Hosted Zone, and the related Domain to use.

        Set up environment

        1. Export few environment variables

        Important: The variables below can be customized to fit your needs for your ingress controller.

        Cross-account Access using Custom OIDC Provider

        Access AWS Cross Account resources using OIDC

        When employing ROSA, a common enterprise pattern involves establishing a cluster in a centralized AWS account while enabling development teams to manage services in their respective AWS accounts. This necessitates granting the ROSA cluster access to services residing in AWS accounts different from its own.

        Various approaches exist to address this challenge, but one straightforward method is to establish a secondary OIDC provider in the AWS account of the development team, enabling direct access for pods.

        Customizing the console URL in ROSA

        UPDATED DOCUMENT: This article has been moved to the official ROSA documentation here .

        Starting with ROSA 4.14.X, it is possible to modify the hostname and TLS certificate of component Routes post-install. These are the OAuth, Console, and Downloads routes. For example, the default ROSA console uses the built-in domain https://console-openshift-console.apps.<cluster_name>.<random>.p1.openshiftapps.com. You can now specify a custom domain, for example test.example.com, and the ROSA console will be available at a URL such as https://console-openshift-console.test.example.com. This guide will walk you through how to customize the console url for a ROSA Classic cluster (not tested on ROSA HCP yet).

        ROSA Break Glass Troubleshooting

        Background

        WARNING: this procedure should only be initiated by a member of the Black Belt team or someone incredibly familiar with ROSA as a whole. THIS IS NOT COMMON!!!

        This guide shows how to access ROSA instances in the situation that a break glass scenario is required in the account where ROSA is deployed. This procedure should only be performed under unusual circumstances like a failed provision in order to collect logs. This may be necessary if the control plane fails and SRE is unable to connect or do much to assist with troubleshooting.

        Patch token-refresher to use a cluster proxy

        Currently, if you deploy a ROSA or OSD cluster with a proxy, the token-refresher pod in the openshift-monitoring namespace will be in crashloopbackoff. There is an RFE open to resolve this, but until then this can affect the ability of the cluster to report telemetry and potentially update. This article provides a workaround on how to patch the token-refresher deployment until that RFE has been fixed using the patch-operator.

        Prerequisites

        • A logged in user with cluster-admin rights to a ROSA or OSD Cluster deployed using a cluster wide proxy

          Setup a VPN Connection into a PrivateLink ROSA Cluster with OpenVPN

          When you configure a Red Hat OpenShift on AWS (ROSA) cluster with a private link configuration, you will need connectivity to this private network in order to access your cluster. This guide will show you how to configute an AWS Client VPN connection so you won’t need to setup and configure Jump Boxes.

          Prerequisites

          • a private link ROSA Cluster - follow this guide to create a private ROSA Cluster
          • jq

          Set Envrionment Variables

          Start by setting environment variables that we will use to setup the VPN connection

          Prerequisites Checklist to Deploy ROSA Cluster with STS

          Background

          This is a quick checklist of prerequisites needed to spin up a classic Red Hat OpenShift Service on AWS (ROSA) cluster with STSexternal link (opens in new tab) . Note that this is a high level checklist and your implementation may vary.

          Before running the installation process, make sure that you deploy this from a machine that has access to:

          Connect to RDS database with STS from ROSA

          The Amazon Web Services Relational Database Service (AWS RDS) can be consumed from Red Hat OpenShift Service on AWS (ROSA) and authenticate to DB with Security Token Service (STS).

          This is a guide to quickly connect to RDS Database (Postgres engine) from ROSA.

          Amazon Web Services Relational Database Service

          Amazon Web Services Relational Database Service (AWS RDS) is a distributed relational database service by Amazon Web Services. It is designed to simplify setup, operation, and scaling of a relational database for use in applications. It supports differents database engines such as Amazon Aurora, MySQL, MariaDB, Oracle, Microsoft SQL Server, and PostgreSQL.

          Deploying ROSA PrivateLink Cluster with Ansible

          Background

          This guide shows an example of how to deploy a classic Red Hat OpenShift Services on AWS (ROSA) cluster with PrivateLinkexternal link (opens in new tab) with STSexternal link (opens in new tab) enabled using Ansibleexternal link (opens in new tab) playbook from our MOBB GitHub repositoryexternal link (opens in new tab) and makefilesexternal link (opens in new tab) to compile them. Note that this is an unofficial Red Hat guide and your implementation may vary.

          Using AWS Secrets Manager CSI on Red Hat OpenShift on AWS with STS

          The AWS Secrets and Configuration Provider (ASCP) provides a way to expose AWS Secrets as Kubernetes storage volumes. With the ASCP, you can store and manage your secrets in Secrets Manager and then retrieve them through your workloads running on ROSA or OSD.

          This is made even easier and more secure through the use of AWS STS and Kubernetes PodIdentity.

          Prerequisites

          Preparing Environment

          1. Validate that your cluster has STS

            What to consider when using Azure AD as IDP?

            Author: Ricardo Macedo Martinsexternal link (opens in new tab)

            May 24, 2023

            In this guide, we will discuss key considerations when using Azure Active Directory (AAD) as the Identity Provider (IDP) for your ARO or ROSA cluster. Below are some helpful references:

            Default Access for All Users in Azure Active Directory

            Once you set up AAD as the IDP for your cluster, it’s important to note that by default, all users in your Azure Active Directory instance will have access to the cluster. They can log in using their AAD credentials through the OpenShift Web Console endpoint:

            Deploy ACM Submariner for connect overlay networks ARO - ROSA clusters

            Submariner is an open source tool that can be used with Red Hat Advanced Cluster Management for Kubernetes to provide direct networking between pods and compatible multicluster service discovery across two or more Kubernetes clusters in your environment, either on-premises or in the cloud.

            This article describes how to deploy ACM Submariner for connecting overlay networks of ARO and ROSA clusters.

            NOTE: Submariner for connecting ARO and ROSA clusters only works from ACM 2.7 onwards!

            Deploy ACM Submariner for connect overlay networks of ROSA clusters

            Submariner is an open source tool that can be used with Red Hat Advanced Cluster Management for Kubernetes to provide direct networking between pods and compatible multicluster service discovery across two or more Kubernetes clusters in your environment, either on-premises or in the cloud.

            This article describes how to deploy ACM Submariner for connecting ROSA clusters overlay networks.

            NOTE: ACM Submariner for ROSA clusters only works with ACM 2.7 or newer!

            Enabling cross account EFS mounting

            The Amazon Web Services Elastic File System (AWS EFS) is a Network File System (NFS) that can be provisioned on Red Hat OpenShift Service on AWS clusters. With the release of OpenShift 4.10 the EFS CSI Driver is now GA and available.

            This is a guide to enable cross-account EFS mounting on ROSA.

            Important: Cross Account EFS is considered an advanced topic, and this article makes various assumptions as to knowledge of AWS terms and techniques across VPCs, Networking, IAM permissions and more.

            Enabling the AWS EFS CSI Driver Operator on ROSA

            The Amazon Web Services Elastic File System (AWS EFS) is a Network File System (NFS) that can be provisioned on Red Hat OpenShift Service on AWS clusters. With the release of OpenShift 4.10 the EFS CSI Driver is now GA and available.

            This is a guide to quickly enable the EFS Operator on ROSA to a Red Hat OpenShift on AWS (ROSA) cluster with STS enabled.

            Note: The official supported installation instructions for the EFS CSI Driver on ROSA are available here .

            Azure DevOps with Managed OpenShift

            Author: Kevin Collins

            Last edited: 03/14/2023

            Adopted from Hosting an Azure Pipelines Build Agent in OpenShift and Kevin Chung Azure Pipelines OpenShift exampleexternal link (opens in new tab)

            Azure DevOps is a very popular DevOps tool that has a host of features including the ability for developers to create CI/CD pipelines.

            In this document, I will show you how to connect your Managed OpenShift Cluster to Azure DevOps end-to-end including running the pipeline build process in the cluster, setting up the OpenShift internal image registry to store the images, and then finally deploy a sample application. To demonstrate the flexibility of Azure DevOps, I will be deploying to a ROSA cluster, however the same procudures will apply to if you choose to deploy to any other OCP Cluster.

            Assign Consistent Egress IP for External Traffic

            It may be desirable to assign a consistent IP address for traffic that leaves the cluster when configuring items such as security groups or other sorts of security controls which require an IP-based configuration. By default, Kubernetes via the OVN-Kubernetes CNI will assign random IP addresses from a pool which will make configuring security lockdowns unpredictable or unnecessarily open. This guide shows you how to configure a set of predictable IP addresses for egress cluster traffic to meet common security standards and guidance and other potential use cases.

            ROSA with Nvidia GPU Workloads

            ROSA guide to running Nvidia GPU workloads.

            Prerequisites

            • ROSA Cluster (4.14+)
            • rosa cli #logged-in
            • oc cli #logged-in-cluster-admin
            • jq

            If you need to install a ROSA cluster, please read our ROSA Quickstart Guide , or better yet Use Terraform to create an HCP Cluster .

            Enter the oc login command, username, and password from the output of the previous command:

            Example login:

            Linux:

            ROSA with Nvidia GPU Workloads - Manual

            This is a guide to install GPU on ROSA cluster manually, which is an alternative to our Helm chart guide .

            Prerequisites

            • ROSA cluster (4.14+)
              • You can install a Classic version using CLI or an HCP one using Terraform .
              • Please be sure you are logged in to the cluster with a cluster admin access.
            • rosa cli
            • oc cli

            1. Setting up GPU machine pools

            In this tutorial, I’m using g5.4xlarge node for the GPU machine pools with auto-scaling enabled up to 4 nodes. Please replace your-cluster-name with the name of your cluster.

            External DNS for ROSA Custom Domain

            Configuring the Custom Domain Operator requires a wildcard CNAME DNS record in your Route53 Hosted Zone. If you do not wish to use a wildcard record, you can use the External DNS Operator to create individual entries for routes.

            This document will guide you through deploying and configuring the External DNS Operator with a Custom Domain in ROSA.

            Important Note: The ExternalDNS Operator does not support STS yet and uses long lived IAM credentials. This guide will be updated once STS is supported.

            VPC and Subnet IP Address Considerations with ROSA

            VPC and Subnet IP Address Considerations with ROSA

            ROSA clusters can be built to be highly available using the fundamental capability that underlies most HA configurations on AWS: Availability Zones. By spreading the resources of a cluster across three separate (but regionally co-located) datacenters, ROSA users can ensure the cluster continues to run even if an entire AWS AZ goes down.

            This capability comes with a few challenges and considerations around IP addressing that this article will attempt to explain and provide options and best practices around.

            AWS Load Balancer Operator On ROSA

            AWS Load Balancer Controllerexternal link (opens in new tab) is a controller to help manage Elastic Load Balancers for a Kubernetes cluster.

            Compared with default AWS In Tree Provider, this controller is actively developed with advanced annotations for both ALBexternal link (opens in new tab) and NLBexternal link (opens in new tab) . Some advanced usecases are:

            Dynamic Certificates for ROSA Custom Domain

            There may be situations when you prefer not to use wild-card certificates. This ROSA guide talks about certificate management with cert-manager and letsencrypt, to dynamically issue certificates to routes created on a custom domain that’s hosted on AWS Route53.

            Prerequisites

            • A Red Hat OpenShift on AWS (ROSA) cluster
            • The oc CLI #logged in.
            • The aws CLI #logged in.
            • The rosa CLI #logged in.
            • jq
            • gettext
            • A Public Route53 Hosted Zone, and the related Domain to use.

            Set up environment

            1. Export few environment variables

              Deploying Red Hat Advanced Cluster Security in ARO/ROSA

              This document is based in the RHACS workshopexternal link (opens in new tab) and in the RHACS official documentation .

              Prerequisites

              1. An ARO cluster or a ROSA cluster .

              Set up the OpenShift CLI (oc)

              1. Download the OS specific OpenShift CLI from Red Hat

              2. Unzip the downloaded file on your local machine

              3. Place the extracted oc executable in your OS path or local directory

                Configure a load balancer service to use a static public IP

                This guide demonstrates how to create and assign a static public IP address to an OpenShift service in Azure Red Hat OpenShift (ARO). By default, the public IP address assigned to an OpenShift service with a type of LoadBalancer created by an ARO cluster is only valid for the lifespan of that resource. If you delete the OpenShift service, the associated load balancer and IP address are also deleted. If you want to assign a specific IP address or retain an IP address for redeployed OpenShift services, you can create and use a static public IP address.

                Verify Permissions for ROSA STS Deployment

                To proceed with the deployment of a ROSA cluster, an account must support the required roles and permissions. AWS Service Control Policies (SCPs) cannot block the API calls made by the installer or operator roles.

                Details about the IAM resources required for an STS-enabled installation of ROSA can be found here: https://docs.openshift.com/rosa/rosa_architecture/rosa-sts-about-iam-resources.html

                This guide is validated for ROSA v4.11.X.

                Prerequisites

                Verify ROSA Permissions

                To verify the permissions required for ROSA we can run the script below without ever creating any AWS resources.

                STS OIDC in ROSA : How it works!

                If you prefer a more visual medium, you can watch this video on YouTubeexternal link (opens in new tab) .

                This short video talks about how the STSexternal link (opens in new tab) OIDC flow work in ROSA (Red Hat OpenShift Service on AWS).

                Security Reference Architecture for ROSA

                The Security Reference Architecture for ROSA is a set of guidelines for deploying Red Hat OpenShift on AWS (ROSA) clusters to support high-security production workloads that align with Red Hat and AWS best practices.

                This overall architectural guidance compliments detailed, specific recommendations for AWS services and Red Hat OpenShift Container Platform.

                The Security Reference Architecture (SRA) for ROSA is a living document and is updated periodically based on new feature releases, customer feedback and evolving security best practices.

                Configure Microsoft Entra ID as an OIDC identity provider for ROSA/OSD

                This guide demonstrates how to configure Azure AD as the cluster identity provider in Red Hat OpenShift Service on AWS (ROSA). This guide will walk through the creation of an Azure Active Directory (Azure AD) application and configure Red Hat OpenShift Service on AWS (ROSA) to authenticate using Azure AD.

                This guide will walk through the following steps:

                1. Register a new application in Azure AD for authentication.
                2. Configure the application registration in Azure AD to include optional and group claims in tokens.
                3. Configure the OpenShift cluster to use Azure AD as the identity provider.
                4. Grant additional permissions to individual groups.

                Before you Begin

                Create a set of security groups and assign users by following the Microsoft documentationexternal link (opens in new tab) .

                Deploying OpenShift API for Data Protection on a ROSA cluster

                Prerequisites

                Getting Started

                1. Create the following environment variables

                  Change the cluster name to match your ROSA cluster and ensure you’re logged into the cluster as an Administrator. Ensure all fields are outputted correctly before moving on.

                Prepare AWS Account

                1. Create an IAM Policy to allow for S3 Access

                  Custom AlertManager in ROSA 4.9.x

                  This page is deprecated. In order to get the best experience for custom alerting in ROSA, please upgrade your cluster to to 4.12 and follow the newer documentation.

                  ROSA 4.9.x introduces a new way to provide custom AlertManager configuration to receive alerts from User Workload Management.

                  The OpenShift Administrator can use the Prometheus Operator to create a custom AlertManager resource and then use the AlertManagerConfig resource to configure User Workload Monitoring to use the custom AlertManager.

                  Configuring the Cluster Log Forwarder for CloudWatch Logs and STS

                  This guide shows how to deploy the Cluster Log Forwarder operator and configure it to use STS authentication to forward logs to CloudWatch.

                  Prerequisites

                  • A ROSA cluster (configured with STS)
                  • The jq cli command
                  • The aws cli command

                  Environment Setup

                  1. Configure the following environment variables

                    Change the cluster name to match your ROSA cluster and ensure you’re logged into the cluster as an Administrator. Ensure all fields are outputted correctly before moving on.

                    Stop default router from serving custom domain routes

                    Note: This page is only valid for clusters using the Custom Domain Operator (CDO), which are ROSA clusters prior to version 4.14

                    OSD and ROSA supports custom domain operator to serve application custom domain, which provisions openshift ingress controller and cloud load balancers. However, when a route with custom domain is created, both default router and custom domain router serve routes. This article describes how to use route labels to stop default router from serving custom domain routes.

                    Using AWS Controllers for Kubernetes (ACK) on ROSA

                    AWS Controllers for Kubernetesexternal link (opens in new tab) (ACK) lets you define and use AWS service resources directly from Kubernetes. With ACK, you can take advantage of AWS-managed services for your Kubernetes applications without needing to define resources outside of the cluster or run services that provide supporting capabilities like databases or message queues within the cluster.

                    Create IAM user and Policy

                    Notes: These are sample commands. Please fill in your own resource parameters E.g. ARN

                    • Create the policy
                    • Create a user and access key and attach the policy

                    Notes: Save access key id and key for later usage

                    Create STS Assume Role

                    About AWS STS and Assume Roleexternal link (opens in new tab)

                    Notes: These are sample commands. Please fill in your own resource parameters E.g. ARN

Interested in contributing to these docs?

Collaboration drives progress. Help improve our documentation The Red Hat Way.

GitHub
Red Hat logo LinkedIn YouTube Facebook Twitter

Products

Tools

Try, buy & sell

Communicate

About Red Hat

We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Subscribe to our newsletter, Red Hat Shares

Sign up now © 2026 Red Hat