Cloud Experts Documentation

AWS

Deploying ROSA in STS mode

Tip The official documentation for installing a ROSA cluster in STS mode can be found here .

Quick Introduction by Ryan Niksch (AWS) and Shaozen Ding (Red Hat) on YouTubeexternal link (opens in new tab)

STS allows us to deploy ROSA without needing a ROSA admin account, instead it uses roles and policies with Amazon STS (secure token service) to gain access to the AWS resources needed to install and operate the cluster.

Ingress to ROSA Virt VMs with Certificate-Based Site-to-Site (S2S) IPsec VPN and Libreswan

Introduction

This solution uses a Site-to-Site (S2S) VPNexternal link (opens in new tab) as a mechanism in OpenShift Virtualization on ROSA to establish an IP route between the virtual overlay network that VMs are attached to, and the VPC outside your clusterexternal link (opens in new tab) without the need for NAT or load balancers. OpenShift Virtualization provides several built-in features to plug VMs directly into outside networks when deployed on-premises , but these depend upon mechanisms that are not exposed in cloud provider networks. This solution should be considered a stop-gap until there is a native OpenShift Virtualization feature to plug VMs into cloud provider networks.

Deploy ROSA + Nvidia GPU + RHOAI with Automation

Getting Red Hat OpenShift AI up and running with NVIDIA GPUs on a Red Hat OpenShift Service on AWS (ROSA) cluster can involve a series of detailed steps, from installing various operators to managing dependencies. While manageable, this process can be time-consuming when you’re eager to start leveraging OpenShift AI for your projects.

This guide and its accompanying Git repository are designed to streamline your setup significantly. We focus on getting you productive faster by using Terraform to deploy a ROSA cluster with GPUs from the start. From there, Ansible scripts take over, automating the deployment and configuration of all necessary operators for both NVIDIA GPUs and Red Hat OpenShift AI. This means less manual configuration for you and more time spent on what matters: innovating with AI.

Using a Private IngressController with CloudFront on a ROSA Cluster

AWS CloudFront is a great choice for a Content Delivery Network in front of your ROSA cluster. In many situations, it may be useful to ensure that traffic to Routes within your cluster must come from your CloudFront Distribution rather than being able to bypass it (for example, if a WAF is attached to the CloudFront Distribution).

This guide shows how to use the CloudFront VPC Origin feature in combination with a Network Load Balancer (NLB), and a private secondary IngressController type NLB, providing a path from the Internet to your applications that only goes through CloudFront.

Accessing the ROSA HCP API Server from a Different AWS Account

Introduction

You can create a ROSA HCP cluster in one AWS account and configure it to allow access from a different AWS account using the oc command. This guide walks you through the actual AWS setup.

pic1 Note: AWS environments vary, so consider this as one possible setup.

Prerequisites

A ROSA HCP cluster has been already deployed in AWS Account-A, and the following AWS resources are available. ROSA HCP 4.19.0 was used for this guide. pic2

Deploy ROSA with Red Hat Advanced Cluster Management for Kubernetes

In the dynamic world of cloud-native development, efficiently managing Kubernetes clusters across diverse environments is paramount. This blog post dives into a powerful combination: deploying Red Hat OpenShift Service on AWS (ROSA) Hosted Control Planes (HCP) clusters, orchestrated and governed by Red Hat Advanced Cluster Management for Kubernetes (RHACM). This approach offers a compelling suite of benefits, including significant cost reductions by offloading control plane management to Red Hat, accelerated cluster provisioning times, and enhanced operational efficiency through a centralized management plane. By leveraging ROSA HCP with RHACM, organizations can achieve a more streamlined, secure, and scalable Kubernetes footprint on AWS, allowing teams to focus more on innovation and less on infrastructure overhead.

Integrating Service Mesh into a ROSA Cluster

This is a simple guide to integrate Red Hat OpenShift Service Mesh into your ROSA cluster. In this scenario, we will install Service Mesh using a custom domain (optional) and expose an app to test it. The first half of the guide will be integrating Service Mesh 2.x and second half will be integrating Service Mesh 3.x .

Prerequisites

Set up environment

Install the necessary operators, i.e. Elasticsearch (optional), Jaeger (distributed tracing platform), Kiali, and Service Mesh (2 or 3 depends on your use case) from OpenShift console.

Adding a Private Ingress Controller and a Public ALB to a ROSA Cluster

Starting with OpenShift 4.14, Red Hat OpenShift Service on AWS (ROSA) supports adding additional Ingress Controllers which can be used to configure a custom domain on a ROSA cluster. This guide shows how to leverage this feature to create a complete routing solution with both a private Ingress Controller (which creates an Network Load Balancer (NLB)) and a public Application Load Balancer (ALB) in front of it, providing a path from the internet to your applications.

ROSA - Federating Metrics to AWS Prometheus

Federating Metrics from ROSA is a bit tricky as the cluster metrics require pulling from its /federated endpoint while the user workload metrics require using the prometheus remoteWrite configuration.

This guide will walk you through using the MOBB Helm Chart to deploy the necessary agents to federate the metrics into AWS Prometheus and then use Grafana to visualize those metrics.

As a bonus it will set up a CloudWatch datasource to view any metrics or logs you have in Cloud Watch.

Using local-zones in ROSA Classic

This guide walks through setting up a local-zone in an existing ROSA Classic cluster. Use this approach when you have latency requirements that can be reduced when using a local zone. Since you are not using the default ingress, you will not be able to use the router strategy the cluster has.

Prerequisites

ROSA Classic Cluster with AWS Load Balancer Operator already installed:

Local Zone validations and configuration in your AWS Account:

Configuring Microsoft Entra ID as an external authentication provider

You can set up Red Hat OpenShift Service on AWS (ROSA) with hosted control planes (HCP) to use an external OpenID Connect (OIDC) identity provider for authentication instead of the built-in OpenShift OAuth server. While the built-in OAuth server supports various identity providers, its capabilities are limited. By integrating external OIDC identity providers directly with ROSA with HCP, you can enable machine-to-machine workflows, like CLI access, and gain features unavailable with the built-in OAuth server.

Securely exposing an application on a private ROSA cluser with an AWS Network Load Balancer

Overview

Red Hat strongly recommends creating a private ROSA cluster with no inbound Internet connectivity, isolating both the cluster API and hosted applications from external access. This configuration is a key part of a multi-layered security strategy to protect clusters and workloads from external threats.

However, some applications may require Internet access to support external users or partners. Even with a private cluster, you can securely expose these applications through various methods. For example, a TCP-based application like Kafka may need connectivity to allow partners on the Internet to send or receive data. This document provides guidance on one specific method to securely expose a TCP-based application while maintaining the security of a private cluster.

Securely exposing an application on a private ROSA cluser with an AWS Network Load Balancer

Continuation of Securely exposing an application on a private ROSA cluser with an AWS Network Load Balancer

These instructions go through setting up an additional VPC as part of the overall blog. If you already have a VPC that you would like to use, you can skip these instructions.

Looking at the overall architecture, this section will setup everything in the blue box.

Create a public VPC

The VPC we will be creating will have a VPC with a private subnet, a public subnet where the Network Load Balancer will sit, an Internet Gateway and a Nat Gateway.

Securely exposing an application on a private ROSA cluser with an AWS Network Load Balancer - Jump Host

Continuation of Securely exposing an application on a private ROSA cluser with an AWS Network Load Balancer

These instructions go through setting up a jump host to connect to the private rosa cluster.

Note: the guide assumes you have set envirionment variables as described in the parent guide.

Create a jumphost instance using the AWS CLI

Create an additional Security Group for the jumphost

Grab the Security Group Id generated in the previous step

Deploying and Running Ollama and Open WebUI in a ROSA Cluster with GPUs

Red Hat OpenShift Service on AWS (ROSA) provides a managed OpenShift environment that can leverage AWS GPU instances. This guide will walk you through deploying Ollama and OpenWebUI on ROSA using instances with GPU for inferences.

Prerequisites

Set up GPU-enabled Machine Pool

First we need to check availability of our instance type used here (g4dn.xlarge), it should be in same region of the cluster. Note you can use also Graviton based instance (ARM64) like g5g* but only on HCP 4.16+ cluster.

Configuring ROSA with HCP Private Cluster API Access

With ROSA with HCP private clusters, the AWS PrivateLink endpoint exposed in the customer’s VPC has a default security group. This security group has access to the PrivateLink endpoint that is limited to only those resources that exist within the VPC or resources that are present with an IP address associated with the VPC CIDR range. In order to grant access to any entities outside of the VPC, through VPC peering and transit gateway, you must create and attach another security group to the PrivateLink endpoint to grant the necessary access.

Maximo Application Suite on ROSA ( Red Hat OpenShift on AWS )

IBM Maximo Application Suite (MAS) is a set of applications for asset monitoring, management, predictive maintenance and reliability planning. When combined with Red Hat OpenShift on AWS ( ROSA ), this frees up your Maximo and operations team to focus on what is important to them ( Maximo ) rather than having to worry about managing and building clusters.

This document outlines how to get quickly get started with ROSA and installing Maximo all through automation.

Migrating EC2 Instances to OpenShift Virtualization

Red Hat OpenShift Service on AWS (ROSA) provides a managed OpenShift environment that can run virtualized workloads using OpenShift Virtualization. This guide will walk you through migrating an existing EC2 instance to OpenShift Virtualization by exporting it to S3, syncing to EFS, and importing as a VM.

Prerequisites

Clone the Repository

First, clone the repository and navigate to the project directory:

Creating a ROSA cluster in AWS GovCloud

This guide outlines the procedure for creating a ROSA cluster in AWS GovCloud. There are some key differences between the ROSA offerings in AWS GovCloud and AWS Commercial. They’re outlined in detail in the AWS documentation hereexternal link (opens in new tab) , but a few requirements in GovCloud that are worth highlighting:

Prerequisites

Create VPC and Subnets

In this guide, we’ll use Terraform to create a VPC to house our cluster, and we’ll opt for a Single-AZ configuration for simplicity. We’ll also create an EC2 jumphost to aid in accessing our cluster once it comes up. Before running it, you’ll need to ensure your AWS CLI is authenticated to a government region in AWS (us-gov-west-1 or us-gov-east-1).

Migrating ROSA Ingress Controllers from a CLB to NLB

This guide will show you how to migrate the default Red Hat OpenShift Service on AWS (ROSA) IngressController from an AWS Classic Load Balancer to an AWS Network Load Balancer.

In version 4.14 of ROSA, Red Hat introduced changes to IngressControllers to give customers more control over their workloads and configuration. The operation below requires a cluster running version 4.14 or higher. To request early access to this additional functionality in version 4.13, please contact Red Hat support and open a case to request access .

Configuring AWS CLB Access Logging

This guide will show you how to enable access logging on the default Classic Load Balancer ingress controller used in Red Hat OpenShift Service on AWS (ROSA) version 4.13 and earlier.

Prerequisites

S3 Bucket Creation

  1. Run the following command, making sure to update the name of the S3 bucket you wish to create and the account number of the Elastic Load Balancing root account (this is not your AWS account):

    Add an Ingress Controller to a ROSA Cluster and optionally with a custom domain.

    Starting with OpenShift 4.14, ROSA supports adding additional Ingress Controllers which can use used to configure a custom domain on a ROSA cluster without having to use the now deprecated Custom Domain Operator. This guide shows how to add an additional Ingress Controller ( public or private ) to a ROSA cluster and optionally also configuring a custom domain.

    Prerequisites

    • A Red Hat OpenShift on AWS (ROSA) cluster
    • The oc CLI #logged in.
    • The aws CLI #logged in.
    • The rosa CLI #logged in.
    • (optional) A Public Route53 Hosted Zone, and the related Domain to use.

    Set up environment

    1. Export few environment variables

    Important: The variables below can be customized to fit your needs for your ingress controller.

    Cross-account Access using Custom OIDC Provider

    Access AWS Cross Account resources using OIDC

    When employing ROSA, a common enterprise pattern involves establishing a cluster in a centralized AWS account while enabling development teams to manage services in their respective AWS accounts. This necessitates granting the ROSA cluster access to services residing in AWS accounts different from its own.

    Various approaches exist to address this challenge, but one straightforward method is to establish a secondary OIDC provider in the AWS account of the development team, enabling direct access for pods.

    Customizing the console URL in ROSA

    UPDATED DOCUMENT: This article has been moved to the official ROSA documentation here .

    Starting with ROSA 4.14.X, it is possible to modify the hostname and TLS certificate of component Routes post-install. These are the OAuth, Console, and Downloads routes. For example, the default ROSA console uses the built-in domain https://console-openshift-console.apps.<cluster_name>.<random>.p1.openshiftapps.com. You can now specify a custom domain, for example test.example.com, and the ROSA console will be available at a URL such as https://console-openshift-console.test.example.com. This guide will walk you through how to customize the console url for a ROSA Classic cluster (not tested on ROSA HCP yet).

    Setup a VPN Connection into a PrivateLink ROSA Cluster with OpenVPN

    When you configure a Red Hat OpenShift on AWS (ROSA) cluster with a private link configuration, you will need connectivity to this private network in order to access your cluster. This guide will show you how to configute an AWS Client VPN connection so you won’t need to setup and configure Jump Boxes.

    Prerequisites

    • a private link ROSA Cluster - follow this guide to create a private ROSA Cluster
    • jq

    Set Envrionment Variables

    Start by setting environment variables that we will use to setup the VPN connection

    Connect to RDS database with STS from ROSA

    The Amazon Web Services Relational Database Service (AWS RDS) can be consumed from Red Hat OpenShift Service on AWS (ROSA) and authenticate to DB with Security Token Service (STS).

    This is a guide to quickly connect to RDS Database (Postgres engine) from ROSA.

    Amazon Web Services Relational Database Service

    Amazon Web Services Relational Database Service (AWS RDS) is a distributed relational database service by Amazon Web Services. It is designed to simplify setup, operation, and scaling of a relational database for use in applications. It supports differents database engines such as Amazon Aurora, MySQL, MariaDB, Oracle, Microsoft SQL Server, and PostgreSQL.

    Using AWS Secrets Manager CSI on Red Hat OpenShift on AWS with STS

    The AWS Secrets and Configuration Provider (ASCP) provides a way to expose AWS Secrets as Kubernetes storage volumes. With the ASCP, you can store and manage your secrets in Secrets Manager and then retrieve them through your workloads running on ROSA or OSD.

    This is made even easier and more secure through the use of AWS STS and Kubernetes PodIdentity.

    Prerequisites

    Preparing Environment

    1. Validate that your cluster has STS

      Enabling cross account EFS mounting

      The Amazon Web Services Elastic File System (AWS EFS) is a Network File System (NFS) that can be provisioned on Red Hat OpenShift Service on AWS clusters. With the release of OpenShift 4.10 the EFS CSI Driver is now GA and available.

      This is a guide to enable cross-account EFS mounting on ROSA.

      Important: Cross Account EFS is considered an advanced topic, and this article makes various assumptions as to knowledge of AWS terms and techniques across VPCs, Networking, IAM permissions and more.

      Enabling the AWS EFS CSI Driver Operator on ROSA

      The Amazon Web Services Elastic File System (AWS EFS) is a Network File System (NFS) that can be provisioned on Red Hat OpenShift Service on AWS clusters. With the release of OpenShift 4.10 the EFS CSI Driver is now GA and available.

      This is a guide to quickly enable the EFS Operator on ROSA to a Red Hat OpenShift on AWS (ROSA) cluster with STS enabled.

      Note: The official supported installation instructions for the EFS CSI Driver on ROSA are available here .

      ROSA with Nvidia GPU Workloads

      ROSA guide to running Nvidia GPU workloads.

      Prerequisites

      • ROSA Cluster (4.14+)
      • rosa cli #logged-in
      • oc cli #logged-in-cluster-admin
      • jq

      If you need to install a ROSA cluster, please read our ROSA Quickstart Guide , or better yet Use Terraform to create an HCP Cluster .

      Enter the oc login command, username, and password from the output of the previous command:

      Example login:

      Linux:

      ROSA with Nvidia GPU Workloads - Manual

      This is a guide to install GPU on ROSA cluster manually, which is an alternative to our Helm chart guide .

      Prerequisites

      • ROSA cluster (4.14+)
        • You can install a Classic version using CLI or an HCP one using Terraform .
        • Please be sure you are logged in to the cluster with a cluster admin access.
      • rosa cli
      • oc cli

      1. Setting up GPU machine pools

      In this tutorial, I’m using g5.4xlarge node for the GPU machine pools with auto-scaling enabled up to 4 nodes. Please replace your-cluster-name with the name of your cluster.

      External DNS for ROSA Custom Domain

      Configuring the Custom Domain Operator requires a wildcard CNAME DNS record in your Route53 Hosted Zone. If you do not wish to use a wildcard record, you can use the External DNS Operator to create individual entries for routes.

      This document will guide you through deploying and configuring the External DNS Operator with a Custom Domain in ROSA.

      Important Note: The ExternalDNS Operator does not support STS yet and uses long lived IAM credentials. This guide will be updated once STS is supported.

      AWS Load Balancer Operator On ROSA

      AWS Load Balancer Controllerexternal link (opens in new tab) is a controller to help manage Elastic Load Balancers for a Kubernetes cluster.

      Compared with default AWS In Tree Provider, this controller is actively developed with advanced annotations for both ALBexternal link (opens in new tab) and NLBexternal link (opens in new tab) . Some advanced usecases are:

      Dynamic Certificates for ROSA Custom Domain

      There may be situations when you prefer not to use wild-card certificates. This ROSA guide talks about certificate management with cert-manager and letsencrypt, to dynamically issue certificates to routes created on a custom domain that’s hosted on AWS Route53.

      Prerequisites

      • A Red Hat OpenShift on AWS (ROSA) cluster
      • The oc CLI #logged in.
      • The aws CLI #logged in.
      • The rosa CLI #logged in.
      • jq
      • gettext
      • A Public Route53 Hosted Zone, and the related Domain to use.

      Set up environment

      1. Export few environment variables

        Configure a load balancer service to use a static public IP

        This guide demonstrates how to create and assign a static public IP address to an OpenShift service in Azure Red Hat OpenShift (ARO). By default, the public IP address assigned to an OpenShift service with a type of LoadBalancer created by an ARO cluster is only valid for the lifespan of that resource. If you delete the OpenShift service, the associated load balancer and IP address are also deleted. If you want to assign a specific IP address or retain an IP address for redeployed OpenShift services, you can create and use a static public IP address.

        Verify Permissions for ROSA STS Deployment

        To proceed with the deployment of a ROSA cluster, an account must support the required roles and permissions. AWS Service Control Policies (SCPs) cannot block the API calls made by the installer or operator roles.

        Details about the IAM resources required for an STS-enabled installation of ROSA can be found here: https://docs.openshift.com/rosa/rosa_architecture/rosa-sts-about-iam-resources.html

        This guide is validated for ROSA v4.11.X.

        Prerequisites

        Verify ROSA Permissions

        To verify the permissions required for ROSA we can run the script below without ever creating any AWS resources.

        STS OIDC in ROSA : How it works!

        If you prefer a more visual medium, you can watch this video on YouTubeexternal link (opens in new tab) .

        This short video talks about how the STSexternal link (opens in new tab) OIDC flow work in ROSA (Red Hat OpenShift Service on AWS).

        Security Reference Architecture for ROSA

        The Security Reference Architecture for ROSA is a set of guidelines for deploying Red Hat OpenShift on AWS (ROSA) clusters to support high-security production workloads that align with Red Hat and AWS best practices.

        This overall architectural guidance compliments detailed, specific recommendations for AWS services and Red Hat OpenShift Container Platform.

        The Security Reference Architecture (SRA) for ROSA is a living document and is updated periodically based on new feature releases, customer feedback and evolving security best practices.

        Deploying OpenShift API for Data Protection on a ROSA cluster

        Prerequisites

        Getting Started

        1. Create the following environment variables

          Change the cluster name to match your ROSA cluster and ensure you’re logged into the cluster as an Administrator. Ensure all fields are outputted correctly before moving on.

        Prepare AWS Account

        1. Create an IAM Policy to allow for S3 Access

          Custom AlertManager in ROSA 4.9.x

          This page is deprecated. In order to get the best experience for custom alerting in ROSA, please upgrade your cluster to to 4.12 and follow the newer documentation.

          ROSA 4.9.x introduces a new way to provide custom AlertManager configuration to receive alerts from User Workload Management.

          The OpenShift Administrator can use the Prometheus Operator to create a custom AlertManager resource and then use the AlertManagerConfig resource to configure User Workload Monitoring to use the custom AlertManager.

          Configuring the Cluster Log Forwarder for CloudWatch Logs and STS

          This guide shows how to deploy the Cluster Log Forwarder operator and configure it to use STS authentication to forward logs to CloudWatch.

          Prerequisites

          • A ROSA cluster (configured with STS)
          • The jq cli command
          • The aws cli command

          Environment Setup

          1. Configure the following environment variables

            Change the cluster name to match your ROSA cluster and ensure you’re logged into the cluster as an Administrator. Ensure all fields are outputted correctly before moving on.

            Using AWS Controllers for Kubernetes (ACK) on ROSA

            AWS Controllers for Kubernetesexternal link (opens in new tab) (ACK) lets you define and use AWS service resources directly from Kubernetes. With ACK, you can take advantage of AWS-managed services for your Kubernetes applications without needing to define resources outside of the cluster or run services that provide supporting capabilities like databases or message queues within the cluster.

            Create IAM user and Policy

            Notes: These are sample commands. Please fill in your own resource parameters E.g. ARN

            • Create the policy
            • Create a user and access key and attach the policy

            Notes: Save access key id and key for later usage

            Create STS Assume Role

            About AWS STS and Assume Roleexternal link (opens in new tab)

            Notes: These are sample commands. Please fill in your own resource parameters E.g. ARN

Interested in contributing to these docs?

Collaboration drives progress. Help improve our documentation The Red Hat Way.

GitHub
Red Hat logo LinkedIn YouTube Facebook Twitter

Products

Tools

Try, buy & sell

Communicate

About Red Hat

We’re the world’s leading provider of enterprise open source solutions—including Linux, cloud, container, and Kubernetes. We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Subscribe to our newsletter, Red Hat Shares

Sign up now © 2026 Red Hat