Below is a complete, copy-pasteable, step-by-step guide to put OIDC (Amazon Cognito) + ALB Ingress + gRPC (EKS) behind your DNS with TLS. I’ve included all commands, manifests, and verification steps. I also note exactly where to substitute your values.
Assumptions you already have:
- An EKS cluster (your output earlier shows
v1.32.6-eks-*
),kubectl
logged in.- AWS Load Balancer Controller installed and healthy.
- Region: ap-northeast-1 (Tokyo).
- Desired host:
raj.dev.aws.k8.rajesh.com
.- Namespace we’ll use:
grpcdemo
.
Citations for critical behaviors (ALB OIDC annotations, callback URL, gRPC with ALB) are embedded where they matter. (Kubernetes SIGs)
0) Quick preflight checks (do these once)
# AWS CLI identity and region
aws sts get-caller-identity
aws configure get region
# EKS context
kubectl version --short
kubectl get nodes -o wide
# AWS Load Balancer Controller is present and healthy
kubectl -n kube-system get deploy aws-load-balancer-controller
kubectl -n kube-system get pods -l app.kubernetes.io/name=aws-load-balancer-controller -o wide
Code language: PHP (php)
✅ If the controller isn’t installed/healthy, fix that first (Helm chart install per AWS docs). The controller must be able to read Ingresses and (for OIDC) read the OIDC client Secret. If your install lacks secret permissions, we’ll add a RoleBinding in Step 5. (There’s a known symptom: “cannot get resource secrets” for the controller.) (GitHub)
1) DNS: create the host name for your ALB
We’ll ultimately point raj.dev.aws.evp.drivemode.com
to the ALB DNS name (e.g., k8s-...ap-northeast-1.elb.amazonaws.com
) provisioned by your Ingress.
If your hosted zone is in Route 53, prepare this:
# (Optional) Identify your public hosted zone ID
aws route53 list-hosted-zones | jq -r '.HostedZones[] | "\(.Name) \(.Id)"'
Code language: PHP (php)
We’ll come back after the ALB is created (Step 4) and add an A/ALIAS to the ALB hostname. Until then, ensure your domain is owned/managed by you.
2) TLS (ACM): request/validate a certificate for your host
ALB in ap-northeast-1 must use an ACM cert in the same region.
# Replace with your exact host; you can also include a wildcard SAN if useful
CERT_ARN=$(aws acm request-certificate \
--domain-name raj.dev.aws.evp.drivemode.com \
--validation-method DNS \
--region ap-northeast-1 \
--query CertificateArn --output text)
echo "CERT_ARN=$CERT_ARN"
# Find the DNS validation record to create
aws acm describe-certificate --certificate-arn "$CERT_ARN" --region ap-northeast-1 \
--query 'Certificate.DomainValidationOptions[].ResourceRecord' --output table
Code language: PHP (php)
Create the returned CNAME in your DNS (Route 53 example):
# Prepare a change batch file (edit with the Name/Value shown in the previous command)
cat > /tmp/acm-validate.json <<'JSON'
{
"Comment": "ACM validation CNAME",
"Changes": [{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "_<from_describe>.raj.dev.aws.evp.drivemode.com.",
"Type": "CNAME",
"TTL": 300,
"ResourceRecords": [{ "Value": "_<target_from_describe>.acm-validations.aws." }]
}
}]
}
JSON
# Apply to your hosted zone (substitute your HostedZoneId)
aws route53 change-resource-record-sets \
--hosted-zone-id Z1234567890ABC \
--change-batch file:///tmp/acm-validate.json
# Wait for ACM to validate
aws acm wait certificate-validated --certificate-arn "$CERT_ARN" --region ap-northeast-1
Code language: PHP (php)
Keep $CERT_ARN
handy—we’ll use it in the Ingress.
3) Amazon Cognito: Create OIDC User Pool + Domain + App Client
ALB OIDC requires a “confidential” app client (i.e., with client secret) and your callback URL must behttps://<your-host>/oauth2/idpresponse
(ALB hard-codes this path). (AWS Documentation)
3.1 Create a user pool (CLI)
# Create a minimal user pool
POOL_ID=$(aws cognito-idp create-user-pool \
--pool-name "grpcdemo-pool" \
--policies '{"PasswordPolicy":{"MinimumLength":8,"RequireUppercase":true,"RequireLowercase":true,"RequireNumbers":true,"RequireSymbols":false}}' \
--region ap-northeast-1 \
--query 'UserPool.Id' --output text)
echo "POOL_ID=$POOL_ID"
Code language: PHP (php)
3.2 Create a user pool domain (Hosted UI)
Pick a unique domain prefix (all lowercase). For example: raj-evp-demo
.
DOMAIN_PREFIX=raj-evp-demo
aws cognito-idp create-user-pool-domain \
--domain $DOMAIN_PREFIX \
--user-pool-id $POOL_ID \
--region ap-northeast-1
Code language: PHP (php)
Your Hosted UI base will be:
https://<DOMAIN_PREFIX>.auth.ap-northeast-1.amazoncognito.com
Code language: HTML, XML (xml)
3.3 Create an app client (with secret), enable Code flow and scopes
CALLBACK_URL="https://raj.dev.aws.evp.drivemode.com/oauth2/idpresponse" # critical!
SIGNOUT_URL="https://raj.dev.aws.evp.drivemode.com/"
APP_OUT=$(aws cognito-idp create-user-pool-client \
--user-pool-id $POOL_ID \
--client-name grpcdemo-app \
--generate-secret \
--allowed-o-auth-flows code \
--allowed-o-auth-scopes "openid" "email" "profile" \
--allowed-o-auth-flows-user-pool-client \
--callback-urls "$CALLBACK_URL" \
--logout-urls "$SIGNOUT_URL" \
--region ap-northeast-1 \
--query 'UserPoolClient.{Id:ClientId,Secret:ClientSecret}' --output json)
echo "$APP_OUT"
APP_CLIENT_ID=$(echo "$APP_OUT" | jq -r .Id)
APP_CLIENT_SECRET=$(echo "$APP_OUT" | jq -r .Secret)
Code language: PHP (php)
OIDC endpoints for your Ingress annotations are:
- issuer:
https://cognito-idp.ap-northeast-1.amazonaws.com/<POOL_ID>
- authorizationEndpoint:
https://<DOMAIN_PREFIX>.auth.ap-northeast-1.amazoncognito.com/oauth2/authorize
- tokenEndpoint:
https://<DOMAIN_PREFIX>.auth.ap-northeast-1.amazoncognito.com/oauth2/token
- userInfoEndpoint:
https://<DOMAIN_PREFIX>.auth.ap-northeast-1.amazoncognito.com/oauth2/userInfo
(These are the required keys for the ALB OIDC annotation. (Kubernetes SIGs))
3.4 (Optional) Create a test user
aws cognito-idp admin-create-user \
--user-pool-id $POOL_ID \
--username rajesh@example.com \
--temporary-password 'MyTempP@ssw0rd' \
--user-attributes Name=email,Value=rajesh@example.com Name=email_verified,Value=true \
--message-action SUPPRESS \
--region ap-northeast-1
# Set a permanent password
aws cognito-idp admin-set-user-password \
--user-pool-id $POOL_ID \
--username rajesh@example.com \
--password 'MyPermP@ssw0rd!' \
--permanent \
--region ap-northeast-1
Code language: PHP (php)
4) Kubernetes namespace + app (gRPC server + HTTP health)
We’ll run moul/grpcbin
(gRPC server) and a tiny HTTP echo container for ALB health checks. grpcbin exposes gRPC on 9000 (plaintext). (Docker Hub)
# Namespace
kubectl create namespace grpcdemo || true
Code language: PHP (php)
4.1 Deploy the app + service
# file: grpcbin.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: whoami
namespace: grpcdemo
spec:
replicas: 1
selector: { matchLabels: { app: whoami } }
template:
metadata: { labels: { app: whoami } }
spec:
containers:
- name: grpcbin
image: moul/grpcbin:latest
ports:
- containerPort: 9000 # gRPC plaintext
# readiness/liveness omitted for simplicity (gRPC)
- name: health
image: mendhak/http-https-echo:30
env:
- name: PORT
value: "8080"
ports:
- containerPort: 8080 # HTTP 200 on "/"
---
apiVersion: v1
kind: Service
metadata:
name: whoami
namespace: grpcdemo
spec:
selector: { app: whoami }
ports:
- name: grpc
port: 9000
targetPort: 9000
protocol: TCP
- name: health
port: 8080
targetPort: 8080
protocol: TCP
Code language: PHP (php)
Apply and verify:
kubectl apply -f grpcbin.yaml
kubectl -n grpcdemo rollout status deploy/whoami
kubectl -n grpcdemo get pods -o wide
kubectl -n grpcdemo get svc whoami -o wide
Code language: JavaScript (javascript)
5) OIDC client secret (and RBAC binding, if needed)
5.1 Create the Secret with Cognito app client creds
# file: oidc-client-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: oidc-client
namespace: grpcdemo
type: Opaque
stringData:
clientID: "<APP_CLIENT_ID>" # paste APP_CLIENT_ID from Step 3
clientSecret: "<APP_CLIENT_SECRET>" # paste APP_CLIENT_SECRET from Step 3
Code language: HTML, XML (xml)
kubectl apply -f oidc-client-secret.yaml
kubectl -n grpcdemo get secret oidc-client -o yaml | sed -n '1,30p' # just to confirm keys exist (values are base64)
Code language: PHP (php)
5.2 (Only if necessary) Bind the controller to read this Secret
Most recent Helm installs of the AWS LB Controller already have cluster-wide get/list/watch
on secrets
. If your logs show cannot get resource "secrets"
, bind it:
# file: alb-controller-can-read-oidc-secret.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: alb-can-read-oidc-secret
namespace: grpcdemo
rules:
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["oidc-client"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: alb-can-read-oidc-secret
namespace: grpcdemo
subjects:
- kind: ServiceAccount
name: aws-load-balancer-controller
namespace: kube-system
roleRef:
kind: Role
name: alb-can-read-oidc-secret
apiGroup: rbac.authorization.k8s.io
kubectl apply -f alb-controller-can-read-oidc-secret.yaml
Code language: CSS (css)
(Reference to the class of issue this avoids.) (GitHub)
6) Ingress (ALB) with TLS, OIDC, and gRPC
This Ingress:
- Terminates TLS at ALB using your ACM cert.
- Uses OIDC with Cognito (code flow +
openid email profile
scopes). (Kubernetes SIGs) - Sends gRPC traffic (HTTP/2) to port 9000.
- Uses an HTTP health check on port 8080.
# file: ingress-grpc-oidc.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: grpcdemo-auth
namespace: grpcdemo
annotations:
# Share an existing ALB across ingresses (optional)
alb.ingress.kubernetes.io/group.name: single-alb
# ALB essentials
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/ip-address-type: ipv4
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80,"HTTPS":443}]'
alb.ingress.kubernetes.io/certificate-arn: "<PUT_YOUR_$CERT_ARN_HERE>"
alb.ingress.kubernetes.io/ssl-redirect: "443"
# Backend is gRPC (HTTP/2)
alb.ingress.kubernetes.io/backend-protocol-version: GRPC
# Health check (HTTP on the sidecar)
alb.ingress.kubernetes.io/healthcheck-port: "8080"
alb.ingress.kubernetes.io/healthcheck-path: /
alb.ingress.kubernetes.io/success-codes: "200-399"
# OIDC authentication (Cognito)
alb.ingress.kubernetes.io/auth-type: oidc
alb.ingress.kubernetes.io/auth-idp-oidc: >
{"issuer":"https://cognito-idp.ap-northeast-1.amazonaws.com/<POOL_ID>",
"authorizationEndpoint":"https://<DOMAIN_PREFIX>.auth.ap-northeast-1.amazoncognito.com/oauth2/authorize",
"tokenEndpoint":"https://<DOMAIN_PREFIX>.auth.ap-northeast-1.amazoncognito.com/oauth2/token",
"userInfoEndpoint":"https://<DOMAIN_PREFIX>.auth.ap-northeast-1.amazoncognito.com/oauth2/userInfo",
"secretName":"oidc-client"}
alb.ingress.kubernetes.io/auth-on-unauthenticated-request: authenticate
alb.ingress.kubernetes.io/auth-scope: "openid email profile"
spec:
ingressClassName: alb
rules:
- host: raj.dev.aws.evp.drivemode.com
http:
paths:
# gRPC endpoint (protected by OIDC)
- path: /grpc
pathType: Prefix
backend:
service:
name: whoami
port:
name: grpc
# Simple HTTP endpoint (also protected) to force the browser login flow
- path: /whoami
pathType: Prefix
backend:
service:
name: whoami
port:
name: health
Code language: HTML, XML (xml)
Apply and watch:
kubectl apply -f ingress-grpc-oidc.yaml
# Inspect the Ingress until an address/hostname appears (ALB created)
kubectl -n grpcdemo get ing grpcdemo-auth -w
kubectl -n grpcdemo describe ing grpcdemo-auth
Code language: PHP (php)
Get the ALB DNS name (you’ll use this for Route 53 if not already set):
ALB_DNS=$(kubectl -n grpcdemo get ing grpcdemo-auth -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
echo "$ALB_DNS"
Code language: PHP (php)
7) Point your DNS name at the ALB
If not yet configured, create an A/ALIAS to the ALB hostname:
cat > /tmp/alb-alias.json <<JSON
{
"Comment": "Alias for EKS ALB",
"Changes": [{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "raj.dev.aws.evp.drivemode.com.",
"Type": "A",
"AliasTarget": {
"HostedZoneId": "Z14GRHDCWA56QT", # ALB zone id varies by region; ap-northeast-1 example, confirm in console
"DNSName": "$ALB_DNS",
"EvaluateTargetHealth": false
}
}
}]
}
JSON
aws route53 change-resource-record-sets \
--hosted-zone-id Z1234567890ABC \
--change-batch file:///tmp/alb-alias.json
Code language: PHP (php)
(You can also CNAME if using a subzone; ALIAS A is preferred for the zone apex.)
8) Verify target group health and OIDC flow
8.1 From Kubernetes side
# Pods, Endpoints
kubectl -n grpcdemo get pods -o wide
kubectl -n grpcdemo get endpoints whoami
# Ingress + annotations rendered correctly
kubectl -n grpcdemo describe ing grpcdemo-auth | sed -n '1,200p'
# Controller logs (helpful if OIDC/Secret isn’t picked up)
kubectl -n kube-system logs -l app.kubernetes.io/name=aws-load-balancer-controller --tail=200
Code language: PHP (php)
8.2 From AWS side
- In EC2 → Target Groups, find the TG(s) created for
grpcdemo-auth
rules. - Check Targets tab → instance/IP health status. If unhealthy:
- Health check path/port not reachable: we set
8080
+/
to return 200. - Wrong success code: we allowed
"200-399"
. - Timeout/No route: confirm Security Groups and Pod is up.
- Health check path/port not reachable: we set
(Health check annotations and OIDC annotations are per AWS LB Controller docs. (Kubernetes SIGs))
8.3 OIDC login & browser test
Open:
https://raj.dev.aws.evp.drivemode.com/whoami
Code language: JavaScript (javascript)
You should be redirected to the Cognito Hosted UI, log in, then be returned to /oauth2/idpresponse
, and finally proxied to your service with an ALB session cookie set. (The callback path requirement and flow are documented by AWS. (AWS Documentation))
If you hit issues:
- redirect_mismatch → your Cognito app client must include the exact
https://raj.dev.aws.evp.drivemode.com/oauth2/idpresponse
(lowercase host). (AWS Documentation) - invalid_scope → ensure
alb.ingress.kubernetes.io/auth-scope
scopes are enabled on your app client (openid
mandatory;email
,profile
optional). (Kubernetes SIGs) - 503 after login → targets unhealthy; fix health check path/port or service wiring.
9) Test gRPC through ALB
ALB will forward HTTP/2 to your backend for gRPC. (Official pattern confirms this setup.) (AWS Documentation)
⚠️ ALB OIDC uses browser redirects + session cookie. Non-browser gRPC clients don’t follow redirects. Two practical ways to test:
- Manual cookie injection: log in via
/whoami
in a browser, copy theAWSELBAuthSessionCookie*
cookie, then call gRPC with that cookie header.- For automated programmatic gRPC auth, consider moving JWT verification to an Envoy/NGINX gateway or API Gateway; ALB OIDC is primarily browser-centric. (Amazon Web Services, Inc.)
9.1 Using grpcurl
with cookie (manual)
In your browser DevTools (Application → Storage → Cookies), copy the full cookie string (often multiple cookie keys). Then:
# Replace cookie string and host
grpcurl -insecure \
-authority raj.dev.aws.evp.drivemode.com \
-H 'cookie: AWSELBAuthSessionCookie=...; AWSELBAuthSessionCookie-0=...' \
raj.dev.aws.evp.drivemode.com:443 list
Code language: PHP (php)
You should see grpcbin services (e.g., grpcbin.GRPCBin
, reflection, etc.). If you prefer an actual RPC:
# Example: server reflection enabled, list methods
grpcurl -insecure \
-authority raj.dev.aws.evp.drivemode.com \
-H 'cookie: AWSELBAuthSessionCookie=...; AWSELBAuthSessionCookie-0=...' \
raj.dev.aws.evp.drivemode.com:443 describe grpcbin.GRPCBin
Code language: PHP (php)
10) Troubleshooting quick table
Symptom | Likely cause | Fix |
---|---|---|
redirect_mismatch at Cognito | Callback URL not exactly https://<host>/oauth2/idpresponse | Add exact callback URL to app client (lowercase host) and save. (AWS Documentation) |
invalid_scope | Ingress requests scopes not enabled in app client | Enable openid (+ email profile ) in app client, or reduce auth-scope . (Kubernetes SIGs) |
401 Authorization Required at /oauth2/idpresponse | Wrong/disabled OAuth flow or scopes | Ensure Authorization code flow + openid allowed on app client. (AWS Documentation) |
503 Service Temporarily Unavailable after login | No healthy targets | Use health sidecar on 8080 + / , widen success codes; confirm Service/Endpoints. |
ALB never provisions | Controller missing permissions/IRSA | Check controller logs; ensure it has required IAM and K8s RBAC. |
Controller can’t read OIDC Secret | RBAC mis-scoped | Apply the Role/RoleBinding in Step 5.2. (GitHub) |
gRPC client gets redirect / cannot auth | ALB OIDC is redirect-based | Use cookie injection (manual) or move JWT validation to an in-cluster gateway/API Gateway. (Amazon Web Services, Inc.) |
11) Clean, minimal re-run checklist
- ACM cert validated in ap-northeast-1; have
$CERT_ARN
. - Cognito:
POOL_ID
,DOMAIN_PREFIX
, App Client (with secret), callback set to/oauth2/idpresponse
. - Kubernetes:
grpcbin.yaml
applied (pod ready, service ports 9000 + 8080).oidc-client-secret.yaml
applied (withclientID/clientSecret
).- (If needed)
alb-controller-can-read-oidc-secret.yaml
applied. ingress-grpc-oidc.yaml
applied; ALB DNS populated.
- DNS: Route 53 A/ALIAS → ALB hostname.
- Browser test:
https://raj.dev.aws.evp.drivemode.com/whoami
→ Cognito login → returns OK. - gRPC test (manual cookie) via
grpcurl
.
Reference docs used
- AWS Load Balancer Controller – Ingress annotations (OIDC, health check, scopes, unauthenticated behavior). (Kubernetes SIGs)
- Application Load Balancer – Authenticate users (OIDC/Cognito) (callback
/oauth2/idpresponse
, flows, permissions). (AWS Documentation) - AWS re:Post – ALB + Cognito callback (explicit
/oauth2/idpresponse
with ALB DNS/your host). (Repost) - EKS Prescriptive Guidance – gRPC behind ALB (HTTP/2 TLS path). (AWS Documentation)
- AWS Containers Blog – Cognito auth for K8s apps (pattern background). (Amazon Web Services, Inc.)
- grpcbin image (ports & usage). (Docker Hub)
I’m a DevOps/SRE/DevSecOps/Cloud Expert passionate about sharing knowledge and experiences. I have worked at Cotocus. I share tech blog at DevOps School, travel stories at Holiday Landmark, stock market tips at Stocks Mantra, health and fitness guidance at My Medic Plus, product reviews at TrueReviewNow , and SEO strategies at Wizbrand.
Do you want to learn Quantum Computing?
Please find my social handles as below;
Rajesh Kumar Personal Website
Rajesh Kumar at YOUTUBE
Rajesh Kumar at INSTAGRAM
Rajesh Kumar at X
Rajesh Kumar at FACEBOOK
Rajesh Kumar at LINKEDIN
Rajesh Kumar at WIZBRAND