Debug private AKS clusters¶

When deploying Opaque via the Azure Marketplace, you can choose to enable a private AKS cluster. This restricts direct access to the Kubernetes API server and limits traffic to within the virtual network. While this enhances security, it can make cluster access trickier—especially for health checks, log inspection, and troubleshooting.

This guide outlines how to debug private AKS workloads securely and efficiently, focusing on options that require minimal setup while preserving cluster isolation.

Why private clusters are tricky to debug¶

Private AKS clusters disable public access to the Kubernetes API server. This means:

• You can’t reach the cluster directly from your laptop.
• You won’t be able to use kubectl unless you're inside the virtual network or routing through a jumpbox.
• You may not have a kubeconfig available or permitted.

But there’s good news: Azure provides secure tools that let you inspect workloads, retrieve logs, and troubleshoot—even without direct access.

Access methods for debugging private AKS clusters¶

When access to the Kubernetes API server is locked down (as in private clusters), you can’t run kubectl from your local machine. The following two methods offer secure alternatives for inspecting and troubleshooting your workloads.

Azure CLI Remote Execution (az aks command invoke)¶

This is the simplest and most secure method for most common tasks.

• Runs kubectl commands inside the control plane. No need to set up a local cluster connection—Azure runs commands on your behalf.
• No need to install kubectl or download kubeconfig. This helps keep your local environment clean and reduces the risk of credential exposure.
• No direct network access to the private cluster required. You can debug even from a laptop that isn’t in the VNet.
• Ideal for quick inspection, logs, and get/describe operations. ****Works best for read-only debugging and basic rollout checks.

Recommended for most basic debugging use cases (e.g., checking pod status, reading logs, verifying deployments).

Jumpbox / Bastion VM access (private network)¶

If you need full cluster access—like opening an interactive shell or using tools not available in az aks command invoke—you can access the cluster from within the private network.

• Virtual machine inside the AKS subnet or peered VNet. Acts as a secure access point into the private environment.
• You SSH into the VM and run kubectl with a local kubeconfig. This gives you full CLI access as if working directly inside the cluster.
• Allows full access to the cluster and internal services. Useful for port-forwarding, shell access, and multi-step debugging.
• Can be combined with Azure Bastion for browser-based SSH. Avoids exposing SSH ports publicly—connect from the Azure Portal.

Recommended for advanced debugging (e.g., kubectl exec, port forwarding) and scenarios requiring persistent tooling.

Which method should I choose?¶

Common Debugging Commands¶

The following examples use the Azure CLI remote execution method (az aks command invoke), which runs commands inside the control plane. Here's what you should know about how it works and what it requires:

• This method is stateless and non-interactive: each command runs independently and does not maintain session state.
• It’s ideal for secure debugging in production environments where direct access or kubeconfig sharing is restricted.
• All commands are executed securely inside the Azure Kubernetes control plane and can be audited via Azure Monitor or Activity Logs.
• The method requires:
  • Azure RBAC permission to invoke AKS commands (Azure Kubernetes Service RBAC Cluster Admin or higher).
  • Does not expose kubectl or cluster credentials locally.

List available namespaces¶

az aks command invoke \
  --resource-group <RESOURCE_GROUP> \
  --name <AKS_CLUSTER_NAME> \
  --command "kubectl get ns"

List pods in a namespace¶

az aks command invoke \
  --resource-group <RESOURCE_GROUP> \
  --name <AKS_CLUSTER_NAME> \
  --command "kubectl get pods -n <NAMESPACE>"

Get last 10 log lines from a pod¶

az aks command invoke \
  --resource-group <RESOURCE_GROUP> \
  --name <AKS_CLUSTER_NAME> \
  --command "kubectl logs <POD_NAME> -n <NAMESPACE> --tail=10"

For more advanced cluster access or session-based debugging, consider setting up a jumpbox or Azure Bastion connection as described above. For production environments, we recommend using az aks command invoke whenever possible for quick, secure, and auditable access.