Training and tutorials

Learn how to dynamically scale Agent Sandbox deployments using the Horizontal Pod Autoscaler (HPA) and standby capacity buffers.

Tutorial Agent Sandbox Agentic AI

Training and tutorials

Learn how to use Cloud Storage FUSE to optimize performance for AI and ML workloads on GKE.

AI/ML Inference AI/ML Training Storage

Training and tutorials

Learn how to use Managed Lustre to optimize performance for AI and ML workloads on GKE.

AI/ML Inference AI/ML Training Storage

Training and tutorials

Learn how to install and run the Agent Sandbox controller on GKE, and deploy a sandboxed environment on the cluster for testing untrusted shell commands.

Tutorial Agent Sandbox Agentic AI

Training and tutorials

Learn how to use open source Kata Containers with Agent Sandbox on GKE to isolate untrusted code execution.

Training Tutorial Agentic AI Agent Sandbox

Training and tutorials

Learn how to deploy and manage a containerized agentic AI application on GKE, using the Agent Development Kit (ADK) and vLLM for scalable inference with Llama 3.1.

Tutorial AI/ML Inference Agentic AI

Training and tutorials

Learn how to deploy and manage a containerized agentic AI application on GKE, using the Agent Development Kit (ADK) and Agent Platform for scalable inference with Gemini 2.0 Flash.

Tutorial AI/ML Inference Agentic AI

Training and tutorials

Learn how to deploy LLMs using Tensor Processing Units (TPUs) on GKE with the Optimum TPU serving framework from Hugging Face.

Tutorial AI/ML Inference TPU

Training and tutorials

Learn how to optimize costs for LLM-serving workloads on GKE using DWS Flex-start.

Cost optimization GPU DWS

Training and tutorials

Learn how to serve large language models (LLMs) with KubeRay on TPUs, and how this can help improve the performance of your models.

Video Ray TPUs

Training and tutorials

Learn how to how to simplify and accelerate the loading of AI/ML model weights on GKE using Hyperdisk ML.

Tutorial AI/ML Data Loading

Training and tutorials

Learn how to serve a LLM using Tensor Processing Units (TPUs) on GKE with JetStream through PyTorch.

Tutorial AI/ML Inference TPUs

Training and tutorials

Learn best practices for optimizing LLM inference performance with GPUs on GKE using the vLLM and Text Generation Inference (TGI) serving frameworks.

Tutorial AI/ML Inference GPUs

Training and tutorials

Learn when to use the NVIDIA GPU operator and how to enable the NVIDIA GPU Operator on GKE.

Tutorial GPUs

Training and tutorials

Learn how to set up your autoscaling infrastructure by using the GKE Horizontal Pod Autoscaler (HPA) to deploy the Gemma LLM using single-host JetStream.

Tutorial TPUs

Training and tutorials

Learn how to fine-tune Gemma LLM using GPUs on GKE with the Hugging Face Transformers library.

Tutorial AI/ML Inference GPUs

Training and tutorials

Learn how to deploy and serve a Stable Diffusion model on GKE using TPUs, Ray Serve, and the Ray Operator add-on.

Tutorial AI/ML Inference Ray TPUs

Training and tutorials

Learn how to set up your autoscaling infrastructure by using the GKE Horizontal Pod Autoscaler (HPA) to deploy the Gemma LLM with the Hugging Face Text Generation Interface (TGI) serving framework.

Tutorial GPUs

Training and tutorials

Learn how to run a container-based, Megatron-LM PyTorch workload on A3 Mega.

Tutorial AI/ML Training GPUs

Training and tutorials

Learn how to request hardware accelerators (GPUs) in your GKE Autopilot workloads.

Tutorial GPUs

Training and tutorials

Learn how to serve Llama 2 70B or Falcon 40B using multiple NVIDIA L4 GPUs with GKE.

Tutorial AI/ML Inference GPUs

Training and tutorials

Learn how to easily start using Ray on GKE by running a workload on a Ray cluster.

Tutorial Ray

Training and tutorials

Learn how to serve Falcon 7b, Llama2 7b, Falcon 40b, or Llama2 70b using the Ray framework in GKE.

Tutorial AI/ML Inference Ray GPUs

Training and tutorials

Learn how to orchestrate a Jax workload on multiple TPU slices on GKE by using JobSet and Kueue.

Tutorial TPUs

Training and tutorials

Learn how to observe GPU workloads on GKE with NVIDIA Data Center GPU Manager (DCGM).

Tutorial AI/ML Observability GPUs

Training and tutorials

This quickstart shows you how to deploy a training model with GPUs in GKE and store the predictions in Cloud Storage.

Tutorial AI/ML Training GPUs

Training and tutorials

This video shows how GKE helps solve common challenges of training large AI models at scale, and the best practices for training and serving large-scale machine learning models on GKE.

Video AI/ML Training AI/ML Inference

Training and tutorials

This blog post is a step-by-step guide to the creation, execution, and teardown of a Tensorflow-enabled Jupiter notebook.

Blog AI/ML Training AI ML Inference GPUs

Training and tutorials

This tutorial uses Kueue to show you how to implement a Job queueing system, and configure workload resource and quota sharing between different namespaces on GKE.

Tutorial AI/ML Batch

Training and tutorials

This tutorial shows you how to integrate a Large Language Model application based on retrieval-augmented generation with PDF files that you upload to a Cloud Storage bucket.

Tutorial AI/ML Data Loading

Training and tutorials

This tutorial shows you how to analyze big datasets on GKE by leveraging BigQuery for data storage and processing, Cloud Run for request handling, and a Gemma LLM for data analysis and predictions.

Tutorial AI/ML Data Loading

Use cases

Learn how to leverage GKE and Ray to efficiently preprocess large datasets for machine learning.

MLOps Training Ray

Use cases

Learn how to speed up data loading times for your machine learning applications on Google Kubernetes Engine.

Inference Hyperdisk ML Cloud Storage FUSE

Use cases

Learn how to optimize your GPU inference costs by fine-tuning GKE's Horizontal Pod Autoscaler for maximum efficiency.

Inference GPU HPA

Use cases

Learn how to deploy cutting-edge NVIDIA NIM microservices on GKE with ease and accelerate your AI workloads.

AI NVIDIA NIM

Use cases

Learn how Ray Operator on GKE simplifies your AI/ML production deployments, boosting performance and scalability.

AI TPU Ray

Use cases

Learn how to maximize large language model (LLM) serving throughput for GPUs on GKE, including infrastructure decisions and model server optimizations.

LLM GPU NVIDIA

Use cases

Learn how to build and optimize batch processing platforms on GKE

Batch Performance Cost optimization

Use cases

Learn how to use Local SSDs to provide high-performance AI/ML storage on GKE.

AI NVMe Local SSD

Use cases

Learn how to run JAX multi-GPU, multi-node applications on GKE with NVIDIA GPUs.

GPUs JAX ML

Use cases

How LiveX AI uses GKE to build AI agents that enhance customer satisfaction and reduce costs.

GenAI NVIDIA GPU

Use cases

Reference architecture for running a generative AI application with retrieval-augmented generation (RAG) using GKE, Cloud SQL, Ray, Hugging Face, and LangChain.

GenAI RAG Ray

Use cases

Reference architecture for a batch processing platform on GKE in Standard mode using Kueue to manage resoure quotas.

AI Kueue Batch

Use cases

How IPRally uses GKE and Ray to build a scalable, efficient ML platform for faster patent searches with better accuracy.

AI Ray GPU

Use cases

Leverage Gemma on Cloud GPUs and Cloud TPUs for inference and training efficiency on GKE.

AI Gemma Performance

Use cases

Use best-in-class Gemma open models to build portable, customizable AI applications and deploy them on GKE.

AI Gemma Performance

Use cases

Orchestrate Ray applications in GKE with KubeRay and Kueue.

Kueue Ray KubeRay

Use cases

Apply security insights and hardening techniques for training AI/ML workloads using Ray on GKE.

AI Ray Security

Use cases

Select the best combination of storage options for AI and ML workloads on Google Cloud.

AI ML Storage

Use cases

Automatically install Nvidia GPU drivers in GKE.

GPU NVIDIA Installation

Use cases

Train generative AI models using GKE and NVIDIA NeMo framework.

GenAI NVIDIA NeMo

Use cases

Improve scalability, cost-efficiency, fault tolerance, isolation, and portability by using GKE for Ray workloads.

AI Ray Scale

Use cases

Simplify the model development and deployment process using Weights & Biases with GKE.

Cost optimization TPUs GPUs

Use cases

Gain improved GPU support, performance, and lower pricing for AI/ML workloads with GKE Autopilot.

GPU Autopilot Performance

Use cases

Startup scales personalized video output with GKE.

GPU Scale Containers

Use cases

How Ray is transforming ML development at Spotify.

ML Ray Containers

Use cases

Ordaōs Bio, one of the leading AI accelerators for biomedical research and discovery, is finding solutions to novel immunotherapies in oncology and chronic inflammatory disease.

Performance TPU Cost optimization

Use cases

How Moloco, a Silicon Valley startup, harnessed the power of GKE and Tensor Flow Enterprise to supercharge its machine learning (ML) infrastructure.

ML Scale Cost optimization

Use cases

Learn how to improve the launch time of Stable Diffusion on GKE.

Performance Scaling PD

Code Samples

View sample applications used in official GKE product tutorials.

Code Samples

View experimental samples for leveraging GKE to accelerate your AI/ML initiatives.

Code Samples

View reference architectures and solutions for deploying accelerated workloads on GKE.