Secure RAG on Google Cloud with RAG Engine and Gemini (2026)
In this article
The build steps in this guide are adapted from Google's official intro_rag_engine.ipynb in the GoogleCloudPlatform/generative-ai repository (Apache-2.0). Code is reproduced as published. Security support statements are from Google's RAG Engine overview as of October 2026.
A RAG system is a data-access system with a language model in front of it. Whatever you put in the corpus, anyone who can query it can potentially read, one well-phrased question at a time. That is why "secure RAG" is less about the model than about four decisions: what goes into the index, who can query which index, where the data and traffic are allowed to go, and what the model is allowed to do with what it retrieves.
On Google Cloud, RAG Engine handles the mechanics: ingestion, chunking, embedding, vector storage and retrieval, and it plugs into Gemini as a grounding tool. Google's documentation describes it as a component of the Gemini Enterprise Agent Platform, and its SDK is published as agentplatform. This guide builds a working pipeline from Google's notebook, then applies the controls RAG Engine supports, and is explicit about the ones it does not.
Prerequisites and a region warning
- A Google Cloud project with billing enabled, and permission to use RAG Engine and Gemini in it.
- Python with the two SDKs the notebook installs:
%pip install --upgrade --quiet google-cloud-aiplatform google-genai
Choose your region before anything else. Google's overview states that new projects must be added to an allowlist to use RAG Engine in us-central1, us-east1 and us-east4. Existing projects are unaffected. Google's own notebook uses us-east1, so on a new project corpus creation can fail there. Pick another region from the supported regions list, and use the same region for every client below.
Step 1: Create the clients
The notebook reads the project ID, creates an Agent Platform client in a region, and imports the types it uses:
# Use the environment variable if the user doesn't provide Project ID.
import os
import agentplatform
# fmt: off
PROJECT_ID = "[your-project-id]" # @param {type: "string", placeholder: "[your-project-id]", isTemplate: true}
# fmt: on
if not PROJECT_ID or PROJECT_ID == "[your-project-id]":
PROJECT_ID = str(os.environ.get("GOOGLE_CLOUD_PROJECT"))
# See https://cloud.google.com/vertex-ai/generative-ai/docs/rag-engine/rag-overview#supported-regions for location options.
client = agentplatform.Client(project=PROJECT_ID, location="us-east1")
from IPython.display import Markdown, display
from agentplatform import types
from google import genai
from google.genai import types as genai_types
Replace us-east1 with your region if your project is not allowlisted. Authentication comes from your environment: Application Default Credentials locally, or the notebook's Colab sign-in cell. The identity you run as is the identity RAG Engine checks against IAM, which matters in Step 6.
Step 2: Create a corpus
A corpus is RAG Engine's index: the container that holds your files, their chunks and their embeddings.
rag_corpus = client.rag.create_corpus(
rag_corpus=types.RagCorpus(
display_name="my-rag-corpus",
)
)
client.rag.list_corpora()
Treat a corpus as a security boundary, not just a folder. Everyone who can query a corpus can retrieve from all of it, so plan one corpus per audience or sensitivity level, for example public product docs, internal engineering and HR, rather than one corpus for everything with filters on top.
Step 3: Import documents
RAG Engine ingests from local files, Cloud Storage and Google Drive. The notebook's Cloud Storage import sets the chunking explicitly:
INPUT_GCS_BUCKET = (
"gs://cloud-samples-data/gen-app-builder/search/alphabet-investor-pdfs/*"
)
response = client.rag.import_files(
name=rag_corpus.name,
import_config=types.ImportRagFilesConfig(
gcs_source=types.GcsSource(uris=[INPUT_GCS_BUCKET]),
rag_file_transformation_config=types.RagFileTransformationConfig(
rag_file_chunking_config=types.RagFileChunkingConfig(
chunk_size=1024,
chunk_overlap=100,
)
), # optional
max_embedding_requests_per_min=900, # optional
),
)
chunk_size=1024 with chunk_overlap=100 controls how documents are split before embedding, and max_embedding_requests_per_min throttles embedding calls so a large import does not exhaust your embedding quota. A single local file can also be added with client.rag.upload_file(corpus_name=rag_corpus.name, path="test.md", display_name="test.md").
This is where most RAG data leaks are actually decided. Once a document is embedded, its contents can be retrieved. Classify documents before import and keep anything a corpus's audience should not see out of it. Google Cloud's Sensitive Data Protection service can inspect and de-identify files in Cloud Storage before they reach the import step. That is a pipeline you build alongside RAG Engine, not a setting in it.
Step 4: Retrieve directly and inspect the results
Before connecting a model, query the corpus yourself and look at what comes back:
# Direct context retrieval
response = client.rag.retrieve_contexts(
vertex_rag_store=genai_types.VertexRagStore(
rag_resources=[
genai_types.VertexRagStoreRagResource(rag_corpus=rag_corpus.name)
]
),
query=types.RagQuery(
text="What is RAG and why it is helpful?",
rag_retrieval_config=types.RagRetrievalConfig(
top_k=10,
filter=types.RagRetrievalConfigFilter(vector_distance_threshold=0.5),
),
),
)
print(response)
top_k=10 caps the number of chunks, and vector_distance_threshold=0.5 drops chunks farther than 0.5 from the query, so a lower value is stricter. This is the cheapest security test you can run: ask the questions an over-curious user would ask and confirm the corpus only returns what that audience may see. Retrieved contexts can also be passed to any model yourself, as the notebook's closing comment notes.
Step 5: Ground Gemini in the corpus
To let Gemini retrieve on its own, wrap the corpus in a retrieval tool:
# Create a tool for the RAG Corpus
rag_retrieval_tool = genai_types.Tool(
retrieval=genai_types.Retrieval(
vertex_rag_store=genai_types.VertexRagStore(
rag_corpora=[rag_corpus.name],
similarity_top_k=10,
vector_distance_threshold=0.5,
)
)
)
Then pass it to Gemini through the Gen AI SDK:
MODEL_ID = "gemini-3.8-flash"
# Create a Gen AI SDK client
genai_client = genai.Client(project=PROJECT_ID, location="your-location")
response = genai_client.models.generate_content(
model=MODEL_ID,
contents="What is RAG?",
config=genai_types.GenerateContentConfig(tools=[rag_retrieval_tool]),
)
display(Markdown(response.text))
Replace "your-location" with the same region as your corpus. The Gen AI SDK works against both the Gemini Developer API and Google Cloud. It uses Google Cloud when created with vertexai=True or when the GOOGLE_GENAI_USE_VERTEXAI environment variable is set to true, so set one of these if the client asks you for an API key.
Step 6: Apply the controls RAG Engine supports
| Control | What it protects against | RAG Engine status |
|---|---|---|
| IAM on the project and corpus | Unauthorised users querying or modifying an index | Core mechanism; scope by giving each audience its own corpus |
| VPC Service Controls | Data being moved out of a defined perimeter, even with valid credentials | Supported, per Google's overview |
| Customer-managed encryption keys (CMEK) | Data at rest under Google-managed keys you cannot revoke | Supported, per Google's overview |
| Data residency guarantees | Data being processed or stored outside a required location | Not supported, per Google's overview |
| Pre-import classification (Sensitive Data Protection) | Sensitive data entering a corpus at all | Separate service, run before import |
| Prompt and response screening (Model Armor) | Prompt injection and sensitive data in responses | Separate Google Cloud service, applied around model calls |
Google's overview also lists what it calls "AXT security controls" as unsupported. Two points deserve emphasis. If you have a regulatory requirement about where data is processed, RAG Engine does not currently meet it on its own; Google lists data residency as unsupported. Resolve that with your compliance team before indexing regulated data. And metadata filters are not access control. RAG Engine supports metadata filtering, but filters are set by the calling code, so a bug or a manipulated request can widen them. Use filters for relevance and separate corpora with IAM for security.
What to watch out for
Retrieved text is untrusted input. A document in the corpus can contain instructions ("ignore the rules above and…"), and a grounded model reads them. Screen prompts and responses, keep the model's tools narrow, and never give a RAG assistant tools that act on systems unless that is the point and you have reviewed it.
Know what you are billed for. Google's overview notes that if you use a RAG Engine-managed Spanner instance as the vector database in a generally available location, that Spanner instance is billed to you. Check the billing page before choosing a vector store.
Test retrieval before trusting answers. A confident Gemini answer grounded in the wrong chunk is worse than "I don't know". Keep a set of test questions per corpus, including questions it should refuse, and re-run them after every import.
Where to go next
For the same problem on AWS, see our guide to a production RAG pipeline with Bedrock and OpenSearch. For architecture choices that apply on any cloud, read designing a production-grade RAG architecture. And to measure whether retrieval is actually good, build a test set with Ragas using our guide to generating synthetic RAG test sets.
Frequently asked questions
Does Google Cloud RAG Engine support VPC Service Controls and CMEK?
Yes. Google's RAG Engine overview states that VPC Service Controls and customer-managed encryption keys (CMEK) are supported. The same page says data residency and AXT security controls are not supported, which matters if you have regulatory location requirements.
Why does creating a RAG corpus fail in us-east1 or us-central1?
Google requires new projects to be added to an allowlist to use RAG Engine in us-central1, us-east1 and us-east4; existing projects are unaffected. Google's official notebook uses us-east1, so on a new project choose another supported region from the RAG Engine regions list.
How do I connect Gemini to a RAG Engine corpus?
Create a Tool with a Retrieval that points VertexRagStore at your corpus name, with similarity_top_k and vector_distance_threshold, and pass it in GenerateContentConfig(tools=[...]) to generate_content. Gemini then retrieves from the corpus and grounds its answer in what it finds.
What do top_k and vector_distance_threshold control in RAG Engine retrieval?
top_k is the maximum number of chunks returned, and vector_distance_threshold drops chunks whose vector distance exceeds the threshold. Google's notebook uses top_k 10 with a threshold of 0.5. A lower threshold is stricter, returning fewer but closer matches.
Can metadata filters in RAG Engine replace access control?
No. Filters narrow what a query returns, but they are set by the calling code, so a bug or a manipulated request can bypass them. Enforce separation with separate corpora per sensitivity level or audience, protected by IAM, and use filters only for relevance.
Related Guides
Gemini Decision 1 Returns Probabilities, Not Text, at $0.10/M
Google's Gemini Decision 1 scores up to 1,000 typed questions with zero output tokens at $0.10 per million input tokens, leading 7 of 9 benchmarks.
Build a Multi-Query RAG Retriever with Elasticsearch and LangChain
Reproduce Elasticsearch Labs' multi-query RAG pipeline: paraphrase user queries, retrieve against Elasticsearch, and feed combined context to an LLM.
Google's Gemini 4 Argon Scores 87% on ERQA, Controls Spot
Google reports Gemini 4 Argon scores 87% on ERQA and uses long prompt harnesses to drive Spot, Omni, and MuJoCo Real2Sim workflows.