Query rewriting prompt for RAG search

Paste this into a chat window and replace each {{variable}}, or open it in the prompt builder to fill them in a form. Sections are marked with XML tags, which every major model reads well.

You turn user questions into search queries for a {{search_type}} over {{corpus}}.

<context>
The conversation so far:
<history>
{{history}}
</history>
</context>

<constraints>
- Resolve references from the history: replace “it”, “that plan” or “the second one” with what they mean.
- Write one to three queries. Use more than one only if the question has separate parts or could use different terms.
- Use the vocabulary the documents are likely to use, not the user’s slang. Keep product names and error codes exactly as written.
- Don’t answer the question.
- If the question doesn’t need a search (a greeting, a thank-you), return an empty list.
</constraints>

<output_format>
JSON only: {"queries": ["...", "..."]}
</output_format>

<task>
Rewrite the latest user question as standalone search queries that will find the documents needed to answer it.

<latest_question>
{{latest_question}}
</latest_question>
</task>

You turn user questions into search queries for a {{search_type}} over {{corpus}}.

<constraints>
- Resolve references from the history: replace “it”, “that plan” or “the second one” with what they mean.
- Write one to three queries. Use more than one only if the question has separate parts or could use different terms.
- Use the vocabulary the documents are likely to use, not the user’s slang. Keep product names and error codes exactly as written.
- Don’t answer the question.
- If the question doesn’t need a search (a greeting, a thank-you), return an empty list.
</constraints>

<output_format>
JSON only: {"queries": ["...", "..."]}
</output_format>

<context>
The conversation so far:
<history>
{{history}}
</history>
</context>

<task>
Rewrite the latest user question as standalone search queries that will find the documents needed to answer it.

<latest_question>
{{latest_question}}
</latest_question>
</task>

{{search_type}}

The retrieval method, such as “keyword search”, “vector search” or “hybrid search”. Keyword search needs exact terms; vector search tolerates paraphrase.

{{corpus}}

What is being searched, such as “our product help centre” or “the engineering wiki”.

{{history}}

The last few turns of the conversation. Three to six turns is usually enough context.

{{latest_question}}

The user’s newest message.

{{search_type}}

hybrid keyword and vector search

{{corpus}}

a hosting company’s help centre

{{history}}

User: How do I add a custom domain?
Assistant: Go to Sites, then Domains, then Add domain, and follow the DNS steps shown.
User: Is SSL included?
Assistant: Yes. A certificate is issued automatically once the DNS records are verified.

{{latest_question}}

and does that work on the free plan?

What to look for. Taken alone, “does that work on the free plan” is unsearchable. Good output resolves “that” to custom domains and SSL, such as custom domain free plan and SSL certificate free plan, keeping “free plan” in the help centre’s wording.

You turn user questions into search queries for a hybrid keyword and vector search over a hosting company’s help centre.

<context>
The conversation so far:
<history>
User: How do I add a custom domain?
Assistant: Go to Sites, then Domains, then Add domain, and follow the DNS steps shown.
User: Is SSL included?
Assistant: Yes. A certificate is issued automatically once the DNS records are verified.
</history>
</context>

<constraints>
- Resolve references from the history: replace “it”, “that plan” or “the second one” with what they mean.
- Write one to three queries. Use more than one only if the question has separate parts or could use different terms.
- Use the vocabulary the documents are likely to use, not the user’s slang. Keep product names and error codes exactly as written.
- Don’t answer the question.
- If the question doesn’t need a search (a greeting, a thank-you), return an empty list.
</constraints>

<output_format>
JSON only: {"queries": ["...", "..."]}
</output_format>

<task>
Rewrite the latest user question as standalone search queries that will find the documents needed to answer it.

<latest_question>
and does that work on the free plan?
</latest_question>
</task>

• All models

  A small, fast model is usually enough for rewriting. It runs on every turn, so its latency adds directly to the time before the answer starts.

• OpenAI GPT

  Use Structured Outputs with an array of strings for queries. OpenAI notes that a refusal may not follow the schema, so check the refusal field before parsing.

• Gemini

  Google recommends few-shot examples for Gemini. Two examples of a follow-up question and its rewritten queries, taken from your own logs, teach the resolution pattern quickly.

• RAG answer with citationsAnswer only from retrieved documents, cite a source for every claim, and admit when the answer isn’t there.
• Classify textSort text into fixed labels with definitions, examples and an “unclear” option, returning JSON.
• Extract structured JSONPull fields from emails, documents or pages into a fixed JSON shape, with null instead of guesses.

Building this into an app that sends the same instructions with every request? Start from a system prompt instead; each comes with design notes, failure modes and a monthly cost. In the system prompt library: Search query rewriter for RAG.

Edited the template? Compare your version with this one in the prompt diff, and count it for your model with the token counter. In the glossary: rag. Browse all 29 prompt templates.