SEO & Content Optimization5.0 · 0 ratings

Meta Robots And Crawl Directive Advisor

Advises on indexation, canonicalization, and crawl directives for a set of URLs to fix index bloat.

Role-BasedChain-of-ThoughtStructured-Output

Prompt

ROLE: You are a technical SEO who manages indexation, canonicalization, and crawl budget.

CONTEXT: Site type: [SITE_TYPE e.g. ecommerce, blog, SaaS]. I have indexation concerns. Below is a list of URL patterns / pages with a short description of each (purpose, parameters, duplication notes).

[PASTE_URL_PATTERNS]

TASK — for each URL or pattern, reason then recommend:
1. Decide the correct indexation directive: index/follow, noindex/follow, noindex/nofollow, or canonical-to-another.
2. Identify canonicalization needs (parameterized, paginated, filtered, or duplicate URLs) and specify the canonical target.
3. Recommend robots.txt disallow vs meta-robots noindex (and explain why crawl-block is not the same as de-index).
4. Flag crawl-budget wasters and faceted-navigation traps.
5. Note any pages wrongly excluded that SHOULD be indexed.

OUTPUT FORMAT:
- Directive table: URL/pattern | Recommended directive | Canonical target | Method (robots.txt / meta / canonical) | Reason
- Crawl-budget warnings
- 'Wrongly blocked — should index' list

CONSTRAINTS: Never recommend robots.txt disallow on URLs that need to be de-indexed (blocked crawl prevents seeing the noindex tag). Preserve link equity via canonical or redirect where relevant. Be explicit about risk and double-check logic before finalizing. Mark uncertain cases for manual verification.

How to use this prompt

  1. 1

    Copy the prompt above and paste it into ChatGPT, Claude, or Gemini — or open it in the visual Studio to edit each part on a canvas and run it with your own key.

  2. 2

    Replace any bracketed placeholders with your specifics. The more concrete your context and constraints, the sharper the result — see the 5-part prompt structure.

  3. 3

    Run it, then refine. Ask the model to critique and improve its own answer with self-critique prompting.

Techniques in this prompt

Role-Based

Assigns the model an expert persona so it adopts the right vocabulary, depth, and standards for the task.

Learn this technique
Chain-of-Thought

Asks the model to reason step by step before answering — ideal for multi-step, logical, or analytical tasks.

Learn this technique
Structured Output

Pins the response to a defined structure so it drops straight into your workflow.

Learn this technique

Recommended models

claudegpt-4ogemini

Build on this prompt

Open it in the visual Studio to wire it into a full workflow with your own API key — or learn the craft behind prompts like this.

Open in Studio How to prompt AI correctly

More in SEO & Content Optimization

Search Intent Classification And Content Brief

Classifies a keyword's dominant search intent and converts it into a structured, ready-to-write content brief.

Read prompt

Title Tag And Meta Description CTR Optimizer

Generates and self-critiques title tags and meta descriptions engineered for pixel limits and click-through rate.

Read prompt

Topic Cluster And Pillar Page Architect

Designs a pillar-and-cluster content map with internal linking logic to build topical authority around a theme.

Read prompt

On-Page SEO Audit Of Existing Copy

Audits a pasted page against on-page ranking factors and returns a prioritized fix list with rewrites.

Read prompt