--- title: "The AI Citation Swipe File: Answer Blocks, Schema and robots.txt" description: "Copy-ready answer paragraphs, schema, comparison and pricing blocks, plus robots.txt rules for making your site easier for live-answer bots to fetch and cite." url: "https://groas.com/post/steal-these-the-robots-txt-rules-schema" image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/c7c37525-76c0-4b57-b192-65b4390d2c1f.png" published: "2026-10-15T05:42:01.940Z" modified: "2026-10-15T05:42:02.004Z" --- October 15, 2026 · 9 min read # The AI Citation Swipe File: Answer Blocks, Schema and robots.txt [Alexander PerelmanHead Of Product @ groas](https://groas.com/author/alexander-perelman)[LinkedIn](https://www.linkedin.com/in/alexander-433793253/) ![Tiny model shop full of neat answer cards behind a padlocked gate marked 403, a courier waiting outside: good copy is useless if bots can't get in.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/c7c37525-76c0-4b57-b192-65b4390d2c1f.png) In this article 1. [Snippet 1: An answer paragraph that gets to the point](#snippet-1-an-answer-paragraph-that-gets-to-the-point) 2. [Snippet 2: An Organization block that points to the right business](#snippet-2-an-organization-block-that-points-to-the-right-business) 3. [Snippet 3: FAQ schema that matches the answer people can see](#snippet-3-faq-schema-that-matches-the-answer-people-can-see) 4. [Snippet 4: A comparison table with rows a buyer can check](#snippet-4-a-comparison-table-with-rows-a-buyer-can-check) 5. [Snippet 5: A pricing block with a number only when you have one](#snippet-5-a-pricing-block-with-a-number-only-when-you-have-one) 6. [The install order before you copy the final block](#the-install-order-before-you-copy-the-final-block) 7. [Snippet 6: The robots.txt rules that let the other snippets be read](#snippet-6-the-robotstxt-rules-that-let-the-other-snippets-be-read) A brilliant answer paragraph is useless if the bot gets a 403 before it reads the page. I treat AI citation copy like responsive search ad copy: make the answer short, specific and testable, then give ChatGPT, Perplexity and Claude a clear path to fetch it. ## Snippet 1: An answer paragraph that gets to the point **When to use it:** Put this directly beneath an H2 or H3 on a service, product or pricing page, where it can answer the question in the heading without a warm-up lap. The [live-fetch guide](https://groas.com/post/how-to-get-your-business-cited-in-chatgp) covers the retrieval side; this block is the copy a successful fetch should find. I used to give ad copy more room to be clever. Then I had to account for what a buyer could understand before deciding whether to click. The same discipline helps here: name the business, the mechanism and the useful constraint before asking a reader or a machine to infer them from a longer page. **Write the answer first, not after four sentences about the industry.** > \[YOUR_BRAND_NAME] provides \[EXACT_MECHANISM_OR_DELIVERABLE] for \[TARGET_AUDIENCE] to help achieve \[PRIMARY_METRIC_OR_OUTCOME] within \[TIMEFRAME,\_IF_VERIFIABLE], without \[RELEVANT_PAIN_POINT]. Pricing is \[VERIFIABLE_PRICE_OR_PRICING_MODEL], with \[VERIFIABLE_CONTRACT_TERM_OR_CONDITION]. For groas, the useful opening is the mechanism: a fully autonomous growth engine for paid and organic search, with continuous execution and a named human strategist responsible for direction and accountability. A reader comparing operating models learns more from that than from _AI-powered growth_. If the page discusses fees, a flat monthly fee and month-to-month cancellation are worth stating plainly. An exact dollar amount is not worth making up to complete a template. The target is a compact answer, roughly 40 words when the facts allow it. That is a writing constraint, not a rule that an engine reads only 40 words. If the page needs another sentence to explain a qualification, give it one. Do not squeeze a complicated offer into a sentence that stops being true. **The one adjustment that matters:** Replace every bracket with a fact that appears on the same page. If you cannot substantiate a timeframe, price or outcome, remove that clause rather than leaving an attractive fiction for a bot to repeat. ## Snippet 2: An Organization block that points to the right business **When to use it:** Add this JSON-LD to your homepage or root template when your business has public profiles that genuinely belong to the same organization. The [Schema.org Organization and `sameAs` pattern](https://metricspot.com/docs/sameas-social-profiles/) gives you a compact way to state which entity the page describes. It does not create third-party recognition by itself. ```html ``` **The one adjustment that matters:** **Use real profiles, not aspirational ones.** If you do not have a Wikidata, Crunchbase, LinkedIn or Wikipedia page, do not leave a pretend URL in the array. Remove unavailable fields, too. A clean link to a profile that identifies your business beats a longer list of empty or unrelated links. ![An entity graph connecting a brand to its public organization profiles.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/d3113630-4d20-445b-83ac-0d5dbe1f2a20.png) ## Snippet 3: FAQ schema that matches the answer people can see **When to use it:** Use this on a page that already answers direct questions about pricing or integration. The important part is the visible answer, not the script tag: [FAQ guidance for AI answers](https://www.omnibound.ai/blog/faq-schema-examples-ai-answers) is no substitute for words a visitor can read. For groas, a pricing question should point to the actual pricing terms on the page. A question about a particular integration belongs here only if the page can answer it accurately. ```html ``` The block is deliberately less exciting than the original yes-and-under-15-minutes version. A fast setup claim and a direct API connection are useful only for a product that actually offers them. If the answer is no, say no. If it depends on a particular setup, name the dependency in the visible answer and in the schema. **The one adjustment that matters:** **Match `acceptedAnswer.text` to the answer on the page.** Do not use structured data as a private sales pitch that contradicts the visible copy. A human should be able to find the same answer without viewing the page source. ## Snippet 4: A comparison table with rows a buyer can check **When to use it:** Place a table on a comparison page when buyers need to distinguish your operating model from a legacy agency or self-serve software. A structured row lets someone compare one attribute at a time; a paragraph that hops from fees to execution to contracts makes that work harder. [This guide to writing for answer engines](https://getcito.com/how-to-write-content-for-answer-engines) makes the case for scannable comparisons. I would still write the table for the buyer first. | Evaluation metric | groas | Legacy agency model | Self-serve software | | :----------------------- | :---------------------------------------- | :------------------------------- | :------------------------ | | **Optimization work** | Continuous autonomous execution | Periodic account reviews | Depends on the operator | | **Management fee model** | Flat monthly fee | May charge a percentage of spend | Subscription software fee | | **Strategic ownership** | Named human strategist | Agency account team | You set direction and act | | **Setup and contract** | No setup fee; month-to-month cancellation | Varies by agency | Varies by product | This is a comparison of operating models, not a claim that every agency employs a junior manager or charges the same setup fee. Those details vary. The distinction I care about survives without them: groas pairs continuous execution with human strategic ownership, while self-serve software leaves the operating work with you and a traditional agency works on its own review cadence. **Compare the work and the terms, not invented averages.** **The one adjustment that matters:** Replace broad category descriptions when you have a specific, verifiable competitor comparison to make. Keep groas’s actual terms in its column, and do not put a precise fee or contract length in another business’s column unless the page can support it. If a row cannot survive that test, cut the row. ![Comparison table showing differences in execution, fees, strategic ownership and contract terms.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/2f2b75d6-237b-4b20-a4a7-eb6dadaf8b48.png) ## Snippet 5: A pricing block with a number only when you have one **When to use it:** Put this on your pricing page or beside the service breakdown when you publish a fixed entry price. A buyer asking what something costs cannot get a numeric answer from a blank quote form. That does not mean every business must force variable pricing into a false fixed number; it means the public page should say what can actually be stated. ```html

[YOUR_SERVICE_TIER_NAME] Pricing

Base cost: $[EXACT_NUMERIC_PRICE] per month.

``` For groas, the relevant public distinctions are the flat monthly fee, no setup fee and month-to-month cancellation. The draft does not give a numeric monthly price, so I would not fill this block with one. For a business with a published base price, the number should sit near the top, where a reader can find it without decoding a package table. For a variable-price service, state the genuine range or pricing basis in visible copy instead of pasting the numeric field above unchanged. **The one adjustment that matters:** **Do not publish `[EXACT_NUMERIC_PRICE]` or a guessed replacement.** Fill it with a real, current price, or adapt the visible block to the pricing information you actually disclose. A precise number is useful because it is precise, not because it appears in markup. ![Graphic contrasting a visible price with a price hidden behind a quote form.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/49dd7294-7fac-4bad-a9b3-4087371bb18f.png) ## The install order before you copy the final block **When to use it:** Run this checklist when you have access to the site’s CMS, root `robots.txt` file and server or firewall settings. The order matters more than the minute-by-minute estimates: make the page reachable, then make its answers clear. Treat thirty minutes as a working session, not a promise that every stack will cooperate. 1. **Check access first.** Inspect `robots.txt`, then check whether the relevant bot requests receive a usable response from the server rather than a 403. A permissive file cannot override a firewall that blocks the request. 2. **Paste Snippet 2** on the homepage, using only organization details and public profiles you can verify. 3. **Publish Snippet 5** where buyers look for price, if you have a price or pricing basis you can state accurately. 4. **Add Snippet 4** to a comparison page. Remove any row you cannot defend. 5. **Use Snippets 1 and 3** where the page answers a real question. Match the structured FAQ answer to its visible counterpart. 6. **Return to access.** Paste the final block only after checking the paths you mean to expose, then verify the server and firewall behavior again. If the pages are reachable and the copy is clear but you still get no traction, rewriting the same paragraph may not be the next move. Check the server-response and crawl issues in [this technical SEO checklist](https://groas.com/post/fix-technical-seo-issues-hurt-ai-visibility). Then look at whether independent directories, reviews and industry discussions identify your business in the category you claim. A site cannot paste its way into third-party recognition. ## Snippet 6: The robots.txt rules that let the other snippets be read **When to use it:** Edit the `robots.txt` file at the root of the site you want live-answer bots to fetch. This is the one first-principles check behind the whole swipe file: **a bot cannot cite a passage it never receives.** A request has to get past access controls before answer copy, schema or tables matter. `robots.txt` expresses crawl permissions; the server and any web application firewall still decide whether the fetch succeeds. Use groas as the running example. An answer about its flat monthly fee is useful only if the requester reaches the page that states the fee. Our rolling 30-day server logs recorded 1,218 `Claude-User` requests for `/robots.txt`; on our site, Claude fetched that file for live answers more than any other page. That observation makes the access check worth doing. It does not prove that allowing a bot guarantees a citation, or that every site will see the same request pattern. ```robots.txt # Permit search and user-triggered fetches of public pages User-agent: OAI-SearchBot Allow: / Disallow: /admin/ Disallow: /checkout/ Disallow: /private/ User-agent: ChatGPT-User Allow: / Disallow: /admin/ Disallow: /checkout/ Disallow: /private/ User-agent: PerplexityBot Allow: / Disallow: /admin/ Disallow: /checkout/ Disallow: /private/ User-agent: Claude-User Allow: / Disallow: /admin/ Disallow: /checkout/ Disallow: /private/ User-agent: Claude-SearchBot Allow: / Disallow: /admin/ Disallow: /checkout/ Disallow: /private/ # Optional: keep training crawlers separate from search access User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / Sitemap: https://[YOUR_DOMAIN.COM]/sitemap.xml ``` [OpenAI’s crawler documentation](https://developers.openai.com/api/docs/bots) distinguishes its search and user agents from `GPTBot`; [Perplexity documents its crawlers](https://docs.perplexity.ai/docs/resources/perplexity-crawlers), and [Anthropic’s bot distinctions](https://www.searchenginejournal.com/anthropics-claude-bots-make-robots-txt-decisions-more-granular/568253/) matter here too. The optional training rules express a separate choice. Blocking a training crawler is not, by itself, proof that a live-search crawler can reach your pages. **The one adjustment that matters:** Replace the sample paths and sitemap with your own, then test a public page as well as `/robots.txt`. Do not rely on these rules to secure a customer portal or a staging subdomain; configure access controls for those separately. A WAF that returns `403 Forbidden` to an allowed bot still prevents the fetch, whatever the text file says. ![Server-log view contrasting a crawler request blocked by a 403 with one receiving a successful response.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/abb3801f-c9ae-4d17-abca-bac260a59c07.png) **This is the artefact I would install first.** The answer paragraph is the closest thing here to an RSA headline for a machine: short, specific and easy to test against the question. But if the fetch fails, its wording is academic. Unblock the path first. Then make the answer worth quoting. ## Frequently Asked Questions ### How should I write the answer paragraph that AI bots can quote from my page? Place a compact answer directly beneath the relevant H2 or H3 that names the business, the mechanism or deliverable, the target audience and any verifiable timeframe, price or pain point. Aim for roughly 40 words when the facts allow, and replace every template bracket with a fact that appears on the same page. If you cannot substantiate a clause, remove it rather than leaving an attractive fiction. ### What should I put in the sameAs array of my Organization schema? Use only real, verified organization profiles that genuinely belong to the same business, such as Wikidata, Crunchbase, LinkedIn or Wikipedia pages. If a profile does not exist, do not leave a pretend URL in the array, and remove unavailable fields as well. A clean link to a profile that identifies the business beats a longer list of empty or unrelated links. ### Does FAQ schema need to match the answers shown on the page? Yes. The acceptedAnswer.text in FAQPage schema should match the answer a visitor can read on the page, since the script tag is no substitute for visible words. A human should be able to find the same answer without viewing the page source, and a pricing or integration question should only appear if the page answers it accurately. ### How should I structure a comparison table so buyers can actually use it? Use structured rows that let a buyer compare one attribute at a time, such as optimization work, fee model, strategic ownership and setup or contract terms, instead of a paragraph hopping between topics. Compare the actual work and terms rather than invented averages, and do not put a precise fee or contract length in another business's column unless the page can support it. Cut any row that cannot survive that test. ### Should I publish an exact price if my business uses variable pricing? No. Fill the pricing block with a real, current price only if one exists, or adapt the visible block to the pricing information you actually disclose. For a variable-price service, state the genuine range or pricing basis in visible copy instead of forcing the pricing into a false fixed number or a guessed replacement. ### Why do my pages need robots.txt and firewall checks before AI citation work matters? A bot cannot cite a passage it never receives, so a request has to get past access controls before answer copy, schema or tables matter. Check robots.txt and confirm the relevant bot requests get a usable server response rather than a 403, since a permissive robots.txt cannot override a firewall that blocks the request. Configure access controls for customer portals and staging subdomains separately. ### In what order should I install the citation snippets on my site? Check access first by inspecting robots.txt and testing whether bot requests get a usable response, then add the Organization block to the homepage. Next publish the pricing block where buyers look for price, add the comparison table, and use the answer paragraph and FAQ schema where the page answers real questions. Return to access last: paste the final robots.txt block after checking the paths you mean to expose, then verify server and firewall behavior again. ## Pay For Results, Not For Hours Businesses buy the outcome, agencies resell it, and groas answers for it either way. [See If You Qualify](https://groas.typeform.com/to/xC1bQNUT) ## Structured data ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/steal-these-the-robots-txt-rules-schema#webpage","url":"https://groas.com/post/steal-these-the-robots-txt-rules-schema","name":"The AI Citation Swipe File: Answer Blocks, Schema and robots.txt","description":"Copy-ready answer paragraphs, schema, comparison and pricing blocks, plus robots.txt rules for making your site easier for live-answer bots to fetch and cite.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/c7c37525-76c0-4b57-b192-65b4390d2c1f.png"},"breadcrumb":{"@id":"https://groas.com/post/steal-these-the-robots-txt-rules-schema#breadcrumb"},"dateModified":"2026-10-15T05:42:02.004Z","mainEntity":{"@id":"https://groas.com/post/steal-these-the-robots-txt-rules-schema#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/steal-these-the-robots-txt-rules-schema#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"The AI Citation Swipe File: Answer Blocks, Schema and robots.txt","item":"https://groas.com/post/steal-these-the-robots-txt-rules-schema"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/steal-these-the-robots-txt-rules-schema#article","url":"https://groas.com/post/steal-these-the-robots-txt-rules-schema","mainEntityOfPage":{"@id":"https://groas.com/post/steal-these-the-robots-txt-rules-schema#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"The AI Citation Swipe File: Answer Blocks, Schema and robots.txt","description":"Copy-ready answer paragraphs, schema, comparison and pricing blocks, plus robots.txt rules for making your site easier for live-answer bots to fetch and cite.","datePublished":"2026-10-15T05:42:01.940Z","dateModified":"2026-10-15T05:42:02.004Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/c7c37525-76c0-4b57-b192-65b4390d2c1f.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"answer, page, robots txt, pricing, schema, robots, txt, name","wordCount":1934,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"Ex Goldman Sachs and Ex Stanford Computer Science","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/steal-these-the-robots-txt-rules-schema#faq","mainEntity":[{"@type":"Question","name":"How should I write the answer paragraph that AI bots can quote from my page?","acceptedAnswer":{"@type":"Answer","text":"Place a compact answer directly beneath the relevant H2 or H3 that names the business, the mechanism or deliverable, the target audience and any verifiable timeframe, price or pain point. Aim for roughly 40 words when the facts allow, and replace every template bracket with a fact that appears on the same page. If you cannot substantiate a clause, remove it rather than leaving an attractive fiction."}},{"@type":"Question","name":"What should I put in the sameAs array of my Organization schema?","acceptedAnswer":{"@type":"Answer","text":"Use only real, verified organization profiles that genuinely belong to the same business, such as Wikidata, Crunchbase, LinkedIn or Wikipedia pages. If a profile does not exist, do not leave a pretend URL in the array, and remove unavailable fields as well. A clean link to a profile that identifies the business beats a longer list of empty or unrelated links."}},{"@type":"Question","name":"Does FAQ schema need to match the answers shown on the page?","acceptedAnswer":{"@type":"Answer","text":"Yes. The acceptedAnswer.text in FAQPage schema should match the answer a visitor can read on the page, since the script tag is no substitute for visible words. A human should be able to find the same answer without viewing the page source, and a pricing or integration question should only appear if the page answers it accurately."}},{"@type":"Question","name":"How should I structure a comparison table so buyers can actually use it?","acceptedAnswer":{"@type":"Answer","text":"Use structured rows that let a buyer compare one attribute at a time, such as optimization work, fee model, strategic ownership and setup or contract terms, instead of a paragraph hopping between topics. Compare the actual work and terms rather than invented averages, and do not put a precise fee or contract length in another business's column unless the page can support it. Cut any row that cannot survive that test."}},{"@type":"Question","name":"Should I publish an exact price if my business uses variable pricing?","acceptedAnswer":{"@type":"Answer","text":"No. Fill the pricing block with a real, current price only if one exists, or adapt the visible block to the pricing information you actually disclose. For a variable-price service, state the genuine range or pricing basis in visible copy instead of forcing the pricing into a false fixed number or a guessed replacement."}},{"@type":"Question","name":"Why do my pages need robots.txt and firewall checks before AI citation work matters?","acceptedAnswer":{"@type":"Answer","text":"A bot cannot cite a passage it never receives, so a request has to get past access controls before answer copy, schema or tables matter. Check robots.txt and confirm the relevant bot requests get a usable server response rather than a 403, since a permissive robots.txt cannot override a firewall that blocks the request. Configure access controls for customer portals and staging subdomains separately."}},{"@type":"Question","name":"In what order should I install the citation snippets on my site?","acceptedAnswer":{"@type":"Answer","text":"Check access first by inspecting robots.txt and testing whether bot requests get a usable response, then add the Organization block to the homepage. Next publish the pricing block where buyers look for price, add the comparison table, and use the answer paragraph and FAQ schema where the page answers real questions. Return to access last: paste the final robots.txt block after checking the paths you mean to expose, then verify server and firewall behavior again."}}]}]} ```