Can customers find your page?
Think about ordinary search results and the pages you want potential customers to visit.
Website owner’s guide · Reviewed October 31, 2026
Three website choices.
One clearer decision.
Being found in search, informing an AI answer and helping train an AI model are different uses of your content. Start with your business goals before changing website controls.
Based on October 30 announcements. These are provider rollout plans and a draft technical proposal—not a new legal filing requirement.
Know the difference
Can your page inform an answer?
“Grounding” means using web content to inform an AI-generated response. It may create a path back to your site.
Can content help train a model?
Training changes a model’s learned parameters. It is a separate use from answering a current question.
What was announced?
Google: plans to support a shared robots.txt training-preference signal in 2027. Existing Google-specific controls remain relevant. Google-Extended choices can also affect Gemini Apps grounding; review Google’s explanation.
Microsoft: new training and grounding preferences begin taking effect on crawl from February 1, 2027. Existing NoArchive/NoCache AI preferences stop being honored on March 1, 2027; affected owners should follow the transition instructions. Eligible sites’ temporary grounding transition settings open in Bing Webmaster Tools on December 15, 2026.
Microsoft separates training from grounding and says these preferences do not change eligibility for ordinary Bing search. Grounding restrictions reduce AI visibility; neither eligibility nor allowing use guarantees traffic.
The shared signal is an individual Internet-Draft, not an adopted IETF standard. A published setting is not proof that every provider already honors it. This guide does not change your website settings.
Google’s October 30 announcement · Microsoft’s announcement and transition steps · Draft status
No purchase or signup required
Your website AI-controls review
Check off each question as you discuss it with your website manager. Checkmarks stay in this browser on this device; no answers are submitted.
Do not use crawler preferences as a way to protect private client information. Keep sensitive material out of public pages.
Message to your website manager
“Please inventory our current crawler, training and AI-answer settings. Explain which providers they affect, whether we use NoArchive or NoCache, and any transition steps. Recommend changes based on our content and customer-discovery goals before implementing them.”
Watch the 45-second explainer
Silent, captioned educational animation. It explains the choices; it does not demonstrate a tested provider configuration.
Read the transcript
Search visibility, AI answers and model training are different uses. Grounding uses your page to inform an AI answer; training changes a model. On October 30, Google and Microsoft announced shared training-preference plans. Google targets 2027. Microsoft begins honoring new preferences on February 1, 2027. Existing NoArchive and NoCache users should review the March 1 transition. Ask your website manager what is already set before changing anything. Use our free checklist and follow Let’s Go SMALL for practical updates.