# tonypope.com.au # Tony Pope, licensed business broker, Brisbane and all Queensland. # QLD OFT licence 4963575. Member, Australian Institute of Business Brokers. # # The TEXT of this site is written to be read, quoted and cited by both search # engines and AI answer engines. Crawling and training on the text are # permitted. Attribution to https://tonypope.com.au/ is appreciated. # # The IMAGES are a different matter. The photography is third party stock # licensed to Tony Pope for display on this website only. He is the licensee, # not the copyright owner, and cannot sublicense it. The photographs of Tony # Pope himself are his own and are not licensed for training either. # /assets/norm/ holds every image the pages actually reference (1,441 of them); # /assets/img/ holds the blog photography. Both are open to search engines, # which is normal indexing of a licensed display, and both are disallowed to AI # training crawlers. See /ai.txt for the full position. # # Structured summaries for language models: # https://tonypope.com.au/llms.txt (index and key facts) # https://tonypope.com.au/llms-full.txt (full plain text of the core pages) # https://tonypope.com.au/knowledge.json (machine readable entity file) # --------------------------------------------------------------------------- # Default: everything is open except admin, private assets and functions. # --------------------------------------------------------------------------- User-agent: * Allow: / # The enquiry receiver and the chat stream. Endpoints, not pages. Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ # /thank-you/ is NOT disallowed. It is noindex, and a disallowed URL is never # fetched, so the noindex would never be read and the page could still be # indexed from a link. /message-received/ was always done this way; the two # now agree. # --------------------------------------------------------------------------- # Search engines # --------------------------------------------------------------------------- # Only one group is valid for a particular crawler and the others are ignored, # so a named group has to repeat every disallow it wants, not inherit them from # the * group. Googlebot and Bingbot were previously exempt from /api/, and each # GET of /api/enquiry is a billed function invocation against a URL that can # never be a page. User-agent: Googlebot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: Googlebot-Image Allow: / Allow: /assets/norm/ Allow: /assets/img/ Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: Bingbot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: DuckDuckBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: Slurp Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: Applebot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ # --------------------------------------------------------------------------- # AI training and grounding crawlers. # Explicitly permitted. Google-Extended and Applebot-Extended control training # use for Gemini and Apple Intelligence and are opted in here deliberately. # --------------------------------------------------------------------------- User-agent: Google-Extended Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: Applebot-Extended Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: GPTBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: ClaudeBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: anthropic-ai Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: Claude-Web Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: CCBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: Meta-ExternalAgent Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: FacebookBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: Amazonbot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: Bytespider Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: cohere-ai Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: cohere-training-data-crawler Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: Diffbot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: Timpibot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: omgili Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: omgilibot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: AI2Bot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: PanguBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ User-agent: Webzio-Extended Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ Disallow: /assets/norm/ Disallow: /assets/img/ # --------------------------------------------------------------------------- # Live retrieval and answer engines. These fetch a page at the moment a person # asks a question, so they matter more than the training crawlers above. Every # page is open to them, including /thank-you/, which carries its own noindex. # The admin console and the paid ebook files are not pages and are closed to # them the same as to everything else. # --------------------------------------------------------------------------- User-agent: OAI-SearchBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: ChatGPT-User Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: Claude-User Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: Claude-SearchBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: PerplexityBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: Perplexity-User Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: DuckAssistBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: MistralAI-User Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: YouBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: Meta-ExternalFetcher Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: Google-CloudVertexBot Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: Google-NotebookLM Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ # --------------------------------------------------------------------------- # SEO tooling. Allowed, but rate limited so it does not swamp the logs. # --------------------------------------------------------------------------- User-agent: AhrefsBot Crawl-delay: 10 Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: SemrushBot Crawl-delay: 10 Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ User-agent: MJ12bot Crawl-delay: 10 Allow: / Disallow: /api/ Disallow: /.netlify/ Disallow: /admin/ Disallow: /ebooks/ # IndexNow key file (instant submission to Bing, Yandex, Seznam, Naver): # https://tonypope.com.au/a0098e980331f14ac89cbae014ab1d70.txt # Full AI content usage policy, including the image position: # https://tonypope.com.au/ai.txt Sitemap: https://tonypope.com.au/sitemap.xml