your site might be invisible to chatgpt, claude and perplexity
one wrong line in robots.txt (or a cloudflare setting) can block their bots, and most makers never check
paste this prompt into cursor or claude and it writes your robots.txt and llms.txt for you π
Create two files for my website [your domain]:
1. robots.txt
- Allow all normal search engines
- Allow these AI bots so my site can show up in AI answers: OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot
- Also allow these if I'm fine with AI training on my site: GPTBot, ClaudeBot
- Block private pages like /admin, /api, /dashboard and /login
- End with my sitemap: https://[your domain]/sitemap.xml
2. llms.txt (follow the https://t.co/HibjI6K7Kg format)
- Start with "# [site name]"
- Then a one-line summary as a "> " quote
- Then a short paragraph on what the site does and who it's for
- Then "## " sections with links like this: - [Page name](full URL): one-line description
- Include my homepage, pricing, docs, blog and main features
- Put less important pages under "## Optional"
What my site does: [2 to 3 lines about your product]
My main pages: [paste your URLs]
Give me both files ready to copy, and tell me where to put them.
#buildinpublic #SEO
Hey X π
Looking to #connect with people building:
- AI agents
- SaaS
- side projects
- Startups from day 0
Say hi and let's connect π€
Building something? Drop it below π
@billgnofficial Itβs only been 20 days since I launched IndieLaunch, and weβve already had 784 visitors, 3.5K page views, and 120 products launched! Still early, but excited to see where this goes π
@naveenmio You should check it out π
https://t.co/zvO6EiVWQu
A launch platform built for SaaS builders and indie makers.
Get your product featured for just $2.99 β only for the next 7 builders!
@coryshoaf@billgnofficial You should check it out π
https://t.co/zvO6EiVWQu
A launch platform built for SaaS builders and indie makers.
Get your product featured for just $2.99 β only for the next 7 builders!
@RektProLLC@nicklaunches You should check it out π
https://t.co/zvO6EiVWQu
A launch platform built for SaaS builders and indie makers.
Get your product featured for just $2.99 β only for the next 7 builders!
@RafVantongerloo@nicklaunches You should check it out π
https://t.co/zvO6EiVWQu
A launch platform built for SaaS builders and indie makers.
Get your product featured for just $2.99 β only for the next 7 builders!
@eduard_pantazi@nicklaunches You should check it out π
https://t.co/zvO6EiVWQu
A launch platform built for SaaS builders and indie makers.
Get your product featured for just $2.99 β only for the next 7 builders!
if you're shipping with claude code, I built LaunchReadyKit for the rest of your launch checklist
/launch-check runs 440 checks (security, payments, SEO and more) and gives you a fix list with the exact file and line
https://t.co/RyYCuGdKw6
your site might be invisible to chatgpt, claude and perplexity
one wrong line in robots.txt (or a cloudflare setting) can block their bots, and most makers never check
paste this prompt into cursor or claude and it writes your robots.txt and llms.txt for you π
Create two files for my website [your domain]:
1. robots.txt
- Allow all normal search engines
- Allow these AI bots so my site can show up in AI answers: OAI-SearchBot, ChatGPT-User, Claude-SearchBot, Claude-User, PerplexityBot
- Also allow these if I'm fine with AI training on my site: GPTBot, ClaudeBot
- Block private pages like /admin, /api, /dashboard and /login
- End with my sitemap: https://[your domain]/sitemap.xml
2. llms.txt (follow the https://t.co/HibjI6K7Kg format)
- Start with "# [site name]"
- Then a one-line summary as a "> " quote
- Then a short paragraph on what the site does and who it's for
- Then "## " sections with links like this: - [Page name](full URL): one-line description
- Include my homepage, pricing, docs, blog and main features
- Put less important pages under "## Optional"
What my site does: [2 to 3 lines about your product]
My main pages: [paste your URLs]
Give me both files ready to copy, and tell me where to put them.
#buildinpublic #SEO