Skip to content

refactor(cloud): write robots.txt at build-time #1922

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
sourishkrout merged 2 commits into from
Jan 14, 2026
Merged

refactor(cloud): write robots.txt at build-time #1922

sourishkrout merged 2 commits into from
Jan 14, 2026

Conversation

@sourishkrout
Copy link
Collaborator

@sourishkrout sourishkrout commented Jan 14, 2026 •
edited
Loading

Here's an example with the current build:

# robots.txt for https://mirascope.com
User-agent: *
Allow: /
Disallow: /login
Disallow: /organizations
Disallow: /dashboard/settings
Disallow: /dashboard
Disallow: /discord-invite

Sitemap: https://mirascope.com/sitemap.xml

This was referenced Jan 14, 2026
Merged
Merged
@sourishkrout Graphite App
Copy link
Collaborator Author

sourishkrout commented Jan 14, 2026 •
edited
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

@sourishkrout sourishkrout mentioned this pull request Jan 14, 2026
Closed
@sourishkrout sourishkrout marked this pull request as ready for review January 14, 2026 19:59
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Jan 14, 2026 •
edited
Loading

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
❌ Deployment failed
View logs		 v2-docs 9351fb2 Jan 14 2026, 10:23 PM

@sourishkrout sourishkrout force-pushed the 01-14-refactor_cloud_write_robots.txt_at_build-time branch from 33d121f to 6a29977 Compare January 14, 2026 20:11
graphite-app[bot]
graphite-app bot reviewed Jan 14, 2026
View reviewed changes
@sourishkrout sourishkrout force-pushed the 01-14-refactor_cloud_write_robots.txt_at_build-time branch from 6a29977 to 0eaca5d Compare January 14, 2026 20:19
@sourishkrout sourishkrout force-pushed the 01-14-refactor_cloud_special_treatment_for_docs_placeholder_ahead_of_dev_content_migration branch from b50e7d8 to 5efa09d Compare January 14, 2026 21:19
@sourishkrout sourishkrout force-pushed the 01-14-refactor_cloud_write_robots.txt_at_build-time branch from 0eaca5d to c1834ff Compare January 14, 2026 21:19
@sourishkrout sourishkrout force-pushed the 01-14-refactor_cloud_special_treatment_for_docs_placeholder_ahead_of_dev_content_migration branch 2 times, most recently from 56e851d to 999b82e Compare January 14, 2026 21:23
@sourishkrout sourishkrout force-pushed the 01-14-refactor_cloud_write_robots.txt_at_build-time branch from c1834ff to a86342f Compare January 14, 2026 21:23
@sourishkrout sourishkrout changed the base branch from 01-14-refactor_cloud_special_treatment_for_docs_placeholder_ahead_of_dev_content_migration to graphite-base/1922 January 14, 2026 21:59
@sourishkrout sourishkrout force-pushed the 01-14-refactor_cloud_write_robots.txt_at_build-time branch from a86342f to 252adf4 Compare January 14, 2026 21:59
@sourishkrout sourishkrout changed the base branch from graphite-base/1922 to v2 January 14, 2026 22:00
@sourishkrout sourishkrout mentioned this pull request Jan 14, 2026
Merged
@sourishkrout sourishkrout force-pushed the 01-14-refactor_cloud_write_robots.txt_at_build-time branch from 252adf4 to 67cd0f3 Compare January 14, 2026 22:06
@sourishkrout Graphite App
Copy link
Collaborator Author

sourishkrout commented Jan 14, 2026 •
edited
Loading

Merge activity

  • Jan 14, 10:06 PM UTC: Graphite rebased this pull request as part of a merge.
  • Jan 14, 10:08 PM UTC: Graphite rebased this pull request as part of a merge.
  • Jan 14, 10:11 PM UTC: @sourishkrout merged this pull request with Graphite.

@sourishkrout sourishkrout force-pushed the 01-14-refactor_cloud_write_robots.txt_at_build-time branch from 67cd0f3 to 9351fb2 Compare January 14, 2026 22:07
@sourishkrout sourishkrout merged commit 90144eb into v2 Jan 14, 2026
10 checks passed
@sourishkrout sourishkrout deleted the 01-14-refactor_cloud_write_robots.txt_at_build-time branch January 14, 2026 22:11
@sourishkrout sourishkrout mentioned this pull request Jan 14, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@graphite-app graphite-app[bot] graphite-app[bot] left review comments

@willbakst willbakst willbakst approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sourishkrout @willbakst