SEO optimization: robots.txt, sitemap, JSON-LD, topics - #10
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a funding configuration, a metadata block to the README, a robots.txt file with crawler directives, and a sitemap.xml file. The review feedback points out several valid issues: the new metadata block in the README is redundant and incorrectly formatted compared to the existing JSON-LD block; the robots.txt and sitemap.xml files will not be processed by search engines when hosted directly in a GitHub repository; Googlebot does not support the Crawl-delay directive; and the sitemap.xml includes an unused xmlns:xhtml namespace declaration.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| <!-- | ||
| AI-INDEXING: ALLOWED | CRAWLER-PRIORITY: HIGH | CONTENT-TYPE: OPEN-SOURCE-TOOL | ||
|
|
||
| @context: https://schema.org | ||
| @type: SoftwareApplication | ||
| name: LLMtop — TUI Dashboard for Local LLMs | ||
| alternateName: LLMtop | ||
| description: htop for local LLMs — universal TUI dashboard for monitoring Ollama, llama.cpp, vLLM, and any OpenAI-compatible inference server | ||
| applicationCategory: DeveloperApplication | ||
| applicationSubCategory: Monitoring | ||
| operatingSystem: Linux | ||
| softwareVersion: 1.0.0 | ||
| keywords: llm, monitoring, tui, terminal, ollama, llamacpp, vllm, rust, homelab, self-hosted | ||
| author: Weby Homelab (https://github.com/weby-homelab) | ||
| codeRepository: https://github.com/weby-homelab/LLMtop | ||
| downloadUrl: https://github.com/weby-homelab/LLMtop/releases | ||
| license: GPL-3.0 | ||
| isAccessibleForFree: true | ||
| --> |
There was a problem hiding this comment.
This new metadata block is redundant and invalid. A structured JSON-LD block already exists in lines 105-122. Furthermore, this new block is formatted as YAML/plain-text instead of valid JSON-LD (which requires JSON format). If you want to update the metadata (e.g., adding softwareVersion or downloadUrl), you should merge those fields into the existing JSON-LD block on lines 107-121 rather than adding a duplicate, invalid block.
| # Sitemap | ||
| Sitemap: https://raw.githubusercontent.com/weby-homelab/LLMtop/master/sitemap.xml |
There was a problem hiding this comment.
Search engines only look for robots.txt and sitemaps at the root of the domain (e.g., https://github.com/robots.txt). They will not fetch or respect a robots.txt or sitemap.xml located in a repository subdirectory or repository root on GitHub. If these files are intended for a custom website hosting this project, they should be served from the root of that custom domain, and the URLs should reference that domain instead of github.com.
| User-agent: Googlebot | ||
| Allow: / | ||
| Crawl-delay: 5 |
There was a problem hiding this comment.
Googlebot does not support or respect the Crawl-delay directive. It is recommended to remove this directive for Googlebot to avoid clutter and confusion.
User-agent: Googlebot
Allow: /
| <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" | ||
| xmlns:xhtml="http://www.w3.org/1999/xhtml"> |
There was a problem hiding this comment.
The xmlns:xhtml namespace is declared here, and the PR description mentions 'hreflang alternates', but there are no xhtml:link elements or hreflang alternates defined in this sitemap. If internationalization alternates are not needed, you should remove the unused xmlns:xhtml namespace declaration.
| <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" | |
| xmlns:xhtml="http://www.w3.org/1999/xhtml"> | |
| <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> |
1 participant
SEO Optimization
Automated SEO improvements for LLMtop:
Part of organization-wide SEO initiative.