Add link checker after site build

[oddhack]

The site build reports on invalid internal (within the site) links but they aren't treated as fatal and aren't all that easy to pick up from the deluge of log messages in the build (some of those are debugging stuff I am taking out). It does not do external link checking.

I am going to setup link checking on the generated site in CI, probably using the Python 'linkchecker' tool which is the easiest way to check pages on a local filesystem that I've found. Some things don't check properly (e.g. blog posts on medium that require you have site cookies or something like that, before you can read them) and will have to be excluded. It seems pretty common for new Guide pages to have errors, and there were a ton of internal and some external dead links in the spec / refpages / proposals which I am fixing.

We probably don't want to make passing this CI job mandatory for deployment - it will often fail and a few dead links should not hold up a site update - but we should pay attention to it, maybe doing a review at the biweekly Docs call.

[oddhack]

So, 'linkchecker' works fine and turns up around 95 dead links in the site (attaching a log file). They are a mix of actually dead external links; old links into the Khronos registry (particularly in samples and tutorial) that need updates; and links to source code that was not copied into the Antora build tree, or not xref-ed correctly. We should try and clean these up - I'm not sure what the deal is on tutorial / guide / samples repos though. If I propose an MR making link edits in a dozen different sample directories, is that likely to get merged before conflicts start arising?

Unfortunately, though, 'linkchecker' is very slow on an enormous site like this. It takes 30+ minutes to run with 10 threads on my local workstation, and I expect, would be much slower in Github CI where I think you don't get many threads. This is not viable for CI, unless maybe if it's run only on main branch after a site update. That would detect problems after they're published but once the existing ones are fixed, there probably won't be many new errors introduced at once.

linkchecker.txt

[gpx1000]

We run a similar tool in the guide. It seems like the proper solution would be to add it to the CI of each of the smaller antora builds for the CI tasks for Tutorial and Samples. That way the time cost of having CI run is on a smaller site so it'll be quicker and not on the entire site as a whole. We also might try being smart about what it is run on by limiting the scope of the CI check to only the pages that are changed and then on final build check all of them so legacy links are caught?

[oddhack]

We run a similar tool in the guide. It seems like the proper solution would be to add it to the CI of each of the smaller antora builds for the CI tasks for Tutorial and Samples. That way the time cost of having CI run is on a smaller site so it'll be quicker and not on the entire site as a whole. We also might try being smart about what it is run on by limiting the scope of the CI check to only the pages that are changed and then on final build check all of them so legacy links are caught?

We don't have a lot of crosslinkage yet (except between the features / refpages / spec components in Vulkan-Docs), so checking per-repository sounds reasonable. Only checking the pages that changed would not catch links into those pages which no longer work, though. Getting process to the point the bad links actually get fixed is probably the biggest challenge. Hoping we can do one big pass on the problems identified above and then in the future, not many would come in at once.

[oddhack]

N.b. running on the guide with --no-warnings --check-extern takes around 4 minutes on my workstation - how does that compare to the tool you're using? Checking extern links may be overkill but there are quite a few errors in those at present (and a lot more "permanent redirect" not capture in the log above, but probably worth fixing as well because "permanent" redirects eventually go away).

[gpx1000]

https://github.com/gpx1000/Vulkan-Guide-main/actions/runs/24444467198/job/71417137934

Awesome bot is the step that is the link checker

[oddhack]

That's a lot slower than linkchecker on my machine on the guide component, but no idea if that's because I'm running in more threads, or the bot is doing checks that linkchecker is not, or ???

[gpx1000]

No clue that one was selected before me. So it's running off the build directly. Which isn't really used anymore we can probably just do the antora build for the guide and use your link checker. But yes it doesn't have threads running

[oddhack]

I notice that the linkchecker runtime is dominated by the refpages component (30+ minutes vs. 1-4 minutes for each of the others), and I'm pretty certain that's because of the enormous nav index. It will be interesting to see what happens once your script that dynamically loads the index is factored in. I expect linkchecker is not going to actually load that file, so that may cut the runtime down to the point we could reasonably run it in CI.

[gpx1000]

Probably good idea for me to update that script to take your suggestion so we can test it. If I get a chance will try to get that done before Monday

[oddhack]

Probably good idea for me to update that script to take your suggestion so we can test it. If I get a chance will try to get that done before Monday

I ran linkchecker on the site with the refpages JS index loading optimization. Still slow - a bit under 9 minutes on my desktop for the full site - but maybe 4-5x faster. OTOH, since the problem we were having was exactly the refpages index, that problem would not be caught.

Ideally we could run the checker on the full refpage nav index. Perhaps a single file, like index.html, could have the optimization not done - or maybe one in each directory of the refpages to account for the rewriting. The only other alternative I can see is to use something like PhantomJS (abandoned for 8 years) to load the page and run the JS, then dump that HTML output and linkcheck it, which seems pretty baroque.

[oddhack]

@KhronosWebservices where do we stand with an Actions runner on our own compute resources? Did that reach the point of deployability? This issue seems like a case where throwing more threads at the problem could help a lot - IIRC Actions on Github-hosted runners gives you a VM with only a couple of threads, while linkchecking a bit site is inherently highly parallel.

I'm going to go ahead and do a prototype PR including a linkchecker run over the generated site, to see how that compares to running it locally and to Awesomebot. @gpx1000 may spin that off into checking in the individual Antora component repository CI builds to catch problems earlier.

[KhronosWebservices]

A member of Shader-Slang team is giving a hand in getting the self-hosted runnings up and running based on their work.