Create docs extension for the flexible retrieval of software library documentation, based on the evaluation of strings representing qualified names of library modules & components #13

This is meant to be a convenient way to retrieve documentation for relevant Python software libraries based on qualified names.

There are two possible approaches for realizing this:

1. Interpret and export docstrings from libraries

Since many Python libraries use docstrings and some Sphinx configuration to generate documentation, we could grab those docstrings directly, and map their library component names to them in a json file. This could be done in a docker container that exposes a tiny API for retrieving that docstring documentation.

Pros

• Full control over which Python libraries to offer docstrings for.

Cons

• Docstrings might not be fully readable (due to missing preprocessing steps by Sphinx).
• Docstrings would be non-uniform, due to varying reST style choices.
• Docstrings may be incomplete or non-existent on some public library components (though unlikely to be a problem).

2. Scrape DevDocs

DevDocs hosts uniformly presented documentation for the most important FOSS software technologies, using on-demand web scraping. The uniformity makes it particularly suitable for scraping using BeautifulSoup. Scraping might also be possible locally in a Docker container, since the project is FOSS itself.

Pros

• Preparsed documentation.
• A lot of uniformity between Sphinx Python projects on DevDocs in terms of how they're presented.

Cons

• Zero control over what is made available to scrape.
• No way to include ecosystem-libraries (e.g. pygame_gui)

The current docs_pre edition of this extension imports modules at runtime to support showing their docstrings, but this is inefficient and has some low hanging fruit for optimization.

As of 2025, the best solution is to mix the current setup (post optimization) and option 2.