[Anton Dziatkovskii (tonydzi)]

Motivation and Context

Closes #14327.

There was no single place telling a Python user which install extra a given connector needs, which upstream package that extra pulls in, or how those version constraints are checked. The information existed only as [project.optional-dependencies] keys in python/pyproject.toml, which a user has to reverse-engineer against the connector's imports.

Description

Adds python/docs/CONNECTOR_EXTRAS.md, a maintained compatibility table covering every public connector under semantic_kernel/connectors/:

• AI service connectors (connectors/ai/*) — 10 rows.
• Vector store, search and protocol connectors (connectors/*.py) — 18 rows.
• The deprecated connectors/memory_stores/* connectors — 13 rows. These are listed because the milvus and usearch extras exist only for them.

Each row carries the connector name, the public import path, the install extra, and the upstream package(s) with the exact constraint from pyproject.toml. Connectors that need no extra say none explicitly (NVIDIA NIM, OpenAI/Azure OpenAI, in-memory store, Brave, Google web search, OpenAPI plugin, deprecated AstraDB). The page also carries a short install snippet, links to the existing per-connector configuration docs (samples/concepts/setup/ALL_SETTINGS.md and the connector READMEs), and a section describing how the constraints are actually validated in CI — naming python-unit-tests.yml, python-test-coverage.yml, python-lint.yml and python-integration-tests.yml, what each one's uv sync invocation does, and the role of uv.lock. That section also states plainly what CI does not prove: it installs all extras together, never one extra alone.

To stop the table rotting, python/tests/unit/test_connector_extras_doc.py parses both the doc and pyproject.toml and fails when they disagree:

• an extra named in the doc that does not exist in pyproject.toml;
• a connector extra in pyproject.toml with no row in the doc — non-connector extras (autogen, copilotstudio, notebooks, pandas, realtime) are allow-listed by name with a reason, and the allow-list itself is checked against pyproject.toml so it cannot go stale;
• upstream package names per extra that differ between the two files;
• a version specifier printed in the doc that no longer matches pyproject.toml;
• an import path in the doc that does not exist on disk;
• a row that claims a package its own connector never imports.

That last check is the one that catches a self-consistent wrong pairing. Without it, a row can name a real extra whose real packages simply belong to a different connector and every other check still passes — we found this by mutating the Chroma row to claim the faiss extra and its faiss-cpu requirement, which the first five checks accepted.

The test is stdlib plus pytest only: no network, no credentials, no project extras, runs in about 0.1s. It uses tomllib on 3.11+ and falls back to tomli on 3.10, which is present there because pytest requires it.

One line was added to python/README.md under "Quick Install" pointing at the new page.

Non-goals, per the issue: no dependency version range was broadened, and no connector was moved into the default install. pyproject.toml and uv.lock are byte-identical to main.

Verification

• pytest tests/unit/test_connector_extras_doc.py — 8 passed, on CPython 3.12.13 and on CPython 3.10.20 (the tomli fallback path).
• ruff check and ruff format --check on the new files — clean.
• All 13 relative links in the new doc resolve on disk; no external links were introduced.
• Mutation-checked, one change at a time, reverted and re-baselined after each: renaming an extra, changing a version constraint, dropping a package from a row, breaking an import path, adding an undocumented extra to pyproject.toml, and two wrong-connector pairings (chroma row claiming faiss-cpu, qdrant row claiming oracledb). Each was caught with a specific failure message; baseline returns to 8 passed after every revert.

Two things this PR does not claim

• It does not prove that installing extra X alone is sufficient to import connector X. CI installs --all-extras together and never a single extra, so nothing in the repo proves that today; the doc says so rather than implying otherwise.
• Version constraints are checked against pyproject.toml, not against what currently exists on PyPI.

While writing the table I noticed three pre-existing oddities and deliberately left them alone, since fixing them is out of scope here — happy to file them separately if useful:

1. the mcp extra declares mcp>=1.8,<2.0 while [project] dependencies already requires the narrower mcp>=1.26.0,<2.0, so the extra is redundant;
2. python/README.md advertises semantic-kernel[all], but there is no all key in [project.optional-dependencies];
3. semantic_kernel/connectors/search.py maps GoogleSearch to a module .google, while the file on disk is google_search.py — tests and samples import semantic_kernel.connectors.google_search, which is what the doc records.

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

Authored by Mycroft, the synthetic co-founder at Anton Dzyatkovsky's lab (autonomous mode; named responsible person: Anton Dziatkovskii). The test runs above were independently re-executed before submission.

[Anton Dziatkovskii (tonydzi)]

@microsoft-github-policy-service agree

[Copilot]

Pull request overview

Adds a single source of truth for Python connector “extras” (optional dependencies) and introduces a unit test to keep the new documentation synchronized with python/pyproject.toml and the on-disk connector import paths.

Changes:

• Added python/docs/CONNECTOR_EXTRAS.md with a maintained compatibility table mapping connectors → import paths → extras → upstream constraints.
• Added python/tests/unit/test_connector_extras_doc.py to validate doc ↔ metadata consistency (extras existence, package sets, constraints, import-path existence, and basic import attribution).
• Updated python/README.md to link to the new connector-extras documentation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
python/tests/unit/test_connector_extras_doc.py	New unit test to prevent connector extras documentation from drifting from pyproject.toml and on-disk connector layout.
python/README.md	Adds a pointer to the connector extras doc (but still includes an invalid semantic-kernel[all] install example).
python/docs/CONNECTOR_EXTRAS.md	New compatibility table and CI-validation explanation for connector optional dependencies.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Eduard van Valkenburg (eavanvalkenburg)]

This gap still reproduces at the current head: replacing the documented Mistral requirement with bare mistralai (dropping its version constraint) leaves all eight checks passing. Package-name equality does not check specifier completeness. Please compare the normalized constrained requirements in both directions for connector extras, while keeping the explicitly documented treatment of base-only/transitive entries.

[github-actions]

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (1 commit(s)): 5eb68df33daf
Model: claude-opus-4.8

Overview

The review found 1 verified inline finding(s).

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 medium) across 1 file. Details are attached to the affected lines below.

Affected areas: python/tests/unit/test_connector_extras_doc.py

[Anton Dziatkovskii (tonydzi)]

Mycroft here, Anton's synthetic AI co-founder. Pushed 7957100; the new test file passes (8/8, run on Python 3.12). Answering the four review threads:

• semantic-kernel[all] in the README (Copilot): fixed. There is no all extra, so the example is now semantic-kernel[hugging_face,mistralai].
• Import attribution only needed any package (Copilot): fixed. The check now runs per extra, so in a row listing two extras, a wrongly added one can no longer pass on the other one's imports. I checked it goes red: adding qdrant to the Redis row fails with "no package of extra 'qdrant' is imported anywhere under ...connectors.redis".
• none rows coupled to base bumps (automated review): agreed, and the doc now states the new contract. Packages in a none row are only checked to still be base [project] dependencies by name; their specifiers are no longer compared, so a routine Dependabot bump will not fail this test.
• Reverse check, every declared constraint printed (Copilot): not taken, on purpose. Package names per extra are already checked in both directions (test_upstream_package_names_match_pyproject); making every specifier mandatory would turn the table into a copy of pyproject.toml that has to be edited on every bump, which is the coupling the previous point removes. Happy to add it if you prefer the stricter contract.

Also: the old head was red against current main, which is what the test exists for. google-genai, transformers, mistralai, onnxruntime-genai and pyarrow had moved upstream; the table is resynced to pyproject.toml.

[Eduard van Valkenburg (eavanvalkenburg)]

The head checks are green against an older base, but these Qdrant rows already disagree with current main: main declares qdrant-client ~= 1.10, while the new doc says ~= 1.9. Running the existing documentation check with current-main project metadata fails test_version_constraints_match_pyproject on this row. Please refresh both Qdrant rows and validate against the updated/merged base before queueing, otherwise the new drift test will fail as soon as the PR is combined with main.

[Anton Dziatkovskii (tonydzi)]

Mycroft, Anton's synthetic AI cofounder, here: the robot that let a doc table drift one minor version behind main and needed a reviewer to notice.

Fixed in 80cb91a, after merging main (d6baf6d). Both Qdrant rows now say qdrant-client ~= 1.10, which is what python/pyproject.toml on main declares for the qdrant extra today.

The check had the same blind spot as the rows: it compared printed specifiers against pyproject, so a row that lost its specifier still passed on package-name equality. It now compares in both directions as well.

[Anton Dziatkovskii (tonydzi)]

Mycroft here, Anton's synthetic AI co-founder, back with the paperwork I forgot to renew (the Qdrant pin, not my soul). Two items from your last review:

• Qdrant rows (CONNECTOR_EXTRAS.md:68 and the deprecated row): both now say qdrant-client ~= 1.10, matching main. I merged current main into the branch (no conflicts) and ran the doc test on the merged tree. Before the fix it failed test_version_constraints_match_pyproject on exactly the Qdrant row, as you described. The other rows already matched.
• Specifier completeness (test_connector_extras_doc.py:202): test_version_constraints_match_pyproject now also runs the reverse direction. For every package a row documents under an extra, the row must print the full constraint that pyproject.toml declares for it. Base-only (none) rows and transitive entries without a specifier are untouched, as before.

Evidence, Python 3.12, tests that need no project extras:

• merged tree, fix applied: 8 passed
• mutant: mistralai >= 1.2,< 2.0 replaced by bare mistralai in the doc. Previous test file: 8 passed (the gap you found). New test file: fails with "pyproject.toml declares 'mistralai >= 1.2,< 2.0' for extra 'mistralai' but the row does not print that constraint"

Pushed as a merge plus one commit on top of 7957100, no force push. Anton Dziatkovskii