Feat/crewai integration

[Shindevrp]

Description

First-class CrewAI integration for Semantica: SemanticaKGTool, SemanticaDecisionTool, and SemanticaKnowledgeSource expose the Semantica context/knowledge graph to CrewAI agents and tasks — including policy-based decision intelligence, checkpoint (de)serialization against real crewai, and a fully functional degraded mode when crewai is not installed.

Type of Change

• New feature (non-breaking change which adds functionality)

Related Issues

1. Closes [FEATURE] Semantica CrewAI Integration #962

Changes Made

• SemanticaKGTool - graph operations for agents: extract_entities, extract_relations, add_to_graph, query_graph, - find_related, with per-graph concurrency locking and confidence scoring
• SemanticaDecisionTool - check_policy / record_decision with bool/numeric coercion, guarded causal-chain tracing, and uniform field-name support (hyphen/dot/space keys)
• SemanticaKnowledgeSource - chunk, embed, and index graph content into CrewAI knowledge; explicit, actionable error when storage is wired but saving fails
• Checkpoint hardening for real crewai 1.15.16: excluded live state survives model_dump(mode="json"), restored tools/sources flag reconstructed_state, knowledge source no longer masks lost live state during restore
• Degraded mode: tools/sources still import and expose run()/arun() when crewai is absent, plus a semantica[crewai] extra
• Docs, READMEs, CHANGELOG, and 243 new/extended tests (crewai + agno integration suites)

Testing

• Tested locally
• Added tests for new functionality
• Package builds successfully (python -m build)

Test Commands

# Build the package
pip install build
python -m build

# Integration suites (real-crewai subprocess tests need a venv with crewai 1.15.16)
pytest tests/integrations/crewai/ tests/integrations/agno/

# Format code
black integrations/crewai/ tests/integrations/crewai/
isort integrations/crewai/ tests/integrations/crewai/
flake8 --max-line-length=88 --extend-ignore=E203,W503 integrations/crewai/ tests/integrations/crewai/

Documentation

• Updated relevant documentation
• Added code examples if applicable
• No documentation changes needed
• Updated API reference if adding new APIs
• Updated cookbook if adding new examples

Breaking Changes

Breaking Changes: No

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• Package builds successfully

Additional Notes

Verified against real crewai 1.15.16 (separate venv) and in degraded mode (import-blocked). 108 crewai + 135 agno tests pass; flake8/black/isort clean; python -m build succeeds.

[qodo-code-review]

ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing

[qodo-free-for-open-source-projects]

PR Summary by Qodo

Add first-class CrewAI integration (tools, knowledge source, checkpoints)

✨ Enhancement 🧪 Tests 📝 Documentation ⚙️ Configuration changes 🕐 40+ Minutes

AI Description

• Add CrewAI tools/knowledge source exposing ContextGraph and AgentContext to crews.
• Harden checkpoint serialization and degraded-mode behavior when CrewAI isn’t installed.
• Add CrewAI docs, install extras, and comprehensive stub + subprocess integration tests.

Diagram

graph TD
  crewai{{"CrewAI (optional)"}} --> kg(["SemanticaKGTool"]) --> sem(["Semantica core (ContextGraph/extractors)"])
  crewai --> dt(["SemanticaDecisionTool"]) --> sem
  crewai --> ks(["SemanticaKnowledgeSource"]) --> sem
  ks --> store[("CrewAI knowledge storage")]

  subgraph Legend
    direction LR
    _ext{{"External"}} ~~~ _svc(["Component"]) ~~~ _db[("Storage")]
  end

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Publish integration as a separate package (semantica-crewai)

• ➕ Hard isolation of optional dependencies and faster base install
• ➕ Allows independent release cadence and compatibility tracking for CrewAI versions
• ➖ More packaging/release overhead and duplicated docs
• ➖ Harder for users to discover; worse DX than semantica[crewai] extra

2. Function-only tool wrappers instead of BaseTool subclasses

• ➕ Less coupling to CrewAI’s Pydantic serialization and lifecycle hooks
• ➕ Potentially simpler surface area if CrewAI supports callable tools uniformly
• ➖ Loses first-class BaseTool metadata/args_schema behavior expected by CrewAI
• ➖ May reduce compatibility across CrewAI versions and tool routing behaviors

3. Centralize optional-dependency handling in a shared integration loader

• ➕ Avoid repeating CREWAI_AVAILABLE / degraded-mode patterns across modules
• ➕ Consistent behavior across future integrations
• ➖ More indirection; can obscure integration-specific restore/self-heal logic
• ➖ Not strictly necessary given current module count

Recommendation: Keep the current in-repo integration with a dedicated semantica[crewai] extra: it provides the best user experience while maintaining safe optional-dependency imports. The chosen approach of excluding live state from CrewAI JSON dumps and rebuilding defaults in model_post_init is appropriate for checkpoint robustness; just ensure docs continue to emphasize re-wiring live graphs/contexts after restore.

Files changed (16) +3380 / -13

Enhancement (4) +1519 / -0

__init__.pyCreate CrewAI integration package exports and availability flag +42/-0

Create CrewAI integration package exports and availability flag

• Defines the public integration surface (KG tool, decision tool, knowledge source) and re-exports CREWAI_AVAILABLE for feature detection.

integrations/crewai/init.py

decision_tool.pyAdd SemanticaDecisionTool with policy checks and checkpoint-safe state +561/-0

Add SemanticaDecisionTool with policy checks and checkpoint-safe state

• Implements a CrewAI BaseTool exposing decision intelligence actions (record, precedents, causal tracing, impact, policy checks) with JSON outputs. Excludes live context from JSON dumps and self-heals defaults on restore, while providing a degraded-mode run/arun path without CrewAI.

integrations/crewai/decision_tool.py

kg_tool.pyAdd SemanticaKGTool for graph extraction, add/query, and related traversal +579/-0

Add SemanticaKGTool for graph extraction, add/query, and related traversal

• Implements a CrewAI BaseTool exposing KG actions (entity/relation extraction, add_to_graph, query_graph, find_related) with JSON outputs. Adds per-graph re-entrant locking for safe concurrent writes, robust field access across dataclass/mock/dict shapes, checkpoint-safe serialization exclusions, and degraded-mode run/arun support.

integrations/crewai/kg_tool.py

knowledge_source.pyAdd SemanticaKnowledgeSource bridging legacy/current CrewAI knowledge APIs +337/-0

Add SemanticaKnowledgeSource bridging legacy/current CrewAI knowledge APIs

• Implements a CrewAI BaseKnowledgeSource that serializes a ContextGraph into readable text, chunks it, and saves it through CrewAI storage. Supports both legacy load_content and newer validate_content/aadd contracts, self-heals graph after restore, and logs actionable errors when storage is wired but saving fails (e.g., missing embedder credentials).

integrations/crewai/knowledge_source.py

Tests (6) +1586 / -0

conftest.pyInstall CrewAI stubs for deterministic integration tests +151/-0

Install CrewAI stubs for deterministic integration tests

• Adds pytest session stubs for CrewAI BaseTool and BaseKnowledgeSource into sys.modules, enabling present-case testing without installing CrewAI.

tests/integrations/crewai/conftest.py

test_decision_tool.pyAdd comprehensive SemanticaDecisionTool tests (actions + serialization) +528/-0

Add comprehensive SemanticaDecisionTool tests (actions + serialization)

• Covers initialization, args schema validation, JSON output shapes, error handling, rule coercion correctness, and checkpoint-safe model_dump/model_validate behavior.

tests/integrations/crewai/test_decision_tool.py

test_degradation.pyVerify degraded mode works when CrewAI is absent (subprocess) +103/-0

Verify degraded mode works when CrewAI is absent (subprocess)

• Runs a clean subprocess without stubs/real CrewAI to ensure imports succeed and run/arun methods remain functional for all integration classes.

tests/integrations/crewai/test_degradation.py

test_kg_tool.pyAdd SemanticaKGTool tests including dataclass shapes and locking +453/-0

Add SemanticaKGTool tests including dataclass shapes and locking

• Validates all KG actions, consistent query result schema, idempotent graph adds, per-graph lock behavior, and compatibility with Semantica Entity/Relation dataclass shapes.

tests/integrations/crewai/test_kg_tool.py

test_knowledge_source.pyAdd SemanticaKnowledgeSource tests for chunking, storage, and restore +228/-0

Add SemanticaKnowledgeSource tests for chunking, storage, and restore

• Tests node/edge serialization, chunking behavior (including manual fallback), storage save behavior and failure logging, async add, and checkpoint-safe restore flags.

tests/integrations/crewai/test_knowledge_source.py

test_real_crewai_integration.pyAdd end-to-end subprocess tests against real CrewAI when installed +123/-0

Add end-to-end subprocess tests against real CrewAI when installed

• Executes a Crew-level serialization round-trip using real CrewAI, verifies live state exclusion from dumps, validates tool restore self-healing, and checks knowledge-source behavior without an embedder.

tests/integrations/crewai/test_real_crewai_integration.py

Documentation (4) +272 / -12

CHANGELOG.mdDocument CrewAI integration feature set and hardening notes +11/-0

Document CrewAI integration feature set and hardening notes

• Adds a detailed changelog entry describing the new CrewAI tools/knowledge source, the new pip extra, degraded-mode behavior, and checkpoint/serialization hardening.

CHANGELOG.md

README.mdPromote CrewAI as a first-class integration and add install snippet +8/-12

Promote CrewAI as a first-class integration and add install snippet

• Updates integration marketing copy and the integrations matrix to list CrewAI as first-class. Adds 'pip install semantica[crewai]' to installation examples.

README.md

crewai.mdAdd CrewAI integration documentation with examples and checkpoint notes +147/-0

Add CrewAI integration documentation with examples and checkpoint notes

• Introduces a full docs page describing installation, SemanticaKGTool/SemanticaDecisionTool/SemanticaKnowledgeSource usage, embedder requirements, and checkpoint restore expectations.

docs/integrations/crewai.md

README.mdAdd integration README for Semantica × CrewAI +106/-0

Add integration README for Semantica × CrewAI

• Adds a repository-local README explaining the three components, installation via extras, usage snippets, and state-sharing/checkpoint caveats.

integrations/crewai/README.md

Other (2) +3 / -1

docs.jsonAdd CrewAI page to Integrations navigation +1/-0

Add CrewAI page to Integrations navigation

• Registers 'integrations/crewai' in the docs navigation so the new page appears under Integrations.

docs/docs.json

pyproject.tomlAdd CrewAI optional extra and include it in the all bundle +2/-1

Add CrewAI optional extra and include it in the all bundle

• Introduces 'crewai' extras ('crewai>=0.80.0', 'crewai-tools>=0.17.0') and adds it to the 'all' extras bundle for one-shot installation.

pyproject.toml

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Confidence cast escapes error ✓ Resolved 🐞 Bug ☼ Reliability

Description

SemanticaDecisionTool._run() calls float(confidence) before entering _record_decision’s try/except,
so a malformed confidence (e.g. "high"/None) can raise ValueError and crash the tool instead of
returning JSON error output (especially in degraded mode where run()/arun() call _run directly).

Code

integrations/crewai/decision_tool.py[R273-281]

+        if action == "record_decision":
+            return self._record_decision(
+                category=category or "general",
+                scenario=scenario or "decision recorded",
+                reasoning=reasoning or "agent decision",
+                outcome=outcome or "recorded",
+                confidence=float(confidence),
+                entities=entities,
+            )

Evidence

The diff shows the float() cast happens in _run() before any try/except, while degraded-mode
run()/arun() directly invoke _run(), so invalid confidence types can raise and escape instead of
returning JSON.

integrations/crewai/decision_tool.py[243-281]
integrations/crewai/decision_tool.py[298-325]
integrations/crewai/decision_tool.py[549-561]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`SemanticaDecisionTool._run()` eagerly casts `confidence` using `float(confidence)` before calling `_record_decision()`. If `confidence` is not convertible, this throws before `_record_decision()` can catch and return structured JSON error output. This is most likely in degraded mode (no CrewAI), where `run()`/`arun()` forward directly to `_run()` without Pydantic validation.
### Issue Context
The tool otherwise intends to always return JSON (including on failures), and `_record_decision()` already has exception handling—but it can’t catch failures that occur before it is invoked.
### Fix Focus Areas
- integrations/crewai/decision_tool.py[243-289]
- integrations/crewai/decision_tool.py[298-327]
- integrations/crewai/decision_tool.py[553-561]
### Suggested fix
Move the `float()` coercion inside `_record_decision()`’s `try:` block (or wrap the conversion in `_run()` with a try/except) and return `{"status":"failed","error":"..."}` JSON on coercion errors. Add/extend a regression test for degraded-mode `tool.run(action="record_decision", confidence="not-a-number")`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Untrimmed string coercion ✓ Resolved 🐞 Bug ≡ Correctness

Description

SemanticaDecisionTool._coerce_value() strips whitespace into text but returns the original
unstripped value for non-numeric strings, so check_policy comparisons can fail unexpectedly when
decision_data contains leading/trailing spaces.

Code

integrations/crewai/decision_tool.py[R533-547]

+        text = value.strip()
+        lowered = text.lower()
+        if lowered in ("true", "1"):
+            return True
+        if lowered in ("false", "0"):
+            return False
+        try:
+            return int(text)
+        except ValueError:
+            pass
+        try:
+            return float(text)
+        except ValueError:
+            pass
+        return value

Evidence

The code explicitly strips into text but returns value for the fallback path, meaning whitespace
normalization is lost for general string comparisons.

integrations/crewai/decision_tool.py[530-547]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`_coerce_value()` computes `text = value.strip()` but returns `value` at the end for non-numeric strings. This defeats the purpose of trimming and can cause rule comparisons like `status == approved` to fail when the actual datum is `' approved '`.
### Issue Context
`_eval_rule()` uses `_coerce_value(actual)` when `actual` is a string, so this affects real policy checks for string fields.
### Fix Focus Areas
- integrations/crewai/decision_tool.py[530-547]
### Suggested fix
Change the final `return value` to `return text` (or otherwise ensure whitespace-stripped string values are returned for the non-numeric path). Add a small unit test covering `decision_data={"status": " approved "}` with a rule `status == approved`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. CREWAI_AVAILABLE not holistic ✓ Resolved 🐞 Bug ⚙ Maintainability

Description

integrations.crewai exports CREWAI_AVAILABLE from decision_tool only, but SemanticaKnowledgeSource
computes its own CrewAI availability via a different import check; callers gating on the
package-level flag may think all components are CrewAI-ready when only the tool import succeeded.

Code

integrations/crewai/init.py[R31-33]

+from .decision_tool import CREWAI_AVAILABLE, SemanticaDecisionTool
+from .kg_tool import SemanticaKGTool
+from .knowledge_source import SemanticaKnowledgeSource

Evidence

__init__.py re-exports CREWAI_AVAILABLE only from decision_tool, while knowledge_source maintains an
independent CREWAI_AVAILABLE computed from a different import path, so a single package-level flag
cannot reliably describe both states.

integrations/crewai/init.py[31-40]
integrations/crewai/decision_tool.py[47-59]
integrations/crewai/knowledge_source.py[49-63]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The package-level `CREWAI_AVAILABLE` is imported from `decision_tool` only. However, `knowledge_source` performs a separate CrewAI import check (`crewai.knowledge...`). These can diverge, so the exported flag may not accurately represent whether *all* CrewAI integration components are operating in CrewAI mode.
### Issue Context
This is mainly a correctness/maintainability problem for downstream code that does `if CREWAI_AVAILABLE: ...` before wiring tools/sources into a Crew.
### Fix Focus Areas
- integrations/crewai/__init__.py[31-40]
- integrations/crewai/knowledge_source.py[49-63]
- integrations/crewai/decision_tool.py[47-59]
### Suggested fix
Option A: compute `CREWAI_AVAILABLE` in `integrations/crewai/__init__.py` as a conjunction of the three modules’ flags.
Option B: export per-component flags (e.g., `CREWAI_TOOLS_AVAILABLE`, `CREWAI_KNOWLEDGE_AVAILABLE`) and document what each means.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Tip of the day

💡 Did you know, you can turn on the rule miner and Qodo learns your standards from review history

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[ZohaibHassan16]

LGTM. All Qodo findings resolved and tests passing.

[ZohaibHassan16]

Looks like the CI build check is failing because requirements-ci.txt is out of date (the code is alright).

This PR adds the new crewai extra in pyproject.toml and also adds it to all. That brings in crewai, crewai-tools and a bunch of other transitive dependencies like chromadb, lancedb, etc

But requirements-ci.txt was not regenerated, so now it doesn't match pyproject.toml. The staleness check from #945 is catching this.

I think running the command from CONTRIBUTING.md should fix it:

pip install uv==0.12.1
uv pip compile pyproject.toml --python-version 3.11 --extra all --generate-hashes -o requirements-ci.txt

Then commit the updated requirements-ci.txt and the build check should pass.

[Shindevrp]

@ZohaibHassan16 thanks for the guidance! Regenerating requirements-ci.txt fixed the build staleness check, but it then exposed the real issue: crewai (all versions) hard-requires chromadb~=1.1.0, which carries a pre-authentication code-injection advisory (CVE-2026-45829 / GHSA-f4j7-r4q5-qw2c) with no fixed release - even the latest chromadb 1.5.9 is still in the affected range. That made pip-audit -r requirements-ci.txt and the safety check fail on the PR.

Since there's no fixed chromadb to pin to, I resolved it by keeping the crewai integration fully installable but out of the locked CI dependency set (commit 5f2fad8):

• Dropped crewai from the all aggregate extra (the standalone semantica[crewai] extra is unchanged and still installs crewai)
• Also dropped crewai-tools from the extra - the integration only uses crewai core (BaseTool, BaseKnowledgeSource), and crewai-tools was only pulling in extra transitive deps (chromadb, lancedb, ...)
• Regenerated requirements-ci.txt with uv==0.12.1 per CONTRIBUTING.md

Verified locally: pip-audit (OSV): 0 vulnerabilities across all 392 packages, safety: 0, staleness check matches, 245 tests pass, lint clean. The audit/security-scan jobs should now be green on the latest head.

Also: the earlier approval was auto-dismissed by the new commits - could you give it another look when you get a chance? 🙏

[ZohaibHassan16]

Thank you for being so thorough, especially vis a vis chromadb CVE. Everything looks right.

Just two things I think we should fix before merging:

1. CHANGELOG.md is outdated. It still says crewai>=0.80.0, crewai-tools>=0.17.0 and mentions it being included in the all extra. Since 5f2fad8d only changed pyproject.toml and requirements-ci.txt, the changelog doesn't match anymore. I think we should update it to mention that crewai-tools was removed and crewai was taken out of all because of CVE-2026-45829.

2. integrations/crewai/README.md should probably have a warning. Right now it only says pip install semantica[crewai]. The CVE is only mentioned in a comment in pyproject.toml, so users installing the extra probably won't see it. Since this is still an unpatched critical CVE, I think we should add a short note saying that installing the extra pulls in chromadb, which is currently affected by CVE-2026-45829.

Once those two are updated, I'm good to re-approve.