Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/graphviz experimental support #237

Closed

Conversation

dgonzo
Copy link

@dgonzo dgonzo commented Dec 28, 2024

Adds tutorial and supporting features

Add support for Graphviz visualization capabilities through pygraphviz,
enabling experimental features in the project. This commit includes:

- Add Taskfile.yml for automated setup of Graphviz and dependencies
- Add environment setup script for Graphviz library paths
- Configure Poetry to install experimental extras with Graphviz support
- Add test script to verify Graphviz installation

The setup process includes:
1. Building Graphviz 8.1.0 from source with custom prefix
2. Setting up environment variables for library paths
3. Installing build dependencies (gcc, python3-dev)
4. Installing project with experimental features via Poetry

To replicate this setup:
1. Install Task (taskfile.dev)
2. Run: task all

Technical details:
- Graphviz Version: 8.1.0
- Installation prefix: /home/gonzo/local
- Required env vars: LD_LIBRARY_PATH, PATH, CFLAGS, LDFLAGS

Resolves the pygraphviz installation issues and enables graph visualization
capabilities for the knowledge graph project.
Add support for building a medical knowledge graph from research papers,
focusing on Systemic Lupus Erythematosus (SLE) and related concepts.

Key changes:
- Add setup_knowledge_graph.py for processing medical PDFs
- Create medical_kg_pipeline_config.json with domain-specific entities
- Update pipeline to use asyncio for async operations
- Define medical entities (diseases, biomarkers, treatments)
- Define medical relationships (causes, treats, indicates)
- Fix prompt template and OpenAI API configuration

The pipeline extracts medical entities and relationships using GPT-4
and stores them in a Neo4j graph database for later retrieval.
Add new components for improved GraphRAG functionality:

- Add EnhancedGraphRAG with post-processing capabilities
- Add OptimizedGraphRAG with performance improvements
- Add MonitoredGraphRAG with logging capabilities
- Add HybridRetriever combining vector and graph-based retrieval
- Add graph visualization utility

Also update .gitignore to exclude test files and data directory.
Update the tutorial documentation to include:
- Examples of medical knowledge graph creation
- Sample output from GraphRAG queries
- Code snippets for enhanced retrievers
- Best practices for graph visualization
@dgonzo dgonzo requested a review from a team as a code owner December 28, 2024 23:49
@dgonzo dgonzo closed this Dec 28, 2024
@dgonzo dgonzo deleted the feat/graphviz-experimental-support branch December 28, 2024 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant