knowledge-networks

In this project, we have created knowledge networks by integrating multimodal knowledge from the FishBase Dataset. The knowledges associated with the world map.

Install

pip install -r requirements.txt

Execution

python create_knowlege_graph.py

Demonstration

Knowledge Network:

Structure of Knowledge Graph:

ScyllaDB Migration (Neo4j → native Scylla/CQL)

This project is migrating from a property graph (Neo4j/Cypher) to ScyllaDB using CQL (Cassandra-compatible).
The design preserves graph-like semantics using wide-column tables.

✅ Migration TODO (native CQL)

Dependencies

• Remove neo4j driver from requirements.txt
• Add cassandra-driver (DataStax Python driver)

Configuration

• Add environment variables:
  • SCYLLA_CONTACT_POINTS (comma-separated list)
  • SCYLLA_KEYSPACE
  • SCYLLA_CONSISTENCY (e.g., LOCAL_QUORUM)
• Add a .env.example file with these keys

Schema

• Create keyspace and tables (see CQL schema below)
• Optionally add edges_by_dst or materialized views for reverse lookups

Data Migration

• Export Neo4j nodes and edges to CSV
• Transform CSVs to use UUIDs and map<text,text> for properties
• Bulk import using cqlsh COPY or a Python loader

Code Changes

• Replace Cypher with CQL:
  • MERGE/CREATE → INSERT (IF NOT EXISTS when needed)
  • MATCH (one hop) → SELECT ... FROM edges WHERE src=?
  • Multi-hop → iterative traversal in application logic
• Batch writes in create_knowlege_graph.py using prepared statements

Indexes and Access

• Prefer partition-key queries over secondary indexes
• Add lookup tables (e.g., species_by_name) for common filters

Testing

• Write unit tests for CRUD operations on nodes and edges
• Validate counts and sample paths against Neo4j output

Docs and Ops

• Add Docker Compose for local Scylla
• Document compaction, replication, and backup strategy

Reference

• Luna, Susan and Barile, Josephine France R. (2016): Data on growth rates of fish from FishBase. University of Reading. Dataset
• ScyllaDB Configuration and Tuning Guide
• ScyllaDB CQL Overview