Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MLH-73 | Cassandra optimisation. Optimise edgeLabel fetching only to root level vertices #4352

Merged
merged 14 commits into from
Mar 6, 2025

Conversation

aarshi0301
Copy link

@aarshi0301 aarshi0301 commented Mar 4, 2025

Change description

Description here

  • contains
  • hotfix for edge retrieval via projected atributes
  • cassandra optimisation in policies
  • casandra optimisation in relationshipAttributes
  • skip edge labels retrieval for 2nd level vertices
  • Atlan PR : https://github.com/atlanhq/atlan/pull/6576
  • JIRA:
  1. https://atlanhq.atlassian.net/browse/MLH-190
  2. https://atlanhq.atlassian.net/browse/MLH-199

Type of change

  • Bug fix (fixes an issue)
  • New feature (adds functionality)

Related issues

Fix #1

Checklists

Development

  • Lint rules pass locally
  • Application changes have been tested thoroughly
  • Automated tests covering modified code pass

Security

  • Security impact of change has been considered
  • Code follows company security practices and guidelines

Code review

  • Pull request has a descriptive title and context useful to a reviewer. Screenshots or screencasts are attached as necessary
  • "Ready for review" label attached and reviewers assigned
  • Changes have been reviewed by at least one other contributor
  • Pull request linked to task tracker where applicable

@aarshi0301 aarshi0301 changed the title Cassandrapoliciesoptimisation MLH-73 | Cassandrapoliciesoptimisation Mar 4, 2025
nikhilbonte21
nikhilbonte21 previously approved these changes Mar 5, 2025
@sumandas0 sumandas0 requested a review from Copilot March 5, 2025 10:39

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Overview

This pull request addresses a bug fix and performance optimisation in Cassandra policies and relationships retrieval by introducing Janus graph optimisations for relationships and adding a fallback mechanism for prefetch failures.

  • Introduces a new configuration flag (ATLAS_INDEXSEARCH_ENABLE_JANUS_OPTIMISATION_FOR_RELATIONS) and related conditional logic for optimised property prefetching in EntityGraphRetriever.
  • Implements a try-catch fallback for prefetch failures when dealing with policy attributes.
  • Removes an obsolete edge retrieval method in GraphHelper and tweaks filtering of edge labels.

Reviewed Changes

File Description
repository/src/main/java/org/apache/atlas/repository/store/graph/v2/EntityGraphRetriever.java Implements Janus optimisation for relationship attributes and a fallback for prefetch errors in policies.
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java Removes the getOnlyActiveEdges method and adjusts label filtering in edge retrieval.
.github/workflows/maven.yml Adds a new branch trigger for cassandrapoliciesoptimisation.
intg/src/main/java/org/apache/atlas/AtlasConfiguration.java Adds a new configuration flag to control Janus optimisation for relationships.

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java:2088

  • [nitpick] The filtering logic for edge labels was changed by removing the check on entry value emptiness; confirm that allowing empty type names is intentional and does not introduce unintended behavior during edge label processing.
.filter(entry -> !entry.getKey().isEmpty())
@atlanhq atlanhq deleted a comment from Copilot bot Mar 5, 2025
@sriram-atlan sriram-atlan changed the title MLH-73 | Cassandrapoliciesoptimisation MLH-73 | Cassandra optimisation. Optimise edgeLabel fetching only to root level vertices Mar 6, 2025
@sriram-atlan sriram-atlan merged commit a1d977e into master Mar 6, 2025
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants