perf(core): implement multi-level caching mechanism for expensive computations and data retrieval

[Diwakar-odds]

Summary

The system currently performs expensive data retrieval and computational tasks without an intermediate caching layer. Implementing an LRU (Least Recently Used) memory cache or a multi-level caching strategy will significantly improve response times and reduce redundant processing.

Motivation

Performance profiling typically reveals that repeated data fetching or re-computation is a primary bottleneck. A configurable caching layer allows us to optimize these hot paths and reduce load on primary data stores.

Acceptance Criteria

• Implement a generic Caching Interface (with .get(), .set(), .invalidate()).
• Provide an in-memory LRU cache implementation of this interface.
• Integrate the cache into a known high-latency or high-frequency read path.
• Add tests to ensure cache hits/misses function correctly and TTL is respected.

How to Test

• Run tests to verify caching behavior (hit vs miss).
• Observe performance metrics or logging to confirm the underlying data source is not hit repeatedly for cached items.

---

Assignment Request (L3/Core Feature)

I would like to be assigned to this issue. Below is my proposed implementation plan:

Step-by-Step Implementation Plan:

1. Interface Design: Create a standardized cache interface to abstract the caching backend.
2. LRU Implementation: Implement an in-memory LRU cache with TTL (Time-To-Live) support to prevent memory leaks and stale data.
3. Integration: Wrap an existing expensive service method or data retrieval function with a cache check.
4. Testing: Write tests simulating repeated calls and verifying that the backend is only invoked on cache misses or after TTL expiry.

Files to be modified/created:

• src/utils/cache.ts (or .js) - Caching interface and LRU implementation.
• src/utils/cache.test.ts - Unit tests for TTL and LRU eviction.
• Refactoring one service to utilize the cache.

[Diwakar-odds]

Hi maintainers! I would love to be assigned to this issue for ECSoC 2026. I am highly interested in optimizing this backend performance bottleneck.

Understanding the Issue

The current system architecture lacks an intermediate caching layer, resulting in performance degradation due to redundant expensive data retrieval and repetitive core computations on high-frequency read paths.

Proposed Implementation Plan

• Define a standardized ICache interface providing get, set, and invalidate contracts for the backend services.
• Implement an in-memory LRU cache adhering to the interface, featuring a robust TTL (Time-To-Live) eviction policy to prevent stale data.
• Refactor the known high-latency read paths (e.g., query resolution) to utilize a cache-aside pattern.
• Write comprehensive unit tests verifying cache hits, deterministic misses, and TTL expiration logic.

Files to be Modified/Created

• src/cache/interface.ts (NEW) - Define the cache abstraction.
• src/cache/lru_cache.ts (NEW) - Implement the TTL-based LRU logic.
• src/services/core_service.ts - Wrap expensive operations with cache checks.
• ests/cache.test.ts (NEW) - Unit tests for caching behavior.

Expected Output

Sub-millisecond response times for cached operations and a significant reduction in underlying database load.

Please assign me so I can start working on the PR!