Skip to content

Implement DataLoader in the GraphQL gateway to batch and cache database queries, preventing N+1 query problems #110

@Oluwaseyi89

Description

@Oluwaseyi89

Description

Implement DataLoader in the GraphQL gateway to batch and cache database queries, preventing N+1 query problems when resolving nested relationships across NFT, Collection, User, Listing, Auction, and Order entities. This optimization will significantly improve GraphQL query performance by reducing the number of database round trips.

Technical Context

  • GraphQL Gateway: Nest.js with Apollo Server (port 3001)
  • Current State: Field resolvers implemented but each nested field triggers individual database queries
  • Problem: Querying a list of 20 NFTs with owner and collection fields results in 1 + 20 + 20 = 41 database queries
  • Target: Reduce to 3-4 batched queries using DataLoader
  • Library: dataloader package

Current State

  • Field resolvers fetch data individually
  • No batching or caching implemented
  • Performance degrades with larger result sets
  • GraphQL resolvers functional but inefficient

Requirements

New/Modified Files:

nftopia-backend/src/graphql/
├── loaders/
│   ├── index.ts                         # NEW - Export all loaders
│   ├── user.loader.ts                   # NEW - Batch user loading
│   ├── nft.loader.ts                    # NEW - Batch NFT loading
│   ├── collection.loader.ts             # NEW - Batch collection loading
│   ├── listing.loader.ts                # NEW - Batch listing loading
│   ├── auction.loader.ts                # NEW - Batch auction loading
│   ├── bid.loader.ts                    # NEW - Batch bid loading
│   └── order.loader.ts                  # NEW - Batch order loading
├── resolvers/
│   ├── nft.resolver.ts                  # MODIFY - Use DataLoader for relationships
│   ├── collection.resolver.ts           # MODIFY - Use DataLoader for relationships
│   ├── user.resolver.ts                 # MODIFY - Use DataLoader for relationships
│   ├── listing.resolver.ts              # MODIFY - Use DataLoader for relationships
│   └── auction.resolver.ts              # MODIFY - Use DataLoader for relationships
└── context/
    └── context.factory.ts               # MODIFY - Initialize DataLoaders per request

Loader Methods to Implement

Loader Batch Method Purpose
UserLoader findByIds(ids: string[]) Batch load users by ID
NFTLoader findByIds(ids: string[]) Batch load NFTs by ID
CollectionLoader findByIds(ids: string[]) Batch load collections by ID
ListingLoader findByNFTIds(nftIds: string[]) Batch load listings by NFT IDs
AuctionLoader findByNFTIds(nftIds: string[]) Batch load active auctions by NFT IDs
BidLoader findByAuctionIds(auctionIds: string[]) Batch load bids by auction IDs
OrderLoader findByNFTIds(nftIds: string[]) Batch load orders by NFT IDs

Performance Improvement

Query Type Before (N+1) After (Batched) Improvement
20 NFTs + owners 21 queries 2 queries 90% reduction
20 NFTs + collections 21 queries 2 queries 90% reduction
20 NFTs + listings 21 queries 2 queries 90% reduction
20 NFTs + all relations 81+ queries 5-6 queries 93% reduction

Acceptance Criteria

  • DataLoader package installed and configured
  • UserLoader batches user queries by ID
  • NFTLoader batches NFT queries by ID
  • CollectionLoader batches collection queries by ID
  • ListingLoader batches listing queries by NFT ID
  • AuctionLoader batches auction queries by NFT ID
  • BidLoader batches bid queries by auction ID
  • OrderLoader batches order queries by NFT ID
  • All field resolvers updated to use DataLoader from context
  • Loaders are request-scoped (new instance per request)
  • Caching works within single request lifecycle
  • Performance test shows reduction from 41+ queries to <10 for typical queries
  • No breaking changes to existing GraphQL schema

Definition of Done

  • PR with DataLoader implementation
  • All batch methods added to services
  • Unit tests for loaders
  • Performance benchmark documented
  • Team review completed

Working Directory:

nftopia-backend

Metadata

Metadata

Assignees

Labels

BackendThis issue is on implementation of a backend feature.Nest.jsThis issue is based on Nest.js framework.Stellar WaveIssues in the Stellar wave programgraphqlThis is issues involves integration of graphql

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions