Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

perf: memoize blobs requests in the request scope #2777

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

perf: memoize blobs requests in the request scope #2777

wants to merge 4 commits into from

Conversation

pieh
Copy link
Contributor

@pieh pieh commented Mar 19, 2025 •
edited
Loading

Description

If site is doing multiple fetches with data caching enabled or is calling method wrapped in unstable_cache resulting in same cache key multiple times to render single page - we currently are repeatedly calling blobStore.get for each of the repeated use.

This PR looks to add memoization of those calls scoped to concrete request. We don't want to memoize them globally due to distributed nature of serverless with potentially many instances running at the same time, each holding different potentially state, but we should be able to safely memoize blobs call while handling same request. This also allow for reading it's own writes within scope of same request handling.

This PR also replace similar handling we already had for tag manifests with more generic one shared for all blobs operations.

Tests

  • Added fetch specific integration test asserting we only fetch same thing from blobs once per request
  • Added memoized blob store unit tests
  • We already did have tests specific to tag manifests that I will not change, but they should continue to pass ( for example

    opennextjs-netlify/tests/integration/revalidate-path.test.ts

    Lines 77 to 81 in 8675b3e

    expect(
    getBlobServerGets(ctx, isTagManifest),
    `expected tag manifests to be retrieved at most once per tag`,
    ).toBeDistinct()
    ctx.blobServerGetSpy.mockClear()
    )

Relevant links (GitHub issues, etc.) or a picture of cute animal

https://linear.app/netlify/issue/FRB-1629/intermittent-proxy-timeouts-on-runtime-v594

@pieh
test: add test case with multiple fetches for same resource and test …
03f168d 
…asserting we only check blobs once per request for same key
@github-actions GitHub Actions
Copy link

github-actions bot commented Mar 19, 2025 •
edited
Loading

📊 Package size report   0.8%↑

File Before (Size / Gzip) After (Size / Gzip)
dist/esm-chunks/package-QK2ZWPVB.js — 3.7 kB / 1.4 kB
dist/esm-chunks/package-UN6EVEHD.js 3.6 kB / 1.4 kB —
dist/run/config.js 1.2 kB / 595 B 13%↑1.3 kB / 9%↑650 B
dist/run/handlers/cache.cjs 22.0 kB / 5.8 kB -5.31%↓20.9 kB / -2.91%↓5.6 kB
dist/run/handlers/request-context.cjs 5.8 kB / 1.8 kB 6%↑6.2 kB / 5%↑1.9 kB
dist/run/handlers/server.js 141.4 kB / 33.2 kB -0.01%↓141.4 kB / -0.02%↓33.2 kB
dist/run/handlers/tracer.cjs 29.9 kB / 6.3 kB 1%↑30.2 kB / 2%↑6.3 kB
dist/run/headers.js 8.0 kB / 2.4 kB -5.06%↓7.6 kB / -5.26%↓2.3 kB
dist/run/next.cjs 23.6 kB / 5.8 kB -0.31%↓23.5 kB / -0.33%↓5.8 kB
dist/run/regional-blob-store.cjs 21.3 kB / 6.1 kB 217%↑67.3 kB / 169%↑16.3 kB
dist/shared/cache-types.cjs 1.3 kB / 566 B 107%↑2.8 kB / 73%↑977 B
package.json 3.2 kB / 1.2 kB 0.9%↑3.2 kB / 0.6%↑1.2 kB
Total (Includes all files) 5.8 MB / 1.2 MB 0.8%↑5.9 MB / 0.9%↑1.2 MB
Tarball size 1.2 MB 0.8%↑1.2 MB
Unchanged files
File Size (Size / Gzip)
dist/build/advanced-api-routes.js 4.3 kB / 1.4 kB
dist/build/cache.js 1.0 kB / 414 B
dist/build/content/next-shims/telemetry-storage.cjs 1.6 kB / 659 B
dist/build/content/prerendered.js 9.4 kB / 2.8 kB
dist/build/content/server.js 8.7 kB / 2.8 kB
dist/build/content/static.js 4.1 kB / 1.4 kB
dist/build/functions/edge.js 20.8 kB / 5.6 kB
dist/build/functions/server.js 5.0 kB / 1.6 kB
dist/build/image-cdn.js 54.0 kB / 11.1 kB
dist/build/plugin-context.js 10.1 kB / 3.0 kB
dist/build/templates/handler-monorepo.tmpl.js 1.7 kB / 703 B
dist/build/templates/handler.tmpl.js 1.6 kB / 655 B
dist/build/verification.js 4.5 kB / 1.5 kB
dist/esm-chunks/chunk-5QSXBV7L.js 2.4 kB / 842 B
dist/esm-chunks/chunk-APO262HE.js 61.2 kB / 11.1 kB
dist/esm-chunks/chunk-GNGHTHMQ.js 55.6 kB / 9.7 kB
dist/esm-chunks/chunk-KGYJQ2U2.js 186.5 kB / 32.9 kB
dist/esm-chunks/chunk-OEQOKJGE.js 2.3 kB / 977 B
dist/index.js 3.4 kB / 1.1 kB
dist/run/constants.js 516 B / 308 B
dist/run/handlers/tracing.js 3.0 MB / 418.4 kB
dist/run/handlers/wait-until.cjs 1.4 kB / 665 B
dist/run/revalidate.js 1.0 kB / 475 B
dist/shared/blobkey.js 742 B / 399 B
edge-runtime/lib/headers.ts 1.9 kB / 841 B
edge-runtime/lib/logging.ts 115 B / 121 B
edge-runtime/lib/middleware.ts 1.9 kB / 807 B
edge-runtime/lib/next-request.ts 3.3 kB / 1.1 kB
edge-runtime/lib/response.ts 9.2 kB / 2.9 kB
edge-runtime/lib/routing.ts 15.1 kB / 3.9 kB
edge-runtime/lib/util.test.ts 1.6 kB / 356 B
edge-runtime/lib/util.ts 3.7 kB / 1.3 kB
edge-runtime/matchers.json 3 B / 23 B
edge-runtime/middleware.ts 2.4 kB / 1.0 kB
edge-runtime/next.config.json 3 B / 23 B
edge-runtime/README.md 992 B / 509 B
edge-runtime/shim/index.js 1.5 kB / 717 B
edge-runtime/vendor.ts 745 B / 312 B
edge-runtime/vendor/deno.land/[email protected]/_util/asserts.ts 854 B / 461 B
edge-runtime/vendor/deno.land/[email protected]/_util/os.ts 644 B / 355 B
edge-runtime/vendor/deno.land/[email protected]/async/abortable.ts 4.0 kB / 1.0 kB
edge-runtime/vendor/deno.land/[email protected]/async/deadline.ts 974 B / 544 B
edge-runtime/vendor/deno.land/[email protected]/async/debounce.ts 2.2 kB / 956 B
edge-runtime/vendor/deno.land/[email protected]/async/deferred.ts 1.5 kB / 798 B
edge-runtime/vendor/deno.land/[email protected]/async/delay.ts 1.8 kB / 845 B
edge-runtime/vendor/deno.land/[email protected]/async/mod.ts 465 B / 241 B
edge-runtime/vendor/deno.land/[email protected]/async/mux_async_iterator.ts 2.5 kB / 1.1 kB
edge-runtime/vendor/deno.land/[email protected]/async/pool.ts 3.2 kB / 1.4 kB
edge-runtime/vendor/deno.land/[email protected]/async/retry.ts 2.4 kB / 1.0 kB
edge-runtime/vendor/deno.land/[email protected]/async/tee.ts 2.1 kB / 924 B
edge-runtime/vendor/deno.land/[email protected]/bytes/index_of_needle.ts 1.4 kB / 668 B
edge-runtime/vendor/deno.land/[email protected]/crypto/timing_safe_equal.ts 875 B / 442 B
edge-runtime/vendor/deno.land/[email protected]/datetime/to_imf.ts 1.3 kB / 681 B
edge-runtime/vendor/deno.land/[email protected]/encoding/base64.ts 2.5 kB / 1.0 kB
edge-runtime/vendor/deno.land/[email protected]/encoding/base64url.ts 2.0 kB / 872 B
edge-runtime/vendor/deno.land/[email protected]/flags/mod.ts 22.6 kB / 5.9 kB
edge-runtime/vendor/deno.land/[email protected]/fmt/colors.ts 12.4 kB / 2.7 kB
edge-runtime/vendor/deno.land/[email protected]/fmt/printf.ts 27.7 kB / 7.7 kB
edge-runtime/vendor/deno.land/[email protected]/http/cookie.ts 11.5 kB / 3.6 kB
edge-runtime/vendor/deno.land/[email protected]/node/_core.ts 2.3 kB / 716 B
edge-runtime/vendor/deno.land/[email protected]/node/_events.d.ts 27.2 kB / 5.8 kB
edge-runtime/vendor/deno.land/[email protected]/node/_events.mjs 28.0 kB / 7.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/_global.d.ts 1.7 kB / 650 B
edge-runtime/vendor/deno.land/[email protected]/node/_next_tick.ts 5.0 kB / 1.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/_process/exiting.ts 138 B / 138 B
edge-runtime/vendor/deno.land/[email protected]/node/_process/process.ts 3.8 kB / 1.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/_process/stdio.mjs 336 B / 233 B
edge-runtime/vendor/deno.land/[email protected]/node/_process/streams.mjs 4.0 kB / 1.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/_stream.d.ts 53.2 kB / 11.9 kB
edge-runtime/vendor/deno.land/[email protected]/node/_stream.mjs 91.2 kB / 25.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/_util/_util_callbackify.ts 4.3 kB / 1.7 kB
edge-runtime/vendor/deno.land/[email protected]/node/_utils.ts 5.9 kB / 2.0 kB
edge-runtime/vendor/deno.land/[email protected]/node/assert.ts 23.1 kB / 4.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/assertion_error.ts 19.6 kB / 6.1 kB
edge-runtime/vendor/deno.land/[email protected]/node/async_hooks.ts 7.7 kB / 2.1 kB
edge-runtime/vendor/deno.land/[email protected]/node/buffer.ts 262 B / 204 B
edge-runtime/vendor/deno.land/[email protected]/node/events.ts 303 B / 221 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/_libuv_winerror.ts 7.8 kB / 1.9 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/_listen.ts 561 B / 342 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/_node.ts 443 B / 335 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/_timingSafeEqual.ts 479 B / 268 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/_utils.ts 2.4 kB / 938 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/_winerror.ts 354.4 kB / 64.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/ares.ts 2.4 kB / 1.1 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/async_wrap.ts 4.0 kB / 1.8 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/buffer.ts 3.5 kB / 1.3 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/cares_wrap.ts 15.2 kB / 3.9 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/config.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/connection_wrap.ts 2.6 kB / 1.3 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/constants.ts 21.5 kB / 5.1 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/contextify.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/credentials.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/crypto.ts 448 B / 244 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/errors.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/fs_dir.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/fs_event_wrap.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/fs.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/handle_wrap.ts 1.8 kB / 1.0 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/heap_utils.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/http_parser.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/icu.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/inspector.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/js_stream.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/messaging.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/mod.ts 3.1 kB / 955 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/module_wrap.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/native_module.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/natives.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/node_file.ts 2.9 kB / 1.5 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/node_options.ts 1.8 kB / 989 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/options.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/os.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/performance.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/pipe_wrap.ts 10.4 kB / 3.3 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/process_methods.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/report.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/serdes.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/signal_wrap.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/spawn_sync.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/stream_wrap.ts 9.3 kB / 2.8 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/string_decoder.ts 504 B / 261 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/symbols.ts 1.4 kB / 828 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/task_queue.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/tcp_wrap.ts 13.1 kB / 3.7 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/timers.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/tls_wrap.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/trace_events.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/tty_wrap.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/types.ts 5.7 kB / 1.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/udp_wrap.ts 12.4 kB / 3.6 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/url.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/util.ts 4.0 kB / 1.8 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/uv.ts 20.1 kB / 3.8 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/v8.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/worker.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal_binding/zlib.ts 87 B / 104 B
edge-runtime/vendor/deno.land/[email protected]/node/internal/buffer.d.ts 73.6 kB / 12.1 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/buffer.mjs 66.1 kB / 10.6 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/crypto/_keys.ts 463 B / 262 B
edge-runtime/vendor/deno.land/[email protected]/node/internal/crypto/constants.ts 252 B / 173 B
edge-runtime/vendor/deno.land/[email protected]/node/internal/error_codes.ts 322 B / 250 B
edge-runtime/vendor/deno.land/[email protected]/node/internal/errors.ts 78.9 kB / 17.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/fixed_queue.ts 4.4 kB / 1.2 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/hide_stack_frames.ts 550 B / 377 B
edge-runtime/vendor/deno.land/[email protected]/node/internal/net.ts 3.1 kB / 1.5 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/normalize_encoding.mjs 2.1 kB / 500 B
edge-runtime/vendor/deno.land/[email protected]/node/internal/options.ts 1.7 kB / 959 B
edge-runtime/vendor/deno.land/[email protected]/node/internal/primordials.mjs 1.8 kB / 431 B
edge-runtime/vendor/deno.land/[email protected]/node/internal/process/per_thread.mjs 7.8 kB / 2.3 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/readline/callbacks.mjs 3.8 kB / 1.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/readline/utils.mjs 14.3 kB / 3.7 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/streams/destroy.mjs 6.9 kB / 1.8 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/streams/end-of-stream.mjs 7.1 kB / 1.9 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/streams/utils.mjs 5.9 kB / 1.2 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/util.mjs 4.0 kB / 1.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/util/comparisons.ts 16.6 kB / 3.8 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/util/debuglog.ts 3.2 kB / 1.4 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/util/inspect.mjs 71.5 kB / 19.8 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/util/types.ts 3.7 kB / 1.3 kB
edge-runtime/vendor/deno.land/[email protected]/node/internal/validators.mjs 8.0 kB / 2.1 kB
edge-runtime/vendor/deno.land/[email protected]/node/process.ts 19.4 kB / 5.2 kB
edge-runtime/vendor/deno.land/[email protected]/node/stream.ts 671 B / 346 B
edge-runtime/vendor/deno.land/[email protected]/node/string_decoder.ts 10.3 kB / 3.3 kB
edge-runtime/vendor/deno.land/[email protected]/node/util.ts 7.8 kB / 2.2 kB
edge-runtime/vendor/deno.land/[email protected]/node/util/types.ts 199 B / 153 B
edge-runtime/vendor/deno.land/[email protected]/path/_constants.ts 2.0 kB / 727 B
edge-runtime/vendor/deno.land/[email protected]/path/_interface.ts 728 B / 369 B
edge-runtime/vendor/deno.land/[email protected]/path/_util.ts 5.0 kB / 1.6 kB
edge-runtime/vendor/deno.land/[email protected]/path/common.ts 1.2 kB / 607 B
edge-runtime/vendor/deno.land/[email protected]/path/glob.ts 12.7 kB / 3.9 kB
edge-runtime/vendor/deno.land/[email protected]/path/mod.ts 1.4 kB / 690 B
edge-runtime/vendor/deno.land/[email protected]/path/posix.ts 13.9 kB / 3.7 kB
edge-runtime/vendor/deno.land/[email protected]/path/separator.ts 259 B / 209 B
edge-runtime/vendor/deno.land/[email protected]/path/win32.ts 28.5 kB / 6.4 kB
edge-runtime/vendor/deno.land/[email protected]/streams/write_all.ts 2.2 kB / 598 B
edge-runtime/vendor/deno.land/[email protected]/testing/_diff.ts 11.6 kB / 3.6 kB
edge-runtime/vendor/deno.land/[email protected]/testing/_format.ts 705 B / 462 B
edge-runtime/vendor/deno.land/[email protected]/testing/asserts.ts 25.5 kB / 5.7 kB
edge-runtime/vendor/deno.land/[email protected]/types.d.ts 4.2 kB / 1.2 kB
edge-runtime/vendor/deno.land/x/[email protected]/pkg/htmlrewriter_bg.wasm 573.2 kB / 262.7 kB
edge-runtime/vendor/deno.land/x/[email protected]/pkg/htmlrewriter.js 31.0 kB / 4.7 kB
edge-runtime/vendor/deno.land/x/[email protected]/src/index.ts 2.6 kB / 989 B
edge-runtime/vendor/deno.land/x/[email protected]/src/types.d.ts 2.1 kB / 446 B
edge-runtime/vendor/deno.land/x/[email protected]/index.ts 15.4 kB / 4.2 kB
edge-runtime/vendor/import_map.json 148 B / 111 B
edge-runtime/vendor/v1-7-0--edge-utils.netlify.app/logger/logger.ts 3.2 kB / 747 B
edge-runtime/vendor/v1-7-0--edge-utils.netlify.app/logger/mod.ts 29 B / 49 B
manifest.yml 31 B / 51 B
README.md 2.8 kB / 1.2 kB

🤖 This report was automatically generated by pkg-size-action

@pieh pieh changed the title perf: memoize blobs requests in the scope of same request perf: memoize blobs requests in the request scope Mar 19, 2025
@pieh pieh force-pushed the michalpiechowiak/frb-1629-intermittent-proxy-timeouts-on-runtime-v594 branch 3 times, most recently from f8a565c to c97fee2 Compare March 20, 2025 15:50
@pieh pieh added the test all versions Run e2e tests against old and canary versions of Next.js label Mar 20, 2025
@pieh pieh force-pushed the michalpiechowiak/frb-1629-intermittent-proxy-timeouts-on-runtime-v594 branch from c97fee2 to 85d4eb9 Compare March 21, 2025 07:16
@pieh pieh marked this pull request as ready for review March 21, 2025 10:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
test all versions Run e2e tests against old and canary versions of Next.js
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@pieh