Skip to content

Commit 42a8b30

Browse files
jeremyevansjeremyevans
committed
Add subset_static_cache plugin for statically caching subsets of a model class
This is useful if the entire model class is not static, but specific subsets of the model class are static. It operates like the static_cache plugin, but restricted to specific subsets. Update the static_cache_cache plugin to handle the subset_static_cache plugin in addition to the static_cache plugin.
1 parent 14fb130 commit 42a8b30

File tree

7 files changed

+781
-21
lines changed

7 files changed

+781
-21
lines changed

‎CHANGELOG

+2
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
=== master
22

3+
* Add subset_static_cache plugin for statically caching subsets of a model class (jeremyevans)
4+
35
* Allow class-level dataset methods to be overridable and call super to get the default behavior (jeremyevans)
46

57
* Support column aliases with data types on PostgreSQL, useful for selecting from functions returning records (jeremyevans)

‎lib/sequel/plugins/static_cache_cache.rb

+43-9
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@
22

33
module Sequel
44
module Plugins
5-
# The static_cache_cache plugin allows for caching the row content for subclasses
6-
# that use the static cache plugin (or just the current class). Using this plugin
7-
# can avoid the need to query the database every time loading the plugin into a
8-
# model, which can save time when you have a lot of models using the static_cache
9-
# plugin.
5+
# The static_cache_cache plugin allows for caching the row content for the current
6+
# class and subclasses that use the static_cache or subset_static_cache plugins.
7+
# Using this plugin can avoid the need to query the database every time loading
8+
# the static_cache plugin into a model (static_cache plugin) or using the
9+
# cache_subset method (subset_static_cache plugin).
1010
#
1111
# Usage:
1212
#
@@ -27,7 +27,27 @@ module ClassMethods
2727
# Dump the in-memory cached rows to the cache file.
2828
def dump_static_cache_cache
2929
static_cache_cache = {}
30-
@static_cache_cache.sort.each do |k, v|
30+
@static_cache_cache.sort do |a, b|
31+
a, = a
32+
b, = b
33+
if a.is_a?(Array)
34+
if b.is_a?(Array)
35+
a_name, a_meth = a
36+
b_name, b_meth = b
37+
x = a_name <=> b_name
38+
if x.zero?
39+
x = a_meth <=> b_meth
40+
end
41+
x
42+
else
43+
1
44+
end
45+
elsif b.is_a?(Array)
46+
-1
47+
else
48+
a <=> b
49+
end
50+
end.each do |k, v|
3151
static_cache_cache[k] = v
3252
end
3353
File.open(@static_cache_cache_file, 'wb'){|f| f.write(Marshal.dump(static_cache_cache))}
@@ -42,12 +62,26 @@ def dump_static_cache_cache
4262
# If not available, load the rows from the database, and
4363
# then update the cache with the raw rows.
4464
def load_static_cache_rows
45-
if rows = Sequel.synchronize{@static_cache_cache[name]}
65+
_load_static_cache_rows(dataset, name)
66+
end
67+
68+
# Load the rows for the subset from the cache if available.
69+
# If not available, load the rows from the database, and
70+
# then update the cache with the raw rows.
71+
def load_subset_static_cache_rows(ds, meth)
72+
_load_static_cache_rows(ds, [name, meth].freeze)
73+
end
74+
75+
# Check the cache first for the key, and return rows without a database
76+
# query if present. Otherwise, get all records in the provided dataset,
77+
# and update the cache with them.
78+
def _load_static_cache_rows(ds, key)
79+
if rows = Sequel.synchronize{@static_cache_cache[key]}
4680
rows.map{|row| call(row)}.freeze
4781
else
48-
rows = dataset.all.freeze
82+
rows = ds.all.freeze
4983
raw_rows = rows.map(&:values)
50-
Sequel.synchronize{@static_cache_cache[name] = raw_rows}
84+
Sequel.synchronize{@static_cache_cache[key] = raw_rows}
5185
rows
5286
end
5387
end

‎lib/sequel/plugins/subset_static_cache.rb

+262
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,262 @@
1+
# frozen-string-literal: true
2+
3+
module Sequel
4+
module Plugins
5+
# The subset_static_cache plugin is designed for model subsets that are not modified at all
6+
# in production use cases, or at least where modifications to them would usually
7+
# coincide with an application restart. When caching a model subset, it
8+
# retrieves all rows in the database and statically caches a ruby array and hash
9+
# keyed on primary key containing all of the model instances. All of these cached
10+
# instances are frozen so they won't be modified unexpectedly.
11+
#
12+
# With the following code:
13+
#
14+
# class StatusType < Sequel::Model
15+
# dataset_module do
16+
# where :available, hidden: false
17+
# end
18+
# cache_subset :available
19+
# end
20+
#
21+
# The following methods will use the cache and not issue a database query:
22+
#
23+
# * StatusType.available.with_pk
24+
# * StatusType.available.all
25+
# * StatusType.available.each
26+
# * StatusType.available.first (without block, only supporting no arguments or single integer argument)
27+
# * StatusType.available.count (without an argument or block)
28+
# * StatusType.available.map
29+
# * StatusType.available.as_hash
30+
# * StatusType.available.to_hash
31+
# * StatusType.available.to_hash_groups
32+
#
33+
# The cache is not used if you chain methods before or after calling the cached
34+
# method, as doing so would not be safe:
35+
#
36+
# StatusType.where{number > 1}.available.all
37+
# StatusType.available.where{number > 1}.all
38+
#
39+
# The cache is also not used if you change the class's dataset after caching
40+
# the subset, or in subclasses of the model.
41+
#
42+
# You should not modify any row that is statically cached when using this plugin,
43+
# as otherwise you will get different results for cached and uncached method
44+
# calls.
45+
module SubsetStaticCache
46+
def self.configure(model)
47+
model.class_exec do
48+
@subset_static_caches ||= ({}.compare_by_identity)
49+
end
50+
end
51+
52+
module ClassMethods
53+
# Cache the given subset statically, so that calling the subset method on
54+
# the model will return a dataset that will return cached results instead
55+
# of issuing database queries (assuming the cache has the necessary
56+
# information).
57+
#
58+
# The model must already respond to the given method before cache_subset
59+
# is called.
60+
def cache_subset(meth)
61+
ds = send(meth).with_extend(CachedDatasetMethods)
62+
cache = ds.instance_variable_get(:@cache)
63+
64+
rows, hash = subset_static_cache_rows(ds, meth)
65+
cache[:subset_static_cache_all] = rows
66+
cache[:subset_static_cache_map] = hash
67+
68+
caches = @subset_static_caches
69+
caches[meth] = ds
70+
model = self
71+
subset_static_cache_module.send(:define_method, meth) do
72+
if (model == self) && (cached_dataset = caches[meth])
73+
cached_dataset
74+
else
75+
super()
76+
end
77+
end
78+
nil
79+
end
80+
81+
Plugins.after_set_dataset(self, :clear_subset_static_caches)
82+
Plugins.inherited_instance_variables(self, :@subset_static_caches=>proc{{}.compare_by_identity})
83+
84+
private
85+
86+
# Clear the subset_static_caches. This is used if the model dataset
87+
# changes, to prevent cached values from being used.
88+
def clear_subset_static_caches
89+
@subset_static_caches.clear
90+
end
91+
92+
# A module for the subset static cache methods, so that you can define
93+
# a singleton method in the class with the same name, and call super
94+
# to get default behavior.
95+
def subset_static_cache_module
96+
return @subset_static_cache_module if @subset_static_cache_module
97+
98+
# Ensure dataset_methods module is defined and class is extended with
99+
# it before calling creating this module.
100+
dataset_methods_module
101+
102+
Sequel.synchronize{@subset_static_cache_module ||= Module.new}
103+
extend(@subset_static_cache_module)
104+
@subset_static_cache_module
105+
end
106+
107+
# Return the frozen array and hash used for caching the subset
108+
# of the given dataset.
109+
def subset_static_cache_rows(ds, meth)
110+
all = load_subset_static_cache_rows(ds, meth)
111+
h = {}
112+
all.each do |o|
113+
o.errors.freeze
114+
h[o.pk.freeze] = o.freeze
115+
end
116+
[all, h.freeze]
117+
end
118+
119+
# Return a frozen array for all rows in the dataset.
120+
def load_subset_static_cache_rows(ds, meth)
121+
ret = super if defined?(super)
122+
ret || ds.all.freeze
123+
end
124+
end
125+
126+
module CachedDatasetMethods
127+
# An array of all of the dataset's instances, without issuing a database
128+
# query. If a block is given, yields each instance to the block.
129+
def all(&block)
130+
return super unless all = @cache[:subset_static_cache_all]
131+
132+
array = all.dup
133+
array.each(&block) if block
134+
array
135+
end
136+
137+
# Get the number of records in the cache, without issuing a database query,
138+
# if no arguments or block are provided.
139+
def count(*a, &block)
140+
if a.empty? && !block && (all = @cache[:subset_static_cache_all])
141+
all.size
142+
else
143+
super
144+
end
145+
end
146+
147+
# If a block is given, multiple arguments are given, or a single
148+
# non-Integer argument is given, performs the default behavior of
149+
# issuing a database query. Otherwise, uses the cached values
150+
# to return either the first cached instance (no arguments) or an
151+
# array containing the number of instances specified (single integer
152+
# argument).
153+
def first(*args)
154+
if !defined?(yield) && args.length <= 1 && (args.length == 0 || args[0].is_a?(Integer)) && (all = @cache[:subset_static_cache_all])
155+
all.first(*args)
156+
else
157+
super
158+
end
159+
end
160+
161+
# Return the frozen object with the given pk, or nil if no such object exists
162+
# in the cache, without issuing a database query.
163+
def with_pk(pk)
164+
if cache = @cache[:subset_static_cache_map]
165+
cache[pk]
166+
else
167+
super
168+
end
169+
end
170+
171+
# Yield each of the dataset's frozen instances to the block, without issuing a database
172+
# query.
173+
def each(&block)
174+
return super unless all = @cache[:subset_static_cache_all]
175+
all.each(&block)
176+
end
177+
178+
# Use the cache instead of a query to get the results.
179+
def map(column=nil, &block)
180+
return super unless all = @cache[:subset_static_cache_all]
181+
if column
182+
raise(Error, "Cannot provide both column and block to map") if block
183+
if column.is_a?(Array)
184+
all.map{|r| r.values.values_at(*column)}
185+
else
186+
all.map{|r| r[column]}
187+
end
188+
else
189+
all.map(&block)
190+
end
191+
end
192+
193+
# Use the cache instead of a query to get the results if possible
194+
def as_hash(key_column = nil, value_column = nil, opts = OPTS)
195+
return super unless all = @cache[:subset_static_cache_all]
196+
197+
if key_column.nil? && value_column.nil?
198+
if opts[:hash]
199+
key_column = model.primary_key
200+
else
201+
return Hash[@cache[:subset_static_cache_map]]
202+
end
203+
end
204+
205+
h = opts[:hash] || {}
206+
if value_column
207+
if value_column.is_a?(Array)
208+
if key_column.is_a?(Array)
209+
all.each{|r| h[r.values.values_at(*key_column)] = r.values.values_at(*value_column)}
210+
else
211+
all.each{|r| h[r[key_column]] = r.values.values_at(*value_column)}
212+
end
213+
else
214+
if key_column.is_a?(Array)
215+
all.each{|r| h[r.values.values_at(*key_column)] = r[value_column]}
216+
else
217+
all.each{|r| h[r[key_column]] = r[value_column]}
218+
end
219+
end
220+
elsif key_column.is_a?(Array)
221+
all.each{|r| h[r.values.values_at(*key_column)] = r}
222+
else
223+
all.each{|r| h[r[key_column]] = r}
224+
end
225+
h
226+
end
227+
228+
# Alias of as_hash for backwards compatibility.
229+
def to_hash(*a)
230+
as_hash(*a)
231+
end
232+
233+
# Use the cache instead of a query to get the results
234+
def to_hash_groups(key_column, value_column = nil, opts = OPTS)
235+
return super unless all = @cache[:subset_static_cache_all]
236+
237+
h = opts[:hash] || {}
238+
if value_column
239+
if value_column.is_a?(Array)
240+
if key_column.is_a?(Array)
241+
all.each{|r| (h[r.values.values_at(*key_column)] ||= []) << r.values.values_at(*value_column)}
242+
else
243+
all.each{|r| (h[r[key_column]] ||= []) << r.values.values_at(*value_column)}
244+
end
245+
else
246+
if key_column.is_a?(Array)
247+
all.each{|r| (h[r.values.values_at(*key_column)] ||= []) << r[value_column]}
248+
else
249+
all.each{|r| (h[r[key_column]] ||= []) << r[value_column]}
250+
end
251+
end
252+
elsif key_column.is_a?(Array)
253+
all.each{|r| (h[r.values.values_at(*key_column)] ||= []) << r}
254+
else
255+
all.each{|r| (h[r[key_column]] ||= []) << r}
256+
end
257+
h
258+
end
259+
end
260+
end
261+
end
262+
end

0 commit comments

Comments
 (0)