Skip to content

Commit c5d3f89

Browse files
committed
[Gem] Fixes ScrollHelper
Fixes #2556 The iteration implementation has been fixed. There was a bug where an additional search (with scroll) request was made to Elasticsearch for each resulting hit. It was rewritten so that the docs are retrieved as needed and the Helper instance doesn't store documents internally, with big savings in memory and requests to Elasticsearch.
1 parent 0e73dfd commit c5d3f89

File tree

1 file changed

+4
-14
lines changed

1 file changed

+4
-14
lines changed

elasticsearch/lib/elasticsearch/helpers/scroll_helper.rb

+4-14
Original file line numberDiff line numberDiff line change
@@ -43,12 +43,8 @@ def initialize(client, index, body, scroll = '1m')
4343
# @yieldparam document [Hash] yields a document found in the search hits.
4444
#
4545
def each(&block)
46-
@docs = []
47-
@scroll_id = nil
48-
refresh_docs
49-
for doc in @docs do
50-
refresh_docs
51-
yield doc
46+
until (docs = results).empty?
47+
docs.each(&block)
5248
end
5349
clear
5450
end
@@ -70,25 +66,19 @@ def results
7066
#
7167
def clear
7268
@client.clear_scroll(body: { scroll_id: @scroll_id }) if @scroll_id
73-
@docs = []
69+
@scroll_id = nil
7470
end
7571

7672
private
7773

78-
def refresh_docs
79-
@docs ||= []
80-
@docs << results
81-
@docs.flatten!
82-
end
83-
8474
def initial_search
8575
response = @client.search(index: @index, scroll: @scroll, body: @body)
8676
@scroll_id = response['_scroll_id']
8777
response['hits']['hits']
8878
end
8979

9080
def scroll_request
91-
@client.scroll(body: {scroll: @scroll, scroll_id: @scroll_id})['hits']['hits']
81+
@client.scroll(body: { scroll: @scroll, scroll_id: @scroll_id })['hits']['hits']
9282
end
9383
end
9484
end

0 commit comments

Comments
 (0)