Create a missing.jp2 thumbnail if thumbnail not able to be generated from replay

[ndushay]

@nullhandle says: "the thumbnail generator that is part of the seed registration process can't complete successfully because it doesn't find a resource with that url indexed into swap

i believe it's because of an indexing issue for the accessioned crawl content in openwayback"

He asks the question here: https://groups.google.com/forum/#!topic/openwayback-dev/lN7fdSL68-c

But sort key is set correctly:

• https://github.com/sul-dlss/was_robot_suite/blob/master/lib/was_crawl_dissemination/cdx_merge_sort_publish_service.rb#L16 uses
  • https://github.com/sul-dlss/shared_configs/blob/was-robots1-prod/config/environments/production.rb#L53

and

• https://github.com/sul-dlss/was_robot_suite/blob/master/lib/was_crawl_dissemination/path_indexer_service.rb#L35

[edsu]

In workcycle planning we discussed this problem and decided that we would try catch these by inspecting the contents of the page, and write them as a missing.jp2 which will:

1. allow the workflow to proceed
2. allow these to be noticed by a scheduled report

[mjgiarlo]

Step 2 in what @edsu writes above is separately ticketed in #431

[ndushay]

pywb will be doing thumbnails in a different way so maybe this isn't usefully done at this time.

[peterchanws]

Here is the ticket showing the thumbnail issue:
#446

[edsu]

Since it appears pywb page view returns a 200 OK items that are not found it ought to be possible to use the "raw" URL or the Memento API to determine if something is there or not?

[lwrubel]

We've created an issue to create a step to check for the site in the web archive here: #475

[lwrubel]

Before proceeding with this, implement #475 and #446 and see how much need remains for a placeholder image. Too many redirects examples may be needed as a remaining use case.

As @andrewjbtw mentioned, "A placeholder to get past the step could be an improvement here because the failure of the workflow step itself is a pain point, in some ways more of a pain point than the steps to manually supply the thumbnail."

[ndushay]

Peter is fine with a temporary thumbnail ... would like to be notified if this happens. See #431 re missing thumbnails reported.

[ndushay]

Decision for now: monitor seed robot workflow failures and see how often this happens, first.