You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Aug 13, 2024. It is now read-only.
The basic idea is that you point a new webhook at a Snowplow Scala Stream Collector. Behind the Snowplow collector, we have a Spark Streaming job running Schema Guru. We would shard based on schema so that only one worker is operating on each schema. We fetch the most recent schema from Iglu, merge the output of our new 5 minutes worth of derivation, and then push the updated schema back to Iglu as a patch.
See #43, #139
The basic idea is that you point a new webhook at a Snowplow Scala Stream Collector. Behind the Snowplow collector, we have a Spark Streaming job running Schema Guru. We would shard based on schema so that only one worker is operating on each schema. We fetch the most recent schema from Iglu, merge the output of our new 5 minutes worth of derivation, and then push the updated schema back to Iglu as a patch.
/cc @chuwy @fblundun, @a1nayak
The text was updated successfully, but these errors were encountered: