-
Notifications
You must be signed in to change notification settings - Fork 13
Open
Labels
Description
Possibly required for #361
Would subsume #333
Mimir has a much cruder internal type system than Spark. In addition to lacking collection types, there's a lot of capabilities (e.g., integer precision) that Mimir just does not support at the moment. Spark already has a well-developed type system, and it would be fantastic if we could replace this part of Mimir with the corresponding parts from Spark.
- Replace Mimir's
Typewith Spark'sDataType - Update or replace
Typecheckerwith corresponding parts of Spark (not sure where this lives... perhaps the Catalyst analyzer - Tack on support for unit-based types (Maybe use User-Defined Types?)
- Move to serialization using
foo.jsonandfoo.as[DataType](make sure to keep track of everywhere this happens) - Add an upgrade path to migrate old, encoded types to the new form