Feature/writable api

README.md

@@ -102,11 +102,24 @@ The admin interface is used to define the components of the Observatory. It is a
 7. Camera - A specific instance of a camera type with a set of optical element groups
 8. Generic modes - A generic definition for a single mode, including an associated overhead and validation schema
 9. Generic mode group - A grouping of one or more generic modes of a single type associated with a camera type. The type is user definable, but some examples used in the Observation Portal include `readout`, `acquisition`, `guiding`, `exposure`, and `rotator`
-10. Instrument - A combination of one or more science cameras and a guide camera on a specific Telescope
+10. Instrument Category - Generic category of instruments. Usually used to differentiate `IMAGE` or `SPECTRA` in the OCS, but you are free to define additional categories and use them differently
+11. Configuration Types - Generic types defining the configurations available on your telescope, i.e. `EXPOSE`, `BIAS`, `FLAT`, etc.
+12. Instrument Type - The generic properties of a single type of instrument
+13. Configuration Type Properties - Links a configuration type to an instrument type with additional settings specific to it
+14. Instrument - A specific instance of an instrument type with a combination of one or more science cameras and a guide camera on a specific Telescope
 
 -   Check out the updated step-by-step setup guide [here](https://observatorycontrolsystem.github.io/deployment/configdb_setup/)
 -   It is recommended that all codes use lowercase characters by convention, except for type codes such as instrument type, camera type, and mode type which should use all upper case. While this convention isn't strictly required, it is useful to choose a convention and apply it consistently when defining your codes.
 
+
+#### Notes on using the API to write
+The API has recently been updated so all data structures should now be writable. Due to the heavily nested structure of the data, it is still highly recommended to write the data structures in the order defined above. There are a few api differencies when writing some structures, including:
+
+1. Optical Element Groups - Can either link existing Optical Elements or create them itself. Set `optical_element_ids` to a list of existing Optical Element ids, or set the `optical_elements` field to a list of objects containing at least the optical elements `code` and `name` field, which will either match an existing optical element by `code` or create a new one. The `default` field should be populated with the code of an existing optical element. If the optical element doesn't exist yet, you must patch the group after creation to add the default.
+2. Generic Mode Groups - Can either link existing Generic Modes or create them itself. Set `mode_ids` to a list of existing Generic Mode ids, or set the `modes` field to a list of objects containing at least the mode `code` and `name` field, which will either match an existing generic mode by `code` or create a new one. The `default` field should be populated with the code of an existing generic mode. If the generic mode doesn't exist yet, you must patch the group after creation to add the default.
+3. Instrument Types - When setting or updating the configuration type properties associated with an Instrument Type, you must first have the Configuration Types created in advance. Then you can send `configuration_types` on creation that contain a list of objects with configuration type property settings and the `configuration_type` field which is the code of the configuration type you want to link. You can alternatively use the configuration type properties API to create those individually, referencing the corresponding configuration type code and instrument type id.
+
+
 #### Generic Mode Validation Schema
 GenericMode structures have a field called `validation_schema` which accepts a dictionary [Cerberus Validation Schema](https://docs.python-cerberus.org/en/stable/schemas.html). This validation schema will be used to provide automatic validation and setting of defaults within the [Observation Portal](https://github.com/observatorycontrolsystem/observation-portal). The validation schema will act on the structure in which the GenericMode is a part of. For example:
 

configdb/hardware/api_views.py

@@ -6,7 +6,8 @@
 from configdb.hardware import serializers
 from .models import (
     Site, Enclosure, Telescope, OpticalElementGroup, Instrument, Camera, OpticalElement,
-    CameraType, GenericMode, GenericModeGroup, InstrumentType
+    CameraType, GenericMode, GenericModeGroup, InstrumentType, ModeType, ConfigurationType,
+    InstrumentCategory, ConfigurationTypeProperties
 )
 
 
@@ -152,6 +153,30 @@ class OpticalElementViewSet(FilterableViewSet):
     filter_fields = ('id', 'name', 'code', 'schedulable')
 
 
+class ModeTypeViewSet(FilterableViewSet):
+    schema = CustomViewSchema(tags=['Mode Types'])
+    queryset = ModeType.objects.all()
+    serializer_class = serializers.ModeTypeSerializer
+
+
+class InstrumentCategoryViewSet(FilterableViewSet):
+    schema = CustomViewSchema(tags=['Instrument Category'])
+    queryset = InstrumentCategory.objects.all()
+    serializer_class = serializers.InstrumentCategorySerializer
+
+
+class ConfigurationTypeViewSet(FilterableViewSet):
+    schema = CustomViewSchema(tags=['Configuration Types'])
+    queryset = ConfigurationType.objects.all()
+    serializer_class = serializers.ConfigurationTypeSerializer
+
+
+class ConfigurationTypePropertiesViewSet(FilterableViewSet):
+    schema = CustomViewSchema(tags=['Configuration Type Properties'])
+    queryset = ConfigurationTypeProperties.objects.all()
+    serializer_class = serializers.ConfigurationTypePropertiesSerializer
+
+
 class GenericModeGroupViewSet(FilterableViewSet):
     schema = CustomViewSchema(tags=['Generic Mode Groups'])
     queryset = GenericModeGroup.objects.all()

configdb/hardware/models.py

@@ -301,6 +301,7 @@ class GenericMode(BaseModel):
     name = models.CharField(max_length=200, help_text='Generic mode name')
     code = models.CharField(max_length=200, help_text='Generic mode code')
     overhead = models.FloatField(
+        default=0.0,
         validators=[MinValueValidator(0)],
         help_text='Overhead associated with the generic mode. Where this overhead is applied depends on what type '
                   'of generic mode this is for. For example, a readout mode is applied per exposure, while an acquisition '

configdb/hardware/serializers.py

@@ -2,7 +2,8 @@
 
 from .models import (
     Site, Enclosure, Telescope, OpticalElement, GenericMode, Instrument, Camera, OpticalElementGroup,
-    CameraType, GenericModeGroup, InstrumentType, ConfigurationTypeProperties
+    CameraType, GenericModeGroup, InstrumentType, ConfigurationTypeProperties, ConfigurationType, ModeType,
+    InstrumentCategory
 )
 from configdb.hardware.validator import OCSValidator
 
@@ -11,35 +12,93 @@ class OpticalElementSerializer(serializers.ModelSerializer):
 
     class Meta:
         model = OpticalElement
-        fields = ('name', 'code', 'schedulable')
+        fields = ('id', 'name', 'code', 'schedulable')
+        read_only_fields = ['id']
+
+
+class OpticalElementNestedSerializer(OpticalElementSerializer):
+    # This nested serializer allows us to choose an existing optical element for the group
+    # Since the groups create method will use get_or_create to link optical elements
+    class Meta(OpticalElementSerializer.Meta):
+        extra_kwargs = {
+            'code': {
+                'validators': [],
+            }
+        }
 
 
 class OpticalElementGroupSerializer(serializers.ModelSerializer):
-    optical_elements = OpticalElementSerializer(many=True, help_text='Optical elements belonging to this optical element group')
-    default = serializers.SerializerMethodField('get_default_code', help_text='Default optical element code')
+    optical_elements = OpticalElementNestedSerializer(
+        many=True, required=False,
+        help_text='Optical elements belonging to this optical element group'
+    )
+    optical_element_ids = serializers.PrimaryKeyRelatedField(
+        many=True, queryset=OpticalElement.objects.all(), required=False, write_only=True,
+        help_text='Existing Optical Element ids to attach to this group'
+    )
+    default = serializers.SlugRelatedField(slug_field='code', queryset=OpticalElement.objects.all(), required=False,
+                                           help_text='Default optical element code')
 
     class Meta:
         model = OpticalElementGroup
-        fields = ('name', 'type', 'optical_elements', 'element_change_overhead', 'default')
-        depth = 1
+        fields = ('id', 'name', 'type', 'optical_elements', 'optical_element_ids', 'element_change_overhead', 'default')
 
-    def get_default_code(self, obj):
-        return obj.default.code if obj.default else ''
+    def to_representation(self, instance):
+        data = super().to_representation(instance)
+        if data.get('default', None) is None:
+            data['default'] = ''
+        return data
 
     def validate(self, data):
-        if data['default'] not in [oe['code'] for oe in data['optical_elements']]:
-            raise serializers.ValidationError("Default must be the code of an optical element in this group")
+        instance = getattr(self, 'instance', None)
+        oe_codes = []
+        if instance:
+            oe_codes = [oe.code for oe in instance.optical_elements.all()]
+        else:
+            if 'optical_elements' in data:
+                oe_codes.extend([oe['code'] for oe in data['optical_elements']])
+            if 'optical_element_ids' in data:
+                oe_codes.extend([oe.code for oe in data['optical_element_ids']])
+        if ('default' in data and data['default'].code not in oe_codes):
+            raise serializers.ValidationError(f"Default optical element {data['default']} must be a member of this groups optical elements")
+        return super().validate(data)
+
+    def create(self, validated_data):
+        optical_elements = validated_data.pop('optical_elements', [])
+        optical_element_instances = validated_data.pop('optical_element_ids', [])
+        optical_element_group = super().create(validated_data)
+
+        for optical_element_instance in optical_element_instances:
+            optical_element_group.optical_elements.add(optical_element_instance)
+
+        for optical_element in optical_elements:
+            optical_element_instance, _ = OpticalElement.objects.get_or_create(code=optical_element.pop('code'), defaults=optical_element)
+            optical_element_group.optical_elements.add(optical_element_instance)
+
+        return optical_element_group
+
+
+class ModeTypeSerializer(serializers.ModelSerializer):
+    class Meta:
+        fields = ('id',)
+        model = ModeType
 
-        return data
+
+class InstrumentCategorySerializer(serializers.ModelSerializer):
+    class Meta:
+        fields = ('code',)
+        model = InstrumentCategory
 
 
 class GenericModeSerializer(serializers.ModelSerializer):
-    validation_schema = serializers.JSONField(help_text='Cerberus styled validation schema used to validate '
+    validation_schema = serializers.JSONField(required=False, default=dict,
+                                              help_text='Cerberus styled validation schema used to validate '
                                                         'instrument configs using this generic mode')
 
     class Meta:
-        fields = ('name', 'overhead', 'code', 'schedulable', 'validation_schema')
+        fields = ('id', 'name', 'overhead', 'code', 'schedulable', 'validation_schema')
         model = GenericMode
+        read_only_fields = ['id']
 
     def validate_validation_schema(self, value):
         try:
@@ -51,21 +110,54 @@ def validate_validation_schema(self, value):
 
 
 class GenericModeGroupSerializer(serializers.ModelSerializer):
-    modes = GenericModeSerializer(many=True, help_text='Set of modes belonging to this generic mode group')
-    default = serializers.SerializerMethodField('get_default_code', help_text='Default mode within this generic mode group')
+    instrument_type = serializers.PrimaryKeyRelatedField(
+        write_only=True, queryset=InstrumentType.objects.all(),
+        help_text='ID for the instrument type associated with this group'
+    )
+    modes = GenericModeSerializer(many=True, required=False,
+                                  help_text='Set of modes belonging to this generic mode group')
+    mode_ids = serializers.PrimaryKeyRelatedField(many=True, queryset=GenericMode.objects.all(), required=False,
+                                                  write_only=True, help_text='Existing mode ids to attach to this group')
+    default = serializers.SlugRelatedField(slug_field='code', queryset=GenericMode.objects.all(), required=False,
+                                           help_text='Default mode within this generic mode group')
 
     class Meta:
-        fields = ('type', 'modes', 'default')
+        fields = ('id', 'type', 'modes', 'mode_ids', 'default', 'instrument_type')
         model = GenericModeGroup
 
-    def get_default_code(self, obj):
-        return obj.default.code if obj.default else ''
+    def to_representation(self, instance):
+        data = super().to_representation(instance)
+        if data.get('default', None) is None:
+            data['default'] = ''
+        return data
 
     def validate(self, data):
-        if data['default'] not in [mode['code'] for mode in data['modes']]:
-            raise serializers.ValidationError("Default must be the code of a mode in this group")
-
-        return data
+        instance = getattr(self, 'instance', None)
+        mode_codes = []
+        if instance:
+            mode_codes = [mode.code for mode in instance.modes.all()]
+        else:
+            if 'modes' in data:
+                mode_codes.extend([mode['code'] for mode in data['modes']])
+            if 'mode_ids' in data:
+                mode_codes.extend([mode.code for mode in data['mode_ids']])
+        if ('default' in data and data['default'].code not in mode_codes):
+            raise serializers.ValidationError(f"Default generic mode {data['default']} must be a member of this groups modes")
+        return super().validate(data)
+
+    def create(self, validated_data):
+        generic_modes = validated_data.pop('modes', [])
+        generic_mode_instances = validated_data.pop('mode_ids', [])
+        generic_mode_group = super().create(validated_data)
+
+        for generic_mode_instance in generic_mode_instances:
+            generic_mode_group.modes.add(generic_mode_instance)
+
+        for generic_mode in generic_modes:
+            generic_mode_instance, _ = GenericMode.objects.get_or_create(**generic_mode)
+            generic_mode_group.modes.add(generic_mode_instance)
+
+        return generic_mode_group
 
 
 class CameraTypeSerializer(serializers.ModelSerializer):
@@ -86,20 +178,36 @@ class Meta:
         model = Camera
 
 
-class ConfigurationTypePropertiesSerializer(serializers.ModelSerializer):
+class ConfigurationTypeSerializer(serializers.ModelSerializer):
+    class Meta:
+        fields = ('name', 'code')
+        model = ConfigurationType
+
+
+class ConfigurationTypePropertiesNestedSerializer(serializers.ModelSerializer):
     name = serializers.ReadOnlyField(source='configuration_type.name', help_text='Configuration type name')
     code = serializers.ReadOnlyField(source='configuration_type.code', help_text='Configuration type code')
+    configuration_type = serializers.PrimaryKeyRelatedField(queryset=ConfigurationType.objects.all(), write_only=True, required=False)
+
+    class Meta:
+        fields = ('name', 'code', 'configuration_type', 'config_change_overhead',
+                  'schedulable', 'force_acquisition_off', 'requires_optical_elements', 'validation_schema')
+        model = ConfigurationTypeProperties
+
 
+class ConfigurationTypePropertiesSerializer(serializers.ModelSerializer):
     class Meta:
-        fields = ('name', 'code', 'config_change_overhead', 'schedulable', 'force_acquisition_off', 'requires_optical_elements', 'validation_schema')
+        fields = ('id', 'configuration_type', 'instrument_type', 'config_change_overhead',
+                  'schedulable', 'force_acquisition_off', 'requires_optical_elements', 'validation_schema')
         model = ConfigurationTypeProperties
 
 
 class InstrumentTypeSerializer(serializers.ModelSerializer):
     mode_types = GenericModeGroupSerializer(many=True, required=False, help_text='Set of generic modes that this instrument type supports')
-    configuration_types = ConfigurationTypePropertiesSerializer(source='configurationtypeproperties_set', many=True, required=False, read_only=True,
-                                                                help_text='Set of configuration types that this instrument type supports')
-
+    configuration_types = ConfigurationTypePropertiesNestedSerializer(
+        source='configurationtypeproperties_set', many=True, required=False,
+        help_text='Set of configuration types that this instrument type supports'
+    )
     class Meta:
         fields = ('id', 'name', 'code', 'fixed_overhead_per_exposure', 'instrument_category',
                   'observation_front_padding', 'acquire_exposure_time', 'default_configuration_type',
@@ -115,6 +223,24 @@ def validate_validation_schema(self, value):
 
         return value
 
+    def update(self, instance, validated_data):
+        if 'configurationtypeproperties_set' in validated_data:
+            configuration_type_properties = validated_data.pop('configurationtypeproperties_set')
+            # Clear all existing configuration type properties if we are setting a new set on an update
+            instance.configuration_types.all().delete()
+            for configuration_type_property in configuration_type_properties:
+                ConfigurationTypeProperties.objects.get_or_create(instrument_type=instance, **configuration_type_property)
+        return super().update(instance, validated_data)
+
+    def create(self, validated_data):
+        configuration_type_properties = validated_data.pop('configurationtypeproperties_set', [])
+        instrument_type = InstrumentType.objects.create(**validated_data)
+
+        for configuration_type_property in configuration_type_properties:
+            ConfigurationTypeProperties.objects.get_or_create(instrument_type=instrument_type, **configuration_type_property)
+
+        return instrument_type
+
 
 class InstrumentSerializer(serializers.ModelSerializer):
     autoguider_camera = CameraSerializer(read_only=True, help_text='Autoguider camera for this instrument')

configdb/hardware/templates/hardware/index.html

@@ -105,9 +105,18 @@ <h2>API</h2>
               <li>
                 Telescopes: <a href="{% url 'telescope-list' %}">{% url 'telescope-list' %}</a>
               </li>
+              <li>
+                Instrument Categories: <a href="{% url 'instrumentcategory-list' %}">{% url 'instrumentcategory-list' %}</a>
+              </li>
+              <li>
+                Instrument Types: <a href="{% url 'instrumenttype-list' %}">{% url 'instrumenttype-list' %}</a>
+              </li>
               <li>
                 Instruments: <a href="{% url 'instrument-list' %}">{% url 'instrument-list' %}</a>
               </li>
+              <li>
+                Camera Types: <a href="{% url 'cameratype-list' %}">{% url 'cameratype-list' %}</a>
+              </li>
               <li>
                 Cameras: <a href="{% url 'camera-list' %}">{% url 'camera-list' %}</a>
               </li>
@@ -123,6 +132,15 @@ <h2>API</h2>
               <li>
                 Generic Modes: <a href="{% url 'genericmode-list' %}">{% url 'genericmode-list' %}</a>
               </li>
+              <li>
+                Mode Types: <a href="{% url 'modetype-list' %}">{% url 'modetype-list' %}</a>
+              </li>
+              <li>
+                Configuration Types: <a href="{% url 'configurationtype-list' %}">{% url 'configurationtype-list' %}</a>
+              </li>
+              <li>
+                Configuration Type Properties: <a href="{% url 'configurationtypeproperties-list' %}">{% url 'configurationtypeproperties-list' %}</a>
+              </li>
             </ul>
             {% endblock %}
           </div>