Plan for backend changes to support Units in Libraries

[bradenmacdonald]

Sketching out the backend work remaining until the backend will fully support the Epic 12: Units in Libraries epic, and the bare-bones version of Epic 14: Units can be published and reused in courses

Learning Core tasks

• Update and merge ADRs, close out ADR for Units and Containers openedx-learning#240
• Review and merge Learning Core Units prototype: Containers + Units in Learning Core openedx-learning#278 (I think it's pretty much ready for review! 🚀)
• Implement fine-grained tracking of changes for drafts & publishes ? - combine containers app into publishing? and Support for grouped changesets of draft modifications openedx-learning#290
• Publishing a unit should auto-publish children - likely requires combining containers app into publishing (test exists but is currently skipped)
• Prevent adding soft-deleted components to a unit (test exists but is currently skipped) (low priority)
• Backend: Mechanism for tracking which components were created within a unit? openedx-learning#284 Requirements TBD.

edx-platform Libraries REST API tasks

• Backend: implement opaque key format for structures in libraries #1704 See What should library container usage keys look like? openedx-learning#282
• Backend: Basic CRUD REST Endpoint for units in libraries #1703
• Backend: Allow adding existing components when creating a unit via the REST API. #1705
• Backend: REST API to create a new component within a unit #1706
• Backend: When a component is edited, all parent Units must show changes #1707
• Backend: Units can be organized with Tags and Collections #1709

Unclear tasks

• Implement a mechanism for storing unit-level settings (visibility, discussions_enabled) - see Slack discussion
• We may need an API to list which children have unpublished changes, to support Need interim publish confirmation modal for components #1188. However, we may also be fine getting this data from the search index or just iterating over the children with an existing API.
• (Probably not for MVP): Implement APIs + tests to get PublishLog history - notes

Tasks that can be done at the same time as the frontend ticket

• Store list of types of the child components in the search index, to support Unit card previews #1618

edx-platform REST API tasks for using units in courses

• Backend: REST API to publish all changes in a unit #1710
• Backend: API to link a unit into a course #1711
• Disallow edits to units in courses that are sourced from a library #1712 (frontend only)
• Sync changes from published unit #1634 (backend and frontend)

[bradenmacdonald]

@ormsbee @kdmccormick @pomegranited Based on my work with our "Units/Containers" prototype so far, here's my attempt to plan the remaining backend work required for the units epic. Can you please review, and let me know your thoughts - especially if I've missed anything or included anything uncessary?

[bradenmacdonald]

Oh, in particular I'm looking for feedback on:

• Decide on / implement opaque key format for structures (like units) in libraries.
• combining containers into the publishing app

More details on both are above in the main description.

[pomegranited]

@bradenmacdonald This is looking great!

I think we're missing backend support for these two use cases, but everything else looks covered above:

• Reorder components within unit #1621
• Remove component from unit #1622 (as opposed to Delete component within unit #1623, which is covered)

...Also update the "GET unit" endpoint to return whether or not the unit has unpublished changes and whether or not it contains unpublished changes (among its children).

1. This GET endpoint can't just return a boolean flag -- we also need to list which children have unpublished changes, to support Need interim publish confirmation modal for components #1188 (this issue is listed as not part of Teak in my notes, but is still in the epic Epic 12: Units can be created and edited in Libraries #1594)

2. Is there a requirement to be able to search/filter "all Units that have unpublished changes in their children"?
   If so, we'll need to make sure the Unit publish_status field in Meilisearch reflects this full unpublished status for the container and its children.

Basic CRUD REST Endpoint for units in libraries. Allow creating a unit, viewing units in library and search results, setting/changing the title, and deleting the unit. No ability to add components yet. Enforce permisisons. Set a "friendly" slug based on the title.

When we do add support for components in units, the "DELETE unit" endpoint also needs to handle re-parenting its child components into the library (if that step is required), a la #1622. This is also mentioned in #1628.

Implement a mechanism for storing #1640 (visibility, discussions_enabled) - see Slack discussion

My notes say this requirement been removed from MVP/Teak?

[bradenmacdonald]

@pomegranited Thanks!

I think we're missing backend support for these two use cases, but everything else looks covered above:

• Reorder components within unit #1621
• Remove component from unit #1622 (as opposed to Delete component within unit #1623, which is covered)

I've updated the wording of the "Update API" story above to explicitly mention re-ordering. It covers removal too (already did): "Allow re-ordering and removing components via the update API."

1. we also need to list which children have unpublished changes, to support Need interim publish confirmation modal for components #1188

I'm a bit reluctant to add that to the "GET unit" API, because it would be a more expensive thing to compute. But we can look at adding a separate API for it if needed. We may also be able to get away with just querying the search index, which should have that information already and already will provide it along with the component titles etc. I will add a note about this above though as a potential story.

2. If so, we'll need to make sure the Unit publish_status field in Meilisearch reflects this full unpublished status for the container and its children.

I think we should do that in any case, so I've noted it above.

When we do add support for components in units, the "DELETE unit" endpoint also needs to handle re-parenting its child components into the library (if that step is required), a la #1622. This is also mentioned in #1628.

I don't think we need any re-parenting. My understanding is that all components will always be visible both in their unit(s) (if any) and separately in the library. When you delete the unit, the child component is still visible separately in the library just as it always was, with no action required on our part.

Implement a mechanism for storing #1640 (visibility, discussions_enabled) - see Slack discussion

My notes say this requirement been removed from MVP/Teak?

Yeah it may very well be - we're still discussing.

[pomegranited]

@bradenmacdonald

Publishing a unit should auto-publish children - likely requires combining containers app into publishing (test exists but is currently skipped)

Looks like your work on publishing.api.publish_from_drafts already covers this point 🚀 Was there anything else that needs doing? If so I'll add them to FAL-4076; if not, I'll just archive that task.

We'll need a little REST API in the platform to publish Units, but I think we can do that as part of #1627 .

[bradenmacdonald]

@pomegranited Yeah, as far as I can think, it's just the REST API that we'll need and everything else is already implemented.