[RFD - 0006] - Semaphore API v2

[AleksandarCole]

Metadata

Authors	State
@radwo @hamir-suspect	Draft

Summary

Our current API is in an alpha version, and many services lack complete documentation. This situation hinders user adoption, makes integration difficult, and complicates future development.

Problems

Copied direct feedback from users. List to be expanded (WIP)

Bad UX for an endpoint

When scheduling a promotion there is no way to then reference that promotion. The only way to try and find it is by manually searching for it on the list of promotions from the workflow which is prone to race conditions.

Fetching latest promotions for a specific project or a specific branch requires multiple API calls.

Missing API endpoints

There's no API for managing artifacts or for managing cache.

Bad API documentation and usability

Discovering how API works and which keys refer to which data requires practically reverse engineering it by opening specific places in UI and then trying to match the UUIDs to the data from the API. There's no clear documentation for most of the API responses and no explanation is given as to why things are this complicated.

User Stories

Copied direct feedback from users. List to be expanded (WIP)

Missing start time - job

If I navigate to https://{domain}/api/v1alpha/jobs/{jobID} to get the job metadata, I see:

"metadata": { "name": "{JOB NAME REDACTED}", "id": "{JOB ID REDACTED}, "create_time": "1669925940", "update_time": "1669926743", "finish_time": "1669926743" }, ... 

I expect to see start_time as a field under metadata

Multiple API calls

For the Jobs api, if there’s a way to list all jobs using the pipeline_id this would vastly improve the performance of the connector being built and reduce a bunch of serial calls to Semaphore’s server to aggregate all the data

Listing pipelines created after certain time

Does Semaphore API supports listing pipelines created after certain time frame?  Are there additional query parameters other than the ones listed in this page?

My use case is around analytics, engineering metrics (DORA). This project here has integrations with multiple CI sources but not Semaphore and some of the reason is likely related to its api. I’m looking into the visibility of this but these created_at filter attributes are required to implement incremental based extraction, and make it more lightweight.

I’m doing an implementation that will have to pull in and paginate through all of the api’s data for our organization on every sync (we’ll likely do it on a low frequency basis).This is not great, and I’m adding certain configurations (like: throttle, and filters [project and branch]) to remediate the load of it, but the correct way to implement is by being able to filter on a timestamped field so we can incrementally bring new records on a high frequency basis as well as choose to load data only from a certain timeframe

RFC link

#224