Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kick off data sources service #5022

Merged
merged 1 commit into from
Nov 25, 2024
Merged

Conversation

JAORMX
Copy link
Contributor

@JAORMX JAORMX commented Nov 21, 2024

Summary

This service will contain all the business logic to deal with data
sources.

For the moment this only kicks off the get calls. But populating this should
allow others to work on other function definitions.

Change Type

Mark the type of change your PR introduces:

  • Bug fix (resolves an issue without affecting existing features)
  • Feature (adds new functionality without breaking changes)
  • Breaking change (may impact existing functionalities or require documentation updates)
  • Documentation (updates or additions to documentation)
  • Refactoring or test improvements (no bug fixes or new functionality)

Testing

Outline how the changes were tested, including steps to reproduce and any relevant configurations.
Attach screenshots if helpful.

Review Checklist:

  • Reviewed my own code for quality and clarity.
  • Added comments to complex or tricky code sections.
  • Updated any affected documentation.
  • Included tests that validate the fix or feature.
  • Checked that related changes are merged.

@JAORMX JAORMX force-pushed the datasourceservice branch 2 times, most recently from 5e93fb8 to cb82429 Compare November 21, 2024 15:02
@JAORMX JAORMX marked this pull request as ready for review November 21, 2024 15:03
@JAORMX JAORMX requested a review from a team as a code owner November 21, 2024 15:03
@JAORMX JAORMX force-pushed the datasourceservice branch 5 times, most recently from 5d646db to f63c683 Compare November 22, 2024 09:49
@coveralls
Copy link

coveralls commented Nov 22, 2024

Coverage Status

coverage: 54.594% (-0.03%) from 54.624%
when pulling f63c683 on JAORMX:datasourceservice
into be22925 on mindersec:main.

blkt
blkt previously requested changes Nov 22, 2024
Copy link
Contributor

@blkt blkt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be wrong here, please keep me honest.

database/query/datasources.sql Outdated Show resolved Hide resolved
database/query/datasources.sql Outdated Show resolved Hide resolved
database/query/datasources.sql Outdated Show resolved Hide resolved
database/query/datasources.sql Outdated Show resolved Hide resolved
@JAORMX
Copy link
Contributor Author

JAORMX commented Nov 22, 2024

@blkt @teodor-yanev and I talked about this and agreed to include the project ID in the functions and relationships tables. @blkt will bring this in and I'll rebase on top of it.

@JAORMX JAORMX mentioned this pull request Nov 23, 2024
10 tasks
@JAORMX JAORMX force-pushed the datasourceservice branch 2 times, most recently from b4643aa to 23885bf Compare November 25, 2024 06:16
@JAORMX JAORMX requested a review from blkt November 25, 2024 06:16
This service will contain all the business logic to deal with data
sources.

Signed-off-by: Juan Antonio Osorio <[email protected]>
@@ -0,0 +1,73 @@
-- CreateDataSource creates a new datasource in a given project.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we have delete on cascade based on the data source id for functions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}

// All data source types should be equal... so we'll just take the first one.
dsfType := dsfuncs[0].Type
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we ensure/validate that all sources have the same type?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should. I was thinking of doing this on the create and update parts though. There's not much we can do on the get side.

})
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
return db.DataSource{}, util.UserVisibleError(codes.NotFound,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it better to return nil instead of empty object in case of errors?

Comment is valid for the other use cases below

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this sub-function returns an object and not a pointer, since that's the type of object that the database wrappers usually returns. we do return pointers and thus nil underneath though.


func (d *dataSourceService) List(
ctx context.Context, project uuid.UUID, opts *ReadOptions) ([]*minderv1.DataSource, error) {
stx, err := d.txBuilder(d, opts)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Why do we want to list within a transaction?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, data sources (and their functions) are part of separate tables, we want the states of these to be consistent when listing/getting them, and not get into funky intermediate states if parallel update happens.

Note that I do set the transaction to be "read-only" so this doesn't block other reads.

@JAORMX JAORMX dismissed blkt’s stale review November 25, 2024 10:01

This has been addressed

@JAORMX JAORMX merged commit 4dff82f into mindersec:main Nov 25, 2024
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants