fix(vulnerability): returning remediated vulnerabilities#1180
Conversation
817ec99 to
e62b880
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 107 out of 107 changed files in this pull request and generated 9 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| decodedKey, err := cache.DecodeKey(key, cache.DefaultKeyHash) | ||
| if err != nil { | ||
| wrappedErr := appErrors.InternalError(string(op), "Remediation", "", err) | ||
| applog.LogError(rh.logger, wrappedErr, logrus.Fields{ | ||
| "remediation": remediation, |
There was a problem hiding this comment.
Cache invalidation decodes keys using cache.DefaultKeyHash, but the cache’s configured KeyHash may differ (SHA256/SHA512/etc.), in which case decoding will fail and invalidation won’t run. Consider exposing the active key-hash (or a decode helper) on the Cache implementation, or avoid decoding entirely by storing non-hashed/tagged keys for invalidation.
There was a problem hiding this comment.
Can you check this comment @kanstantsinbuklis-sap
There was a problem hiding this comment.
Added method to get key hash depending on config
| stmt, filterParameters, err := s.buildServiceStatement(context.Background(), baseQuery, filter, false, order, l) | ||
| if err != nil { | ||
| l.Error("Error preparing statement: ", err) | ||
| return nil, err |
There was a problem hiding this comment.
getServiceAttr accepts a ctx, but it currently uses context.Background() when preparing the statement. This breaks request context propagation (timeouts/cancellation) and defeats the purpose of adding ctx. Use the passed ctx here.
| // Execute the query | ||
| rows, err := stmt.Queryx(filterParameters...) | ||
| rows, err := stmt.QueryxContext(context.Background(), filterParameters...) | ||
| if err != nil { | ||
| l.Error("Error executing query: ", err) | ||
| return nil, err |
There was a problem hiding this comment.
getServiceAttr uses stmt.QueryxContext(context.Background(), ...) instead of the provided ctx, so query cancellation/timeouts won’t propagate. Use ctx for the query call as well.
| rows, err := performListScan( | ||
| context.Background(), | ||
| stmt, | ||
| filterParameters, | ||
| l, |
There was a problem hiding this comment.
GetAllIssueCursors builds the statement with the passed ctx, but then calls performListScan(context.Background(), ...). This drops request context propagation for the query (timeouts/cancellation). Pass ctx through to performListScan.
| if strings.Contains(decodedKey, fmt.Sprintf("\"issue_id\":[%d]\"", newRemediation.IssueId)) && | ||
| (strings.Contains(decodedKey, "GetIssuesWithAggregations") || strings.Contains(decodedKey, "GetIssues") || | ||
| strings.Contains(decodedKey, "GetAllIssueCursors")) { |
There was a problem hiding this comment.
The invalidation match uses "issue_id" in the decoded cache key, but entity.IssueFilter serializes the issue id field as "id" (json tag id). This condition will never match, so the stale Issue cache entries won’t be invalidated after remediation creation. Update the match to use the correct serialized field name (and consider matching the function name + structured JSON instead of brittle substrings).
There was a problem hiding this comment.
Can you check this comment @kanstantsinbuklis-sap
There was a problem hiding this comment.
Added additional check
| if err := rows.Err(); err != nil { | ||
| msg := "Error while iterating over result rows" | ||
| l.WithFields( | ||
| logrus.Fields{ | ||
| "error": err.Error(), | ||
| "parameters": filterParameters, | ||
| }).Error(msg) | ||
|
|
||
| return nil, fmt.Errorf("%s", msg) | ||
| } |
There was a problem hiding this comment.
When rows.Err() is non-nil, performListScan logs the underlying error but returns fmt.Errorf("%s", msg), losing the original error and making troubleshooting harder. Return an error that wraps the underlying cause (e.g., include %w).
637fec5 to
445b392
Compare
| }) | ||
|
|
||
| issues, err := app.ListIssues(f, opt) | ||
| // TODO: remove it |
There was a problem hiding this comment.
Deleted redundant comment
| RUN mockery | ||
| # generate graphql code | ||
| RUN cd internal/api/graphql && go run github.com/99designs/gqlgen generate | ||
| # # generate mock code files |
There was a problem hiding this comment.
I'm not sure we need to generate mocks and GraphQL in the final image, It has to be earlier
BWT I've mentioned that we avoid some dependencies caching, ldflags and dockerignore that also can reduce the final image size
FROM --platform=${BUILDPLATFORM:-linux/amd64} golang:1.26 AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build \
-trimpath \
-ldflags="-s -w" \
-o /bin/heureka \
cmd/heureka/main.go
FROM --platform=${TARGETPLATFORM:-linux/amd64} gcr.io/distroless/static-debian12:nonroot
LABEL source_repository="https://github.com/cloudoperators/heureka"
USER nonroot:nonroot
COPY --from=builder /bin/heureka /heureka
ENTRYPOINT ["/heureka"]Maybe this Dockerfile will be more applicable?
There was a problem hiding this comment.
The mocks are only required for tests, but the GraphQL generation is required for the app, so it needs to be generated before the build.
| key, err := c.CacheKey(fnname, fn, filterArgs(args...)...) | ||
| if err != nil { | ||
| return zero, fmt.Errorf("cache (key): Could not create cache key") | ||
| return zero, fmt.Errorf("cache (key): could not create cache key") |
There was a problem hiding this comment.
I have to say I am not convinced about this particular solution. It checks for args to find context in there to delete it. In the other function (getCallParameters) it checks for context type I am not sure why it is necessary...that is solution with too many conditions and a bit complex logic behind it and I am not sure we want it here.
Consider two other options:
- Define new template function: CallCachedWithContext, where context will be passed explicitly in named argument, this way we will be able to skip context in key easily, and add it to the call if necessary. The only requirement is that context will be first argument of called function, but this is what we have in all functions and as far as I know most (if not all) library functions using context have it in first argument.
If we do not use cache other way (without context) we can even remove callcached (but I would not do that). - Second option is to use lambda function to bind call with context and omit it in parameters
I think you can easily create generic wrapper to remove context parameter and inject it in original call
so in CallCached you have param:
....
LocalContext(ctx, ir.database.GetIssueRepositories)
....
which will remove context from the function and use local context if necessary.
This way we will avoid changes in current cache code.
and this option seems to be very good usage of first class citizen function in golang
445b392 to
60cb523
Compare
54241d1 to
1cb9f6f
Compare
michalkrzyz
left a comment
There was a problem hiding this comment.
Is context change in cache related to the fix of the problem with returning remediated vulnerabilities? If not maybe we should split it, because both changes seems that they need to be worked out.
| return nil, wrappedErr | ||
| } | ||
|
|
||
| if rh.cache != nil { |
There was a problem hiding this comment.
Is this cache handling function implemented in handler? This is not acceptable, we cannot mix those layers.
If we really want such handler we need to think about the proper pattern to handle that. We can't just mix logical layers this way.
I think that proper solution in here is to change GetAll interface function which is exposing too much and is not acceptable solution, to interface function named Invalidate(.*) which will take some regexp/wildcard/list of strings/wildcard/regex or whatever to filter through keys (decoded inside cache layer)
BTW GetAll is not a good name
'Get All Cache' - what do you expect to be done under such statement?
I would prefer to use GetAllKeys or even GetAllValidKeys
| return getReturnValues[T](out) | ||
| } | ||
|
|
||
| func callDisabledWithContext[T any](ctx context.Context, fn any, args ...any) (T, error) { |
There was a problem hiding this comment.
this is redundant function, please remove it and use common one
| if err == nil { | ||
| c.IncHit() | ||
| c.LaunchRefresh(func() { | ||
| _, _ = callFn[T](c, ttl, key, v, in) |
There was a problem hiding this comment.
If you are not using any special handling in refresh procedure, why don't you just use bind-like lambda to hide the context?
| func callEnabledWithContext[T any](ctx context.Context, c Cache, ttl time.Duration, fnname string, fn any, args ...any) (T, error) { | ||
| var zero T | ||
|
|
||
| v, in, err := getCallParametersWithContext(fn, ctx, args...) |
There was a problem hiding this comment.
this is something I wanted to avoid, why don't you use fn, args... with the old function?
| return v, in, nil | ||
| } | ||
|
|
||
| func getCallParametersWithContext(fn any, ctx context.Context, args ...any) (reflect.Value, []reflect.Value, error) { |
There was a problem hiding this comment.
This is redundant function
| return valStr, true, nil | ||
| } | ||
|
|
||
| func (imc *InMemoryCache) GetAll() ([]string, error) { |
There was a problem hiding this comment.
Is this getting all keys? I don't think we want invalid (ttl elapsed) keys to be considered here
There was a problem hiding this comment.
When getting keys from valkey and redis, you won't get invalid keys. It makes sense to rename the function to GetAllKeys, but GetAllValidKeys doesn't make sense
…emediated-vulnerabilities
1cb9f6f to
be04783
Compare
| }) | ||
|
|
||
| // TODO: add context | ||
| ctx := context.TODO() |
There was a problem hiding this comment.
I think we can use Background in here, because that is what we want.
There was a problem hiding this comment.
Changed
|
|
||
| // Trigger autopatch whenever a scanner run has completed successfully | ||
| if _, err := srh.database.Autopatch(); err != nil { | ||
| if _, err := srh.database.Autopatch(context.TODO()); err != nil { |
There was a problem hiding this comment.
context.Background() instead of TODO
There was a problem hiding this comment.
Changed
|
|
||
| // Fetch IssueRepositories | ||
| issueRepositories, err := db.GetIssueRepositories(&entity.IssueRepositoryFilter{ | ||
| issueRepositories, err := db.GetIssueRepositories(context.TODO(), &entity.IssueRepositoryFilter{ |
There was a problem hiding this comment.
context.Background() - because we will never have foreground context here anyway
There was a problem hiding this comment.
Changed
|
|
||
| // Get Issue Variants | ||
| issueVariants, err := db.GetServiceIssueVariants( | ||
| context.TODO(), |
There was a problem hiding this comment.
context.TODO() -> context.Background()
There was a problem hiding this comment.
Changed
| Debug("Building map of IssueVariants for issues related to assigned Component Version") | ||
|
|
||
| issueVariantMap, err := shared.BuildIssueVariantMap(db, &entity.ServiceIssueVariantFilter{ | ||
| issueVariantMap, err := shared.BuildIssueVariantMap(context.TODO(), db, &entity.ServiceIssueVariantFilter{ |
There was a problem hiding this comment.
context.TODO() -> context.Background()
There was a problem hiding this comment.
Changed
| Debug("Fetching issue matches related to assigned Component Instance") | ||
|
|
||
| issue_matches, err := db.GetIssueMatches(&entity.IssueMatchFilter{ | ||
| issue_matches, err := db.GetIssueMatches(context.TODO(), &entity.IssueMatchFilter{ |
There was a problem hiding this comment.
context.TODO() -> context.Background()
There was a problem hiding this comment.
Changed
|
|
||
| // Fetch services | ||
| services, err := db.GetServices(&entity.ServiceFilter{}, []entity.Order{}) | ||
| services, err := db.GetServices(context.TODO(), &entity.ServiceFilter{}, []entity.Order{}) |
There was a problem hiding this comment.
context.TODO() -> context.Background()
There was a problem hiding this comment.
Changed
| return vc.client.Do(vc.ctx, vc.client.B().Del().Key(key).Build()).Error() | ||
| } | ||
|
|
||
| func (vc *ValkeyCache) InvalidateByMatch(keys []string, keyMatcher func(string) bool) error { |
There was a problem hiding this comment.
I think this is some unused code and should be removed. Either we will extend interface or we define this function in cache.go both options are ok but we have to decide to use one.
Anyway why don't we implement it as interface function instead of GetAllKeys?
There was a problem hiding this comment.
Moved this as the method for valkey and in memory caches
…emediated-vulnerabilities
0985985 to
6847be4
Compare
There was a problem hiding this comment.
It is a very good job! Thank you. @kanstantsinbuklis-sap
Description
In this PR I've returned cache to the remediation handler and added logic to invalidate cached entries during remediation creation. Also I've added context propagation from the request to database layer
What type of PR is this? (check all applicable)
Related Tickets & Documents
Added tests?
Added to documentation?