Skip to content

Commit

Permalink
test(e2e): change consitently to eventually (#12097)
Browse files Browse the repository at this point in the history
## Motivation

I've noticed a flake in a specific test. We are using `Consistently`,
which expects that error code `52` will always be returned. This is a
correct assumption, unless something happens on the network and we
receive a different error code.

```
  [FAILED] Failed after 0.250s.
  The function passed to Consistently failed at github.com/kumahq/kuma/test/e2e/reachableservices/auto_reachable_services_k8s.go:100 with:
  Expected
      <int>: 56
  to equal
      <int>: 52
Error: It 11/22/24 10:02:24.833
```

## Implementation information

To avoid this flake, I changed `Consistently` to `Eventually` and added
`MustPassRepeatedly`. We cannot guarantee that the network is 100%
reliable, so we have two options:

* Add another error code to check, or
* If it fails, wait for 5 consecutive successes.

## Supporting documentation

https://github.com/kumahq/kuma/actions/runs/11969768862/job/33372432545

<!--
> Changelog: skip
-->
<!--
Uncomment the above section to explicitly set a [`> Changelog:` entry
here](https://github.com/kumahq/kuma/blob/master/CONTRIBUTING.md#submitting-a-patch)?
-->

Signed-off-by: Lukasz Dziedziak <[email protected]>
  • Loading branch information
lukidzi authored Nov 27, 2024
1 parent c542d02 commit 7521c57
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions test/e2e/reachableservices/auto_reachable_services_k8s.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ spec:
g.Expect(stdout).To(ContainSubstring("first-test-server_reachable-services_svc_80"))
}, "30s", "1s").Should(Succeed())

Consistently(func(g Gomega) {
Eventually(func(g Gomega) {
failures, err := client.CollectFailure(
KubeCluster,
"second-test-server",
Expand All @@ -98,7 +98,7 @@ spec:
)
g.Expect(err).To(Not(HaveOccurred()))
g.Expect(failures.Exitcode).To(Equal(52))
}, "5s", "1s").Should(Succeed())
}, "15s", "1s", MustPassRepeatedly(5)).Should(Succeed())
})

It("should connect to ExternalService when MeshTrafficPermission defined", func() {
Expand Down

0 comments on commit 7521c57

Please sign in to comment.