race condition opportunities in writing tools and verification tags #477
Comments
|
Overall though, I am thoroughly surprised at how well it works to run That said, in our implementation, we only build the actual images of each variant as we offload all package building to another process that builds the kit itself. I imagine starting 4 twoliter processes against the same source tree including overlapping package builds would've ended in disaster..? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi!
As you may know, we have created an unholy beast by wrapping
twoliterin Bazel. An implementation that I am sure makes even you, perhaps ask "why".A fun thing that Bazel does, is that if you have several inputs to a rule where the dependency graph has put them firmly parallel to each other, it will build those in parallel, whether you want Bazel to or not. (It is incredibly difficult to serialize actions in Bazel based on an engineer's opinion)
This has put me in the weird position of having found a couple of race issues in
twoliter, and also suffering from their existence. Here, race issue means "if I for some stupid reason start more than onetwoliterprocess, this will break".The issues are two, one is writing out the tools/ folder. I understand that you do want to do this every time you run
twoliterto avoid a host of "this is broken on my machine". I ended up implementing a file lock for writing it out to avoid the odd double-remove race condition.For the second one, which is removing the verification tags for the SDKs, I ended up with a bit larger implementation - this implementation is that I am actually hashing the contents of the verification tags and comparing before removing/writing out again. I also found that in a few places you gratuitously remove the verification tags several times instead of just writing them out again, perhaps squarely to stop me from doing what I am doing. It happens as you can see in this log excerpt:
There was a third call to
::project::lock::verificationoriginally, but this was part through my 'fix', it does however still paint a pretty good picture of it being called multiple times (removing the verification tags) a bit redundantly.I will attach a PR that contains these fixes for your amusement. We are carrying this patch for now (also hence the binary is called
twoliter__bin, an artefact of building it using Bazel).The text was updated successfully, but these errors were encountered: