Repository navigation
feat: Obfuscate collected must-gather output with must-gather-clean [RHIDP-16944 ] #411
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Fortune-Ndlovu
wants to merge
6
commits into
redhat-developer:main
Choose a base branch
from
Fortune-Ndlovu:RHIDP-16944-integrate-must-gather-clean-as-automatic-sanitization-for-must-gather-output
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+1,091
−20
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
148c50b
feat: obfuscate must-gather output with must-gather-clean
Fortune-Ndlovu b35a875
refactor: drop the unused in-process obfuscation helper
Fortune-Ndlovu efc3e21
fix: keep collected resources named report
Fortune-Ndlovu 4d762a6
fix: accept positional arguments after adding the obfuscate command
Fortune-Ndlovu becf057
fix: ignore cleanup error when removing the obfuscation work directory
Fortune-Ndlovu cb15337
fix: discover Ingress and Route hosts when OpenShift domains are missing
Fortune-Ndlovu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| package cli | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "os" | ||
| "os/exec" | ||
| "runtime" | ||
| "strconv" | ||
|
|
||
| "github.com/spf13/cobra" | ||
|
|
||
| "github.com/redhat-developer/rhdh-must-gather/internal/obfuscate" | ||
| ) | ||
|
|
||
| func newObfuscateCmd() *cobra.Command { | ||
| var input, output, reportDir, config string | ||
| var workers int | ||
|
|
||
| cmd := &cobra.Command{ | ||
| Use: "obfuscate", | ||
| Short: "Obfuscate an existing must-gather directory", | ||
| Hidden: true, | ||
| RunE: func(cmd *cobra.Command, args []string) error { | ||
| return obfuscate.Clean(config, input, output, reportDir, workers) | ||
| }, | ||
| SilenceUsage: true, | ||
| SilenceErrors: true, | ||
| } | ||
| flags := cmd.Flags() | ||
| flags.StringVar(&config, "config", "", "Path to the must-gather-clean config file") | ||
| flags.StringVar(&input, "input", "", "Directory of the collected must-gather") | ||
| flags.StringVar(&output, "output", "", "Directory for the obfuscated output") | ||
| flags.StringVar(&reportDir, "report-dir", "", "Directory for report.yaml, which must stay out of the published gather") | ||
| flags.IntVar(&workers, "workers", runtime.GOMAXPROCS(0), "Number of must-gather-clean workers") | ||
| for _, name := range []string{"config", "input", "output", "report-dir"} { | ||
| if err := cmd.MarkFlagRequired(name); err != nil { | ||
| panic(err) | ||
| } | ||
| } | ||
| return cmd | ||
| } | ||
|
|
||
| // runCleanSubprocess re-executes this binary so must-gather-clean's klog.Exitf | ||
| // cannot terminate the collector. That Exitf runs on the library's own error | ||
| // goroutine, so a BehaviorOnFatal hook in this process does not catch it. | ||
| // The parent keeps the original tree when the child fails. | ||
| func runCleanSubprocess(configPath, inputPath, outputPath, reportDir string) error { | ||
| exe, err := os.Executable() | ||
| if err != nil { | ||
| return fmt.Errorf("finding gather executable: %w", err) | ||
| } | ||
| cmd := exec.Command(exe, obfuscateCommandArgs(configPath, inputPath, outputPath, reportDir, runtime.GOMAXPROCS(0))...) | ||
| cmd.Stdout = os.Stdout | ||
| cmd.Stderr = os.Stderr | ||
| if err := cmd.Run(); err != nil { | ||
| return fmt.Errorf("must-gather-clean failed: %w", err) | ||
| } | ||
| return nil | ||
| } | ||
|
|
||
| func obfuscateCommandArgs(configPath, inputPath, outputPath, reportDir string, workers int) []string { | ||
| return []string{ | ||
| "obfuscate", | ||
| "--config", configPath, | ||
| "--input", inputPath, | ||
| "--output", outputPath, | ||
| "--report-dir", reportDir, | ||
| "--workers", strconv.Itoa(workers), | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would suggest not exposing this flag for now. In my understanding, the consistent replacements (
x-ipv4-...,domain0000000001) should preserve the structure we need for debugging, so I guess most analysis should still work on obfuscated output.If the need or complaints come later, we could consider adding it, but for now, I think it should just be the opinionated behavior to obfuscate (similar to the automatic sanitization which is done with no option to skip). WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would like to keep the flag. Obfuscation is already the default, the same way sanitization always runs. The consistent tokens are enough for most debugging. Heap dumps collected with --with-heap-dumps go through this pass, and looking at a memory snapshot needs the real addresses. The flag is also the way out when the clean step fails and the command refuses to publish the result. The docs say to pass --no-obfuscate for heap snapshots and for local debugging.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One of the general rules for must-gathers is to make it straightforward for customers to run when they are already facing issues with the main application like RHDH. So if our must-gather fails because of an issue in must-gather-clean, it should just fall back gracefully and and return whatever it collected. It is usually advised not to require users to re-run with another flag, as this would add friction here.
So IMO our must-gather must always ensure to fall back gracefully and not block publishing even if there are failures in the middle and return whatever it was able to capture.
So this means disabling all obfuscation entirely for collecting heap snapshots, which sounds a bit confusing IMO. Maybe a more targeted solution would be to exclude
.heapsnapshotfiles from obfuscation via the must-gather-clean config instead?As for local debugging, I don't think that's an issue. If you're debugging locally, you already have access to the real data; the must-gather ourput is for customers to share with support, where obfuscation matters.
Why would this be needed? Node heap snapshot files are just regular text files, so it should be okay if any IP, MAC addresses are obfuscated. Obfuscation returns consistent strings that will show up in the heapsnapshot files, which is fine IMO. I've just checked and the structural integrity of the JSON is preserved, and only strings held in JS memory that happen to match IP/MAC/domain patterns would be replaced. This should be the default behavior IMO even for heap snapshots.
So all this means to me that, similar to the secret sanitization, obfuscation should always be run as part of the must-gather, not opt-out.