All journeys export failures

[atomicsamurai]

Frodo CLI version

You are running the homebrew release.
cli: v3.0.3
lib: v3.0.3
node: v20.18.0

Describe the issue

When exporting all journeys with -A, if one or more of the journeys fail, nothing is exported, even though the export seems to continue after the failed journey.

$ frodo journey export -A -D . --curlirize yyc
curl -X GET -H "Accept:application/json, text/plain, */*" -H "Content-Type:application/json" -H "User-Agent:@rockcarver/frodo-lib/3.0.3" -H "X-ForgeRock-TransactionId:frodo-98676a15-e027-4f86-b7bc-f9c975f35f11" -H "Accept-API-Version:resource=1.1" "https://openam-yyc-dev.forgeblocks.com/am/json/serverinfo/*"
200 OK
.
.
.
200 OK
curl -X GET -H "Accept:application/json, text/plain, */*" -H "Content-Type:application/json" -H "User-Agent:@rockcarver/frodo-lib/3.0.3" -H "X-ForgeRock-TransactionId:frodo-98676a15-e027-4f86-b7bc-f9c975f35f11" -H "Accept-API-Version:protocol=2.0,resource=1.0" -H "Auth"
curl -X GET -H "Accept:application/json, text/plain, */*" -H "Content-Type:application/json" -H "User-Agent:@rockcarver/frodo-lib/3.0.3" -H "X-ForgeRock-TransactionId:frodo-98676a15-e027-4f86-b7bc-f9c975f35f11" -H "Authorization:Bearer eyJ0eXAiOiJKV1QiLCJlbmMiOiJBMTI4Q"
200 OK
[==================----------------------] 45% | 37/82 | Exporting journey ExportTesting404 Not Found - Not Found
Object [AxiosHeaders] {
  date: 'Mon, 24 Mar 2025 16:05:04 GMT',
  'cache-control': 'no-store',
  'content-api-version': 'protocol=2.1,resource=1.0',
  'content-security-policy': "default-src 'none';frame-ancestors 'none';sandbox",
  'content-type': 'application/json;charset=utf-8',
  'cross-origin-opener-policy': 'same-origin',
  'cross-origin-resource-policy': 'same-origin',
  expires: '0',
  pragma: 'no-cache',
  'x-content-type-options': 'nosniff',
  'x-frame-options': 'DENY',
  'content-length': '55',
  'x-forgerock-transactionid': 'frodo-98676a15-e027-4f86-b7bc-f9c975f35f11',
  'strict-transport-security': 'max-age=31536000; includeSubDomains; preload;',
  'x-robots-tag': 'none',
  via: '1.1 google',
  'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000'
}
200 OK
curl -X GET -H "Accept:application/json, text/plain, */*" -H "Content-Type:application/json" -H "User-Agent:@rockcarver/frodo-lib/3.0.3" -H "X-ForgeRock-TransactionId:frodo-98676a15-e027-4f86-b7bc-f9c975f35f11" -H "Accept-API-Version:protocol=2.1,resource=1.0" -H "Auth"
200 OK
.
.
.
200 OK
Error exporting journeys
  Error exporting journey ExportTesting
  HTTP client error
    Code: ERR_BAD_REQUEST
    Status: 404
    Reason: Not Found
[========================================] 100% | 82/82 | Error exporting journeys.

At the end of the process, no files are created.
STDOUT/STDERR from the above is attached.

[atomicsamurai]

export.log

[vscheuber]

@hfranklin you said you would have your team take a look. Assigning this to you for tracking.

[phalestrivir]

I looked into it, and I can't reproduce the problem since it appears that you have a journey that is being read that is referencing a script that doesn't exist. Normally, this shouldn't be possible to have since they prevent deleting scripts if they are being used somewhere (such as a journey), so I'm not sure how this happened in the first place and how to reproduce it.

However, regarding the issue of exporting other journeys that don't have any errors, this problem appears not only for journey exports but most other exports as well.

What's happening is that for most exports using the -A flag, it will fail when exporting if any error occurs, regardless of whether or not any singular exports are successful.

Journeys in particular do export all the journeys regardless of whether an error occurs during one of them. However it will still throw an error at the end if any errors do occur, essentially canceling the export.

The question is how do we want to fix this. I can think of two ways:

1. If we want an error to be thrown whenever an error occurs, we can update the journey export code (and any other similar export code) to stop when it encounters the first error instead of continuing. This would be the easiest and fastest solution since most exports already do this, with Journeys, Scripts, and maybe a few others being the exception.
2. If we still want to export what we can despite any errors that occur, then we'd need to change how we handle exports. Instead of throwing the errors, we could either print the errors to the console when they happen or we could do something where we pass in an optional array to the export function that can then be populated with any errors that do occur during the export so that they can be handled by the client if the client wants to handle them in a special way. Currently, the client can handle errors in a special fashion by using try-catch statements around the export function if anything is thrown, but this prevents us from returning any entities that were successfully exported.
3. Another option is we add a flag where the user can decide whether to ignore errors or not during exports. If they ignore errors, we don't throw anything (and maybe instead we just print them out in this case instead of throwing them).

I'm thinking the best option would be number 2 where we add optional array parameter to the export functions so that they return the exported entities along with any errors if the client wants to handle them. However, this will involve some refactoring to the various export functions in Frodo to allow for this. In some cases where exports are done by doing a single HTTP request to get all the entities, we won't be able to do this. We can for sure do it for journeys, scripts, and probably a few other ones that make separate HTTP requests for each export.

@vscheuber @hfranklin Do you guys think this is the a good way to do it, or do you guys have a better idea for how we want to handle it?

[vscheuber]

@phalestrivir @atomicsamurai @hfranklin I am so torn about this. A lot of the reasoning why we did what we did is coming back to me as I think about this:
Back in the early days of having frodo split into lib and cli one of our partners in the UK made contributions to the project and their biggest concern was our error handling at the time, which was to only print errors even at the library level but not actually report errors. It was in discussions with that partner (and I believe you were on, too, @hfranklin ) that we declared that our contract would be:
The library always throws
It was that decision that made us remove all screen output from the library and moved all error printouts to throwing exceptions. We also introduced the array of errors or FrodoError, which can store an array of errors, to preserve all state even in a multiple error scenario.
I am pretty certain that in one or some ops layer modules I introduced the option to pass an array as the last argument of a function and instead of throwing an exception, the exception(s) would be added to that array. I cannot remember if that code is still in play or if it was just an interim state of our current end state.

So here is the reason I struggle: If we decide to change the current behavior, I advocate strongly for implementing it consistently across all our modules in the ops layer, which is a lot of work and will generate a lot of new test cases.

As I am thinking through this: I believe there is an option #4:

4. Instead of returning results and swallowing exceptions or populating a caller-provided error array, how about we introduce a new field in FrodoError called PartialResults or something like that. We then populate that field with what the function would return if there was no error. In the final FrodoError we throw, we then include the results. That way the CLI can decide what to do about it and we don't really need to make any significant changes. Still changes to every OPS layer module but a much simpler change and we don't break our own contract (the library always throws).

Thoughts?

[vscheuber]

So in FrodoError we would do something like this:

And then in every OPS layer module we would add line 1416 in this next screenshot to every function that is is capable of returning partial results:

The CLI can then do with those whatever it sees fit. In this particular case the current issue is about, it could write the journeys it got to file and log the error and clearly indicate that some files might be corrupt and not importable.

[phalestrivir]

@vscheuber Yes, thanks for reminding me about The library always throws rule, I remember @hfranklin mentioning it to me before. If we want to continue with following that rule, then returning the partial results as part of the error is probably our only option since I can't think of any other way to return those results back without breaking that rule.

I will put together a PR later this week which makes these changes in frodo-lib and updates any exports where it would apply. Same for the changes for the frodo-cli.

[vscheuber]

Fantastic! Thanks, @phalestrivir !

[phalestrivir]

@vscheuber I created PRs earlier today for the changes where we return the partial results as part of the FrodoError. I talked with @hfranklin today as well and he had some other ideas as well for returning partial results which I'll share here:

One option is returning a result set. For example, we would call exportJourneys() as usual, but instead of return just the exported object it would return any errors that were thrown. I'm already doing something similar to this in my current PR's with the getResults helper method I created. It essentially calls the export functions, and returns an object with the results in addition to any errors that occurred in a single object. This means if we want to do it this way, I wouldn't need to make too many changes to the PRs. This is the way @hfranklin said he probably would've done it.

Another option is sending callback functions as part of the export functions. For example, we could do exportJourneys(handleError), where handleError is a function that gets called when an error is thrown so that we can handle any errors that do occur. What we could then do is if no callback is provided, then the default behavior is to just throw the error as normal.

I like both these options, but I'm not sure if they would follow the The library always throws rule. If we return the errors like we do in the first option, the library no longer is throwing (at least for the export functions), but just returning them. The second option I think does follow the rule though because it still throws the errors, but provides an option to handle them if you want to via a callback. Then again, using a callback would mean that the function no longer throws the errors (since they are being handled in the callbacks), so maybe it still breaks that rule. However, callbacks are a pretty common way to do this type of thing, so it may be a good option.

Let me know what you think @vscheuber, I'm happy to change my PRs if we want to do it differently.