`value_and_gradient` with Enzyme forward-mode returns a different type

[penelopeysm]

Forward-mode Enzyme returns its own array type for the gradient, compared to other backends which return regular Vector{Float64}:

julia> f(x) = x[1] + x[2]
f (generic function with 1 method)

julia> import DifferentiationInterface as DI

julia> using Enzyme

julia> typeof(DI.value_and_gradient(f, AutoEnzyme(mode=Enzyme.Forward), [0.5, 0.2])[2])
Enzyme.TupleArray{Float64, (2,), 2, 1}

julia> typeof(DI.value_and_gradient(f, AutoEnzyme(mode=Enzyme.Reverse), [0.5, 0.2])[2])
Vector{Float64} (alias for Array{Float64, 1})

It's easy enough to handle with a collect on my side and/or using value_and_gradient! (the copyto! gives back the Vector{Float64}). Enzyme officially doesn't promise what type of AbstractArray it returns, but wondered if this was something worth looking at and/or documenting in DI?

[gdalle]

DI's approach is to be as un-opinionated as possible and rely on the backend's API whenever we can (in this case Enzyme). Besides, calling convert or collect would incur a perfomance penalty.
I think this is not something I want to change inside DI, because standardizing outputs is a vicious can of worms. In the same way, for x::StaticArray, some backends may preserve the staticness while others may not.

[penelopeysm]

No problem - very happy with that. I can add collect on my end so not a problem at all (it's just a testing utility not performance sensitive). I think it might be nice to have a note in DI's FAQ though - would you like me to add one?

[gdalle]

Yeah I guess we could add a warning somewhere that the precise type returned is a property of the backend and not a DI guarantee