added: linearize! is allocation-free once again

[franckgaga]

Re-designed the Jacobian and linearization buffers for an implementation that is a bit simpler, cleaner and also modular (will also work with gradients and hessians, normally)

Also prepare the ground for migration to DifferentiationInterface.jl.

There is no type-instability at runtime, tested on my side with @code_warntype.

Related to #162

[franckgaga]

Something that I still don't like

typeof(nonlinmodel) is extremely verbose because of parametric types with the AD buffers, e.g.:

NonLinModel{Float64, ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, typeof(h3!), @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, RungeKutta, ModelPredictiveControl.LinearizationBuffer{Float64, ModelPredictiveControl.JacobianBuffer{ModelPredictiveControl.var"#f_at_u_d!#192"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, ForwardDiff.JacobianConfig{ForwardDiff.Tag{ModelPredictiveControl.var"#f_at_u_d!#192"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, Float64}, Float64, 2, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{ModelPredictiveControl.var"#f_at_u_d!#192"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, Float64}, Float64, 2}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{ModelPredictiveControl.var"#f_at_u_d!#192"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, Float64}, Float64, 2}}}}}, ModelPredictiveControl.JacobianBuffer{ModelPredictiveControl.var"#f_at_x_d!#193"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, ForwardDiff.JacobianConfig{ForwardDiff.Tag{ModelPredictiveControl.var"#f_at_x_d!#193"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, Float64}, Float64, 1, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{ModelPredictiveControl.var"#f_at_x_d!#193"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{ModelPredictiveControl.var"#f_at_x_d!#193"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, Float64}, Float64, 1}}}}}, ModelPredictiveControl.JacobianBuffer{ModelPredictiveControl.var"#f_at_x_u!#194"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, ForwardDiff.JacobianConfig{ForwardDiff.Tag{ModelPredictiveControl.var"#f_at_x_u!#194"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, Float64}, Float64, 1, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{ModelPredictiveControl.var"#f_at_x_u!#194"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{ModelPredictiveControl.var"#f_at_x_u!#194"{ModelPredictiveControl.var"#rk4_solver!#21"{DataType, RungeKutta, typeof(f3!), PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, PreallocationTools.DiffCache{Vector{Float64}, Vector{Float64}}, Float64}, @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}, Vector{Float64}}, Float64}, Float64, 1}}}}}, ModelPredictiveControl.JacobianBuffer{ModelPredictiveControl.var"#h_at_d!#195"{typeof(h3!), @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}}, ForwardDiff.JacobianConfig{ForwardDiff.Tag{ModelPredictiveControl.var"#h_at_d!#195"{typeof(h3!), @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}}, Float64}, Float64, 2, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{ModelPredictiveControl.var"#h_at_d!#195"{typeof(h3!), @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}}, Float64}, Float64, 2}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{ModelPredictiveControl.var"#h_at_d!#195"{typeof(h3!), @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}}, Float64}, Float64, 2}}}}}, ModelPredictiveControl.JacobianBuffer{ModelPredictiveControl.var"#h_at_x!#196"{typeof(h3!), @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}}, ForwardDiff.JacobianConfig{ForwardDiff.Tag{ModelPredictiveControl.var"#h_at_x!#196"{typeof(h3!), @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}}, Float64}, Float64, 1, Tuple{Vector{ForwardDiff.Dual{ForwardDiff.Tag{ModelPredictiveControl.var"#h_at_x!#196"{typeof(h3!), @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}}, Float64}, Float64, 1}}, Vector{ForwardDiff.Dual{ForwardDiff.Tag{ModelPredictiveControl.var"#h_at_x!#196"{typeof(h3!), @NamedTuple{A::Matrix{Float64}, Bu::Vector{Float64}, Bd::Vector{Float64}, C::Matrix{Float64}, Dd::Vector{Int64}}, Vector{Float64}}, Float64}, Float64, 1}}}}}}}

I'm pretty sure it cannot be avoided when pre-allocating the work buffer of any AD tools. Normally the user should not see this extremely verbose type.

@gdalle I see that there is a similar "problem" in the type of the object returned by prepare_gradient in DifferentiationInterface tutorial here. Do you know if there is way of hiding some parameters when showing the type of an object e.g. with typeof ?

[gdalle]

Why would you need to hide them? If the user prints the preparation result, which is an internal, that's on them.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.02%. Comparing base (2e7ab0e) to head (896a35d).
Report is 4 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #165   +/-   ##
=======================================
  Coverage   99.01%   99.02%           
=======================================
  Files          24       24           
  Lines        3770     3790   +20     
=======================================
+ Hits         3733     3753   +20     
  Misses         37       37           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[franckgaga]

Yep, that's true if the user explicitly print them.

The real problem in my opinion is when the target function is not differentiable for some reasons. ForwardDiff stacktraces are already super verbose and hard to read, even with simple target functions. It's even worse when there is parametric types like that printed throughout the stacktrace.

Oh well, I think I will have to live with this.

[gdalle]

Also, overriding printing on types seems dangerously close to piracy

[gdalle]

There are options to abbreviate stack traces using third party packages

[franckgaga]

Alright thanks @gdalle, I'll take a look at the third party packages.

[gdalle]

The one I had in mind is https://github.com/BioTurboNick/AbbreviatedStackTraces.jl. Note that it shouldn't become a dependency of your package, it's better suited as a dev dependency installed in your global environment (like Revise).