As/emulate sample with nn hmc

[sandreza]

Adds an emulate-sample example using a neural network and hamiltonian monte-carlo. Enzyme is used for autodiff. The example runs in about 2 minutes on a local laptop. This is done as a separate directory and is meant as a discussion point rather than user-friendly "emulate sample" framework (which would better go in the CalibrateEmulateSample package)

[glwagner]

Does the example produce a posterior eestimate? What is it?

[sandreza]

It produces the following figures:

[glwagner]

Epic! What do the bimodal results mean (parameters 1 and 5). Noise or something meaningful?

[sandreza]

I think it's just noise. More sampling might suffice to get rid of it (or perhaps a better neural network surrogate). I don't expect this to be the "end" of the story, but rather a starting point for further work

[sandreza]

Here it is with 100,000 samples:

[sandreza]

and the regularization distribution (basically the prior)

[glwagner]

How expensive is it? Can we experiment changing the prior/regularization distribution?

[sandreza]

It runs in a few minutes. It's easy to change

[sandreza]

The likelihood (no regularization)

[sandreza]

Note that these are the marginal distributions as given by the neural network (we aren't see the higher order joint pdf structures with these figures

[glwagner]

For reference I think these are the names of the parameters:

julia> eki.inverse_problem[1].free_parameters.names
(:CᵂwΔ, :Cᵂu★, :Cʰⁱc, :Cʰⁱu, :Cʰⁱe, :CʰⁱD, :Cˢ, :Cˡᵒc, :Cˡᵒu, :Cˡᵒe, :CˡᵒD, :CRi⁰, :CRiᵟ, :Cᵘⁿc, :Cᵘⁿu, :Cᵘⁿe, :CᵘⁿD, :Cᶜc, :Cᶜu, :Cᶜe, :CᶜD, :Cᵉc, :Cˢᵖ)

or

1 CᵂwΔ
2 Cᵂu★
3 Cʰⁱc
4 Cʰⁱu
5 Cʰⁱe
6 CʰⁱD
7 Cˢ
8 Cˡᵒc
9 Cˡᵒu
10 Cˡᵒe
11 CˡᵒD
12 CRi⁰
13 CRiᵟ
14 Cᵘⁿc
15 Cᵘⁿu
16 Cᵘⁿe
17 CᵘⁿD
18 Cᶜc
19 Cᶜu
20 Cᶜe
21 CᶜD
22 Cᵉc
23 Cˢᵖ

[glwagner]

Couple thoughts: parameters 1 and 2 probably don't matter much. Parameter 10 (has to do with diffusivity of TKE for low Ri) seems sketch, since the prior is up against the upper bound. I'm surprised that parameters 8-9 are not more well constrained, I think they are crucial (along with 11) for shear mixing (together they determined the model's critical Ri). It's nice to see constraint on parameter 18, which controls tracer mixing during convection.

[sandreza]

The updated plot now looks like the following for convenience: