Skip to content

47 general minimal benchmark #60

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 16 commits into
base: main
Choose a base branch
from
Draft

47 general minimal benchmark #60

wants to merge 16 commits into from

Conversation

jbcaillau
Copy link
Member

@jbcaillau jbcaillau commented Oct 9, 2025 •
edited
Loading

@ocots src/mini.jl

Problem: space_shuttle

N      :   100
JuMP   :   1.487 s (7280 allocations: 163.16 KiB)
adnlp  :   1.107 s (5413732 allocations: 255.31 MiB)
exa    :   409.254 ms (3421 allocations: 243.07 KiB)

N      :   200
JuMP   :   4.529 s (10513 allocations: 235.48 KiB)
adnlp  :   4.150 s (18640172 allocations: 878.30 MiB)
exa    :   1.546 s (5727 allocations: 363.73 KiB)

N      :   500
JuMP   :   6.913 s (6512 allocations: 145.84 KiB)
adnlp  :   7.572 s (28696287 allocations: 1.32 GiB)
exa    :   1.823 s (3607 allocations: 479.29 KiB)

ocots and others added 6 commits September 20, 2025 15:34
@ocots
foo
459939b
@ocots
Merge 459939b into b722e68
2977985
@github-actions
📊 Add minimal benchmark results
a03a06b 
Generated by automated benchmark workflow

Results saved to docs/src/assets/benchmark-minimal/data.json

Ready for documentation generation
@jbcaillau
added mini.jl
be15439
@jbcaillau
added mini.jl
9ec66b9
@jbcaillau
added mini.jl
d08ac87
@jbcaillau jbcaillau self-assigned this Oct 9, 2025
@github-actions github-actions bot requested a review from ocots October 9, 2025 21:04
@jbcaillau
Copy link
Member Author

jbcaillau commented Oct 10, 2025 •
edited
Loading

@ocots

problem: beam, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   21.662 ms (875 allocations: 18.05 KiB)
adnlp  :   35.370 ms (240436 allocations: 9.99 MiB)
exa    :   18.499 ms (1109 allocations: 89.04 KiB)

problem: chain, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   17.966 ms (702 allocations: 15.69 KiB)
adnlp  :   26.595 ms (225337 allocations: 9.44 MiB)
exa    :   12.337 ms (687 allocations: 84.55 KiB)

problem: double_oscillator, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   11.093 ms (536 allocations: 11.97 KiB)
adnlp  :   26.057 ms (324921 allocations: 13.44 MiB)
exa    :   14.432 ms (615 allocations: 117.29 KiB)

problem: ducted_fan, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   2.452 s (3986 allocations: 89.00 KiB)
adnlp  :   616.885 ms (3447189 allocations: 148.50 MiB)
exa    :   438.306 ms (3366 allocations: 251.68 KiB)

problem: electric_vehicle, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   8.668 ms (436 allocations: 9.84 KiB)
adnlp  :   14.450 ms (107917 allocations: 4.68 MiB)
exa    :   6.513 ms (573 allocations: 72.80 KiB)

problem: glider, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   6.978 s (33394 allocations: 747.56 KiB)
adnlp  :   9.813 s (22935247 allocations: 1.07 GiB)
exa    :   2.039 s (12194 allocations: 521.68 KiB)

problem: insurance, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   654.405 ms (9473 allocations: 212.73 KiB)
adnlp  :   974.306 ms (5593414 allocations: 271.51 MiB)
exa    :   508.204 ms (4700 allocations: 279.59 KiB)

problem: glider, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   7.256 s (33394 allocations: 747.56 KiB)
adnlp  :   8.675 s (22935247 allocations: 1.07 GiB)
exa    :   2.003 s (12194 allocations: 521.68 KiB)

problem: jackson, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   77.280 ms (3492 allocations: 78.22 KiB)
adnlp  :   145.451 ms ([1253078](tel:1253078) allocations: 49.58 MiB)
exa    :   89.067 ms (1966 allocations: 136.12 KiB)

problem: robbins, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   47.967 ms (1588 allocations: 32.31 KiB)
adnlp  :   87.781 ms (653954 allocations: 27.13 MiB)
exa    :   44.361 ms (1724 allocations: 125.30 KiB)

problem: robot, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   168.593 ms (2030 allocations: 45.44 KiB)
adnlp  :   284.124 ms ([1955518](tel:1955518) allocations: 83.95 MiB)
exa    :   103.166 ms (1299 allocations: 200.48 KiB)

problem: rocket, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   79.080 ms (1864 allocations: 41.72 KiB)
adnlp  :   109.223 ms (923921 allocations: 40.75 MiB)
exa    :   36.209 ms (1223 allocations: 118.54 KiB)

problem: space_shuttle, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   4.372 s (10513 allocations: 235.48 KiB)
adnlp  :   4.450 s (18640172 allocations: 878.30 MiB)
exa    :   1.467 s (5727 allocations: 363.76 KiB)

problem: steering, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   46.401 ms (1300 allocations: 29.09 KiB)
adnlp  :   63.786 ms (503540 allocations: 21.44 MiB)
exa    :   36.543 ms (964 allocations: 123.40 KiB)

problem: vanderpol, solver: ipopt, disc_method: trapeze

N      :   200
JuMP   :   6.480 ms (356 allocations: 8.03 KiB)
adnlp  :   11.248 ms (89547 allocations: 3.67 MiB)
exa    :   5.392 ms (535 allocations: 67.93 KiB)

@jbcaillau
Copy link
Member Author

@ocots cannot run dielectrophoretic_particle (not known): issue?

@ocots
Copy link
Member

ocots commented Oct 10, 2025

@ocots cannot run dielectrophoretic_particle (not known): issue?

https://github.com/control-toolbox/OptimalControlProblems.jl/blob/1bcb6368609fb842b51b7092e23ca97897451743/src/OptimalControlProblems.jl#L82-L86

@jbcaillau
Copy link
Member Author

jbcaillau commented Oct 10, 2025 •
edited
Loading

@ocots run with MadNLP instead of Ipopt: systematically better / much better

problem: beam, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   15.443 ms (4705 allocations: 1.84 MiB)
adnlp  :   30.289 ms (282456 allocations: 13.13 MiB)
exa    :   15.569 ms (4312 allocations: 1.75 MiB)

problem: chain, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   17.884 ms (79784 allocations: 6.02 MiB)
adnlp  :   22.349 ms (230362 allocations: 10.64 MiB)
exa    :   10.298 ms (2060 allocations: 1.43 MiB)

problem: double_oscillator, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   10.130 ms (2083 allocations: 2.61 MiB)
adnlp  :   27.357 ms (331400 allocations: 15.15 MiB)
exa    :   10.602 ms (1821 allocations: 2.71 MiB)

problem: ducted_fan, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   1.856 s ([1668225](tel:1668225) allocations: 144.29 MiB)
adnlp  :   569.967 ms (3651381 allocations: 164.26 MiB)
exa    :   198.645 ms (9748 allocations: 9.51 MiB)

problem: electric_vehicle, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   9.113 ms (38962 allocations: 4.32 MiB)
adnlp  :   12.874 ms (111577 allocations: 5.54 MiB)
exa    :   4.570 ms (1594 allocations: 1.33 MiB)

problem: glider, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   4.159 s (436710 allocations: 70.23 MiB)
adnlp  :   7.922 s (14862425 allocations: 731.30 MiB)
exa    :   1.195 s (59766 allocations: 31.28 MiB)

problem: insurance, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   646.263 ms (193746 allocations: 35.93 MiB)
adnlp  :   1.073 s (7105133 allocations: 367.69 MiB)
exa    :   469.365 ms (27872 allocations: 26.32 MiB)

problem: glider, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   4.393 s (436710 allocations: 70.23 MiB)
adnlp  :   6.042 s (14862425 allocations: 731.30 MiB)
exa    :   1.203 s (59766 allocations: 31.28 MiB)

problem: jackson, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   60.110 ms (9004 allocations: 7.19 MiB)
adnlp  :   152.030 ms ([1259635](tel:1259635) allocations: 55.17 MiB)
exa    :   63.305 ms (8305 allocations: 6.74 MiB)

problem: robbins, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   33.772 ms (6701 allocations: 2.89 MiB)
adnlp  :   74.944 ms (658616 allocations: 29.46 MiB)
exa    :   34.329 ms (6152 allocations: 2.65 MiB)

problem: robot, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   182.498 ms (236154 allocations: 23.97 MiB)
adnlp  :   293.801 ms ([1967164](tel:1967164) allocations: 90.88 MiB)
exa    :   98.515 ms (5059 allocations: 7.92 MiB)

problem: rocket, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   69.936 ms (186334 allocations: 19.67 MiB)
adnlp  :   114.138 ms (931375 allocations: 44.21 MiB)
exa    :   29.695 ms (4628 allocations: 4.41 MiB)

problem: space_shuttle, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   3.350 s ([1489842](tel:1489842) allocations: 148.27 MiB)
adnlp  :   4.987 s (21634794 allocations: 1.03 GiB)
exa    :   1.007 s (34014 allocations: 44.72 MiB)

problem: steering, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   45.255 ms (107218 allocations: 9.06 MiB)
adnlp  :   65.952 ms (511288 allocations: 23.70 MiB)
exa    :   32.953 ms (3405 allocations: 2.57 MiB)

problem: vanderpol, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   7.904 ms (45977 allocations: 4.23 MiB)
adnlp  :   9.911 ms (93067 allocations: 4.50 MiB)
exa    :   3.777 ms (1417 allocations: 1.33 MiB)

@ocots ocots mentioned this pull request Oct 11, 2025
Open
@jbcaillau
Copy link
Member Author

close #47

@jbcaillau
Copy link
Member Author

@ocots minor point: stale dependencies issue with Aqua test. don't know what to do

@jbcaillau
Copy link
Member Author

@ocots partial results (ongoing re-run with N = 5000; issue with ducted_fan, very long - not converging? - with JuMP + MadNLP)

problem: beam, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   15.181 ms (4705 allocations: 1.84 MiB)
adnlp  :   31.505 ms (282456 allocations: 13.13 MiB)
exa    :   15.546 ms (4312 allocations: 1.75 MiB)
exa GPU:   0.029953 seconds (9.14 k CPU allocations: 2.020 MiB)

N      :   1000
JuMP   :   80.710 ms (5917 allocations: 9.47 MiB)
adnlp  :   173.699 ms ([1820713](tel:1820713) allocations: 83.93 MiB)
exa    :   82.892 ms (5429 allocations: 8.99 MiB)
exa GPU:   0.085811 seconds (5.46 k CPU allocations: 8.992 MiB)

N      :   2000
JuMP   :   264.032 ms (8079 allocations: 24.21 MiB)
adnlp  :   583.040 ms (5051742 allocations: 231.19 MiB)
exa    :   284.704 ms (7281 allocations: 22.32 MiB)
exa GPU:   0.314519 seconds (7.31 k CPU allocations: 22.321 MiB, 1.27% gc time)

problem: chain, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   17.688 ms (79784 allocations: 6.02 MiB)
adnlp  :   22.268 ms (230362 allocations: 10.64 MiB)
exa    :   10.366 ms (2060 allocations: 1.43 MiB)
exa GPU:   0.011260 seconds (2.09 k CPU allocations: 1.429 MiB)

N      :   1000
JuMP   :   77.084 ms (485452 allocations: 31.87 MiB)
adnlp  :   96.387 ms ([1011234](tel:1011234) allocations: 47.01 MiB)
exa    :   39.048 ms (1901 allocations: 6.42 MiB)
exa GPU:   0.040772 seconds (1.93 k CPU allocations: 6.422 MiB)

N      :   2000
JuMP   :   222.192 ms (985468 allocations: 61.84 MiB)
adnlp  :   261.965 ms (2019250 allocations: 93.86 MiB)
exa    :   136.463 ms (1906 allocations: 12.71 MiB)
exa GPU:   0.143513 seconds (1.93 k CPU allocations: 12.713 MiB)

problem: double_oscillator, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   10.239 ms (2083 allocations: 2.61 MiB)
adnlp  :   28.100 ms (331400 allocations: 15.15 MiB)
exa    :   10.773 ms (1821 allocations: 2.71 MiB)
exa GPU:   0.011577 seconds (1.85 k CPU allocations: 2.708 MiB)

N      :   1000
JuMP   :   49.954 ms (2366 allocations: 11.26 MiB)
adnlp  :   166.803 ms ([1876324](tel:1876324) allocations: 85.02 MiB)
exa    :   54.412 ms (2072 allocations: 13.26 MiB)
exa GPU:   0.059203 seconds (2.10 k CPU allocations: 13.264 MiB, 5.42% gc time)

N      :   2000
JuMP   :   163.114 ms (2402 allocations: 23.92 MiB)
adnlp  :   481.804 ms (3748344 allocations: 169.83 MiB)
exa    :   230.295 ms (2100 allocations: 26.38 MiB)
exa GPU:   0.234061 seconds (2.13 k CPU allocations: 26.376 MiB)

problem: electric_vehicle, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   9.342 ms (38962 allocations: 4.32 MiB)
adnlp  :   13.215 ms (111577 allocations: 5.54 MiB)
exa    :   4.559 ms (1594 allocations: 1.33 MiB)
exa GPU:   0.005453 seconds (1.62 k CPU allocations: 1.332 MiB)

N      :   1000
JuMP   :   42.967 ms (250205 allocations: 21.89 MiB)
adnlp  :   62.488 ms (548380 allocations: 27.18 MiB)
exa    :   19.490 ms (1598 allocations: 6.15 MiB)
exa GPU:   0.020522 seconds (1.62 k CPU allocations: 6.150 MiB)

N      :   2000
JuMP   :   113.625 ms (512219 allocations: 42.13 MiB)
adnlp  :   178.991 ms ([1094380](tel:1094380) allocations: 54.24 MiB)
exa    :   68.038 ms (1610 allocations: 12.17 MiB)
exa GPU:   0.069446 seconds (1.64 k CPU allocations: 12.175 MiB)

problem: glider, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   4.139 s (436710 allocations: 70.23 MiB)
adnlp  :   7.289 s (14862425 allocations: 731.30 MiB)
exa    :   1.194 s (59766 allocations: 31.28 MiB)
exa GPU:   1.219796 seconds (59.79 k CPU allocations: 31.272 MiB)

N      :   1000
JuMP   :   63.016 s (2148684 allocations: 513.59 MiB)
adnlp  :   59.892 s (116243461 allocations: 5.59 GiB)
exa    :   12.693 s (142919 allocations: 306.54 MiB)
exa GPU:  12.919529 seconds (142.95 k CPU allocations: 306.526 MiB, 0.38% gc time)

N      :   2000
JuMP   :   44.077 s (4058550 allocations: 635.90 MiB)
adnlp  :   141.331 s (190317527 allocations: 9.15 GiB)
exa    :   25.719 s (128569 allocations: 559.83 MiB)
exa GPU:  25.858778 seconds (128.60 k CPU allocations: 559.793 MiB, 0.31% gc time)

problem: insurance, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   662.206 ms (193746 allocations: 35.93 MiB)
adnlp  :   1.067 s (7105133 allocations: 367.69 MiB)
exa    :   446.394 ms (27872 allocations: 26.32 MiB)
exa GPU:   0.446624 seconds (27.90 k CPU allocations: 26.353 MiB)

N      :   1000
JuMP   :   15.066 s (945341 allocations: 468.70 MiB)
adnlp  :   31.611 s (162773576 allocations: 8.26 GiB)
exa    :   16.299 s (124654 allocations: 550.03 MiB)
exa GPU:  16.492541 seconds (124.68 k CPU allocations: 550.028 MiB, 0.47% gc time)

N      :   2000
JuMP   :   13.544 s ([1737674](tel:1737674) allocations: 415.71 MiB)
adnlp  :   22.478 s (92212508 allocations: 4.68 GiB)
exa    :   9.586 s (36339 allocations: 321.34 MiB)
exa GPU:   9.643864 seconds (36.37 k CPU allocations: 321.334 MiB, 0.24% gc time)

problem: glider, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   4.208 s (436710 allocations: 70.23 MiB)
adnlp  :   5.810 s (14862425 allocations: 731.30 MiB)
exa    :   1.224 s (59766 allocations: 31.28 MiB)
exa GPU:   1.227025 seconds (59.79 k CPU allocations: 31.268 MiB)

N      :   1000
JuMP   :   64.168 s (2148684 allocations: 513.58 MiB)
adnlp  :   68.145 s (116243461 allocations: 5.59 GiB)
exa    :   12.857 s (142919 allocations: 306.54 MiB)
exa GPU:  12.944845 seconds (142.95 k CPU allocations: 306.523 MiB, 0.27% gc time)

N      :   2000
JuMP   :   45.163 s (4058550 allocations: 635.90 MiB)
adnlp  :   138.991 s (190317527 allocations: 9.15 GiB)
exa    :   25.846 s (128569 allocations: 559.84 MiB)
exa GPU:  25.872030 seconds (128.60 k CPU allocations: 559.806 MiB, 0.30% gc time)

problem: jackson, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   65.238 ms (9004 allocations: 7.19 MiB)
adnlp  :   147.029 ms ([1259635](tel:1259635) allocations: 55.17 MiB)
exa    :   63.626 ms (8305 allocations: 6.74 MiB)
exa GPU:   0.066657 seconds (8.33 k CPU allocations: 6.754 MiB)

N      :   1000
JuMP   :   123.425 ms (5426 allocations: 26.43 MiB)
adnlp  :   350.558 ms (3684709 allocations: 163.11 MiB)
exa    :   126.184 ms (4916 allocations: 22.83 MiB)
exa GPU:   0.128923 seconds (4.94 k CPU allocations: 22.828 MiB)

N      :   2000
JuMP   :   301.187 ms (5420 allocations: 45.98 MiB)
adnlp  :   806.718 ms (7359698 allocations: 325.79 MiB)
exa    :   345.484 ms (4913 allocations: 45.39 MiB)
exa GPU:   0.355381 seconds (4.94 k CPU allocations: 45.388 MiB)

problem: robbins, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   34.227 ms (6701 allocations: 2.89 MiB)
adnlp  :   77.381 ms (658616 allocations: 29.46 MiB)
exa    :   34.595 ms (6152 allocations: 2.65 MiB)
exa GPU:   0.035288 seconds (6.18 k CPU allocations: 2.647 MiB)

N      :   1000
JuMP   :   220.135 ms (9659 allocations: 16.79 MiB)
adnlp  :   508.116 ms (4695372 allocations: 208.60 MiB)
exa    :   224.277 ms (8897 allocations: 15.45 MiB)
exa GPU:   0.237455 seconds (8.92 k CPU allocations: 15.451 MiB)

N      :   2000
JuMP   :   477.625 ms (8528 allocations: 28.43 MiB)
adnlp  :   1.053 s (8293688 allocations: 369.22 MiB)
exa    :   543.265 ms (7790 allocations: 27.55 MiB)
exa GPU:   0.546389 seconds (7.82 k CPU allocations: 27.556 MiB)

problem: robot, solver: madnlp, disc_method: trapeze

N      :   200
JuMP   :   189.048 ms (236154 allocations: 23.97 MiB)
adnlp  :   296.670 ms ([1967164](tel:1967164) allocations: 90.88 MiB)
exa    :   105.214 ms (5059 allocations: 7.92 MiB)
exa GPU:   0.109417 seconds (5.09 k CPU allocations: 7.920 MiB)

N      :   1000
JuMP   :   1.322 s ([1206320](tel:1206320) allocations: 125.14 MiB)
adnlp  :   2.142 s (11798722 allocations: 543.02 MiB)
exa    :   814.708 ms (6019 allocations: 43.15 MiB)
exa GPU:   0.812553 seconds (6.05 k CPU allocations: 43.149 MiB)

N      :   2000
JuMP   :   4.336 s (2429655 allocations: 268.15 MiB)
adnlp  :   7.534 s (42003159 allocations: 1.89 GiB)
exa    :   3.968 s (15515 allocations: 189.36 MiB)
exa GPU:   4.014517 seconds (15.54 k CPU allocations: 189.360 MiB, 0.37% gc time)

@ocots
Copy link
Member

ocots commented Oct 11, 2025

We can add limit time.

We will discuss how to handle GPU vs CPU.

I am working in parallel on the code.

@jbcaillau
Copy link
Member Author

@ocots

**** problem: beam (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   29.716 ms (4459 allocations: 3.90 MiB)
adnlp  :   61.008 ms (605950 allocations: 28.31 MiB)
exa    :   29.862 ms (4082 allocations: 3.74 MiB)
exa GPU:   0.045124 seconds (8.91 k CPU allocations: 4.006 MiB)

N      :   1000
JuMP   :   79.701 ms (5917 allocations: 9.47 MiB)
adnlp  :   170.049 ms (1820713 allocations: 83.93 MiB)
exa    :   81.674 ms (5429 allocations: 8.99 MiB)
exa GPU:   0.083564 seconds (5.46 k CPU allocations: 8.993 MiB)

N      :   5000
JuMP   :   1.298 s (15626 allocations: 103.63 MiB)
adnlp  :   3.159 s (24622281 allocations: 1.09 GiB)
exa    :   1.573 s (14333 allocations: 98.83 MiB)
exa GPU:   1.623616 seconds (14.36 k CPU allocations: 98.833 MiB, 0.45% gc time)

N      :   10000
JuMP   :   4.835 s (24264 allocations: 307.59 MiB)
adnlp  :   10.490 s (76834399 allocations: 3.39 GiB)
exa    :   5.601 s (24257 allocations: 318.97 MiB)
exa GPU:   5.870847 seconds (24.28 k CPU allocations: 318.973 MiB, 0.42% gc time)

**** problem: chain (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   80.295 ms (221999 allocations: 15.74 MiB)
adnlp  :   100.330 ms (1068174 allocations: 47.63 MiB)
exa    :   46.007 ms (3293 allocations: 3.98 MiB)
exa GPU:   0.048620 seconds (3.32 k CPU allocations: 3.978 MiB)

N      :   1000
JuMP   :   79.321 ms (485452 allocations: 31.87 MiB)
adnlp  :   90.201 ms (1011234 allocations: 47.01 MiB)
exa    :   40.285 ms (1901 allocations: 6.42 MiB)
exa GPU:   0.041169 seconds (1.93 k CPU allocations: 6.422 MiB)

N      :   5000
JuMP   :   668.512 ms (2485509 allocations: 158.23 MiB)
adnlp  :   750.646 ms (5043265 allocations: 234.40 MiB)
exa    :   395.250 ms (1917 allocations: 31.58 MiB)
exa GPU:   0.424170 seconds (1.94 k CPU allocations: 31.584 MiB)

N      :   10000
JuMP   :   1.473 s (4985554 allocations: 313.20 MiB)
adnlp  :   1.622 s (10083271 allocations: 468.63 MiB)
exa    :   881.889 ms (1934 allocations: 63.03 MiB)
exa GPU:   0.892483 seconds (1.96 k CPU allocations: 63.034 MiB)

**** problem: double_oscillator (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   25.779 ms (2393 allocations: 6.63 MiB)
adnlp  :   74.709 ms (940371 allocations: 42.62 MiB)
exa    :   28.219 ms (2107 allocations: 6.72 MiB)
exa GPU:   0.029416 seconds (2.13 k CPU allocations: 6.716 MiB)

N      :   1000
JuMP   :   50.344 ms (2366 allocations: 11.26 MiB)
adnlp  :   158.591 ms (1876324 allocations: 85.02 MiB)
exa    :   54.080 ms (2072 allocations: 13.26 MiB)
exa GPU:   0.054705 seconds (2.10 k CPU allocations: 13.264 MiB)

N      :   5000
JuMP   :   456.464 ms (2435 allocations: 55.28 MiB)
adnlp  :   1.255 s (9364357 allocations: 424.27 MiB)
exa    :   671.522 ms (2108 allocations: 65.70 MiB)
exa GPU:   0.713668 seconds (2.13 k CPU allocations: 65.702 MiB, 0.83% gc time)

N      :   10000
JuMP   :   1.039 s (2448 allocations: 113.70 MiB)
adnlp  :   2.595 s (18724354 allocations: 848.32 MiB)
exa    :   1.402 s (2105 allocations: 131.25 MiB)
exa GPU:   1.442901 seconds (2.13 k CPU allocations: 131.247 MiB)

**** problem: electric_vehicle (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   21.834 ms (110261 allocations: 10.39 MiB)
adnlp  :   30.044 ms (275384 allocations: 13.65 MiB)
exa    :   10.445 ms (1601 allocations: 3.14 MiB)
exa GPU:   0.011741 seconds (1.63 k CPU allocations: 3.139 MiB)

N      :   1000
JuMP   :   43.083 ms (250205 allocations: 21.89 MiB)
adnlp  :   63.697 ms (548380 allocations: 27.18 MiB)
exa    :   18.985 ms (1598 allocations: 6.15 MiB)
exa GPU:   0.020157 seconds (1.62 k CPU allocations: 6.150 MiB)

N      :   5000
JuMP   :   309.583 ms (1298256 allocations: 107.89 MiB)
adnlp  :   524.232 ms (2732381 allocations: 135.40 MiB)
exa    :   173.160 ms (1610 allocations: 30.24 MiB)
exa GPU:   0.177411 seconds (1.64 k CPU allocations: 30.243 MiB)

N      :   10000
JuMP   :   714.361 ms (2608299 allocations: 209.91 MiB)
adnlp  :   1.083 s (5462397 allocations: 270.67 MiB)
exa    :   391.526 ms (1622 allocations: 60.36 MiB)
exa GPU:   0.394228 seconds (1.65 k CPU allocations: 60.356 MiB)

**** problem: glider (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   28.989 s (1135462 allocations: 255.57 MiB)
adnlp  :   51.572 s (85753157 allocations: 4.12 GiB)
exa    :   6.318 s (134018 allocations: 148.33 MiB)
exa GPU:   6.235604 seconds (134.04 k CPU allocations: 148.341 MiB)

N      :   1000
JuMP   :   64.235 s (2148684 allocations: 513.59 MiB)
adnlp  :   57.108 s (116243461 allocations: 5.59 GiB)
exa    :   12.751 s (142919 allocations: 306.54 MiB)
exa GPU:  12.823894 seconds (142.95 k CPU allocations: 306.531 MiB, 0.34% gc time)

N      :   5000
JuMP   :   200.923 s (10136662 allocations: 1.87 GiB)
adnlp  :   991.130 s (1361560303 allocations: 65.40 GiB)
exa    :   63.552 s (101028 allocations: 1.08 GiB)
exa GPU:  65.130692 seconds (101.06 k CPU allocations: 1.080 GiB, 0.17% gc time)

N      :   10000
JuMP   :   343.922 s (20194963 allocations: 3.55 GiB)
adnlp  :   771.153 s (1108100909 allocations: 53.20 GiB)
exa    :   114.562 s (87478 allocations: 1.87 GiB)
exa GPU: 116.079787 seconds (87.50 k CPU allocations: 1.870 GiB, 0.16% gc time)

**** problem: insurance (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   3.998 s (480664 allocations: 152.21 MiB)
adnlp  :   9.393 s (47159648 allocations: 2.39 GiB)
exa    :   3.071 s (53928 allocations: 118.29 MiB)
exa GPU:   3.049749 seconds (53.95 k CPU allocations: 118.291 MiB)

N      :   1000
JuMP   :   14.907 s (945341 allocations: 468.70 MiB)
adnlp  :   30.045 s (162773576 allocations: 8.26 GiB)
exa    :   16.380 s (124654 allocations: 550.03 MiB)
exa GPU:  16.509949 seconds (124.68 k CPU allocations: 550.026 MiB, 0.49% gc time)

N      :   5000
JuMP   :   151.911 s (4419502 allocations: 3.47 GiB)
adnlp  :   224.620 s (939907589 allocations: 47.68 GiB)
exa    :   112.619 s (144270 allocations: 3.07 GiB)
exa GPU: 112.791990 seconds (144.30 k CPU allocations: 3.074 GiB, 0.33% gc time)

N      :   10000
JuMP   :   299.365 s (8670142 allocations: 6.25 GiB)
adnlp  :   504.940 s (1846405288 allocations: 93.67 GiB)
exa    :   248.339 s (142863 allocations: 5.95 GiB)
exa GPU: 247.832171 seconds (142.89 k CPU allocations: 5.955 GiB, 0.26% gc time)

**** problem: glider (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   29.400 s (1135462 allocations: 255.56 MiB)
adnlp  :   42.939 s (85753157 allocations: 4.12 GiB)
exa    :   6.501 s (134018 allocations: 148.33 MiB)
exa GPU:   6.365962 seconds (134.04 k CPU allocations: 148.311 MiB)

N      :   1000
JuMP   :   64.698 s (2148684 allocations: 513.58 MiB)
adnlp  :   70.725 s (116243461 allocations: 5.59 GiB)
exa    :   12.933 s (142919 allocations: 306.54 MiB)
exa GPU:  13.057218 seconds (142.95 k CPU allocations: 306.522 MiB, 0.29% gc time)

N      :   5000
JuMP   :   199.034 s (10136662 allocations: 1.87 GiB)
adnlp  :   1012.734 s (1361560303 allocations: 65.40 GiB)
exa    :   64.958 s (101028 allocations: 1.08 GiB)
exa GPU:  64.954681 seconds (101.06 k CPU allocations: 1.080 GiB, 0.22% gc time)

N      :   10000
JuMP   :   344.156 s (20194963 allocations: 3.55 GiB)
adnlp  :   770.901 s (1108100909 allocations: 53.20 GiB)
exa    :   113.218 s (87478 allocations: 1.87 GiB)
exa GPU: 116.966066 seconds (87.50 k CPU allocations: 1.870 GiB, 0.19% gc time)

**** problem: jackson (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   63.464 ms (5236 allocations: 12.38 MiB)
adnlp  :   157.930 ms (1773348 allocations: 78.58 MiB)
exa    :   62.982 ms (4748 allocations: 11.28 MiB)
exa GPU:   0.067803 seconds (4.78 k CPU allocations: 11.281 MiB)

N      :   1000
JuMP   :   123.821 ms (5426 allocations: 26.43 MiB)
adnlp  :   347.001 ms (3684709 allocations: 163.11 MiB)
exa    :   126.812 ms (4916 allocations: 22.83 MiB)
exa GPU:   0.130331 seconds (4.94 k CPU allocations: 22.827 MiB)

N      :   5000
JuMP   :   872.674 ms (5493 allocations: 126.88 MiB)
adnlp  :   2.320 s (18384743 allocations: 814.15 MiB)
exa    :   979.152 ms (4970 allocations: 113.38 MiB)
exa GPU:   0.984086 seconds (5.00 k CPU allocations: 113.378 MiB)

N      :   10000
JuMP   :   2.012 s (5691 allocations: 236.51 MiB)
adnlp  :   5.726 s (38230107 allocations: 1.65 GiB)
exa    :   2.291 s (5136 allocations: 231.68 MiB)
exa GPU:   2.326110 seconds (5.16 k CPU allocations: 231.679 MiB, 0.61% gc time)

**** problem: robbins (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   66.455 ms (6108 allocations: 6.05 MiB)
adnlp  :   160.028 ms (1513169 allocations: 67.77 MiB)
exa    :   74.034 ms (5635 allocations: 5.67 MiB)
exa GPU:   0.073922 seconds (5.66 k CPU allocations: 5.670 MiB)

N      :   1000
JuMP   :   228.458 ms (9659 allocations: 16.79 MiB)
adnlp  :   535.823 ms (4695372 allocations: 208.60 MiB)
exa    :   220.755 ms (8897 allocations: 15.45 MiB)
exa GPU:   0.224292 seconds (8.92 k CPU allocations: 15.453 MiB)

N      :   5000
JuMP   :   2.263 s (14520 allocations: 105.69 MiB)
adnlp  :   5.548 s (35573283 allocations: 1.54 GiB)
exa    :   2.372 s (13162 allocations: 103.60 MiB)
exa GPU:   2.702661 seconds (13.19 k CPU allocations: 103.602 MiB)

N      :   10000
JuMP   :   5.103 s (13392 allocations: 198.46 MiB)
adnlp  :   10.260 s (63740986 allocations: 2.75 GiB)
exa    :   5.099 s (11817 allocations: 188.78 MiB)
exa GPU:   5.090411 seconds (11.84 k CPU allocations: 188.780 MiB)

**** problem: robot (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   636.970 ms (596947 allocations: 63.45 MiB)
adnlp  :   985.400 ms (6281477 allocations: 288.88 MiB)
exa    :   318.322 ms (6270 allocations: 22.42 MiB)
exa GPU:   0.320931 seconds (6.30 k CPU allocations: 22.413 MiB)

N      :   1000
JuMP   :   1.280 s (1206320 allocations: 125.14 MiB)
adnlp  :   1.956 s (11798722 allocations: 543.02 MiB)
exa    :   790.971 ms (6019 allocations: 43.15 MiB)
exa GPU:   0.880067 seconds (6.05 k CPU allocations: 43.149 MiB)

N      :   5000
JuMP   :   15.357 s (6095663 allocations: 738.12 MiB)
adnlp  :   17.424 s (83100353 allocations: 3.72 GiB)
exa    :   9.541 s (10778 allocations: 324.38 MiB)
exa GPU:   9.571541 seconds (10.80 k CPU allocations: 324.380 MiB, 0.17% gc time)

N      :   10000
JuMP   :   32.038 s (12200501 allocations: 1.43 GiB)
adnlp  :   63.063 s (264077301 allocations: 11.78 GiB)
exa    :   24.123 s (14003 allocations: 797.07 MiB)
exa GPU:  24.157101 seconds (14.03 k CPU allocations: 797.072 MiB, 0.41% gc time)

**** problem: rocket (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   197.128 ms (485830 allocations: 49.65 MiB)
adnlp  :   316.444 ms (2411890 allocations: 114.42 MiB)
exa    :   74.126 ms (4801 allocations: 10.86 MiB)
exa GPU:   0.082028 seconds (4.83 k CPU allocations: 10.862 MiB)

N      :   1000
JuMP   :   478.877 ms (984442 allocations: 98.89 MiB)
adnlp  :   753.924 ms (5212863 allocations: 246.98 MiB)
exa    :   161.871 ms (5178 allocations: 22.40 MiB)
exa GPU:   0.167000 seconds (5.21 k CPU allocations: 22.402 MiB)

N      :   5000
JuMP   :   3.184 s (4972565 allocations: 491.19 MiB)
adnlp  :   4.942 s (26012878 allocations: 1.20 GiB)
exa    :   1.119 s (5244 allocations: 110.88 MiB)
exa GPU:   1.146107 seconds (5.27 k CPU allocations: 110.881 MiB, 1.78% gc time)

N      :   10000
JuMP   :   6.616 s (9957188 allocations: 980.31 MiB)
adnlp  :   10.963 s (48011903 allocations: 2.23 GiB)
exa    :   2.361 s (4857 allocations: 212.15 MiB)
exa GPU:   2.361021 seconds (4.88 k CPU allocations: 212.149 MiB)

**** problem: space_shuttle (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   6.577 s (3739304 allocations: 343.08 MiB)
adnlp  :   15.393 s (50784531 allocations: 2.41 GiB)
exa    :   2.170 s (29934 allocations: 100.04 MiB)
exa GPU:   2.146658 seconds (29.96 k CPU allocations: 100.026 MiB)

N      :   1000
JuMP   :   118.468 s (7659765 allocations: 1.29 GiB)
adnlp  :   38.052 s (139574193 allocations: 6.61 GiB)
exa    :   13.911 s (98870 allocations: 518.21 MiB)
exa GPU:  14.008857 seconds (98.90 k CPU allocations: 518.205 MiB, 0.44% gc time)

N      :   5000
JuMP   :   215.428 s (37640344 allocations: 4.10 GiB)
adnlp  :   282.800 s (692164752 allocations: 32.92 GiB)
exa    :   35.746 s (35958 allocations: 1.15 GiB)
exa GPU:  35.731898 seconds (35.98 k CPU allocations: 1.148 GiB, 0.36% gc time)

N      :   10000
JuMP   :   382.287 s (75264340 allocations: 7.84 GiB)
adnlp  :   623.083 s (1694966392 allocations: 80.30 GiB)
exa    :   76.398 s (45147 allocations: 2.82 GiB)
exa GPU:  85.230000 seconds (45.17 k CPU allocations: 2.816 GiB, 6.31% gc time)

**** problem: steering (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   122.466 ms (291118 allocations: 22.94 MiB)
adnlp  :   164.541 ms (1346513 allocations: 62.24 MiB)
exa    :   81.567 ms (3769 allocations: 6.24 MiB)
exa GPU:   0.082140 seconds (3.80 k CPU allocations: 6.241 MiB)

N      :   1000
JuMP   :   267.627 ms (593023 allocations: 44.47 MiB)
adnlp  :   353.445 ms (2685530 allocations: 124.14 MiB)
exa    :   174.861 ms (3787 allocations: 12.26 MiB)
exa GPU:   0.182660 seconds (3.81 k CPU allocations: 12.265 MiB)

N      :   5000
JuMP   :   2.160 s (3009524 allocations: 222.26 MiB)
adnlp  :   2.489 s (14958392 allocations: 688.73 MiB)
exa    :   1.350 s (4204 allocations: 63.33 MiB)
exa GPU:   1.343241 seconds (4.23 k CPU allocations: 63.329 MiB)

N      :   10000
JuMP   :   5.258 s (6029909 allocations: 447.59 MiB)
adnlp  :   6.524 s (33029177 allocations: 1.48 GiB)
exa    :   3.162 s (4537 allocations: 131.92 MiB)
exa GPU:   3.460843 seconds (4.56 k CPU allocations: 131.924 MiB)

**** problem: vanderpol (solver: madnlp, disc_method: trapeze)

N      :   500
JuMP   :   17.435 ms (128032 allocations: 10.20 MiB)
adnlp  :   22.801 ms (229566 allocations: 11.07 MiB)
exa    :   8.266 ms (1420 allocations: 3.15 MiB)
exa GPU:   0.009786 seconds (1.45 k CPU allocations: 3.151 MiB)

N      :   1000
JuMP   :   36.988 ms (286055 allocations: 21.62 MiB)
adnlp  :   45.884 ms (457099 allocations: 22.03 MiB)
exa    :   15.755 ms (1454 allocations: 6.19 MiB)
exa GPU:   0.016547 seconds (1.48 k CPU allocations: 6.191 MiB)

N      :   5000
JuMP   :   257.774 ms (1478110 allocations: 106.78 MiB)
adnlp  :   336.348 ms (2277104 allocations: 109.71 MiB)
exa    :   155.279 ms (1466 allocations: 30.50 MiB)
exa GPU:   0.159592 seconds (1.49 k CPU allocations: 30.499 MiB)

N      :   10000
JuMP   :   543.075 ms (2968153 allocations: 211.99 MiB)
adnlp  :   712.355 ms (4552120 allocations: 219.31 MiB)
exa    :   331.810 ms (1478 allocations: 60.88 MiB)
exa GPU:   0.338586 seconds (1.50 k CPU allocations: 60.880 MiB)

@jbcaillau
Copy link
Member Author

@ocots tol = 1e-6, CPU vs. GPU only

**** problem: beam (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   892.921 ms (4095 allocations: 70.95 MiB)
exa GPU:   0.860197 seconds (8.93 k CPU allocations: 71.213 MiB, 2.32% gc time)

N      :   50000
exa    :   4.611 s (3749 allocations: 333.26 MiB)
exa GPU:   5.193481 seconds (3.78 k CPU allocations: 333.261 MiB, 11.79% gc time)

**** problem: chain (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   751.801 ms (1774 allocations: 61.50 MiB)
exa GPU:   0.764188 seconds (1.80 k CPU allocations: 61.500 MiB, 1.33% gc time)

N      :   50000
exa    :   7.694 s (3026 allocations: 363.50 MiB)
exa GPU:   8.185146 seconds (3.05 k CPU allocations: 363.497 MiB, 6.78% gc time)

**** problem: double_oscillator (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   1.257 s (1925 allocations: 128.42 MiB)
exa GPU:   1.268943 seconds (1.95 k CPU allocations: 128.417 MiB, 0.61% gc time)

N      :   50000
exa    :   7.035 s (1929 allocations: 641.49 MiB)
exa GPU:   7.616793 seconds (1.96 k CPU allocations: 641.484 MiB, 8.59% gc time)

**** problem: electric_vehicle (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   333.627 ms (1433 allocations: 59.20 MiB)
exa GPU:   0.348489 seconds (1.46 k CPU allocations: 59.204 MiB, 3.76% gc time)

N      :   50000
exa    :   1.861 s (1437 allocations: 295.53 MiB)
exa GPU:   1.876430 seconds (1.46 k CPU allocations: 295.535 MiB, 0.28% gc time)

**** problem: glider (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   131.173 s (95253 allocations: 2.02 GiB)
exa GPU: 131.774736 seconds (95.28 k CPU allocations: 2.020 GiB, 0.73% gc time)

N      :   50000
exa    :   1154.282 s (147250 allocations: 15.28 GiB)
exa GPU: 1172.009957 seconds (147.28 k CPU allocations: 15.284 GiB, 0.22% gc time)

**** problem: insurance (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   170.319 s (112191 allocations: 4.81 GiB)
exa GPU: 168.877553 seconds (112.22 k CPU allocations: 4.812 GiB, 0.31% gc time)

N      :   50000
exa    :   795.133 s (117805 allocations: 26.57 GiB)
exa GPU: 796.589644 seconds (117.83 k CPU allocations: 26.573 GiB, 0.45% gc time)

**** problem: glider (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   118.283 s (95253 allocations: 2.02 GiB)
exa GPU: 120.245296 seconds (95.28 k CPU allocations: 2.020 GiB, 0.81% gc time)

N      :   50000
exa    :   1154.607 s (147250 allocations: 15.28 GiB)
exa GPU: 1163.008571 seconds (147.28 k CPU allocations: 15.284 GiB, 0.22% gc time)

**** problem: jackson (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   1.641 s (4082 allocations: 199.90 MiB)
exa GPU:   2.557133 seconds (4.11 k CPU allocations: 199.897 MiB, 30.86% gc time)

N      :   50000
exa    :   9.335 s (3917 allocations: 972.61 MiB)
exa GPU:   9.888734 seconds (3.94 k CPU allocations: 972.609 MiB, 8.18% gc time)

**** problem: robbins (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   3.327 s (7807 allocations: 136.28 MiB)
exa GPU:   3.331390 seconds (7.83 k CPU allocations: 136.280 MiB)

N      :   50000
exa    :   67.190 s (28510 allocations: 1.98 GiB)
exa GPU:  66.043324 seconds (28.54 k CPU allocations: 1.982 GiB, 0.29% gc time)

**** problem: robot (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   21.950 s (13049 allocations: 752.40 MiB)
exa GPU:  21.780091 seconds (13.08 k CPU allocations: 752.393 MiB, 0.33% gc time)

N      :   50000
exa    :   106.070 s (10668 allocations: 3.09 GiB)
exa GPU: 105.773356 seconds (10.70 k CPU allocations: 3.089 GiB, 0.95% gc time)

**** problem: rocket (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   1.688 s (3999 allocations: 190.44 MiB)
exa GPU:   1.726517 seconds (4.03 k CPU allocations: 190.442 MiB, 1.54% gc time)

N      :   50000
exa    :   8.693 s (3660 allocations: 907.01 MiB)
exa GPU:  10.295476 seconds (3.69 k CPU allocations: 907.010 MiB, 15.18% gc time)

**** problem: space_shuttle (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   97.437 s (48112 allocations: 2.92 GiB)
exa GPU:  98.027192 seconds (48.14 k CPU allocations: 2.923 GiB, 1.13% gc time)

N      :   50000
exa    :   394.553 s (31333 allocations: 10.46 GiB)
exa GPU: 396.039100 seconds (31.36 k CPU allocations: 10.464 GiB, 0.78% gc time)

**** problem: steering (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   2.913 s (4369 allocations: 129.25 MiB)
exa GPU:   3.228157 seconds (4.40 k CPU allocations: 129.245 MiB)

N      :   50000
exa    :   124.838 s (28207 allocations: 2.56 GiB)
exa GPU: 124.950677 seconds (28.23 k CPU allocations: 2.563 GiB, 0.89% gc time)

**** problem: vanderpol (solver: madnlp, disc_method: trapeze)

N      :   10000
exa    :   326.933 ms (1478 allocations: 60.88 MiB)
exa GPU:   0.350939 seconds (1.50 k CPU allocations: 60.880 MiB, 4.74% gc time)

N      :   50000
exa    :   1.651 s (1297 allocations: 298.20 MiB)
exa GPU:   2.436675 seconds (1.32 k CPU allocations: 298.204 MiB, 32.11% gc time)

@jbcaillau
Copy link
Member Author

@horasio regarding Ipopt vs. MadNLP, check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ocots ocots Awaiting requested review from ocots

At least 1 approving review is required to merge this pull request.

Assignees

@jbcaillau jbcaillau

@ocots ocots

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jbcaillau @ocots