Add optimization notebook #358

cluhmann · 2023-08-17T23:48:37Z

Initial version of the optimization notebook. I have not yet added all the "bells and whistles" that are possible, because I want to get feedback on the core first. Then we can make it pretty.

One thing that I think can immediately be improved is to replace much of the manual contribution/ROAS calculation code with things like mmm.channel_contributions_forward_pass() etc. These helper functions didn't exist when the notebook was first created.

Also, I had planned on omitting all the MMM example notebook code, but realized I need to serialize the model, the inferenceData object, and the input data (and possibly more?). So I just kept the code but stripped it down to the essentials. Keeping all these cells also prevents "syncing" issues between this notebook and the MMM example notebook.

Todos:

Replace objective function code with pymc-marketing functionality as possible
Make objective function a compiled pytensor function to avoid eval()s
Add additional exploration of optimization results (e.g., risk attitudes, diversification penalty, etc.)

📚 Documentation preview 📚: https://pymc-marketing--358.org.readthedocs.build/en/358/

review-notebook-app · 2023-08-17T23:48:42Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

codecov · 2023-08-18T00:24:32Z

Codecov Report

Attention: Patch coverage is 92.61391% with 154 lines in your changes missing coverage. Please review.

Project coverage is 94.58%. Comparing base (73765d4) to head (3b5c92a).
Report is 542 commits behind head on main.

Files with missing lines	Patch %	Lines
pymc_marketing/mmm/delayed_saturated_mmm.py	73.92%	115 Missing ⚠️
pymc_marketing/clv/models/pareto_nbd.py	92.61%	11 Missing ⚠️
pymc_marketing/mmm/plot.py	93.47%	6 Missing ⚠️
pymc_marketing/mmm/budget_optimizer.py	92.15%	4 Missing ⚠️
pymc_marketing/mmm/components/base.py	97.08%	4 Missing ⚠️
pymc_marketing/model_config.py	92.30%	3 Missing ⚠️
pymc_marketing/clv/utils.py	98.26%	2 Missing ⚠️
pymc_marketing/mmm/base.py	98.33%	2 Missing ⚠️
pymc_marketing/mmm/fourier.py	97.72%	2 Missing ⚠️
pymc_marketing/mmm/lift_test.py	97.87%	2 Missing ⚠️
... and 3 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #358      +/-   ##
==========================================
+ Coverage   93.49%   94.58%   +1.08%     
==========================================
  Files          20       33      +13     
  Lines        1646     3378    +1732     
==========================================
+ Hits         1539     3195    +1656     
- Misses        107      183      +76

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

* add scaling into lift test method * test scaling methods * test change of MMM likelihood * link up beta_channel parameters * change name to sigma * reorganize and edit * closes #406 * address the feedback in docstrings * add more to docstring * format the docstring * be verbose for future devs * be explicit for the column max values * incorporate the feedback * hide cells and add to intro * add conclusion * switch to header 2 * run notebook through * move the types together * separate model estimated from empirical * fix typos

* drop python 3.9 * try python 3.12 * undo try python 3.12

* improve nb * rm warnings and add link to lifetimes quickstart * address comments * feedback part 3 * remove warnings manually

* add more info to the notebook * hide plots code * fix plot y labels * fix plot outputs and remove model build * improve final note probability plots * address comments * use quickstart dataset * feedback part 3 * remowe warnings manually * feedback part 4

* improve mmm docs init * add more code examples to docstrings * minor improvemeents * typo * better phrasing * add thomas suggestion

* move fixtures to conftest * docstrings and moved set_model_fit to conftest * fixed pandas quickstart warnings * revert to MockModel and add ParetoNBD support * quickstart edit for issue 609 * notebook edit

remove ruff E501 ignore

updates: - [github.com/astral-sh/ruff-pre-commit: v0.3.5 → v0.3.7](astral-sh/ruff-pre-commit@v0.3.5...v0.3.7) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Juan Orduz <[email protected]>

* fix potential bugs * minor improvements * remove rule for nb * fix test * improve tests syntax * use stacklevel 2 for warnings * use stacklevel 1 for warnings as they are used in public methods * ignore B904 * undo change * ricardos feedback * use fit_posterior * Update pymc_marketing/clv/models/gamma_gamma.py Co-authored-by: Ricardo Vieira <[email protected]> * last fix XD --------- Co-authored-by: Ricardo Vieira <[email protected]>

* notebook opening and imports * model definition markdown * Data Load Notebook section * WIP model fitting section * added notebook to docs directory * notebook edits and graph code * ppc section and nb cleanup * demz sampling and WIP plotting * WIP predictive plots * WIP heatmap plots * predictive plots * WIP covariates and nbqa-ruff edits * covariate section * plot additions * fig sizes * remove model file

…book (#651) * add spaces, increase indentation, and fix number order * explicit with 6

* Fix plotting by evaluating tensors. * Add space after sphinx directive. * Remove indentation from blank line. * Add shared y axis for subplots. --------- Co-authored-by: Patrick Robotham <[email protected]> Co-authored-by: Will Dean <[email protected]>

* add original scale implementation * add plot nb * change location * undo * make mypy happy * test plot * add test * update plot readme * fix test * improve variable description

* infernece changed to dataset * inference changed dataset for plot_allocated_contribution_by_channel --------- Co-authored-by: Will Dean <[email protected]>

* feat: adding root_saturation to transformers.py * feat: adding RootSaturation class to saturation.py * chore: adding missing RootSaturation to SATURATION_TRANSFORMATIONS * feat: adding root_saturation to transformers.py * feat: adding RootSaturation class to saturation.py * chore: adding missing RootSaturation to SATURATION_TRANSFORMATIONS * chore: linting edits * chore: adding coefficient to function * chore: linting corrections * chore: removed empty References section of docstring * chore: produce visual examples of root saturation * chore: adding root to test_saturation.py * chore: adding RootSaturation to init file --------- Co-authored-by: ruari.walker <[email protected]> Co-authored-by: Will Dean <[email protected]>

williambdean · 2024-07-25T17:17:45Z

This seems stale. Shall we close and turn into smaller issues?
Thoughts @cluhmann @juanitorduz

juanitorduz · 2024-07-25T17:21:05Z

This notebook has many great ideas that I would like to explore further. Can we leave it open while I try to get some time? Maybe we can add a special label?

juanitorduz · 2024-07-25T17:22:09Z

Or actually, as it is a dev notebook, can we merge it into the dev folder (as the clv dev ones)?

cluhmann · 2024-07-25T17:26:09Z

Happy to help hammer this into something (more) useful.

williambdean · 2024-07-25T17:48:37Z

How about merge in since it is dev notebook and then create an issue or two?

cluhmann · 2024-07-25T17:55:28Z

Fine with me.

* separate the attr creation from attachment and perform check * remove data for CLV * fix model_builder tests * fix clv tests * more specific model builder checks * rework with no args and kwargs * rework common load method * Update pymc_marketing/model_builder.py

* loads doesnt support boolean * defaults for the media transformation * test for the time_varyign

updates: - [github.com/astral-sh/ruff-pre-commit: v0.5.4 → v0.5.5](astral-sh/ruff-pre-commit@v0.5.4...v0.5.5) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update docs in transformers.py docs: Fix the docstring * Update docs in transformers.py docs: fix other docstrings * change the plot default --------- Co-authored-by: Will Dean <[email protected]> Co-authored-by: Will Dean <[email protected]>

* to_dict via lookup_name * parse to and from dict for attrs * improve the codecov * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * change test with change in default behavior * increase the MMM model version --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

williambdean · 2024-12-16T15:09:59Z

Did the latest optimization notebooks kill the need for this?

cluhmann · 2024-12-16T16:31:33Z

The one thing that I will want out of this PR is a notebook in which we compare one model/data set and various approaches to optimization, including:

optimization using point estimates (i.e., the old method of optimization)
optimization using full posterior but neutral utility (e.g., mean/expected response)
optimization using full posterior but utility functions reflecting various risk profiles

I think this that seeing how these different approaches yield different optimized spends would really drive home the value of this new approach to optimization and introduce the Bayesian decision theory stuff at the same time. I talked to @carlosagostini about this when he was first working on the optimization.

cetagostini · 2025-03-21T22:51:31Z

I'll close here, because we address all comments here. Current optimizer works with all posterior, and allow risk profile management.

cluhmann added 2 commits August 17, 2023 19:38

add initial optimization nb

5dca79d

move notebook to dev dir

be7df56

cluhmann requested review from juanitorduz and ricardoV94 August 17, 2023 23:48

williambdean and others added 24 commits April 2, 2024 15:07

Drop python 3.9 support (#615)

8590b45

* drop python 3.9 * try python 3.12 * undo try python 3.12

add lift tests check

27a55d5

Add more content to the Gamma-Gamma Notebook (#573)

f3ec2cf

* improve nb * rm warnings and add link to lifetimes quickstart * address comments * feedback part 3 * remove warnings manually

Improve MMM Docs (#612)

da9f025

* improve mmm docs init * add more code examples to docstrings * minor improvemeents * typo * better phrasing * add thomas suggestion

Fix clv plotting bugs and edits to Quickstart (#601)

38b55a2

* move fixtures to conftest * docstrings and moved set_model_fit to conftest * fixed pandas quickstart warnings * revert to MockModel and add ParetoNBD support * quickstart edit for issue 609 * notebook edit

[pre-commit.ci] pre-commit autoupdate (#616)

ffef5a6

improve coords matching (#623)

a151330

python 3.12 attempt (#618)

6f33cef

closes #520 (#621) dataset -> data

c40c8b8

mmm example notebook small fixes (#626)

ff937f4

Remove ruff E501 ignore (#619)

e2dba27

remove ruff E501 ignore

Update pypi.yml (dash change) (#633)

b777eb5

UP rule ruff (#635)

2be2f63

Add ruff RUF rules (#636)

582d2ed

Update version.txt (#640)

78964f0

[pre-commit.ci] pre-commit autoupdate (#642)

18d926c

Fix build badge (#645)

ce41f5f

Add downloads stats to README

818781a

add spaces, increase indentation, and fix number order to Pareto note…

7034e28

…book (#651) * add spaces, increase indentation, and fix number order * explicit with 6

GiannisApost and others added 5 commits July 25, 2024 06:55

Skip coords with scalar value (#868)

8ee9254

Allow plot MMM components in the original scale (#870)

de5679f

* add original scale implementation * add plot nb * change location * undo * make mypy happy * test plot * add test * update plot readme * fix test * improve variable description

Inference changed to dataset (#873)

19aea61

* infernece changed to dataset * inference changed dataset for plot_allocated_contribution_by_channel --------- Co-authored-by: Will Dean <[email protected]>

Merge branch 'main' into budget_optimzation

13c1a94

cluhmann mentioned this pull request Jul 25, 2024

Remove unnecessary calls to PyTensor eval() from user-facing methods #386

Closed

williambdean and others added 6 commits July 28, 2024 09:34

json.loads with python types bug (#881)

43131db

* loads doesnt support boolean * defaults for the media transformation * test for the time_varyign

[pre-commit.ci] pre-commit autoupdate (#883)

24e9833

updates: - [github.com/astral-sh/ruff-pre-commit: v0.5.4 → v0.5.5](astral-sh/ruff-pre-commit@v0.5.4...v0.5.5) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Fix default ConvMode in docstring (#864)

7a89f27

* Update docs in transformers.py docs: Fix the docstring * Update docs in transformers.py docs: fix other docstrings * change the plot default --------- Co-authored-by: Will Dean <[email protected]> Co-authored-by: Will Dean <[email protected]>

Merge branch 'main' into budget_optimzation

3b5c92a

ulfaslak force-pushed the main branch 4 times, most recently from f9a38fd to 818ba39 Compare September 8, 2024 12:38

twiecki force-pushed the main branch from 35dc7f1 to 9f9c67f Compare September 10, 2024 15:25

williambdean assigned juanitorduz and cetagostini Dec 16, 2024

cetagostini closed this Mar 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add optimization notebook #358

Add optimization notebook #358

Uh oh!

cluhmann commented Aug 17, 2023 •

edited

Loading

Uh oh!

review-notebook-app bot commented Aug 17, 2023

Uh oh!

codecov bot commented Aug 18, 2023 •

edited

Loading

Uh oh!

williambdean commented Jul 25, 2024

Uh oh!

juanitorduz commented Jul 25, 2024

Uh oh!

juanitorduz commented Jul 25, 2024 •

edited

Loading

Uh oh!

cluhmann commented Jul 25, 2024

Uh oh!

williambdean commented Jul 25, 2024

Uh oh!

cluhmann commented Jul 25, 2024

Uh oh!

williambdean commented Dec 16, 2024

Uh oh!

cluhmann commented Dec 16, 2024

Uh oh!

cetagostini commented Mar 21, 2025

Uh oh!

Uh oh!

Add optimization notebook #358

Add optimization notebook #358

Uh oh!

Conversation

cluhmann commented Aug 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

review-notebook-app bot commented Aug 17, 2023

Uh oh!

codecov bot commented Aug 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

williambdean commented Jul 25, 2024

Uh oh!

juanitorduz commented Jul 25, 2024

Uh oh!

juanitorduz commented Jul 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cluhmann commented Jul 25, 2024

Uh oh!

williambdean commented Jul 25, 2024

Uh oh!

cluhmann commented Jul 25, 2024

Uh oh!

williambdean commented Dec 16, 2024

Uh oh!

cluhmann commented Dec 16, 2024

Uh oh!

cetagostini commented Mar 21, 2025

Uh oh!

Uh oh!

cluhmann commented Aug 17, 2023 •

edited

Loading

codecov bot commented Aug 18, 2023 •

edited

Loading

juanitorduz commented Jul 25, 2024 •

edited

Loading