Skip to content
/ eevi Public

Estimators of Entropy and Information in Probabilistic Programs

License

Notifications You must be signed in to change notification settings

probsys/eevi

Folders and files

NameName
Last commit message
Last commit date

Latest commit

b9f70dc · Dec 28, 2024

History

1 Commit
Dec 28, 2024
Dec 28, 2024
Dec 28, 2024
Dec 28, 2024
Dec 28, 2024
Dec 28, 2024
Dec 28, 2024
Dec 28, 2024
Dec 28, 2024

Repository files navigation

Estimators of Entropy and Information via Inference in Probabilistic Models

This repository contains experiment code for

Estimators of Entropy and Information via Inference in Probabilistic Models_. Feras A. Saad, Marco Cusumano Towner, Vikash K. Mansinghka. Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, PMLR 151:5604-5621, 2022. https://proceedings.mlr.press/v151/saad22a.html

Getting started

  1. Install julia v1.6.2 from

    https://github.com/JuliaLang/julia/releases/tag/v1.6.2

  2. Set current directory to the Julia project using

    export JULIA_PROJECT=.

  3. Instantiate the package dependencies using

    julia -e 'using Pkg; Pkg.instantiate()'

    The main dependency is the Gen.jl package,

Running experiments

Please navigate to ./examples and follow the README.

These experiments show how to estimate the entropy of random variables or the (conditional) mutual information between groups of random variables in a probabilistic program written in Gen.jl. The two applications in ./examples directory are based on Gen probabilistic programs that encode models for blood glucose monitoring and the HEPAR expert system for liver disease.

A further reference of the program analysis implementation in Gen can be found in Section 8.6 of the following dissertation:

Scalable Structure Learning, Inference, and Analysis with Probabilistic Programs. Feras A. K. Saad. PhD Thesis, Massachusetts Institute of Technology, 2022. Pages 178–182. https://dspace.mit.edu/handle/1721.1/147226

About

Estimators of Entropy and Information in Probabilistic Programs

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages