SOI: Score-based O-INFORMATION Estimation

Bounoua, Mustapha; Franzese, Giulio; Michiardi, Pietro
ICML 2024, 41st International Conference on Machine Learning, 21-27 July 2024, Vienna, Austria, PMLR 235:4444-4471, 2024

The analysis of scientific data and complex multivariate systems requires information quantities that capture relationships among multiple random variables. Recently, new information-theoretic measures have been developed to overcome the shortcomings of classical ones, such as mutual information, that are restricted to considering pairwise interactions. Among them, the concept of information synergy and redundancy is crucial for understanding the high-order dependencies between variables. One of the most prominent and versatile measures based on this concept is OINFORMATION, which provides a clear and scalable way to quantify the synergy-redundancy balance in multivariate systems. However, its practical application is limited to simplified cases. In this work, we introduce SΩI, which allows — for the first time — to compute O-INFORMATION without restrictive assumptions about the system. Our experiments validate our approach on synthetic data, and demonstrate the effectiveness of SΩI in the context of a real-world use case. 


HAL
Type:
Conférence
City:
Vienna
Date:
2024-07-21
Department:
Data Science
Eurecom Ref:
7618
Copyright:
© 2024 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

PERMALINK : https://www.eurecom.fr/publication/7618