dowhy.gcm package

Subpackages

Submodules

dowhy.gcm.anomaly module

dowhy.gcm.anomaly.anomaly_scores(causal_model: ~dowhy.gcm.cms.ProbabilisticCausalModel, anomaly_data: ~pandas.core.frame.DataFrame, num_samples_conditional: int = 10000, num_samples_unconditional: int = 10000, anomaly_scorer_factory: ~typing.Callable[[], ~dowhy.gcm.anomaly_scorer.AnomalyScorer] = <class 'dowhy.gcm.anomaly_scorers.RescaledMedianCDFQuantileScorer'>) → Dict[Any, ndarray][source]
dowhy.gcm.anomaly.attribute_anomalies(causal_model: InvertibleStructuralCausalModel, target_node: Any, anomaly_samples: DataFrame, anomaly_scorer: Optional[AnomalyScorer] = None, attribute_mean_deviation: bool = False, num_distribution_samples: int = 5000, shapley_config: Optional[ShapleyConfig] = None) → Dict[Any, ndarray][source]

Estimates the contributions of upstream nodes to the anomaly score of the target_node for each sample in anomaly_samples. By default, the anomaly score is based on the information theoretic (IT) score -log(P(g(X) >= g(x))), where g is the anomaly_scorer, X samples from the marginal distribution of the target_node and x an observation of the target_node in anomaly_samples. If attribute_mean_deviation is set to True, the contribution to g(x) - E[g(X)] is estimated instead, i.e. the feature relevance for the given scoring function. The underlying algorithm utilizes the reconstructed noise of upstream nodes (including the target_node itself) for the given anomaly_samples. By this, it is possible to estimate how much of the anomaly score can be explained by upstream anomalies with respect to anomalous noise values.

Note: This function requires that the noise can be recovered from samples, i.e. the causal models of non-root nodes need to be an InvertibleNoiseModel (e.g. AdditiveNoiseModel).

Related paper: Janzing, D., Budhathoki, K., Minorics, L., & Bloebaum, P. (2019). Causal structure based root cause analysis of outliers https://arxiv.org/abs/1912.02724

Parameters

  • causal_model – The fitted InvertibleStructuralCausalModel.

  • target_node – Target node for which the contributions are estimated.

  • anomaly_samples – Anomalous observations for which the contributions are estimated.

  • anomaly_scorer – Anomaly scorer g. If None is given, a MedianCDFQuantileScorer is used.

  • attribute_mean_deviation – If set to False, the contribution is estimated based on the IT score and if it is set to True, the contribution is based on the feature relevance with respect to the given scoring function.

  • num_distribution_samples – Number of samples from X, the marginal distribution of the target. These are used for evaluating the tail probability in case of the IT score (attribute_mean_deviation is False) or as samples for randomization in case of feature relevance (attribute_mean_deviation is True).

  • shapley_config – ShapleyConfig for the Shapley estimator.

Returns

A dictionary that assigns a numpy array to each upstream node including the target_node itself. The i-th entry of an array indicates the contribution of the corresponding node to the anomaly score of the target for the i-th observation in anomaly_samples.

dowhy.gcm.anomaly.attribute_anomaly_scores(anomaly_samples: ndarray, distribution_samples: ndarray, anomaly_scoring_func: Callable[[ndarray], ndarray], attribute_mean_deviation: bool, shapley_config: Optional[ShapleyConfig] = None) → ndarray[source]

Estimates the contributions of the features for each sample in anomaly_samples to the anomaly score obtained by the anomaly_scoring_func. If attribute_mean_deviation is set to False, the anomaly score is based on the information theoretic (IT) score -log(P(g(X) >= g(x))), where g is the anomaly_scoring_func, X samples from the marginal distribution of the target_node and x an observation of the target_node in anomaly_samples. If attribute_mean_deviation is set to True, the contribution to g(x) - E[g(X)] is estimated instead, i.e. the feature relevance for the given scorer.

Note that the anomaly scoring function needs to handle the dimension and modality of the data. An example for a function for multidimensional continues data would be:

density_estimator = GaussianMixtureDensityEstimator() density_estimator.fit(original_observations) anomaly_scoring_func = lambda x, y: estimate_inverse_density_score(x, y, density_estimator)

Related paper: Janzing, D., Budhathoki, K., Minorics, L., & Bloebaum, P. (2022). Causal structure based root cause analysis of outliers https://arxiv.org/abs/1912.02724

Parameters

  • anomaly_samples – Samples x for which the contributions are estimated. The dimensionality of these samples doesn’t matter as long as the anomaly_scoring_func supports it.

  • distribution_samples – Samples from the (non-anomalous) distribution X.

  • anomaly_scoring_func – A function g that takes a sample from X as input and returns an anomaly score.

  • attribute_mean_deviation – If set to False, the contribution is estimated based on the IT score and if it is set to True, the contribution is based on the feature relevance with respect to the given scoring function.

  • shapley_config – ShapleyConfig for the Shapley estimator.

Returns

A numpy array with the feature contributions to the anomaly score for each sample in anomaly_samples.

dowhy.gcm.anomaly.conditional_anomaly_scores(parent_samples: ~numpy.ndarray, target_samples: ~numpy.ndarray, causal_mechanism: ~dowhy.gcm.graph.ConditionalStochasticModel, anomaly_scorer_factory: ~typing.Callable[[], ~dowhy.gcm.anomaly_scorer.AnomalyScorer] = <class 'dowhy.gcm.anomaly_scorers.MedianCDFQuantileScorer'>, num_samples_conditional: int = 10000) → ndarray[source]

Estimates the conditional anomaly scores based on the expected outcomes of the causal model.

Parameters

  • parent_samples – Samples from all parents of the target node.

  • target_samples – Samples from the target node.

  • causal_mechanism – Causal mechanism of the target node.

  • anomaly_scorer_factory – A callable that returns an anomaly scorer.

  • num_samples_conditional – Number of samples drawn from the conditional distribution based on the given parent samples. The more samples, the more accurate the results.

Returns

The conditional anomaly score for each sample in target_samples.

dowhy.gcm.anomaly_scorer module

class dowhy.gcm.anomaly_scorer.AnomalyScorer[source]

Bases: ABC

abstract fit(X: ndarray) → None[source]

Fits the anomaly scorer to the given data. Depending on the definition of the scorer, this can imply different things, such as fitting a (parametric) distribution to the data or estimating certain properties such as mean, variance, median etc. that are used for computing a score.

Parameters

X – Samples from the underlying data distribution.

abstract score(X: ndarray) → ndarray[source]

dowhy.gcm.anomaly_scorers module

This module contains implementations of different anomaly scorers.

Classes and functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

class dowhy.gcm.anomaly_scorers.ITAnomalyScorer(anomaly_scorer: AnomalyScorer)[source]

Bases: AnomalyScorer

Transforms any anomaly scorer into an information theoretic (IT) score. This means, given a scorer S(x), an anomalous observation x and samples from the distribution of X, this scorer class represents:

score(x) = -log(P(S(X) >= S(x)))

This is, the negative logarithm of the probability to get the same or a higher score with (random) samples from X compared to the score obtained based on the anomalous observation x. By this, the score of arbitrarily different anomaly scorers become comparable information theoretic quantities. The new score -log(P(S(X) >= S(x))) can also be seen as “The higher the score, the rarer the anomaly event”. For instance, if we have S(x) = c, but observe the same or higher scores in 50% or even 100% of all samples in X, then this is not really a rare event, and thus, not an anomaly. As mentioned above, transforming it into an IT score makes arbitrarily different anomaly scorer with potentially completely different scaling comparable. For example, one could compare the IT score of isolation forests with z-scores.

For more details about IT scores, see:

Causal structure based root cause analysis of outliers Kailash Budhathoki, Patrick Bloebaum, Lenon Minorics, Dominik Janzing (2022)

The higher the score, the higher the likelihood that the observations is an anomaly.

fit(X: ndarray) → None[source]

Fits the anomaly scorer to the given data. Depending on the definition of the scorer, this can imply different things, such as fitting a (parametric) distribution to the data or estimating certain properties such as mean, variance, median etc. that are used for computing a score.

Parameters

X – Samples from the underlying data distribution.

score(X: ndarray) → ndarray[source]
class dowhy.gcm.anomaly_scorers.InverseDensityScorer(density_estimator: Optional[DensityEstimator] = None)[source]

Bases: AnomalyScorer

Estimates an anomaly score based on 1 / p(x), where x is the data to score. The density value p(x) is estimated using the given density estimator. If None is given, a Gaussian mixture model is used by default.

Note: The given density estimator needs to support the data types, i.e. if the data has categorical values, the density estimator needs to be able to handle that. The default Gaussian model can only handle numeric data.

Note: If the density p(x) is 0, a nan or inf could be returned.

fit(X: ndarray) → None[source]

Fits the anomaly scorer to the given data. Depending on the definition of the scorer, this can imply different things, such as fitting a (parametric) distribution to the data or estimating certain properties such as mean, variance, median etc. that are used for computing a score.

Parameters

X – Samples from the underlying data distribution.

score(X: ndarray) → ndarray[source]
class dowhy.gcm.anomaly_scorers.MeanDeviationScorer[source]

Bases: AnomalyScorer

Given an anomalous observation x and samples from the distribution of X, this score represents:

score(x) = |x - E[X]| / std[X]

This scores the given sample based on its distance to the mean of X and scaled by the standard deviation of X. This is also equivalent to the Z-score in Gaussian variables.

The higher the score, the higher the deviation of the observation from the mean of X.

fit(X: ndarray) → None[source]

Fits the anomaly scorer to the given data. Depending on the definition of the scorer, this can imply different things, such as fitting a (parametric) distribution to the data or estimating certain properties such as mean, variance, median etc. that are used for computing a score.

Parameters

X – Samples from the underlying data distribution.

score(X: ndarray) → ndarray[source]
class dowhy.gcm.anomaly_scorers.MedianCDFQuantileScorer[source]

Bases: AnomalyScorer

Given an anomalous observation x and samples from the distribution of X, this score represents:

score(x) = 1 - 2 * min[P(X >= x), P(X <= x)]

It scores the observation based on the quantile of x with respect to the distribution of X. Here, if the sample x lies in the tail of the distribution, we want to have a large score. Since we apriori don’t know whether the sample falls on the left or right side of the median of X, we estimate the quantile on both sides and take the minimum. Here, these probabilities are estimated by counting and since half of the samples are on one side from the median, we need to multiply this by a factor of two to obtain the two-sided quantile. For example:

X = [-3, -2, -1, 0, 1, 2, 3] x = 2.5

Then, x falls in the right sided-quantile and only one sample in X is larger than x. Therefore, we get

p(X >= x) = 1 / 7 P(X <= x) = 6 / 7

With the end score of:

1 - 2 * min[P(X >= x), P(X <= x)] = 1 - 2 / 7 = 0.71

Note: For equal samples, we contribute half of the count to the left and half of the count the right side.

The higher the score, the less likely the sample comes from the distribution of X.

fit(X: ndarray) → None[source]

Fits the anomaly scorer to the given data. Depending on the definition of the scorer, this can imply different things, such as fitting a (parametric) distribution to the data or estimating certain properties such as mean, variance, median etc. that are used for computing a score.

Parameters

X – Samples from the underlying data distribution.

score(X: ndarray) → ndarray[source]
class dowhy.gcm.anomaly_scorers.MedianDeviationScorer[source]

Bases: AnomalyScorer

Given an anomalous observation x and samples from the distribution of X, this score represents:

score(x) = |x - med[X]| / mad[X]

This scores the given sample based on its distance to the median of X and scaled by the median absolute deviation of X.

The higher the score, the higher the deviation of the observation from the median of X.

fit(X: ndarray) → None[source]

Fits the anomaly scorer to the given data. Depending on the definition of the scorer, this can imply different things, such as fitting a (parametric) distribution to the data or estimating certain properties such as mean, variance, median etc. that are used for computing a score.

Parameters

X – Samples from the underlying data distribution.

score(X: ndarray) → ndarray[source]
class dowhy.gcm.anomaly_scorers.RescaledMedianCDFQuantileScorer[source]

Bases: AnomalyScorer

Given an anomalous observation x and samples from the distribution of X, this score represents:

score(x) = -log(2 * min[P(X >= x), P(X <= x)])

This is a rescaled version of the score s obtained by the MedianCDFQuantileScorer by calculating the negative log-probability -log(1 - s). This has the advantage that small differences in the probabilities are amplified, especially when they are close to 0. For instance, the difference between probabilities 0.02 and 0.01 seems to be small and insignificant, but the rescaled difference would be significantly larger: -log(0.02) - log(0.01) −= 8.5

The higher the score, the less likely the sample comes from the distribution of X.

fit(X: ndarray) → None[source]

Fits the anomaly scorer to the given data. Depending on the definition of the scorer, this can imply different things, such as fitting a (parametric) distribution to the data or estimating certain properties such as mean, variance, median etc. that are used for computing a score.

Parameters

X – Samples from the underlying data distribution.

score(X: ndarray) → ndarray[source]

dowhy.gcm.auto module

class dowhy.gcm.auto.AssignmentQuality(value)[source]

Bases: Enum

An enumeration.

BEST = 3
BETTER = 2
GOOD = 1
dowhy.gcm.auto.assign_causal_mechanisms(causal_model: ProbabilisticCausalModel, based_on: DataFrame, quality: AssignmentQuality = AssignmentQuality.GOOD, override_models: bool = False) → None[source]

Automatically assigns appropriate causal models. If causal models are already assigned to nodes and override_models is set to False, this function only validates the assignments with respect to the graph structure. Here, the validation checks whether root nodes have StochasticModels and non-root ConditionalStochasticModels assigned.

Parameters

  • causal_model – The causal model to whose nodes to assign causal models.

  • based_on – Jointly sampled data corresponding to the nodes of the given graph.

  • quality – AssignmentQuality for the automatic model selection and model accuracy. This changes the type of

prediction model and time spent on the selection. Options are:
  • AssignmentQuality.GOOD: Compares a linear, polynomial and gradient boost model on small test-training split

    of the data. The best performing model is then selected. Model selection speed: Fast Model training speed: Fast Model inference speed: Fast Model accuracy: Medium

  • AssignmentQuality.BETTER: Compares multiple model types and uses the one with the best performance

    averaged over multiple splits of the training data. By default, the model with the smallest root mean squared error is selected for regression problems and the model with the highest F1 score is selected for classification problems. For a list of possible models, see _LIST_OF_POTENTIAL_REGRESSORS_BETTER and _LIST_OF_POTENTIAL_CLASSIFIERS_BETTER, respectively. Model selection speed: Medium Model training speed: Fast Model inference speed: Fast Model accuracy: Good

  • AssignmentQuality.BEST: Uses an AutoGluon (auto ML) model with default settings defined by the AutoGluon

    wrapper. While the model selection itself is fast, the training and inference speed can be significantly slower than in the other options. NOTE: This requires the optional autogluon.tabular dependency. Model selection speed: Instant Model training speed: Slow Model inference speed: Slow-Medium Model accuracy: Best

param override_models

If set to True, existing model assignments are replaced with automatically selected

ones. If set to False, the assigned models are only validated with respect to the graph structure.

Returns

None

dowhy.gcm.auto.find_best_model(prediction_model_factories: List[Callable[[], PredictionModel]], X: ndarray, Y: ndarray, metric: Optional[Callable[[ndarray, ndarray], float]] = None, max_samples_per_split: int = 10000, model_selection_splits: int = 5, n_jobs: Optional[int] = None) → Callable[[], PredictionModel][source]
dowhy.gcm.auto.has_linear_relationship(X: ndarray, Y: ndarray, max_num_samples: int = 3000) → bool[source]
dowhy.gcm.auto.select_model(X: ndarray, Y: ndarray, model_selection_quality: AssignmentQuality) → Union[PredictionModel, ClassificationModel][source]

dowhy.gcm.cms module

This module defines the fundamental classes for graphical causal models (GCMs).

Classes in this module should be considered experimental, meaning there might be breaking API changes in the future.

class dowhy.gcm.cms.InvertibleStructuralCausalModel(graph: ~typing.Optional[~dowhy.gcm.graph.DirectedGraph] = None, graph_copier: ~typing.Callable[[~dowhy.gcm.graph.DirectedGraph], ~dowhy.gcm.graph.DirectedGraph] = <class 'networkx.classes.digraph.DiGraph'>)[source]

Bases: StructuralCausalModel

Represents an invertible structural graphical causal model, as required e.g. by counterfactual_samples(). This is a subclass of StructuralCausalModel and has further restrictions on the class of causal mechanisms. Here, the mechanisms of non-root nodes need to be invertible with respect to the noise, such as PostNonlinearModel.

Parameters

  • graph – Optional graph object to be used as causal graph.

  • graph_copier – Optional function that can copy a causal graph. Defaults to a networkx.DiGraph constructor.

causal_mechanism(node: Any) → Union[StochasticModel, InvertibleFunctionalCausalModel][source]

Returns the generative causal model of node in the causal graph.

Parameters

node – Target node whose causal model is to be assigned.

Returns

The causal mechanism for this node. A root node is of type StochasticModel, whereas a non-root node is of type ConditionalStochasticModel.

set_causal_mechanism(target_node: Any, mechanism: Union[StochasticModel, InvertibleFunctionalCausalModel]) → None[source]

Assigns the generative causal model of node in the causal graph.

Parameters

  • node – Target node whose causal model is to be assigned.

  • mechanism – Causal mechanism to be assigned. A root node must be a StochasticModel, whereas a non-root node must be a ConditionalStochasticModel.

class dowhy.gcm.cms.ProbabilisticCausalModel(graph: ~typing.Optional[~dowhy.gcm.graph.DirectedGraph] = None, graph_copier: ~typing.Callable[[~dowhy.gcm.graph.DirectedGraph], ~dowhy.gcm.graph.DirectedGraph] = <class 'networkx.classes.digraph.DiGraph'>)[source]

Bases: object

Represents a probabilistic graphical causal model, i.e. it combines a graphical representation of causal causal relationships and corresponding causal mechanism for each node describing the data generation process. The causal mechanisms can be any general stochastic models.

Parameters

  • graph – Optional graph object to be used as causal graph.

  • graph_copier – Optional function that can copy a causal graph. Defaults to a networkx.DiGraph constructor.

causal_mechanism(node: Any) → Union[StochasticModel, ConditionalStochasticModel][source]

Returns the generative causal model of node in the causal graph.

Parameters

node – Target node whose causal model is to be assigned.

Returns

The causal mechanism for this node. A root node is of type StochasticModel, whereas a non-root node is of type ConditionalStochasticModel.

clone()[source]

Clones the causal model, but keeps causal mechanisms untrained.

set_causal_mechanism(node: Any, mechanism: Union[StochasticModel, ConditionalStochasticModel]) → None[source]

Assigns the generative causal model of node in the causal graph.

Parameters

  • node – Target node whose causal model is to be assigned.

  • mechanism – Causal mechanism to be assigned. A root node must be a StochasticModel, whereas a non-root node must be a ConditionalStochasticModel.

class dowhy.gcm.cms.StructuralCausalModel(graph: ~typing.Optional[~dowhy.gcm.graph.DirectedGraph] = None, graph_copier: ~typing.Callable[[~dowhy.gcm.graph.DirectedGraph], ~dowhy.gcm.graph.DirectedGraph] = <class 'networkx.classes.digraph.DiGraph'>)[source]

Bases: ProbabilisticCausalModel

Represents a structural causal model (SCM), as required e.g. by counterfactual_samples(). As compared to a ProbabilisticCausalModel, an SCM describes the data generation process in non-root nodes by functional causal models.

Parameters

  • graph – Optional graph object to be used as causal graph.

  • graph_copier – Optional function that can copy a causal graph. Defaults to a networkx.DiGraph constructor.

causal_mechanism(node: Any) → Union[StochasticModel, FunctionalCausalModel][source]

Returns the generative causal model of node in the causal graph.

Parameters

node – Target node whose causal model is to be assigned.

Returns

The causal mechanism for this node. A root node is of type StochasticModel, whereas a non-root node is of type ConditionalStochasticModel.

set_causal_mechanism(node: Any, mechanism: Union[StochasticModel, FunctionalCausalModel]) → None[source]

Assigns the generative causal model of node in the causal graph.

Parameters

  • node – Target node whose causal model is to be assigned.

  • mechanism – Causal mechanism to be assigned. A root node must be a StochasticModel, whereas a non-root node must be a ConditionalStochasticModel.

dowhy.gcm.confidence_intervals module

This module provides functionality to estimate confidence intervals via bootstrapping.

Functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

dowhy.gcm.confidence_intervals.confidence_intervals(estimation_func: ~typing.Union[~typing.Callable[[], ~numpy.ndarray], ~typing.Callable[[], ~typing.Dict[~typing.Any, float]]], confidence_level: float = 0.95, num_bootstrap_resamples: int = 20, bootstrap_results_summary_func: ~typing.Callable[[~numpy.ndarray], ~numpy.ndarray] = <function estimate_geometric_median>, n_jobs: int = 1) → Tuple[Union[ndarray, Dict[Any, ndarray]], Union[ndarray, Dict[Any, ndarray]]][source]

Estimates confidence intervals based on the outputs generated by calling the given estimation_func. Since one result for each repetition is produced, all results can be summarized by the method defined in summary_method_of_bootstrap_results. For instance, summary_method_of_bootstrap_results = lambda x: numpy.mean(x, axis=0) to get the mean over all runs. By default, the geometric median is returned.

Currently, the confidence intervals are empirically estimated based on the n-th estimated quantiles (without bias correction) of the results, where the quantiles are determined by the given confidence_level.

NOTE: The outputs of estimation_func are assumed to be pairwise independent. For multidimensional outputs of estimation_func, this could be violated and should be kept in mind. For instance, when evaluating the outcome of interventions in a graph like X -> Y -> Z, the confidence intervals are estimate independently for X, Y and Z although they have a strong dependency. If estimation_func returns one dimensional results, as for instance when estimating the direct arrow strength, then there should be no problem.

Example usage with numpy array output:

>>> def estimation_func() -> np.ndarray:
>>>     return direct_arrow_strength_of_model(causal_model, parent_data)
>>>
>>> arrow_strengths, confidence_intervals = confidence_intervals(estimation_func)

Example usage with dictionary output:

>>> def estimation_func() -> Dict[Any, float]:
>>>     return distribution_change(
>>>             causal_dag, original_observations, outlier_observations, 'X3')
>>>
>>> mean_contributions, confidence_intervals = confidence_intervals(estimation_func)

More details about the estimation of confidence intervals via bootstrapping can be found here.

Parameters

  • estimation_func – Function that generates a non-deterministic output for which the confidence interval(s) are estimated.

  • confidence_level – Confidence level of the interval.

  • num_bootstrap_resamples – Number of samples generated by estimation_func, i.e. number of times is called. The higher the number, the more accurate the results and intervals, but the slower the runtime.

  • bootstrap_results_summary_func – Function that takes a numpy array with all results as an input and returns a single (potentially multidimensional) value/vector. For instance, the mean or median over all results.

  • n_jobs – Number of parallel jobs. Each repetition can be estimated in parallel. However, since many other functions of the library are already running in parallel ( such as distribution change), this is set to 1 by default. Only if it is certain that the estimation_func is not running in parallel internally (e.g. when performing interventions), this should be set to a different value.

Returns

A tuple (summarized result over all repetitions based on summary_method_of_bootstrap_results, confidence interval for each dimension/variable)

dowhy.gcm.confidence_intervals.estimate_geometric_median(X: ndarray) → ndarray[source]

dowhy.gcm.confidence_intervals_cms module

This module provides functionality to estimate confidence intervals via bootstrapping the fitting and sampling.

Functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

dowhy.gcm.confidence_intervals_cms.fit_and_compute(f: Callable[[Union[ProbabilisticCausalModel, StructuralCausalModel, InvertibleStructuralCausalModel], Any], Dict[Any, Union[ndarray, float]]], causal_model: Union[ProbabilisticCausalModel, StructuralCausalModel, InvertibleStructuralCausalModel], bootstrap_training_data: DataFrame, bootstrap_data_subset_size_fraction: float = 0.75, *args, **kwargs)[source]

A convenience function when computing confidence intervals specifically for causal queries. This function specifically bootstraps training and sampling.

Example usage:

>>> scores_median, scores_intervals = gcm.confidence_intervals(
>>>     gcm.fit_and_compute(gcm.arrow_strength,
>>>                         causal_model,
>>>                         bootstrap_training_data=data,
>>>                         target_node='Y'))
Parameters

  • f – The causal query to perform. A causal query is a function taking a graphical causal model as first parameter and an arbitrary number of remaining parameters. It must return a dictionary with attribution-like data.

  • causal_model – A graphical causal model to perform the causal query on. It need not be fitted.

  • bootstrap_training_data – The training data to use when fitting. A random subset from this data set is used in every iteration when calling fit.

  • bootstrap_data_subset_size_fraction – The fraction defines the fractional size of the subset compared to the total training data.

  • args – Args passed through verbatim to the causal queries.

  • kwargs – Keyword args passed through verbatim to the causal queries.

Returns

A tuple containing (1) the median of causal query results and (2) the confidence intervals.

dowhy.gcm.config module

dowhy.gcm.config.disable_progress_bars()[source]
dowhy.gcm.config.enable_progress_bars()[source]
dowhy.gcm.config.set_default_n_jobs(n_jobs: int) → None[source]

dowhy.gcm.constant module

dowhy.gcm.density_estimator module

class dowhy.gcm.density_estimator.DensityEstimator[source]

Bases: ABC

abstract density(X: ndarray) → ndarray[source]

Returns the density of each input.

abstract fit(X: ndarray) → None[source]

dowhy.gcm.density_estimators module

This module contains implementations of different density estimators.

Classes and functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

class dowhy.gcm.density_estimators.GaussianMixtureDensityEstimator(num_components: Optional[int] = None)[source]

Bases: DensityEstimator

Represents a density estimator based on a Gaussian mixture model. The estimator uses the sklearn BayesianGaussianMixture model internally.

density(X: ndarray) → ndarray[source]

Returns the density of each input.

fit(X: ndarray) → None[source]
class dowhy.gcm.density_estimators.KernelDensityEstimator1D[source]

Bases: DensityEstimator

Represents a kernel based density estimator. The estimator uses the sklearn KernelDensity class internally.

density(X: ndarray) → ndarray[source]

Returns the density of each input.

fit(X: ndarray) → None[source]

dowhy.gcm.distribution_change module

This module defines functions to attribute distribution changes.

Functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

dowhy.gcm.distribution_change.distribution_change(causal_model: ~dowhy.gcm.cms.ProbabilisticCausalModel, old_data: ~pandas.core.frame.DataFrame, new_data: ~pandas.core.frame.DataFrame, target_node: ~typing.Any, num_samples: int = 2000, difference_estimation_func: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray], float] = <function auto_estimate_kl_divergence>, independence_test: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray], float] = <function kernel_based>, conditional_independence_test: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray, ~numpy.ndarray], float] = <function kernel_based>, mechanism_change_test_significance_level: float = 0.05, mechanism_change_test_fdr_control_method: ~typing.Optional[str] = 'fdr_bh', auto_assignment_quality: ~typing.Optional[~dowhy.gcm.auto.AssignmentQuality] = None, return_additional_info: bool = False, shapley_config: ~typing.Optional[~dowhy.gcm.shapley.ShapleyConfig] = None, graph_factory: ~typing.Callable[[~typing.Any], ~dowhy.gcm.graph.DirectedGraph] = <class 'networkx.classes.digraph.DiGraph'>) → Union[Dict[Any, float], Tuple[Dict[Any, float], Dict[Any, bool], ProbabilisticCausalModel, ProbabilisticCausalModel]][source]

Attributes the change in the marginal distribution of the target_node to nodes upstream in the causal DAG.

Note that this method creates two copies of the causal DAG. The causal models of one causal DAG are learned from old data and those of another DAG are learned from new data.

Research Paper: Kailash Budhathoki, Dominik Janzing, Patrick Bloebaum, Hoiyi Ng. Why did the distribution change?. Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR 130:1666-1674, 2021.

Parameters

  • causal_model – Reference causal model.

  • old_data – Joint samples from the ‘old’ distribution.

  • new_data – Joint samples from the ‘new’ distribution.

  • target_node – Target node of interest for attributing the marginal distribution change.

  • num_samples – Number of samples used for estimating Shapley values. This can have a significant influence on runtime and accuracy.

  • difference_estimation_func – Function for quantifying the distribution change. This function should expect two inputs which represent samples from two different distributions, e.g. difference in average values.

  • independence_test – Unconditional independence test. This is used to identify mechanism changes in root nodes.

  • conditional_independence_test – Conditional independence test. This is used to identify mechanism changes in non-root nodes.

  • mechanism_change_test_significance_level – A significance level for rejecting the null hypothesis that the causal mechanism of a node has not changed.

  • mechanism_change_test_fdr_control_method – The false discovery rate control method for mechanism change tests. For more options, checkout statsmodels manual.

  • auto_assignment_quality – If set to None, the assigned models from the given causal models are used for the old and new graph. However, they are re-fitted on the given data. If set to a valid assignment quality, new models are automatically assigned to the old and new graph based on the respective data.

  • return_additional_info – If set to True, three additional items are returned: a dictionary indicating whether each node’s mechanism changed, the causal DAG whose causal models are learned from old data, and the causal DAG whose causal models are learned from new data.

  • shapley_config – Configuration for the Shapley estimator.

  • graph_factory – Allows customization in case a graph class different than networkx.DiGraph should be used. This function must copy nodes and edges. Attributes of nodes will be overridden in the copy, so the algorithm is independent of the attribute copy behavior of this factory.

Returns

By default, if return_additional_info is set to False, only the dictionary containing contribution of each upstream node is returned. If return_additional_info is set to True, three additional items are returned: a dictionary indicating whether each node’s mechanism changed, the causal DAG whose causal models learned from old data, and the causal DAG whose causal models are learned from new data.

dowhy.gcm.distribution_change.distribution_change_of_graphs(causal_model_old: ~dowhy.gcm.cms.ProbabilisticCausalModel, causal_model_new: ~dowhy.gcm.cms.ProbabilisticCausalModel, target_node: ~typing.Any, num_samples: int = 2000, difference_estimation_func: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray], float] = <function auto_estimate_kl_divergence>, shapley_config: ~typing.Optional[~dowhy.gcm.shapley.ShapleyConfig] = None, graph_factory: ~typing.Callable[[~typing.Any], ~dowhy.gcm.graph.DirectedGraph] = <class 'networkx.classes.digraph.DiGraph'>) → Dict[Any, float][source]

Attributes the change of the marginal distribution of target_node to upstream nodes based on the distributions generated by the ‘old’ and ‘new’ causal graphs. These graphs are assumed to represent the same causal structure and to be fitted on the respective data.

Note: This method creates a copy of the given causal models, i.e. the original objects will not be modified.

Related paper: Budhathoki, K., Janzing, D., Bloebaum, P., & Ng, H. (2021). Why did the distribution change? arXiv preprint arXiv:2102.13384.

Parameters

  • causal_model_old – The ProbabilisticCausalModel fitted on the ‘old’ data.

  • causal_model_new – The ProbabilisticCausalModel fitted on the ‘new’ data.

  • target_node – Node of interest for attributing the marginal distribution change.

  • num_samples – Number of samples used for the estimation. This can have a significant influence on the runtime and accuracy.

  • difference_estimation_func – Function for quantifying the distribution change. This function should expect two inputs which represent samples from two different distributions. An example could be the KL divergence.

  • shapley_config – Config for the Shapley estimator.

  • graph_factory – Allows customization in case a graph class different than networkx.DiGraph should be used. This function must copy nodes and edges. Attributes of nodes will be overridden in the copy, so the algorithm is independent of the attribute copy behavior of this factory.

Returns

A dictionary containing the contributions of upstream nodes to the marginal distribution change in the target node.

dowhy.gcm.distribution_change.estimate_distribution_change_scores(causal_model: ~dowhy.gcm.cms.ProbabilisticCausalModel, original_data: ~pandas.core.frame.DataFrame, new_data: ~pandas.core.frame.DataFrame, difference_estimation_func: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray], ~typing.Union[~numpy.ndarray, float]] = <function auto_estimate_kl_divergence>, max_num_evaluation_samples: int = 1000, num_joint_samples: int = 500, early_stopping_percentage: float = 0.01, independence_test: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray], float] = <function kernel_based>, conditional_independence_test: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray, ~numpy.ndarray], float] = <function kernel_based>, mechanism_change_test_significance_level: float = 0.05, mechanism_change_test_fdr_control_method: ~typing.Optional[str] = 'fdr_bh') → Dict[Any, float][source]

Given newly observed and original samples from the joint distribution of the given causal graphical model, this method estimates a score for each node that quantifies how much the distribution of the node has changed. For this, it first checks whether the underlying causal mechanism has changed at all and, if this is the case, it estimates the difference between the new and original distributions. The score is based on the quantity measured by the provided difference_estimation_func or 0 if no mechanism change has been detected.

Note that for each parent sample, num_joint_samples conditional samples are generated based on the original and new causal mechanism and evaluated by the given difference_estimation_func function. These results are then averaged over multiple different parent samples.

Parameters

  • causal_model – The underlying causal model based on the original data.

  • original_data – Samples from the original data.

  • new_data – Samples from the new data.

  • difference_estimation_func – Function for quantifying the distribution change. This function should expect two inputs which represent samples from two different distributions. An example could be the KL divergence.

  • max_num_evaluation_samples – Maximum number of (parent) samples for evaluating the difference in distributions.

  • num_joint_samples – Number of samples generated in a node per parent sample.

  • early_stopping_percentage – If the change in percentage between multiple consecutive runs is below this threshold, the evaluation stops before evaluating all max_num_evaluation_samples.

  • independence_test – Unconditional independence test. This is used to identify mechanism changes in root nodes.

  • conditional_independence_test – Conditional independence test. This is used to identify mechanism changes in non-root nodes.

  • mechanism_change_test_significance_level – A significance level for rejecting the null hypothesis that the causal mechanism of a node has not changed.

  • mechanism_change_test_fdr_control_method –

    The false discovery rate control method for mechanism change tests. For more options, checkout statsmodels manual.

Returns

A dictionary assining a score to each node in the causal graph.

dowhy.gcm.distribution_change.mechanism_change_test(target_original_data: ~numpy.ndarray, target_new_data: ~numpy.ndarray, parents_original_data: ~typing.Optional[~numpy.ndarray] = None, parents_new_data: ~typing.Optional[~numpy.ndarray] = None, independence_test: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray], float] = <function kernel_based>, conditional_independence_test: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray, ~numpy.ndarray], float] = <function kernel_based>) → float[source]

Estimates a p-value for the null hypothesis that the original and new data were generated by the same mechanism. Here, we check the dependency between binary labels indicating whether a sample is from the original or a new data set. If the labels do not provide information to determine if a sample is coming from the original/new distribution, then it is likely that the mechanism has not changed.

For non-root nodes, samples from parent variables are needed as conditioning variables. This is, testing the null hypothesis that the data were generated by the same mechanism given the parent samples. By this, we incorporate upstream changes that might have impacted the parents, but not the target node itself.

Parameters

  • target_original_data – Samples of the node from the original data set.

  • target_new_data – Samples of the node from the new data set.

  • parents_original_data – Samples from parents of the node from the original data set.

  • parents_new_data – Samples from parents of the node from the new data set.

  • independence_test – Unconditional independence test. This is used to identify mechanism changes in nodes without parents.

  • conditional_independence_test – Conditional independence test. This is used to identify mechanism changes in nodes with parents.

Returns

A p-value for the null hypothesis that the mechanism has not changed.

dowhy.gcm.divergence module

Functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

dowhy.gcm.divergence.auto_estimate_kl_divergence(X: ndarray, Y: ndarray) → float[source]
dowhy.gcm.divergence.estimate_kl_divergence_categorical(X: ndarray, Y: ndarray) → float[source]
dowhy.gcm.divergence.estimate_kl_divergence_continuous(X: ndarray, Y: ndarray) → float[source]

Estimates KL-Divergence using k-nearest neighbours (Wang et al., 2009).

Q. Wang, S. R. Kulkarni, and S. Verdú, “Divergence estimation for multidimensional densities via k-nearest-neighbor distances”, IEEE Transactions on Information Theory, vol. 55, no. 5, pp. 2392-2405, May 2009.

Parameters

  • X – (N_1,D) Sample drawn from distribution P_X

  • Y – (N_2,D) Sample drawn from distribution P_Y

return: Estimated value of D(P_X||P_Y).

dowhy.gcm.divergence.estimate_kl_divergence_of_probabilities(X: ndarray, Y: ndarray) → float[source]

Estimates the Kullback-Leibler divergence between each pair of probability vectors (row wise) in X and Y separately and returns the mean over all results.

dowhy.gcm.divergence.is_probability_matrix(X: ndarray) → bool[source]

dowhy.gcm.fcms module

This module defines multiple implementations of the abstract class FunctionalCausalModel (FCM)

Classes in this module should be considered experimental, meaning there might be breaking API changes in the future.

class dowhy.gcm.fcms.AdditiveNoiseModel(prediction_model: PredictionModel, noise_model: Optional[StochasticModel] = None)[source]

Bases: PostNonlinearModel

Represents the continuous functional causal model of the form

Y = f(X) + N,

where X is the input (typically, direct causal parents of Y) and the noise N is assumed to be independent of X. This is a special instance of a PostNonlinearModel where the function g is the identity function.

Given joint samples from (X, Y), this model can be fitted by first training a model f (e.g. using least squares regression) and then reconstruct N by N = Y - f(X), i.e. using the residual.

Parameters

  • prediction_model – The prediction model f.

  • invertible_function – The invertible function g.

  • noise_model – The StochasticModel to describe the distribution of the noise N.

clone()[source]
class dowhy.gcm.fcms.ClassificationModel[source]

Bases: PredictionModel

abstract property classes: List[str]
abstract predict_probabilities(X: array) → ndarray[source]
class dowhy.gcm.fcms.ClassifierFCM(classifier_model: Optional[ClassificationModel] = None)[source]

Bases: FunctionalCausalModel, ProbabilityEstimatorModel

Represents the categorical functional causal model of the form

Y = f(X, N),

where X is the input (typically, direct causal parents of Y) and the noise N here is uniform on [0, 1]. The model is mostly based on a standard classification model that outputs probabilities. In order to generate a new random sample given an input x, the return value y is uniformly sampled based on the class probabilities p(y | x). Here, the noise is used to make this sampling process deterministic by using the cumulative distribution functions defined by the given inputs.

property classifier_model: ClassificationModel
clone()[source]
draw_noise_samples(num_samples: int) → ndarray[source]

Returns uniformly sampled values on [0, 1].

Parameters

num_samples – Number of noise samples.

Returns

Noise samples on [0, 1].

estimate_probabilities(parent_samples: ndarray) → ndarray[source]

Returns the class probabilities for the given parent_samples.

Parameters

parent_samples – Samples from inputs X.

Returns

A nxd numpy matrix with class probabilities for each sample, where n is the number of samples and d the number of classes. Here, array entry A[i][j] corresponds to the i-th sample indicating the probability of the j-th class.

evaluate(parent_samples: ndarray, noise_samples: ndarray) → ndarray[source]

Evaluates the model Y = f(X, N), where X are the parent_samples and N the noise_samples. Here, the cumulative distribution functions are defined by the parent_samples. For instance, lets say we have 2 classes, n = 0.7 and an input x with p(y = 0| x) = 0.6 and p(y = 1| x) = 0.4, then we get y = 1 as a return value. This is because p(y = 0| x) < n <= 1.0, i.e. n falls into the bucket that is spanned by p(y = 1| x).

Parameters

  • parent_samples – Samples from the inputs X.

  • noise_samples – Samples from the noise on [0, 1].

Returns

Class labels Y based on the inputs and noise.

fit(X: ndarray, Y: ndarray) → None[source]

Fits the underlying classification model.

Parameters

  • X – Input samples.

  • Y – Target labels.

Returns

None

get_class_names(class_indices: ndarray) → List[str][source]
class dowhy.gcm.fcms.InvertibleFunction[source]

Bases: object

abstract evaluate(X: ndarray) → ndarray[source]

Applies the function on the input.

abstract evaluate_inverse(X: ndarray) → ndarray[source]

Returns the outcome of applying the inverse of the function on the inputs.

class dowhy.gcm.fcms.PostNonlinearModel(prediction_model: PredictionModel, noise_model: StochasticModel, invertible_function: InvertibleFunction)[source]

Bases: InvertibleFunctionalCausalModel

Represents an post nonlinear FCM, i.e. models of the form:

Y = g(f(X) + N),

where X are parent nodes of the target node Y, f an arbitrary prediction model expecting inputs from the parents X, N a noise variable and g an invertible function.

Parameters

  • prediction_model – The prediction model f.

  • invertible_function – The invertible function g.

  • noise_model – The StochasticModel to describe the distribution of the noise N.

clone()[source]
draw_noise_samples(num_samples: int) → ndarray[source]

Draws samples from the noise distribution N.

Parameters

num_samples – Number of noise samples.

Returns

A numpy array containing num_samples samples from the noise.

estimate_noise(target_samples: ndarray, parent_samples: ndarray) → ndarray[source]

Reconstruct the noise given samples from (X, Y). This is done by:

  1. Transform Y via the inverse of g: g^-1(Y) = f(X) + N

  2. Return the residual g^-1(Y) - f(X)

Parameters

  • target_samples – Samples from the input X.

  • parent_samples – Samples from the target Y.

Returns

The reconstructed noise based on the given samples.

evaluate(parent_samples: ndarray, noise_samples: ndarray) → ndarray[source]

Evaluates the post non-linear model given samples (X, N). This is done by:

  1. Evaluate f(X)

  2. Evaluate f(X) + N

  3. Return g(f(X) + N)

Parameters

  • parent_samples – Samples from the inputs X.

  • noise_samples – Samples from the noise N.

Returns

The Y values based on the given samples.

fit(X: ndarray, Y: ndarray) → None[source]

Fits the post non-linear model of the form Y = g(f(X) + N). Here, this consists of three steps given samples from (X, Y):

  1. Transform Y via the inverse of g: g^-1(Y) = f(X) + N

  2. Fit the model for f on (X, g^-1(Y))

  3. Reconstruct N based on the residual N = g^-1(Y) - f(X)

Note that the noise here can be inferred uniquely if the model assumption holds.

Parameters

  • X – Samples from the input X.

  • Y – Samples from the target Y.

Returns

None

property invertible_function: InvertibleFunction
property noise_model: StochasticModel
property prediction_model: PredictionModel
class dowhy.gcm.fcms.PredictionModel[source]

Bases: object

Represents general prediction model implementations. Each prediction model should provide a fit and a predict method.

abstract clone()[source]

Clones the prediction model using the same hyper parameters but not fitted.

Returns

An unfitted clone of the prediction model.

abstract fit(X: ndarray, Y: ndarray) → None[source]
abstract predict(X: ndarray) → ndarray[source]
class dowhy.gcm.fcms.ProbabilityEstimatorModel[source]

Bases: ABC

abstract estimate_probabilities(parent_samples: ndarray) → ndarray[source]

dowhy.gcm.feature module

This module allows to estimate the feature relevance of inputs with respect to a given model. While these models can be blackbox prediction models, it is also possible to explain causal mechanisms with respect to the direct parents. In these cases, it would be possible to incorporate the noise to represent the part of the generation process that cannot be explained by the parents.

Functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

dowhy.gcm.feature.feature_relevance_distribution(prediction_method: Callable[[ndarray], ndarray], feature_samples: ndarray, subset_scoring_func: Callable[[ndarray, ndarray], Union[ndarray, float]], max_num_samples_randomization: int = 5000, max_num_baseline_samples: int = 500, max_batch_size: int = 100, randomize_features_jointly: bool = True, shapley_config: Optional[ShapleyConfig] = None) → ndarray[source]

Estimates the population based feature relevance of the input features for the given prediction_method. This method uses all samples given in feature_samples by comparing the output of the prediction_method given certain features are randomized with the outputs when no features are randomized. The subset_scoring_func defines how these predictions are compared. For instance, the variance of deviations.

If the randomized predictions should rather be compared to the original data, this has (and can) be defined via the set function by ignoring the second input parameter (the predicted values using all feauters). Instead, the original data can be used.

Note: The distribution level relevance is estimated by taking the expectation of the outcome of the set functions when applied to multiple samples. Due to the linearity of the Shapley value estimation, this is equivalent to taking the expectation over the Shapley values.

Related paper: Janzing, D., Minorics, L., & Bloebaum, P. (2020). Feature relevance quantification in explainable AI: A causal problem. In International Conference on Artificial Intelligence and Statistics (pp. 2907-2916). PMLR.

Parameters

  • prediction_method – A callable that is expected to return a prediction for given samples.

  • feature_samples – Samples from the joint distribution.

  • subset_scoring_func – Set function for estimating the quantity of interest based on the model outcomes. This function expects two inputs; the outcome of the prediction model for some samples if certain features are permuted and the outcome of the model for the same samples when no features were permuted. The set functions represents the comparison between the samples, for instance, the variance of deviations. This is then used as the ‘characteristic function’ in coalition games when estimating the Shapley values.

  • max_num_samples_randomization – Maximum number of samples used for randomizing the feature that are not in the susbet. Consider increasing this number for more accurate results (if enough samples are available) or reducing it for less memory consumption and faster runtime.

  • max_num_baseline_samples – Maximum number of samples on which the set function is evaluated on. These samples are used as fixed observations for features that are in the subset. For instance, in case of taking the mean as set_function_summary_func, this defines the maximum number of samples used to estimate the mean. Consider increasing this number for more accurate results (if enough samples are available) or reducing it for less memory consumption and faster runtime.

  • max_batch_size – Maximum batch size for a estimating the predictions. This has a significant influence on the overall memory usage. If set to -1, all samples are used in one batch.

  • randomize_features_jointly – If set to True, features that are not in a subset are jointly permuted. Note that this still represents an interventional distribution. If set to False, features that are not in a subset are independently permuted. Note: The theory in the linked publication assumes that this is set to True.

  • shapley_config – Config for the Shapley estimator.

Returns

A numpy array with the feature relevance of each input feature.

dowhy.gcm.feature.feature_relevance_sample(prediction_method: Callable[[ndarray], ndarray], feature_samples: ndarray, baseline_samples: ndarray, subset_scoring_func: Callable[[ndarray, ndarray], Union[ndarray, float]], baseline_target_values: Optional[ndarray] = None, average_set_function: bool = False, max_batch_size: int = 100, randomize_features_jointly: bool = True, shapley_config: Optional[ShapleyConfig] = None) → ndarray[source]

Estimates the feature relevance of the prediction_method for each sample in baseline_noise_samples. This method uses all samples given in feature_samples as ‘background’ samples. This is, they should represent samples from the joint distribution of the input features. The subset_scoring_func defines the comparison between the output of the prediction_method when certain features are randomized and the outputs when no features are randomized. The most common function would be the difference between the expectations.

If the randomized predictions should rather be compared to the original data, this has (and can) be defined via the set function by ignoring the second input parameter (the predicted values using all feauters). Instead, the original data can be used.

Related paper: Janzing, D., Minorics, L., & Bloebaum, P. (2020). Feature relevance quantification in explainable AI: A causal problem. In International Conference on Artificial Intelligence and Statistics (pp. 2907-2916). PMLR.

Parameters

  • prediction_method – A callable that is expected to return a prediction for given samples.

  • feature_samples – Samples from the joint distribution. These are used as ‘background samples’ to randomize features that are not in a subset.

  • baseline_samples – Samples for which the feature relevance should be estimated.

  • subset_scoring_func – Set function for estimating the quantity of interest based on the model outcomes. This function expects two inputs; the outcome of the prediction model for some samples if certain features are permuted and the outcome of the model for the same samples when no features were permuted. A typical choice for regression models would be the difference between expectations. This is then used as the ‘characteristic function’ in coalition games when estimating the Shapley values.

  • baseline_target_values – These baseline values are compared with the subset specific outcomes of the prediction method. If set to None (default), the baseline values are the outcomes of the given prediction_method applied to the baseline_noise_samples, i.e. the outcome of the empty subset.

  • max_batch_size – Maximum batch size for a estimating the predictions. This has a significant influence on the overall memory usage. If set to -1, all samples are used in one batch.

  • average_set_function – If set to True, the averaged result of the set function applied to each sample of interest is used for estimating the Shapley values. If set to False, Shapley values for each sample of interest are estimated separately.

  • randomize_features_jointly – If set to True, features that are not in a subset are jointly permuted. Note that this still represents an interventional distribution. If set to False, features that are not in a subset are independently permuted. Note: The theory in the linked publication assumes that this is set to True.

  • shapley_config – Config for the Shapley estimator.

Returns

A numpy array with the feature relevance for each sample in baseline_noise_samples.

dowhy.gcm.feature.parent_relevance(causal_model: StructuralCausalModel, target_node: Any, parent_samples: Optional[DataFrame] = None, subset_scoring_func: Optional[Callable[[ndarray, ndarray], Union[ndarray, float]]] = None, num_samples_randomization: int = 5000, num_samples_baseline: int = 500, max_batch_size: int = 100, shapley_config: Optional[ShapleyConfig] = None) → Tuple[Dict[Any, Any], ndarray][source]

Estimates the distribution based relevance of the direct parents of the given target_node. This is, the relevance of direct parents as input features of the the underlying causal model of target_node. Here, the unobserved noise is considered as a direct parent (input) as well. Samples utilized for the estimation are drawn from the given causal graph.

By default, the used subset_scoring_func is based on the variance between Y and Y’, where Y is the outputs of the causal model and Y’ the outputs of the models when certain features are randomized. In case of continuous data, the feature relevance adds up to Var(Y - Y’).

Note: The feature relevance based on the distribution cannot be directly compared with the feature relevance for single samples. If this is desired, the set function needs to be defined accordingly.

Related paper: Janzing, D., Minorics, L., & Bloebaum, P. (2020). Feature relevance quantification in explainable AI: A causal problem. In International Conference on Artificial Intelligence and Statistics (pp. 2907-2916). PMLR.

Parameters

  • causal_model – The fitted structural causal model.

  • target_node – Node with the causal model of interest.

  • parent_samples – Samples for the parents of the given target_node. If None is given, new samples are generated based on the graph. These samples are used for randomizing features that are not in the subset.

  • subset_scoring_func – Set function for estimating the quantity of interest based on the model outcomes. This function expects two inputs; the outcome of the causal model for some samples if certain features are permuted and the outcome of the model for the same samples when no features were permuted. The set functions represents the comparison between the samples, for instance, the variance of deviations. This is then used as the ‘characteristic function’ in coalition games when estimating the Shapley values.

  • num_samples_randomization – Number of samples used as background parent samples for evaluating the set function. If no parent_samples are given, this represents the number of generated samples from the joint distribution of the parents and are used for randomizing features that are not in the subset. Consider increasing this number for more accurate results or reducing it for less memory consumption and faster runtime.

  • num_samples_baseline – Number of samples on which the set functions are evaluated on. These samples are used as fixed observations for parents that are in the subset. Consider increasing this number for more accurate results or reducing it for less memory consumption and faster runtime.

  • max_batch_size – Maximum batch size for estimating multiple predictions at once. This has a significant influence on the overall memory usage. If set to -1, all samples are used in one batch.

  • shapley_config – ShapleyConfig for the Shapley estimator.

Returns

There are two return vales. A dictionary with the feature relevance for each direct parent of the given target_node and the feature relevance of noise.

dowhy.gcm.fitting_sampling module

This module provides functionality for fitting probabilistic causal models and drawing samples from them.

Functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

dowhy.gcm.fitting_sampling.draw_samples(causal_model: ProbabilisticCausalModel, num_samples: int) → DataFrame[source]

Draws new joint samples from the given graphical causal model. This is done by first generating random samples from root nodes and then propagating causal downstream effects through the graph.

Parameters

  • causal_model – New samples are generated based on the given causal model.

  • num_samples – Number of samples to draw.

Returns

A pandas data frame where columns correspond to the nodes in the graph and rows to the drawn joint samples.

dowhy.gcm.fitting_sampling.fit(causal_model: ProbabilisticCausalModel, data: DataFrame)[source]

Learns generative causal models of nodes in the causal graph from data.

Parameters

  • causal_model – The causal model containing the mechanisms that will be fitted.

  • data – Observations of nodes in the causal model.

dowhy.gcm.fitting_sampling.fit_causal_model_of_target(causal_model: ProbabilisticCausalModel, target_node: Any, training_data: DataFrame) → None[source]

Fits only the causal mechanism of the given target node based on the training data.

Parameters

  • causal_model – The causal model containing the target node.

  • target_node – Target node for which the mechanism is fitted.

  • training_data – Training data for fitting the causal mechanism.

Returns

None

dowhy.gcm.graph module

This module defines the fundamental interfaces and functions related to causal graphs in graphical causal models.

Classes and functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

class dowhy.gcm.graph.ConditionalStochasticModel[source]

Bases: ABC

A conditional stochastic model represents a model used for causal mechanisms for non-root nodes in a graphical causal model.

abstract clone()[source]
abstract draw_samples(parent_samples: ndarray) → ndarray[source]

Draws samples for the fitted model.

abstract fit(X: ndarray, Y: ndarray) → None[source]

Fits the model according to the data.

class dowhy.gcm.graph.DirectedGraph(*args, **kwargs)[source]

Bases: HasNodes, HasEdges, Protocol

A protocol representing a directed graph as needed by graphical causal models.

This protocol specifically defines a subset of the networkx.DiGraph class, which make that class automatically compatible with DirectedGraph. While in most cases a networkx.DiGraph is the class of choice when constructing a causal graph, anyone can choose to provide their own implementation of the DirectGraph interface.

abstract predecessors(node)[source]
class dowhy.gcm.graph.FunctionalCausalModel[source]

Bases: ConditionalStochasticModel

Represents a Functional Causal Model (FCM), a specific type of conditional stochastic model, that is defined as:

Y := f(X, N), N: Noise

abstract draw_noise_samples(num_samples: int) → ndarray[source]
draw_samples(parent_samples: ndarray) → ndarray[source]

Draws samples for the fitted model.

abstract evaluate(parent_samples: ndarray, noise_samples: ndarray) → ndarray[source]
class dowhy.gcm.graph.HasEdges(*args, **kwargs)[source]

Bases: Protocol

This protocol defines a trait for classes having edges.

abstract property edges

:returns a Dict[Tuple[Any, Any], Dict[Any, Any]]

class dowhy.gcm.graph.HasNodes(*args, **kwargs)[source]

Bases: Protocol

This protocol defines a trait for classes having nodes.

abstract property nodes

:returns Dict[Any, Dict[Any, Any]]

class dowhy.gcm.graph.InvertibleFunctionalCausalModel[source]

Bases: FunctionalCausalModel, ABC

abstract estimate_noise(target_samples: ndarray, parent_samples: ndarray) → ndarray[source]
class dowhy.gcm.graph.StochasticModel[source]

Bases: ABC

A stochastic model represents a model used for causal mechanisms for root nodes in a graphical causal model.

abstract clone()[source]
abstract draw_samples(num_samples: int) → ndarray[source]

Draws samples for the fitted model.

abstract fit(X: ndarray) → None[source]

Fits the model according to the data.

dowhy.gcm.graph.clone_causal_models(source: HasNodes, destination: HasNodes)[source]
dowhy.gcm.graph.get_ordered_predecessors(causal_graph: DirectedGraph, node: Any) → List[Any][source]

This function returns predecessors of a node in a well-defined order.

This is necessary, because we select subsets of columns in Dataframes by using a node’s parents, and these parents might not be returned in a reliable order.

dowhy.gcm.graph.is_root_node(causal_graph: DirectedGraph, node: Any) → bool[source]
dowhy.gcm.graph.node_connected_subgraph_view(g: DirectedGraph, node: Any) → Any[source]

Returns a view of the provided graph g that contains only nodes connected to the node passed in

dowhy.gcm.graph.validate_acyclic(causal_graph: DirectedGraph) → None[source]
dowhy.gcm.graph.validate_causal_dag(causal_graph: DirectedGraph) → None[source]
dowhy.gcm.graph.validate_causal_graph(causal_graph: DirectedGraph) → None[source]
dowhy.gcm.graph.validate_causal_model_assignment(causal_graph: DirectedGraph, target_node: Any) → None[source]
dowhy.gcm.graph.validate_local_structure(causal_graph: DirectedGraph, node: Any) → None[source]
dowhy.gcm.graph.validate_node(causal_graph: DirectedGraph, node: Any) → None[source]
dowhy.gcm.graph.validate_node_has_causal_model(causal_graph: HasNodes, node: Any) → None[source]
dowhy.gcm.graph.validate_node_in_graph(causal_graph: HasNodes, node: Any) → None[source]

dowhy.gcm.influence module

This module provides functions to estimate causal influences.

Functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

dowhy.gcm.influence.arrow_strength(causal_model: ProbabilisticCausalModel, target_node: Any, parent_samples: Optional[DataFrame] = None, num_samples_conditional: int = 1000, max_num_runs: int = 5000, tolerance: float = 0.0001, n_jobs: int = 1, difference_estimation_func: Optional[Callable[[ndarray, ndarray], Union[ndarray, float]]] = None) → Dict[Tuple[Any, Any], float][source]

Computes the causal strength of each edge directed to the target node. The strength of an edge is quantified in terms of distance between conditional distributions of the target node in the original graph and the imputed graph wherein the edge has been removed and the target node is fed a random permutation of the observations of the source node. For more scientific details behind this API, please refer to the research paper below.

Research Paper: Dominik Janzing, David Balduzzi, Moritz Grosse-Wentrup, Bernhard Schölkopf. Quantifying Causal Influences. The Annals of Statistics, Vol. 41, No. 5, 2324-2358, 2013.

Parameters

  • causal_model – The probabilistic causal model for whose target node we compute the strength of incoming edges for.

  • target_node – The target node whose incoming edges’ strength is to be computed.

  • parent_samples – Optional samples from the parents of the target_node. If None are given, they are generated based on the provided causal model. Providing observational data can help to mitigate misspecifications in the graph, such as missing interactions between root nodes or confounders.

  • num_samples_conditional – Sample size to use for estimating the distance between distributions.

  • max_num_runs – The maximum number of times to resample and estimate the strength to report the average strength.

  • tolerance – The difference in average strength between two successive runs to terminate early without running it max_num_runs times.

  • n_jobs – The number of jobs to run in parallel. Set it to -1 to use all processors.

  • difference_estimation_func – Optional: How to measure the distance between two distributions. By default, the difference of the variance is estimated for a continuous target node and the KL divergence for a categorical target node.

Returns

Causal strength of each edge.

dowhy.gcm.influence.arrow_strength_of_model(conditional_stochastic_model: ConditionalStochasticModel, input_samples: ndarray, num_samples_from_conditional: int = 1000, max_num_runs: int = 5000, tolerance: float = 0.0001, n_jobs: int = 1, difference_estimation_func: Optional[Callable[[ndarray, ndarray], Union[ndarray, float]]] = None, input_subsets: Optional[List[List[int]]] = None) → ndarray[source]
dowhy.gcm.influence.intrinsic_causal_influence(causal_model: StructuralCausalModel, target_node: Any, prediction_model: Union[PredictionModel, ClassificationModel, str] = 'approx', attribution_func: Optional[Callable[[ndarray, ndarray], float]] = None, num_training_samples: int = 100000, num_samples_randomization: int = 7500, num_samples_baseline: int = 1000, max_batch_size: int = 100, auto_assign_quality: AssignmentQuality = AssignmentQuality.GOOD, shapley_config: Optional[ShapleyConfig] = None) → Dict[Any, float][source]

Computes the causal contribution of each upstream noise term of the target node (including the noise of the target itself) to the statistical property (e.g. mean, variance) of the target. We call this contribution intrinsic as noise terms, by definition, do not inherit properties of observed parents. The contribution of each noise term is then the intrinsic causal contribution of the corresponding node. For more scientific details, please refer to the paper below.

Research Paper: Janzing et al. Quantifying causal contributions via structure preserving interventions. arXiv:2007.00714, 2021.

Parameters

  • causal_model – The structural causal model for whose target node we compute the intrinsic causal influence of its ancestors.

  • target_node – Target node whose statistical property is to be attributed.

  • prediction_model – Prediction model for estimating the functional relationship between subsets of ancestor noise terms and the target node. This can be an instance of a PredictionModel, the string ‘approx’ or the string ‘exact’. With ‘exact’, the underlying causal models in the graph are utilized directly by propagating given noise inputs through the graph. This is generally more accurate but slow. With ‘approx’, an appropriate model is selected and trained based on sampled data from the graph, which is less accurate but faster. A more detailed treatment on why we need this parameter is also provided in icc.

  • attribution_func – Optional attribution function to measure the statistical property of the target node. This function expects two inputs; predictions after the randomization of certain features (i.e. samples from noise nodes) and a baseline where no features were randomized. The baseline predictions can be typically ignored if one is interested in uncertainty measures such as entropy or variance, but they might be relevant if, for instance, these shall be estimated based on the residuals. By default, entropy is used if prediction model is a classifier, variance otherwise.

  • num_training_samples – Number of samples drawn from the graphical causal model that are used for fitting the prediction_model (if necessary).

  • num_samples_randomization – Number of noise samples drawn from the graphical causal model that are used for evaluating the set function. Here, these samples are samples from the noise distributions used for randomizing features that are not in the subset.

  • num_samples_baseline – Number of noise samples drawn from the graphical causal model that are used for evaluating the set function. Here, these samples are used as fixed observations for features that are in the subset.

  • max_batch_size – Maximum batch size for estimating the predictions from evaluation samples. This has a significant impact on the overall memory usage. If set to -1, all samples are used in one batch.

  • auto_assign_quality – Auto assign quality for the ‘approx’ prediction_model option.

  • shapley_config – ShapleyConfig for the Shapley estimator.

Returns

Intrinsic causal contribution of each ancestor node to the statistical property defined by the attribution_func of the target node.

dowhy.gcm.shapley module

This module provides functionality for shapley value estimation.

Classes and functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

class dowhy.gcm.shapley.ShapleyApproximationMethods(value)[source]

Bases: Enum

AUTO: Using EXACT when number of players is below 6 and EARLY_STOPPING otherwise. EXACT: Generate all possible subsets and estimate Shapley values with corresponding subset weights. EXACT_FAST: Generate all possible subsets and estimate Shapley values via weighed least squares regression. This can

be faster, but, depending on the set function, numerically less stable.

SUBSET_SAMPLING: Randomly samples subsets and estimate Shapley values via weighed least squares regression. Here,

only a certain number of randomly drawn subsets are used.

EARLY_STOPPING: Estimate Shapley values based on a few randomly generated permutations. Stop the estimation process

when the the Shapley values do not change much on average anymore between runs.

PERMUTATION: Estimates Shapley values based on a fixed number of randomly generated permutations. By fine tuning

hyperparameters, this can be potentially faster than the early stopping approach due to a better utilization of the parallelization.

AUTO = (0,)
EARLY_STOPPING = (3,)
EXACT = (1,)
EXACT_FAST = (2,)
PERMUTATION = (4,)
SUBSET_SAMPLING = 5
class dowhy.gcm.shapley.ShapleyConfig(approximation_method: ShapleyApproximationMethods = ShapleyApproximationMethods.AUTO, num_samples: int = 5000, min_percentage_change_threshold: float = 0.01, n_jobs: Optional[int] = None)[source]

Bases: object

Config for estimating Shapley values.

Parameters

  • approximation_method – Type of approximation methods (see ShapleyApproximationMethods).

  • num_samples – Number of samples used for approximating the Shapley values. Depending on the approximation method, this can either represent the number of drawn subsets (in SUBSET_SAMPLING) or the number of drawn permutations (in EARLY_STOPPING and PERMUTATION). In case of EARLY_STOPPING, this also represents a limit on the evaluation runs.

  • min_percentage_change_threshold – This parameter is only relevant for EARLY_STOPPING and indicates the minimum required change of the Shapley values between two runs (i.e. evaluation of permutations) before the estimation stops. For instance, if the Shapley value changes less than the given value for a certain number of consecutive runs, the algorithm stops and returns the current result.

  • n_jobs – Number of parallel jobs.

dowhy.gcm.shapley.estimate_shapley_values(set_func: Callable[[ndarray], Union[float, ndarray]], num_players: int, shapley_config: Optional[ShapleyConfig] = None) → ndarray[source]

Estimates the Shapley values based on the provided set function. A set function here is defined by taking a (subset) of players and returning a certain utility value. This is in the context of attributing the value of the i-th player to a subset of players S by evaluating v(S u {i}) - v(S), where v is the set function and i is not in S. While we use the term ‘player’ here, this is often a certain feature/variable.

The input of the set function is a binary vector indicating which player is part of the set. For instance, given 4 players (1,2,3,4) and a subset only contains players 1,2,4, then this is indicated by the vector [1, 1, 0, 1]. The function is expected to return a numeric value based on this input.

Note: The set function can be arbitrary and can resemble computationally complex operations. Keep in mind that the estimation of Shapley values can become computationally expensive and requires a lot of memory. If the runtime is too slow, consider changing the default config.

Parameters

  • set_func – A set function that expects a binary vector as input which specifies which player is part of the subset.

  • num_players – Total number of players.

  • shapley_config – A config object for indicating the approximation method and other parameters. If None is given, a default config is used. For faster runtime or more accurate results, consider creating a custom config.

Returns

A numpy array representing the Shapley values for each player, i.e. there are as many Shapley values as num_players. The i-th entry belongs to the i-th player. Here, the set function defines which index belongs to which player and is responsible to keep it consistent.

dowhy.gcm.stats module

Functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

dowhy.gcm.stats.estimate_ftest_pvalue(X_training_a: ndarray, X_training_b: ndarray, Y_training: ndarray, X_test_a: ndarray, X_test_b: ndarray, Y_test: ndarray) → float[source]

Estimates the p-value for the null hypothesis that the same regression error with less parameters can be achieved. This is, a linear model trained on a data set A with d number of features has the same performance (in terms of squared error) relative to the number of features as a model trained on a data set B with k number features, where k < d. Here, both data sets need to have the same target values. A small p-value would indicate that the model performances are significantly different.

Note that all given test samples are utilized in the f-test.

See https://en.wikipedia.org/wiki/F-test#Regression_problems for more details.

Parameters

  • X_training_a – Input training samples for model A.

  • X_training_b – Input training samples for model B. These samples should have less features than samples in X_training_a.

  • Y_training – Target training values.

  • X_test_a – Test samples for model A.

  • X_test_b – Test samples for model B.

  • Y_test – Test values.

Returns

A p-value on [0, 1].

dowhy.gcm.stats.marginal_expectation(prediction_method: Callable[[ndarray], ndarray], feature_samples: ndarray, baseline_samples: ndarray, baseline_feature_indices: List[int], return_averaged_results: bool = True, feature_perturbation: str = 'randomize_columns_jointly', max_batch_size: int = -1) → ndarray[source]

Estimates the marginal expectation for samples in baseline_noise_samples when randomizing features that are not part of baseline_feature_indices. This is, this function estimates

y^i = E[Y | do(x^i_s)] := int_x_s’ E[Y | x^i_s, x_s’] p(x_s’) d x_s’,

where x^i_s is the i-th sample from baseline_noise_samples, s denotes the baseline_feature_indices and x_s’ ~ X_s’ denotes the randomized features that are not in s. For an approximation of the integral, the given prediction_method is evaluated multiple times for the same x^i_s, but different x_s’ ~ X_s’.

Parameters

prediction_method – Prediction method of interest. This should expect a numpy array as input for making

predictions. :param feature_samples: Samples from the joint distribution. These are used for randomizing the features that are not in

baseline_feature_indices.

Parameters

  • baseline_samples – Samples for which the marginal expectation should be estimated.

  • baseline_feature_indices – Column indices of the features in s. These values for these features are remain constant when estimating the expectation.

  • return_averaged_results – If set to True, the expectation over all evaluated samples for the i-th

baseline_noise_samples is returned. If set to False, all corresponding results for the i-th sample are returned. :param feature_perturbation: Type of feature permutation:

‘randomize_columns_independently’: Each feature not in s is randomly permuted separately. ‘randomize_columns_jointly’: All features not in s are jointly permuted. Note that this still represents an interventional distribution.

Parameters

max_batch_size – Maximum batch size for a estimating the predictions. This has a significant influence on the

overall memory usage. If set to -1, all samples are used in one batch. :return: If return_averaged_results is False, a numpy array where the i-th entry belongs to the marginal expectation of x^i_s when randomizing the remaining features. If return_averaged_results is True, a two dimensional numpy array where the i-th entry contains all predictions for x^i_s when randomizing the remaining features.

dowhy.gcm.stats.permute_features(feature_samples: ndarray, features_to_permute: Union[List[int], ndarray], randomize_features_jointly: bool) → ndarray[source]
dowhy.gcm.stats.quantile_based_fwer(p_values: Union[ndarray, List[float]], p_values_scaling: Optional[ndarray] = None, quantile: float = 0.5) → float[source]

Applies a quantile based family wise error rate (FWER) control to the given p-values. This is based on the approach described in:

Meinshausen, N., Meier, L. and Buehlmann, P. (2009). p-values for high-dimensional regression. J. Amer. Statist. Assoc.104 1671–1681

Parameters

  • p_values – A list or array of p-values.

  • p_values_scaling – An optional list of scaling factors for each p-value.

  • quantile – The quantile used for the p-value adjustment. By default, this is the median (0.5).

Returns

The p-value that lies on the quantile threshold. Note that this is the quantile based on scaled values p_values / quantile.

dowhy.gcm.stochastic_models module

This module defines multiple implementations of the abstract class StochasticModel.

Classes in this module should be considered experimental, meaning there might be breaking API changes in the future.

class dowhy.gcm.stochastic_models.BayesianGaussianMixtureDistribution[source]

Bases: StochasticModel

clone()[source]
draw_samples(num_samples: int) → ndarray[source]

Draws samples for the fitted model.

fit(X: ndarray) → None[source]

Fits the model according to the data.

class dowhy.gcm.stochastic_models.EmpiricalDistribution[source]

Bases: StochasticModel

An implementation of a stochastic model that uniformly samples from data samples. By randomly returning a sample from the training data set, this model represents a parameter free representation of the marginal distribution of the training data. However, it will not generate unseen data points. For this, consider BayesianGaussianMixtureDistribution.

clone()[source]
property data: ndarray
draw_samples(num_samples: int) → ndarray[source]

Draws samples for the fitted model.

fit(X: ndarray) → None[source]

Fits the model according to the data.

class dowhy.gcm.stochastic_models.ScipyDistribution(scipy_distribution: Optional[Union[rv_continuous, rv_discrete]] = None, **parameters)[source]

Bases: StochasticModel

Represents any parametric distribution that can be modeled by scipy.

Initializes a stochastic model that allows to sample from a parametric distribution implemented in Scipy.

For instance, to use a beta distribution with parameters a=2 and b=0.5:

ScipyDistribution(stats.beta, a=2, b=0.5)

Or a Gaussian distribution with mean=0 and standard deviation 2:

ScipyDistribution(stats.norm, loc=2, scale=0.5)

Note that the parameter names need to coincide with the parameter names in the corresponding Scipy implementations. See https://docs.scipy.org/doc/scipy/tutorial/stats.html for more information.

Parameters

  • scipy_distribution – A continuous or discrete distribution parametric distribution implemented in Scipy.

  • parameters – Set of parameters of the parametric distribution.

clone()[source]
draw_samples(num_samples: int) → ndarray[source]

Draws samples for the fitted model.

static find_suitable_continuous_distribution(distribution_samples: ndarray, divergence_threshold: float = 0.01) → Tuple[rv_continuous, Dict[str, float]][source]

Tries to find the best fitting continuous parametric distribution of given samples. This is done by fitting different parametric models and selecting the one with the smallest KL divergence between observed and generated samples.

fit(X: ndarray) → None[source]

Fits the model according to the data.

static map_scipy_distribution_parameters_to_names(scipy_distribution: Union[rv_continuous, rv_discrete], parameters: Tuple[float]) → Dict[str, float][source]

Helper function to obtain a mapping from parameter name to parameter value. Depending whether the distribution is discrete or continuous, there are slightly different parameter names. The given parameters are assumed to follow the order as provided by the scipy fit function.

Parameters

  • scipy_distribution – The scipy distribution.

  • parameters – The values of the corresponding parameters of the distribution. Here, it is expected to follow the same order as defined by the scipy fit function.

Returns

A dictionary that maps a parameter name to its value.

property parameters: Dict[str, float]
property scipy_distribution: Optional[Union[rv_continuous, rv_discrete]]

dowhy.gcm.uncertainty module

Functions to estimate uncertainties such as entropy, KL divergence etc.

Functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

dowhy.gcm.uncertainty.estimate_entropy_discrete(X: ndarray) → float[source]

Estimates the entropy assuming the data in X is discrete.

Parameters

X – Discrete samples.

Returns

Entropy of X.

dowhy.gcm.uncertainty.estimate_entropy_kmeans(X: ndarray) → float[source]

Related paper: Kozachenko, L., & Leonenko, N. (1987). Sample estimate of the entropy of a random vector. Problemy Peredachi Informatsii, 23(2), 9–16.

dowhy.gcm.uncertainty.estimate_entropy_of_probabilities(X: ndarray) → float[source]

Estimates the entropy of each probability vector (row wise) in X separately and returns the mean over all results.

dowhy.gcm.uncertainty.estimate_entropy_using_discretization(X: ndarray, bin_width: float = 1) → float[source]
dowhy.gcm.uncertainty.estimate_gaussian_entropy(X: ndarray) → float[source]

Entropy with respect to standardized variables.

dowhy.gcm.uncertainty.estimate_variance(X: ndarray) → float[source]

dowhy.gcm.unit_change module

This module provides the APIs for attributing the change in the output value of a deterministic mechanism for a statistical unit.

class dowhy.gcm.unit_change.LinearPredictionModel[source]

Bases: object

abstract property coefficients: ndarray
class dowhy.gcm.unit_change.SklearnLinearRegressionModel(sklearn_mdl: LinearModel)[source]

Bases: SklearnRegressionModel, LinearPredictionModel

property coefficients: ndarray
dowhy.gcm.unit_change.unit_change(background_df: DataFrame, foreground_df: DataFrame, input_column_names: List[str], background_mechanism: PredictionModel, foreground_mechanism: Optional[PredictionModel] = None, shapley_config: Optional[ShapleyConfig] = None) → DataFrame[source]

This function attributes the change in the output value of a deterministic mechanism for a statistical unit to each input and optionally for the mechanism if foreground_mechanism is provided. The technical method is described in the following research paper: Kailash Budhathoki, George Michailidis, Dominik Janzing. Explaining the root causes of unit-level changes. arXiv, 2022.

Parameters

  • background_df – The background dataset.

  • foreground_df – The foreground dataset.

  • input_column_names – The names of the input columns.

  • background_mechanism – The background mechanism. If the mechanism does not change, then this mechanism is used for attribution.

  • foreground_mechanism – The foreground mechanism. If provided, the method also attributes the output change to the change in the mechanism.

  • shapley_config – The configuration for calculating Shapley values.

Returns

A dataframe containing the contributions of each input and optionally the mechanism to the change in the output values of the deterministic mechanism(s) for given inputs.

dowhy.gcm.unit_change.unit_change_linear(background_mechanism: LinearPredictionModel, background_df: DataFrame, foreground_mechanism: LinearPredictionModel, foreground_df: DataFrame, input_column_names: List[str]) → DataFrame[source]

Calculates the contributions of mechanism and each input to the change in the output values of a linear deterministic mechanism.

Parameters

  • background_mechanism – The linear background mechanism.

  • background_df – The background data.

  • foreground_mechanism – The linear foreground mechanism.

  • foreground_df – The foreground data.

  • input_column_names – The names of the input columns in both dataframes.

Returns

A pandas dataframe with attributions to each cause for the change in each output row of provided dataframes.

dowhy.gcm.unit_change.unit_change_linear_input_only(mechanism: LinearPredictionModel, background_df: DataFrame, foreground_df: DataFrame, input_column_names: List[str]) → DataFrame[source]

Calculates the contributions of each input to the change in the output values of a linear deterministic mechanism.

Parameters

  • mechanism – The linear mechanism.

  • background_df – The background data.

  • foreground_df – The foreground data.

  • input_column_names – The names of the input (features) columns in both dataframes.

Returns

A pandas dataframe with attributions to each cause for the change in each output row of provided dataframes.

dowhy.gcm.unit_change.unit_change_nonlinear(background_mechanism: PredictionModel, background_df: DataFrame, foreground_mechanism: PredictionModel, foreground_df: DataFrame, input_column_names: List[str], shapley_config: Optional[ShapleyConfig] = None) → DataFrame[source]

Calculates the contributions of mechanism and each input to the change in the output values of a non-linear deterministic mechanism. The technical method is described in the following research paper: Kailash Budhathoki, George Michailidis, Dominik Janzing. Explaining the root causes of unit-level changes. arXiv, 2022.

Parameters

  • background_mechanism – The background mechanism.

  • background_df – The background data.

  • foreground_mechanism – The foreground mechanism.

  • foreground_df – The foreground data.

  • input_column_names – The names of the input (features) columns in both dataframes.

  • shapley_config – The configuration for calculating Shapley values.

Returns

A pandas dataframe with attributions to each cause for the change in each output row of provided dataframes.

dowhy.gcm.unit_change.unit_change_nonlinear_input_only(mechanism: PredictionModel, background_df: DataFrame, foreground_df: DataFrame, input_column_names: List[str], shapley_config: Optional[ShapleyConfig] = None) → DataFrame[source]

Calculates the contributions of each input to the change in the output values of a non-linear deterministic mechanism. The technical method is a modification of the attribution method described in the following research paper, without mechanism as a player: Kailash Budhathoki, George Michailidis, Dominik Janzing. Explaining the root causes of unit-level changes. arXiv, 2022.

Parameters

  • mechanism – The mechanism.

  • background_df – The background data.

  • foreground_df – The foreground data.

  • input_column_names – The names of the input (features) columns in both dataframes.

  • shapley_config – The configuration for calculating Shapley values.

Returns

A pandas dataframe with attributions to each cause for the change in each output row of provided dataframes.

dowhy.gcm.validation module

Contains a method to reject the causal graph and validate causal mechanisms such as post non-linear models.

Classes and functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

class dowhy.gcm.validation.RejectionResult(value)[source]

Bases: Enum

An enumeration.

NOT_REJECTED = (<enum.auto object>,)
REJECTED = 1
dowhy.gcm.validation.refute_causal_structure(causal_graph: ~dowhy.gcm.graph.DirectedGraph, data: ~pandas.core.frame.DataFrame, independence_test: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray], float] = <function kernel_based>, conditional_independence_test: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray, ~numpy.ndarray], float] = <function kernel_based>, significance_level: float = 0.05, fdr_control_method: ~typing.Optional[str] = 'fdr_bh') → Tuple[RejectionResult, Dict[str, Dict[str, Dict[str, Union[bool, float, Dict[str, Union[bool, float]]]]]]][source]

Validates the assumptions in a causal graph against data. To this end, at each node, we test if the node is dependent on each of its parents, and test the local Markov condition. Note that valid local Markov conditions also imply a valid global Markov condition.

Parameters

  • causal_graph – A directed acyclic graph (DAG).

  • data – Observations of variables in the DAG.

  • independence_test – Independence test to use for checking edge dependencies.

  • conditional_independence_test – Conditional independence test to use for checking local Markov condition.

  • significance_level – Significance level for (conditional) independence tests.

  • fdr_control_method – Method for false discovery rate (FDR) control. For various options, please refer to this page.

Returns

Outcome of the validation process. The first element of the tuple indicates whether the graph is valid w.r.t. given data, and the second element gives the summary of tests at each node. An example for X->Y->Z:

[True, {'X': {'local_markov_test': {}, 'edge_dependence_test': {}},
        'Y': {'local_markov_test': {}, 'edge_dependence_test': {'X': {'p_value': 0.5, 'fdr_adjusted_p_value': 0.5, 'success': True}}},
        'Z': {'local_markov_test': {'p_value': 0.0, 'fdr_adjusted_p_value': 0.5, 'success': False},
              'edge_dependence_test': {'Y': {'p_value': 0.5, 'fdr_adjusted_p_value': 0.5, 'success': True}}}}]
dowhy.gcm.validation.refute_invertible_model(causal_model: ~dowhy.gcm.cms.InvertibleStructuralCausalModel, data: ~pandas.core.frame.DataFrame, independence_test: ~typing.Callable[[~numpy.ndarray, ~numpy.ndarray], float] = <function kernel_based>, significance_level: float = 0.05, fdr_control_method: ~typing.Optional[str] = None) → RejectionResult[source]

Validate the assumption that the structural causal models can be represented by a InvertibleFunctionalCausalModel (e.g. the causal mechanisms are AdditiveNoiseModels and/or PostNonlinearModels). For this, it is checked if the residual of a causal mechanism is independent of the mechanism’s input (i.e. we assume causal sufficiency here). For instance, PostNonlinearModels represent

Y = f(g(X) + N),

where f is invertible (g does not need to be), X are the parents of Y and N is (assumed to be) independent noise. The latter point is important here. For given data, we can then reconstruct N and perform an independence test between X and N.

Note that this method only validates the causal mechanisms and not the graph structure.

For the case of post non-linear models, see the following paper for more details:

Zhang, K., and A. Hyvärinen. On the Identifiability of the Post-Nonlinear Causal Model. 25th Conference on Uncertainty in Artificial Intelligence (UAI 2009). AUAI Press, 2009.

Parameters

  • causal_model – A fitted invertible structural causal model.

  • data – Observations of variables in the DAG.

  • independence_test – Independence test to use for checking if residual and input are dependent.

  • significance_level – Significance level for deciding whether input and residual is dependent.

  • fdr_control_method –

    Method for false discovery rate (FDR) control. For various options, please refer to this page.

Returns

The outcome of the validation. The causal model can not be rejected if all causal mechanisms are consistent with the invertible model assumption.

dowhy.gcm.whatif module

This module provides functionality to answer what-if questions.

Functions in this module should be considered experimental, meaning there might be breaking API changes in the future.

dowhy.gcm.whatif.average_causal_effect(causal_model: ProbabilisticCausalModel, target_node: Any, interventions_alternative: Dict[Any, Callable[[ndarray], Union[float, ndarray]]], interventions_reference: Dict[Any, Callable[[ndarray], Union[float, ndarray]]], observed_data: Optional[DataFrame] = None, num_samples_to_draw: Optional[int] = None) → float[source]

Estimates the average causal effect (ACE) on the target of two different sets of interventions. The interventions can be specified through the parameters interventions_alternative and interventions_reference. For example, if the alternative intervention is do(T := 1) and the reference intervention is do(T := 0), then the average causal effect is given by ACE = E[Y | do(T := 1)] - E[Y | do(T := 0)]:

>>> average_causal_effect(causal_model, 'Y', {'T': lambda _ : 1}, {'T': lambda _ : 0})
We can also specify more complex interventions on multiple nodes:
>>> average_causal_effect(causal_model,
>>>                       'Y',
>>>                       {'T': lambda _ : 1, 'X0': lambda x : x + 1},
>>>                       {'T': lambda _ : 0, 'X0': lambda x : x * 2})

In the above, we would estimate ACE = E[Y | do(T := 1), do(X0 := X0 + 1)] - E[Y | do(T := 0), do(X0 := X0 * 2)].

Note: The target node can be a continuous real-valued variable or a categorical variable with at most two classes (i.e. binary).

Parameters

  • causal_model – The probabilistic causal model we perform this intervention on .

  • target_node – Target node for which the ACE is estimated.

  • interventions_alternative – Dictionary defining the interventions for the alternative values.

  • interventions_reference – Dictionary defining the interventions for the reference values.

  • observed_data – Factual data that we observe for the nodes in the causal graph. By default, new data is sampled using the causal model. If observational data is available, providing them might improve the accuracy by mitigating issues due to a misspecified graph and/or causal models.

  • num_samples_to_draw – Number of samples drawn from the causal model for estimating ACE if no observed data is given.

Returns

The estimated average causal effect (ACE).

dowhy.gcm.whatif.counterfactual_samples(causal_model: Union[StructuralCausalModel, InvertibleStructuralCausalModel], interventions: Dict[Any, Callable[[ndarray], Union[float, ndarray]]], observed_data: Optional[DataFrame] = None, noise_data: Optional[DataFrame] = None) → DataFrame[source]

Estimates counterfactual data for observed data if we were to perform specified interventions. This function implements the 3-step process for computing counterfactuals by Pearl (see https://ftp.cs.ucla.edu/pub/stat_ser/r485.pdf).

Parameters

  • causal_model – The (invertible) structural causal model we perform this intervention on. If noise_data is None and observed_data is provided, this must be an invertible structural model, otherwise, this can be either a structural causal model or an invertible one.

  • interventions – Dictionary containing the interventions we want to perform keyed by node name. An intervention is a function that takes a value as input and returns another value. For example, {‘X’: lambda x: 2} mimics the atomic intervention do(X:=2).

  • observed_data – Factual data that we observe for the nodes in the causal graph.

  • noise_data – Data of noise terms corresponding to nodes in the causal graph. If not provided, these have to be estimated from observed data. Then we require causal models of nodes to be invertible.

Returns

Estimated counterfactual data.

dowhy.gcm.whatif.interventional_samples(causal_model: ProbabilisticCausalModel, interventions: Dict[Any, Callable[[ndarray], Union[float, ndarray]]], observed_data: Optional[DataFrame] = None, num_samples_to_draw: Optional[int] = None) → DataFrame[source]

Performs intervention on nodes in the causal graph.

Parameters

  • causal_model – The probabilistic causal model we perform this intervention on .

  • interventions – Dictionary containing the interventions we want to perform, keyed by node name. An intervention is a function that takes a value as input and returns another value. For example, {‘X’: lambda x: 2} mimics the atomic intervention do(X:=2). A soft intervention can be formulated as {‘X’: lambda x: 0.2 * x}.

  • observed_data – Optionally, data on which to perform interventions. If None are given, data is generated based on the generative models.

  • num_samples_to_draw – Sample size to draw from the interventional distribution.

Returns

Samples from the interventional distribution.

Module contents

The gcm sub-package provides features built on top of graphical causal model (GCM) based inference. The status of this addition and its API is considered experimental, meaning there might be breaking changes to its API in the future.