NIPY logo

Site Navigation

NIPY Community

Table Of Contents

Previous topic

nipy.neurospin.utils.mask.compute_mask_sessions

This Page

Empirical null

The nipy.neurospin.utils.emp_null module contains a class that fits a gaussian model to the central part of an histogram, following Schwartzman et al, 2009. This is typically necessary to estimate a fdr when one is not certain that the data behaves as a standard normal under H_0.

The ENN class learns its null distribution on the data provided at initialisation. Two different methods can be used to set a threshold from the null distribution: the ENN.threshold() method returns the threshold for a given false discovery rate, and thus accounts for multiple comparisons with the given dataset; the ENN.uncorrected_threshold() returns the threshold for a given uncorrected p-value, and as such does not account for multiple comparisons.

Example

If we use the empirical normal null estimator on a two gaussian mixture distribution, with a central gaussian, and a wide one, it uses the central distribution as a null hypothesis, and returns the threshold followingr which the data can be claimed to belong to the wide gaussian:

[source code, hires.png, pdf]

../_images/enn_demo.png

The threshold evaluated with the ENN.threshold() method is around 2.8 (using the default p-value of 0.05). The ENN.uncorrected_threshold() return, for the same p-value, a threshold of 1.9. It is necessary to use a higher p-value with uncorrected comparisons.

Class documentation

class nipy.neurospin.utils.emp_null.ENN(x)

Class to compute the empirical null normal fit to the data.

The data which is used to estimate the FDR, assuming a gaussian null from Schwartzmann et al., NeuroImage 44 (2009) 71–82

Methods

fdr
fdrcurve
learn
plot
threshold
uncorrected_threshold
__init__(x)

Initiate an empirical null normal object.

Parameters :

x : 1D ndarray

The data used to estimate the empirical null.

fdr(theta)

given a threshold theta, find the estimated fdr

fdrcurve()

Returns the fdr associated with any point of self.x

learn(left=0.2, right=0.8)

Estimate the proportion, mean and variance of a gaussian distribution for a fraction of the data

Parameters :

left : float, optional

Left cut parameter to prevent fitting non-gaussian data

right : float, optional

Right cut parameter to prevent fitting non-gaussian data

Notes

This method stores the following attributes:
  • mu = mu
  • p0 = min(1, np.exp(lp0))
  • sqsigma : standard deviation of the estimated normal distribution
  • sigma = np.sqrt(sqsigma) : variance of the estimated normal distribution
plot(efp=None, alpha=0.05, bar=1, mpaxes=None)

plot the histogram of x

Parameters :

efp : float, optional

The empirical fdr (corresponding to x) if efp==None, the false positive rate threshod plot is not drawn.

alpha : float, optional

The chosen fdr threshold

bar=1 : bool, optional

mpaxes=None: if not None, handle to an axes where the fig. :

will be drawn. Avoids creating unnecessarily new figures. :

threshold(alpha=0.05, verbose=0)

Compute the threshold correponding to an alpha-level fdr for x

Parameters :

alpha : float, optional

the chosen false discovery rate threshold.

verbose : boolean, optional

the verbosity level, if True a plot is generated.

uncorrected_threshold(alpha=0.001, verbose=0)

Compute the threshold correponding to a specificity alpha for x

Parameters :

alpha : float, optional

the chosen false discovery rate threshold.

verbose : boolean, optional

the verbosity level, if True a plot is generated.


Reference: Schwartzmann et al., NeuroImage 44 (2009) 71–82