The MultipleTesting package offers common algorithms for p-value adjustment and combination and more…



The MultipleTesting package offers common algorithms for p-value adjustment and combination as well as the estimation of the proportion π₀ of true null hypotheses.

xkcd p-value guide


Adjustment of p-values

  • Bonferroni
  • Benjamini-Hochberg
  • Adaptive Benjamini-Hochberg with known π₀ or π₀ estimation method (see section below)
  • Benjamini-Yekutieli
  • Benjamini-Liu
  • Hochberg
  • Holm
  • Hommel
  • Sidak
  • Forward Stop
  • Barber-Candès adjust(pvals, Bonferroni()) adjust(pvals, BenjaminiHochberg()) adjust(pvals, BenjaminiHochbergAdaptive(0.9)) adjust(pvals, BenjaminiHochbergAdaptive(Storey())) adjust(pvals, BenjaminiYekutieli()) adjust(pvals, BenjaminiLiu()) adjust(pvals, Hochberg()) adjust(pvals, Holm()) adjust(pvals, Hommel()) adjust(pvals, Sidak()) adjust(pvals, ForwardStop()) adjust(pvals, BarberCandes())

The adjustment can also be performed on the k smallest out of n p-values:

adjust(pvals, n, PValueAdjustmentMethod)

Estimation of π₀

  • Storey
  • Storey's closed-form bootstrap
  • Least Slope
  • Two Step
  • RightBoundary (Storey's estimate with dynamically chosen λ)
  • Beta-Uniform Mixture (BUM)
  • Censored BUM
  • Flat Grenander
  • Oracle for known π₀ estimate_pi0(pvals, Storey()) estimate_pi0(pvals, StoreyBootstrap()) estimate_pi0(pvals, LeastSlope()) estimate_pi0(pvals, TwoStep()) estimate_pi0(pvals, TwoStep(0.05)) estimate_pi0(pvals, TwoStep(0.05, BenjaminiHochbergAdaptive(0.9)) estimate_pi0(pvals, RightBoundary()) estimate_pi0(pvals, CensoredBUM()) estimate_pi0(pvals, BUM()) estimate_pi0(pvals, FlatGrenander()) estimate_pi0(pvals, Oracle(0.9))

Combination of p-values

  • Fisher
  • Stouffer, optionally with weights
  • Logit
  • Tippett
  • Simes
  • Wilkinson
  • Minimum of adjusted p-values combine(pvals, FisherCombination()) combine(pvals, StoufferCombination()) combine(pvals, weights, StoufferCombination()) combine(pvals, LogitCombination()) combine(pvals, TippettCombination()) combine(pvals, SimesCombination()) combine(pvals, WilkinsonCombination(rank)) combine(pvals, MinimumCombination(PValueAdjustment()))

Higher criticism

  • Higher criticism scores
  • Higher criticism threshold estimate(pvals, HigherCriticismScores()) estimate(pvals, HigherCriticismThreshold())


The MultipleTesting package is part of the Julia ecosphere and the latest release version can be installed with


More details on packages and how to manage them can be found in the package section of the Julia documentation.

Feedback and Contributions

Contributions of any kind are very welcome. Please feel free to open pull requests or issues with your questions or ideas.

Package Status



Package Status

Linux/Mac Build Status Windows Build Status Coverage Status

The package uses semantic versioning.

First Commit


Last Touched

about 2 months ago


344 commits

Used By: