TorchDrift: drift detection for PyTorch

TorchDrift is a data and concept drift library for PyTorch. It lets you monitor your PyTorch models to see if they operate within spec.

We focus on practical application and strive to seamlessly integrate with PyTorch.

Get started:

• Installation

Examples:

• Drift detection on image classifiers
  • Load data
  • Build a model
  • Simulating drifted data
  • The drift detector
• Deploying Drift Detection

Background:

• Practical drift detection
  • What is drift?
  • How do we detect drift?
  • Dimension reduction
  • Hypothesis testing and beyond
  • Drift detection as classification
• Comparing drift detectors
  • Feature extractor
  • Simulating drifted data
  • Kernel MMD drift detector
  • Dimension Reduction & Kolmogorov-Smirnov test
  • Untrained Autoencoder
• Intuition for the Maximum Mean Discrepancy two-sample test
  • Detecting different distributions
  • Bootstrapping

torchdrift API:

• detectors
  • Kernels for MMD
• reducers
• data.functional
• utils

Authors

TorchDrift is a joint project of Orobix Srl, Bergamo, Italy and MathInf GmbH, Garching b. München, Germany.

The TorchDrift Team: Thomas Viehmann, Luca Antiga, Daniele Cortinovis, Lisa Lozza

Acknowledgements

We were inspired by

• Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift, NeurIPS 2019 https://github.com/steverab/failing-loudly

• Hendrycks & Dietterich: Benchmarking Neural Network Robustness to Common Corruptions and Perturbations ICLR 2019 https://github.com/hendrycks/robustness/

• Van Looveren et al.: Alibi Detect https://github.com/SeldonIO/alibi-detect/

Indices and tables

• Index

• Module Index