Foreword
0 Introduction
Scope
Normative references
Terms and definitions
General principles
General requirements for statistical methods
Basic model
General approaches for the evaluation of performance
Guidelines for the statistical design of proficiency testing schemes
Introduction to the statistical design of proficiency testing schemes
Basis of a statistical design
Considerations for the statistical distribution of results
Considerations for small numbers of participants
Guidelines for choosing the reporting format
General requirements for reporting format
Reporting of replicate measurements
Reporting of ‘less than’ or ‘greater than’ a limit (censored data)
Number of significant digits
Guidelines for the initial review of proficiency testing items and results
Homogeneity and stability of proficiency test items
Considerations for different measurement methods
Blunder removal
Visual review of data
Robust statistical methods
Outlier techniques for individual results
Determination of the assigned value and its standard uncertainty
Choice of method of determining the assigned value
Determining the uncertainty of the assigned value
Formulation
Certified reference material
Results from one laboratory
Consensus value from expert laboratories
Consensus value from participant results
Comparison of the assigned value with an independent reference value
Determination of criteria for evaluation of performance
Approaches for determining evaluation criteria
By perception of experts
By experience from previous rounds of a proficiency testing scheme
By use of a general model
Using the repeatability and reproducibility standard deviations from a previous collaborative study of precision of a measurement method
From data obtained in the same round of a proficiency testing scheme
Monitoring interlaboratory agreement
Calculation of performance statistics
General considerations for determining performance
Limiting the uncertainty of the assigned value
Estimates of deviation (measurement error)
z scores
z′ scores
Zeta scores (ζ)
E scores
Evaluation of participant uncertainties in testing
Combined performance scores
Graphical methods for describing performance scores
Application of graphical methods
Histograms of results or performance scores
Kernel density plots
Bar-plots of standardized performance scores
Youden plot
Plots of repeatability standard deviations
Split samples
Graphical methods for combining performance scores over several rounds of a proficiency testing scheme
Design and analysis of qualitative proficiency testing schemes (including nominal and ordinal properties)
Types of qualitative data
Statistical design
Assigned values for qualitative proficiency testing schemes
Performance evaluation and scoring for qualitative proficiency testing schemes
Symbols (normative)
Homogeneity and stability of proficiency test items (informative)
General procedure for a homogeneity check
Assessment criteria for a homogeneity check
Formulae for homogeneity check
Procedures for checking stability
General considerations for checking stability
Procedure for checking stability during the course of a proficiency testing scheme round
Assessment criterion for a stability check
Stability in transport conditions
Robust analysis (informative)
General
Simple outlier-resistant estimators for the population mean and standard deviation
The median
Scaled median absolute deviation MAD
Normalized interquartile range nIQR
Robust analysis: Algorithm A
Algorithm A with iterated scale
Variants of Algorithm A
Robust analysis: Algorithm S
Computationally intensive robust estimators: Q method and Hampel estimator
Rationale for computationally intensive estimators
Determination of a robust standard deviation using Q and Q methods
Determination of a robust mean using the Hampel estimator
The Q/Hampel method
Other robust techniques
Additional guidance on statistical procedures (informative)
Procedures for small numbers of participants
General considerations
Procedures for identifying outliers
Procedures for estimates of location
Procedures for estimates of dispersion
Efficiency and breakdown points for robust procedures
Different statistical estimators (e.g. robust techniques) can be compared on three key characteristics:
Breakdown point
Relative efficiency
Use of proficiency testing data for evaluating the reproducibility and repeatability of a measurement method
Illustrative examples (informative)
Effect of censored values (see 5.5.3.3)
Homogeneity and stability test – Arsenic (As) in chocolate (see 6.1)
Comprehensive example of atrazine in drinking water
Comprehensive example for mercury in animal feed
Reference value from a single laboratory: Los Angeles value of aggregates (see 7.5)
Example of bootstrap technique for Coliforms in food sample (see 7.7.6)
Comparison of reference value and consensus mean (see 7.8)
Determination of evaluation criteria by experience with previous rounds of a proficiency testing scheme: toxaphene in drinking water (see 8.3)
From a general model: Horwitz equation (see 8.4)
Determining performance from a precision experiment: Determination of the cement content of hardened concrete (see 8.5)
Bar-plots of standardized biases: Antibody concentrations (see 10.4)
Youden Plot — antibody concentrations (see 10.5)
Plot of repeatability standard deviations: Antibody concentrations (see 10.6)
Graphical methods for tracking performance over time (see 10.8)
Qualitative Data Analysis; example of an ordinal quantity: skin reaction to a cosmetic (see Clause 11)
Example of computer code for plotting and resampling analysis (“bootstrapping”) of PT results (Informative)
Bibliography