Preface

Notation and Abbreviations

1. Introduction

1.1 On Permutation Analysis
1.2 Conditionality and Exchangeability
1.3 The Permutation Testing Principle
1.4 Permutation Approaches
1.5 Randomization and Permutation
1.6 When Conditioning Is Appropriate
1.7 Computational Aspects
1.8 Basic Notation

2 Discussion of a Simple Testing Problem

2.1 A Problem with Paired Observations
2.1.1 Introduction
2.1.2 A Motivating Example
2.1.3 Modelling Responses
2.1.4 Symmetry Induced by Exchangeability
2.1.5 Further General Aspects
2.2 The Student t Paired Solution
2.2.1 Paired Normal Data
2.2.2 An Extension
2.3 The Signed Rank Test Solution
2.3.1 Paired Continuous Data
2.3.2 Generalized Scores
2.4 The McNemar Solution
2.4.1 Test on Signs of Differences
2.4.2 An Extension
26 2.5 The Permutation Solution
2.5.1 General Aspects
2.5.2 The Permutation Sample Space
2.6 The Conditional Monte Carlo Method
2.6.1 A Simulation Algorithm for Inspecting Permutation Sample Spaces
2.6.2 A Routine for Random Permutations
29 2.6.3 Approximating the Permutation Distribution
2.7 Permutation Confidence Interval for delta
2.7.1 Introductory Aspects
2.7.2 An Approximate Solution
2.7.3 A Conditional Monte Carlo Algorithm
2.8 Problems and Exercises

3 Theory of Permutation Tests for One-Sample Problems

3.1 Introduction
3.1.1 Basic Concepts
3.1.2 On Sufficient Statistics for Symmetric Data
3.2 Equivalence of Permutation Statistics
3.3 Formal Definition of Permutation Tests
3.3.1 Randomized Permutation Tests
3.3.2 Non-randomized Permutation Tests
3.4 Arguments Related to Permutation Tests
3.4.1 General Aspects
3.4.2 Arguments for Selecting a Test Statistic T
3.5 Examples of One-Sample Problems
3.6 Other Properties of Permutation Tests
3.6.1 Stochastic Ordering of p-values
3.6.2 Conditional and Unconditional Properties
3.6.3 Further Unconditional Properties
3.6.4 Some Consequences of the Theorems
3.6.5 Comments on Power Functions
3.6.6 An Algorithm for Evaluating the Conditional Power
3.6.7 Permutation Testing for Composite Hypotheses
3.7 Optimal Properties
3.7.1 Introductory Remarks
3.7.2 Further Remarks
3.8 Problems and Exercises

4 Examples of Univariate Multi-Sample Problems

4.1 Introduction
4.1.1 General Aspects
4.1.2 On Permutation Distributions
4.2 Inspection of the Permutation Sample Space
4.2.1 Conditional Monte Carlo Method
4.2.2 An Example
4.2.3 Rank Solutions
4.2.4 A Simple Routine for Random Permutations
4.2.5 Permutation Confidence Interval for delta
4.2.6 Problems and Exercises
4.3 Permutation One-Way ANOVA
4.3.1 An Example
4.3.2 Modelling Responses
4.3.3 Permutation Solutions
4.3.4 Problems and Exercises
4.4 Goodness-of-Fit for Ordered Categorical Variables
4.4.1 Introduction
4.4.2 Goodness-of-Fit Tests for Ordered Categorical Variables
4.4.3 A Solution Based on Score Transformations
4.4.4 Typical Goodness-of-Fit Solutions
4.4.5 Extension to Non-dominance Alternatives and C Groups
4.4.6 Problems and Exercises
4.5 A Problem with Repeated Observations
4.5.1 Introduction
4.5.2 Friedman's Rank Test
4.5.3 A Permutation Solution
4.5.4 An Example
4.5.5 Problems and Exercises

5 Theory of Permutation Tests for Multi-Sample Problems

5.1 Introduction
5.1.1 Recalling Basic Notions
5.1.2 Recalling the Non-randomized Version
5.2 Examples of Multi-Sample Problems
5.3 Unbiasedness and Power of Some Multi-Sample Tests
5.3.1 Unbiasedness and Power for Two-Sample Location Problems
5.3.2 Some Corollaries
5.3.3 Unbiasedness of One-Way ANOVA
5.4 Some Asymptotic Properties
5.4.1 Introduction
5.4.2 Two Basic Theorems
5.5 Permutation Central Limit Theorems
5.5.1 Basic Notions
5.5.2 Permutation Central Limit Theorems
5.6 Problems and Exercises

6 Nonparametric Combination Methodology

6.1 Introduction
6.1.1 General Aspects
6.1.2 Bibliographic Notes
6.1.3 Main Assumptions and Notation
6.1.4 Some Comments
6.2 The Nonparametric Combination Methodology
6.2.1 Assumptions on Partial Tests
6.2.2 Desirable Properties of Combining Functions
6.2.3 A Two-Phase Algorithm for Nonparametric Combination
6.2.4 Some Useful Combining Functions
6.2.5 Four Examples of Nonparametric Combination
6.2.6 Problems and Exercises
6.3 Consistency and Unbiasedness of Combined Tests
6.3.1 Consistency
6.3.2 Unbiasedness
6.3.3 A Not Consistent Combining Function
6.3.4 Problems and Exercises
6.4 Some Further Asymptotic Properties
6.4.1 General Conditions
6.4.2 Further Asymptotic Properties
6.5 Comments on Nonparametric Combination
6.5.1 General Comments
6.5.2 Final Remarks

7 Examples of Nonparametric Combination

7.1 Introduction
7.2 Permutation Testing with Multivariate Paired Observations
7.2.1 Formal Description of Testing Problem
7.2.2 An Example
7.2.3 A Multivariate Extension of McNemar's Test
7.2.4 Asymptotic Behaviour of Multivariate Tests with Paired Observations
7.2.5 Problems and Exercises
7.3 Permutation MANOVA with Mixed Data
7.3.1 A Formal Description
7.3.2 An Example with Four Categorical Variables
7.3.3 A Two-Sample Multivariate Test
7.3.4 A Multivariate Extension of Fisher's Exact Probability Test
7.3.5 A Cross-over Design
7.3.6 Problems and Exercises
7.4 Goodness-of-Fit Tests for Ordered Categorical Variables
7.4.1 Revisiting the Problem
7.4.2 Some Extensions
7.4.3 Two Examples
7.4.4 Some Power Evaluations
7.5 A Problem of Isotonic Inference
7.5.1 Introduction
7.5.2 Conditions for Nonparametric Combination
7.5.3 An Application from Genetics
7.5.4 An Example
7.5.5 A Multivariate Extension
7.6 A Problem with Multivariate Homoscedastic Repeated Responses
7.6.1 A Formal Description
7.6.2 An Algorithm for Conditional Simulation
7.6.3 An Example
7.6.4 Problems and Exercises
7.7 Power Behaviour and Remarks on Restricted Alternatives
7.7.1 Power Behaviour of Combined Tests
7.7.2 Remarks on Restricted Alternatives
7.7.3 A Few Simulation Results

8 Permutation Analysis in Factorial Designs

8.1 Introduction
8.1.1 General Aspects
8.1.2 Solutions Based on Residuals
8.2 Exact Separate Tests for Replicated 2^2 Factorial Designs
8.2.1 Separate Sets of Hypotheses
8.2.2 Synchronized Permutations
8.3 Exact Tests for 2^2 Unbalanced Designs
8.3.1 Weighting Intermediate Statistics for Factor A
8.3.2 Weighting Intermediate Statistics for Factors B and AB
8.3.3 An Extension of Welch's Test for Heteroscedastic Models
8.4 Exact Tests in I*J Balanced Designs
8.4.1 Balanced Two-Way Layout
8.4.2 Separate Testing
8.4.3 Some Comments
8.4.4 Multivariate Extension of I*J Designs
8.4.5 Extension to a Kind of Unbalanced Heteroscedastic Model
8.5 Synchronized Tests in Replicated 2^k Factorials
8.5.1 Extension to 2^k Factorial Designs
8.5.2 Realignments
8.5.3 Construction of Test Statistics
8.6 General Characterization of Synchronized Tests
8.6.1 Characterizing Synchronized Permutations
8.6.2 Synchronized Permutation Tests for Fractional Designs of Different Resolution
8.7 A Comparative Simulation Study
8.7.1 Comparing Solutions in H_0
8.7.2 Power Results
8.8 An Application
8.9 Permutation Tests in Unreplicated Factorials
264 8.9.1 Introduction
264 8.9.2 Paired Permutation Tests
267 8.9.3 Paired Permutation Testing Algorithm
268 8.9.4 A Simulation Study
269 8.10 Problems and Exercises
271 8.11 Appendices
272 8.11.1 Sufficient Statistics for Replicated $2^2 $ Factorial Designs
8.11.2 A Brief Review of Two-Level Factorial Designs

9 Permutation Testing with Missing Data

9.1 Introduction
9.1.1 General Aspects
9.1.2 Bibliographic Notes
9.2 On Missing Data Processes
9.2.1 Introduction
9.2.2 Data Missing Completely at Random
9.2.3 Data Missing Not at Random
9.3 The Permutation Approach
9.3.1 Introduction
9.3.2 Breaking Down the Hypotheses
9.4 The Structure of Testing Problems
9.4.1 Hypotheses for Non-MAR Models
9.4.2 Hypotheses for MCAR Models
9.4.3 Permutation Structure with Missing Values
9.5 Permutation Analysis of Missing Values
9.5.1 Partitioning the Permutation Sample Space
9.5.2 Solution for Two-Sample MCAR Problems
9.5.3 Extensions to Multivariate C-sample Problems
9.5.4 Extension to Non-MAR Models
9.5.5 Some Comments
9.6 An Example of a Non-MAR Model
9.6.1 Introduction
9.6.2 The Example
9.6.3 The Permutation Solution
9.7 An Example with Multivariate Paired Observations
9.7.1 The Problem
9.7.2 An Example with Fictitious Data
9.8 Power Behaviour of Some Tests with Missing Values
9.8.1 Introduction
9.8.2 Simulation Results and Comments
9.9 Problems and Exercises
 

10.1 Introduction
10.1.1 General Aspects
10.1.2 Bibliographic Notes
10.1.3 Formal Description of the Problem
10.2 Parametric Solutions
10.2.1 Basic Statistics
10.2.2 Known Covariance Matrices
10.2.3 Unknown Proportional Covariance Matrices
10.2.4 Unknown Covariance Matrices
10.3 A First Approximate Permutation Solution
10.3.1 Approximate Solution Based on Aspin and Welch Statistics
10.3.2 An Example
10.3.3 Extension of Approximate Solution to Multivariate Situations
10.4 An Almost Exact Permutation Solution
10.4.1 A Univariate Solution by Testing for Symmetry
10.4.2 Combining Two Tests of Symmetry
10.4.3 Asymptotic Behaviour
10.5 Extension to C>2 Groups
10.6 Multivariate Permutation Solutions
10.6.1 Multivariate Solutions for Symmetric Distributions
10.6.2 Three Multivariate Testing Problems
10.7 Distributional Behaviour
10.7.1 A Simulation Study in H_0 
10.7.2 A Simulation Study in H_1 
10.8 Permutation Testing for Location and Scale Coefficients
10.8.1 An Approximate Solution for Scale Coefficients
10.8.2 Joint Permutation Testing for Location and Scale Coefficients
10.9 Problems and Exercises
 

11.1 Introduction
11.2 Modelling Repeated Measurements
11.2.1 A General Additive Model
330 11.2.2 The Hypotheses of Interest
11.3 Testing Solutions
11.3.1 Solutions by the Nonparametric Combination Approach
333 11.3.2 Analysis of Two-Sample Dominance Problems 
335 11.3.3 An Example from the Literature
335 11.3.4 Some Power Evaluations
337 11.4 Testing for Repeated Measurements with Missing Data
11.4.1 Introduction
11.4.2 A Formal Description of the Problem
11.4.3 An Example
11.4.4 A Comparative Simulation Study
11.5 Tests for Balanced and Unbalanced Repeated Measures Designs
11.5.1 Repeated Measures on 2^2 Unbalanced Factorials
11.5.2 On I*J Balanced Designs
11.6 Problems and Exercises
 

12.1 Introduction
12.2 Morphological Differences in Biting Flies
12.2.1 Description of the Problem
12.2.2 Results of Analysis
12.3 A Clinical Trial on a Respiratory Drug
12.3.1 Description of the Problem
12.3.2 Results of Analysis
12.4 A Medical Experiment on Diabetic Patients
12.4.1 Description of the Problem
12.4.2 Results of Analysis
12.5 A Case Study of Recovery Wards
12.5.1 Description of the Problem
12.5.2 Results of Analysis
12.6 Analysis of Experimental Tumour Growth Curves
12.6.1 Description of the Problem
12.6.2 Results of Analysis
12.7 An Epidemiological Survey
12.7.1 Description of the Problem
12.7.2 Results of Analysis
12.8 Appendix: NPC Test 2.0 Software for Multivariate and Multistrata Permutation Analysis