We continued the introduction of the Analysis of Variance by focusing on the differences between the different designs. We also mentioned about the assumptions underlying the ANOVA and the plots for a visual inspection of this assumptions on the data at hand. We discussed about multi comparisons and the different choices for the adjustment of the p-values. This part was substantiated by practical examples in R on the projector. We used the functions aov, pairwise.t.test, TukeyHSD, friedman.test, pairwise.wilcox.test.
We then introduced sequential testing and the racing algorithm. Beside the algorithmic procedure the method has been illustrated with an example in practice. We commented the code for the wrapper file required by the race R library.
Finally we skimmed over other methods such as Kolmogorov Smirnov tests and Multiple Regression, Non Linear Regression, and Smoothing. These methods become useful in cases of more fine grained analysis of algorithms. We mentioned R library functions to accomplish these analyses.
The article by Birattari is the only specific literature from this lecture. All other notions can be found from the books of applied statistics.
In the last lecture we will discuss shortly Multiobjective Optimisation and Stochastic Optimisation. This part is also not included in the text book and I will provide at the lecture few pointers for further notions on these themes.