#4 - Recompiling the kernel can introduce all sorts of variables. While they mention that they did this to unload modules, they do not give specifics as to what other settings they may have changed even inadvertantly. This was a mistake, they should have left it alone, or at least ran a test showing how the changes made impacted the test.
They do not give specific details on the configuration of the machines, but on the Dell 4100 the results show the Samba server putting out far more network traffic than a single 100baseT connection is capable of. If you are talking about dual network cards, or even Gigabit, the configuration of the network settings can play a major role.
Above all, they break the fundamental rule of testing... there is no documentation detailing their exact configuration, the test scenarios, etc. so there is no way to reproduce the tests.
I don't care whether or not Samba outperforms Win2k, but I do care when the tests are not executed properly and give the appearance of bias.
|