Could you explain this graph:
Sure. Each post-tune dyno graph has variations in the power gained at various rpms points over the baseline run. If a tuner had exactly the average amount of variation, they would be at 0 on the graph. GIAC has almost average variation, ESS has a little less than GIAC. OE on dynojet and AA both have slightly more variation than average. OE on Gintani's dyno had much, much less variation.
What normal distribution looks like... 95.4% of the data should fall within +- 2 standard deviations of the mean. 99.7% should fall within +- 3 standard deviations, and 99.99% should fall within 4, and 99.9999% should fall within 5. Gintani fell outside of 5 standard deviations. (6 is the statistical limit -- anything outside of that is essentially impossible rather than just really really really really unlikely).
1) Sample size is too small...
Response: The calculation of standard deviation factors in sample size (small samples will have a larger standard deviation than larger samples of the same distribution).
2) This assumes a normal distribution
Response: It does assume a normal distribution... However, a normal distribution seems to hold when I add additional tuners to the existing graph:
or if I use them to calculate a new standard deviation:
3) There is a dynojet bias to the data
Response: Possible, but as you can see above Evolve's DD data fit the normal distribution -- as did GIAC's load-bearing mustang dyno. However, both of those graphs also included an aftermarket air filter and Evolve's before/after was done on 2 separate cars/days so they are imperfect comparisons (I would of course welcome additional data)