Dates and Times

Poster: How alerting affects can bias speed skating competitions!
Thursday, 17 December, 17:00 – 19:00 (Poster Session 1), Lounge 1 or 2

OpenSesame booth: With cool demos and answers to all your questions!
Thursday to Saturday, 17-19 December, all day (staffed during breaks), Lounge 1 or 2

Abstract

Life is unfair, and so is speed skating: Some athletes can randomly benefit from alerting effects due to inconsistent starting procedures

Edwin S. Dalmaijer ¹, Beorn Nijenhuis ², Stefan Van der Stigchel ²

¹ Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom
² Department of Experimental Psychology, Utrecht University, Utrecht, Netherlands

The Olympics are the world’s largest sporting events, attracting billions of viewers worldwide. An important part is speed skating. In this sport, athletes compete against each other in paired races to determine who wins the gold. At each race’s start, the referee waits for a variable interval between cueing athletes to get “Ready” and the starting shot. This closely resembled an alerting experiment, in which reaction times are lowest for an optimal interval, and progressively slower with longer intervals. Here we examine skaters’ performance in the 500 meter speed-skating competition at the 2010 Winter Olympics. We demonstrate that the time between “Ready” and the start (the ready-start interval) is a significant predictor of speed skaters’ times at both 100 meters and at the finish, with longer ready-start intervals resulting in higher race times. This suggests high-level speed-skating competitions are biased to randomly disadvatage some athletes. This bias could be removed by simple alterations to current starting procedures. The proposed change would greatly improve racing sport fairness, which currently suffers from an injustice that disadvantages not only athletes, but entire nations rooting for them.

Poster

The poster we presented at NVP 2015 can be found on FigShare.

Publications

Some of the findings we included in our poster, have been published in Perspective and Commentary articles in Frontiers in Experimental Psychology:

• Dalmaijer, E.S., Nijenhuis, B.G., & Van der Stigchel, S. (2015). Life is unfair, and so are racing sports: Some athletes can randomly benefit from alerting effects due to inconsistent starting procedures. Frontiers in Psychology, 6(1618). doi: 10.3389/fpsyg.2015.01618
• Dalmaijer, E.S., Nijenhuis, B.G., & Van der Stigchel, S. (2016). Commentary: Life is unfair, and so are racing sports: Some athletes can randomly benefit from alerting effects due to inconsistent starting procedures. Frontiers in Psychology, 7(119). doi: 10.3389/fpsyg.2016.00119

Blog posts

In addition to the articles mentioned above, we have also blogged on the topic:

1. Post on the original article: Life is unfair, and so are racing sports
2. Post on an additional analysis: Is skating really unfair? Yes, even in extra stringent analysis.
3. Post on 100-meter times: Is skating really unfair? Yes, even at 100m times.

Two weeks ago, we published a Perspective article on how the starting procedure in racing sports could bias competitions. The article was met with very positive attention (even in the Huffington Post!), and also some scepticism. We answered to the sceptics with a follow-up analysis of our data, which underscored our original point. In addition, some speed skating enthusiasts suggested we analyse the 100-meter times from the races we reported on. So we did! The results are very similar to our earlier results: Longer ready-start intervals lead to slower 100-meter times in Olympic speed skating.

What happened?

In a Perspective article in Frontiers in Psychology, we argued that random variability in the starting procedure of many racing sports could bias the competition. These variabilities arise from referees using a different interval between cueing the athletes to get “Ready”, and to fire the starting pistol once the athletes are in position. This duration of this ready-start interval is determined by how quick athletes get ready, and by the added temporal jitter (random variability) that the referee adds. The issue with this, is that the “Ready” cue will alert the athletes. From research in experimental psychology, we know that alerting cues make people quicker to respond to a target stimulus (such as the starting shot). This benefit is optimal with an interval of 500 milliseconds, and slowly decreases after that. So, we argued, athletes starting with a shorter ready-start interval will be more alerted than athletes with a longer interval.

When an athlete is more alerted at the time of the starting shot, they will be able to respond to it quicker. This will also make him finish earlier. This is a strong claim to make based on just theory, so in the article we included data from the 500 meter speed skating event at the 2010 Winter Olympics. We observed a clear relationship between ready-start interval and race time: On average, the longer an athletes ready-start interval was, the slower they were to finish.

Some people argued that the causality might be opposite: maybe a better race time was causing a shorter ready-start interval. They argued that, maybe, quicker skaters were quicker to assume their starting positions. Although there is no scientific literature to support this, and although the speed-skating rules state that the referee must add random temporal variability, we were intrigued by this argument. Therefore, we did an extra analysis on the same data, and demonstrated that we were right about the causality: longer ready-start intervals lead to slower race times.

Some skating enthusiasts were also intrigued by our findings, and they were asking what would happen if we looked at the 100 meter times of the current competition. Those times is what I analyse here.

100 meter times

If speed skating is not your sport, you might be confused. There is a 500 meter competition, but not a 100 meter one. This is true. However, during the 500 meter competition, athletes also get timed 100 meters after the start. This gives a preliminary indication of how they are doing, and athletes can use this information in the rest of the race (e.g. to put in extra effort, or to pace themselves).

Dr. F. van de Bult was curious to find out whether the ready-start interval had an effect on the 100 meter times, and he had learned that we were keen to share our data. So he asked for our data, and he collected the 100 meter times. He then shared these with us, and the analyses here are based on the data that he supplied. Thanks!

Ready-start interval negatively affects 100 meter times

We did the exact same things as we did in the original article, and in the additional analysis on the data from that article. For details on the methods, please refer to those.

The longer a referee waits between “Ready” and “Start”, the slower athletes reach 100 meters!

There is a correlation between the duration of the ready-start interval and the 100 meter times. The ready-start interval can explain 6.3% of the variance in men’s 100 meter times, and 13.9% in women’s. In the current dataset, on average, an extra second of ready-start interval relates to an extra 123 milliseconds in men’s 100 meter times, and an extra 200 milliseconds in women’s.

Shorter ready-start intervals cause quicker 100m times, even when controlling for potential confounds.

To answer the causality question, we performed a linear regression of the inter-individual differences in ready-start interval and in 100-meter time. The difference between the ready-start interval in athletes’ first race and their second race is a significant predictor of the difference between their 100-meter times in the first and second races. The difference in ready-start interval explained 13.1% of the variance in the differences in 100-meter times. In the current dataset, on average, an extra second of ready-start interval difference caused 59 milliseconds of extra difference in the 100-meter times.

One important observation here is that the ready-start interval differences explained roughly the same amount of variance in the 100-meter and complete-race times. In addition, the effect that the ready-start interval has, scales from 56 milliseconds at the 100-meter point to 174 milliseconds at the end of the race (500 meter). This is interesting, as it suggests that a longer ready-start interval does not just cause a worse reaction time (due to less alerting benefits), but continues to plague the skaters during their races. Now, it should be clear that there is none to little scientific literature to explain this. But if we had to speculate, we would suggest it might be due to issues with athletes’ muscles, or with their motivation. It could be that a longer ready-start interval increases muscle fatigue, because it forces athletes to maintain their starting position for longer. It could also be that athletes realise when respond slower to the starting pistol (due to less alerting following a longer ready-start interval), and lose motivation for the rest of the race, which could suppress their overall speed.

Conclusion

The time between when the referee cues speed skaters to get “Ready” and when he fires the starting pistol, has an effect on skaters’ performance. The longer the ready-start interval, the slower skaters are to reach the 100 meter point, and the slower they are to finish the race.

Reference

• Dalmaijer, E.S., Nijenhuis, B.G., & Van der Stigchel, S. (2015). Life is unfair, and so are racing sports: Some athletes can randomly benefit from alerting effects due to inconsistent starting procedures. Frontiers in Psychology, 6(1618). doi: 10.3389/fpsyg.2015.01618

Yesterday, we reported that random variability in the starting procedure of racing sports can bias competitions, even at Olympic events. Not everyone was keen to believe this, and some people have made suggestions for things we should control for. Some even went so far as to criticise our methods. In this post we address all questions, and provide an extra analysis that looks at within-athlete effects of changes in the ready-start interval on changes in race times. This analysis is robust to differences between skaters’ individual qualities, and has causal power. Our results indicate that there still is evidence that random differences in ready-start intervals might bias competitions. At the very least, this calls for future research into the starting procedure of racing sports. Which is exactly what we intended to provoke with yesterday’s publication.

What happened?

Perspective article in Frontiers in Psychology

Yesterday, we published a Perspective article in the academic journal Frontiers in Psychology. The article made a theoretical point about a potential bias in the starting procedure in some racing sports, including speed skating. We explained about the alerting effect, that makes people quicker to respond when they are just alerted:

The starting procedure in racing sports closely resembles a classical experiment, where participants receive an alerting cue before having to respond to a target stimulus.The cue is a general, non-spatial signal that precedes the target stimulus by a variable interval. In the lab, participants are quicker (Posner and Boies, 1971; Adams and Lambos, 1986) and more precise(Klein and Kerr, 1974) to respond after an optimal interval duration of around 500ms, and are progressively slower and less precise after longer durations.

Inconsistent starts in racing sports

In some racing sports, the referee signals the competing athletes to get ready, and fires the starting shot after a certain interval. We refer to this as the ready-start interval. The crucial point is that this interval is variable. It depends on how quick athletes assume their starting position, but also on regulations that specifically call for variability. The latter is true in speed skating. In addition, speed skaters compete in pairs, and whoever ends up with the lowest total wins the gold. Crucially, each skater starts with different ready-start intervals.

Alerting differences could bias competitions

In our article, we argued that athletes that start with a shorter ready-start interval should have a theoretical benefit over those that start with a longer ready-start interval, due to the alerting effect we describe above. Of course, we realised that this was quite a strong claim to make based on data from psychological studies that is collected in a highly controlled environment. Surely, at Olympic events, factors like talent and training should strongly outweigh the potential benefits of a short ready-start interval? This is why we collected data from the 500 meter speed skating competition at the 2010 Winter Olympics. We collected millisecond accurate ready-start intervals from the audio trace of the event’s broadcast, and chose to correlate that with the race times of each athlete.

What we show, is that ready-start intervals and race times correlate. And the effect was quite large at that: several hundreds of milliseconds. That does not sound like much, but can make a real difference in racing sports. Enough, in some cases, to mean the difference between winning the gold, and not winning a medal at all!

The longer a referee waits between “Ready” and “Start”, the slower athletes finish!

So what’s wrong?

Lots of attention for our article…

The story got picked up by media and was reported on in newspapers, websites, and radio shows in several countries. In the Netherlands, where Dutch people live, reporting was especially pronounced: Beorn (second author and former speed skater) even made it to national television. There are two things you should know about the Dutch: they love speed skating, and they are a naturally sceptic kind of people. This quality makes them very good at evaluating science, and so they did: Mere minutes from publication, I was already receiving emails from colleagues that suggested further analyses, and what kind of control studies could be run. This was great! Post-publication discussion of the starting procedure is exactly what we wanted!

Some bad attention…

In addition to the constructive comments, some people simply dismissed our findings. Some did so because they simply “do not believe them”, but some had more substantiated criticisms. The most prevalent of these was that we relied on what is essentially a correlation between ready-start intervals and race times. As we have al heard, correlation does not imply causation! What people argued was that the causality might be opposite: maybe good skaters take less time to get ready, and thus have shorter ready-start intervals!

In addition, some people argued that we should not have been using a correlation in the first place, because our data included at most two data points from the same athletes. The assumption here is that the data points coming from the same athlete might cause non-independence in our data set. At least, that is what I assume they meant, because they did not do a very good job at explaining their point. They also did not make any suggestions about what would be the appropriate analysis, and they hid behind an anonymous website. Furthermore, they did not contact us before publication, and they fail to recognise that our manuscript has passed scientific peer review, which should vote for its methodological and statistical soundness. Instead, TopSport Topics decided to post a rather shallow discussion of our paper, that came to a very strong conclusion, without more than simply handwaving towards potential issues.

Cool scientific discussion

From the previous paragraph, you might conclude that we were a bit upset. And we were. But not because they criticised us! Other people have contacted us about similar issues, and we had very good discussions about our data. For example, Prof. Lex Borhans of Maastricht University contacted us with the question highlighted before: what direction was causality in? Were skaters really quicker due to shorter ready-start intervals, or could it be that good skaters assume their starting positions quicker and thus have shorter ready-start intervals? I should note that Lex was not the only one asking this question, but he was the only one (to my knowledge) that wanted to write a blog post on it. He asked for our data, and we were very happy to send it over for Lex to re-analyse. The resulting blog post can be found here.

The method that Lex applied was really clever: he tried to predict the race times of skaters’ second race by using the ready-start interval from the first time. This allowed him to infer the direction of causality. If shorter ready-start intervals really cause shorter race times, there should be no correlation in Lex’ analysis. If, on the other hand, short ready-start intervals were caused by athletes being really good, there should be a correlation. Lex’ findings were more in line with the latter suggestion: there were correlations between the ready-start intervals on one race correlated with the race time of the other! The correlation was less strong than the correlation we found, but that difference was not significant.

We have to go deeper!

So where does that leave us? Did we publish too soon? Not necessarily, because Lex’ method is a bit like a statistical sledgehammer: It’s not really sensitive to the kind of subtle effects that our alerting hypothesis would predict. In addition, if you assume skaters are causing both their ready-start intervals and their race times to be longer or shorter at the same time, then the skater would be a moderator. In Lex’ analysis, this would mean that the relationship between the ready-start interval of one race and the finish time of the other race is inflated. For example, if a skater performs bad (e.g. due to a long ready-start interval) during the first race and gets demotivated after the first race, they will show both a longer ready-start interval and a slower race time. Thus the correlation between the first ready-start interval and the finish time of the second race would be moderated by the skater’s state. In the new analysis we describe below, this is not an issue: When the state of the skater is assumed to have an effect on both ready-start interval and race time, this does not change the direction of the relationship between ready-start interval and race time. And that direction is exactly the problem in the current starting procedure. Please do read on if you want to learn about our new super-stringent analysis.

So we set out to do what every scientist does when they are being critiqued: we dove back into our own data, and we scienced the shit out of it. The results are presented below.

What’s really going on?

One analysis to rule them all

There is one way in which we can address all criticisms at the same time, and that is by looking at individual differences within each speed skater. In that way we can measure the effect that differences in ready-start interval have on each individual skater’s performance. This answers the causality question, because we use a difference in ready-start interval (caused by the referee) to explain a difference in race times using a linear regression. It also bypasses the issue of differences in skaters’ abilities, because we are looking at differences in each individual’s performance. Finally, it bypasses all other confounds that others have brought up here or there, for example the idea that both referees and skaters get quicker as the competition progresses due to excitement building up. The following analysis addresses those issues, because it is a direct test of the effect of within-skater differences in ready-start interval on differences in within-skater race times.

Methods, methods, methods

We want to be absolutely clear about our methods here, so here they are. We excluded all skaters that fell or nearly fell during their race, and also those that did not complete one of either. These are Mitchell Whitmore (nearly fell in first race), Maciej Biega (nearly fell in second race), Shani Davis (gave up after first race), Annette Gerritsen (fell in first race), and Yulia Nemaya (fell in second race). Falls or near falls have such a massive impact on race time that they obscure everything else, including athletes’ talent and training, but also the effects of ready-start intervals. Therefore we strongly feel that we should exclude these races from further analysis. For all remaining skaters, 70 in total, we calculated the difference between the first and the second race (race 1 minus race 2) in both their ready-start interval and their race time. We combined the data for men and women, because we have no theoretical reason to split them up, because their individual differences are on the same scale (unlike their race times), and because we need the sample to be sufficiently large for any regression or correlation to be sensitive enough to pick up the effects that we predicted.

Way more stringent analysis, but same results

The linear regression between the individual differences in ready-start intervals and race times demonstrates that there is a significant positive effect of ready-start interval on race time. When the difference in ready-start interval is negative (i.e. the second race had a longer ready-start interval), the difference in race times was also negative (i.e. the second race time was longer). The Pearson correlation is significant (p = 0.003), and explains about 12% of the variance in race time differences.

In the current dataset, assuming a linear relationship, one extra second of ready-start interval difference caused 174 ms of difference in race times. Both the explained variance and the magnitude of the effect are less than what we demonstrated in the analysis in our article. This means that at least some variance in that data can be explained by what several people, including Lex, suggested: quicker skaters are quicker to assume their starting position, and thus have shorter ready-start intervals. However, we can still explain 12% of the variance, whereas this should be 0% in a fair competition. And the remaining effect of 174 ms of added race time per extra second of ready-start interval is still very worrying in sports where the difference between winning gold or silver (or nothing!) can sometimes be only a few milliseconds.

Shorter ready-start intervals do cause shorter race times, even when controlling for confounds.

Is this sample too small?

After collecting data and computing a correlation, you can calculate the statistical power of your results. Ours is 91.42%. That number indicates that our sample was big enough to reliably test the effect that we found.

What happened to the differences between men and women?

They are not there in the current analysis, with the individual Pearson R for men being 0.21, and 0.26 for women. This means they were likely due to noise in the men’s analysis from our article.

Final note

The most sceptic of people might now argue something along the lines of the following: “But wait a minute… Maybe skaters that are very good are both more constant in assuming their position (and thus their ready-start interval), and in their race times! So you ARE wrong! HA!“. In that case, one would still expect the minor differences you assume to correlate in the same direction. So they should still be picked up by our regression. In other words: our concerns with starting procedures still hold.

Conclusion

The theoretical issues that we put forward in our article are valid, and the data we provided to support our claims are still valid. This is after we corrected for skaters’ individual qualities, and using a regression on within-participant differences from which causal inferences can be made. Our article was a Perspective, which are intended to highlight important areas of future research. Our article and the discussion following its publication illustrate precisely that: There is a need to thoroughly investigate the starting procedure of racing sports, and speed skating in particular.

Further comments

If you have any comments, objections, compliments, or tips for funny cat videos, please post them in the comment section below. You can also direct them to me directly, using my email address: edwin.dalmaijer@psy.ox.ac.uk.

Reference

• Dalmaijer, E.S., Nijenhuis, B.G., & Van der Stigchel, S. (2015). Life is unfair, and so are racing sports: Some athletes can randomly benefit from alerting effects due to inconsistent starting procedures. Frontiers in Psychology, 6(1618). doi: 10.3389/fpsyg.2015.01618

At the Olympics, you might think that practice and skill are what determines who gets the gold. However, inconsistencies in the starting procedure of racing sports can randomly disadvantage some athletes. Due to the alerting effect, the longer the referee waits between saying “Ready?” and firing the starting pistol, the slower athletes will be able to respond to the starting shot. This might seem trivial, but we show that it can explain an unexpectedly large amount of variance in the 2010 Winter Olympic’s 500 meter sprint event. Here, one second of extra delay can mean 700 milliseconds of extra race time: Enough to mean the difference between first and fifth! We propose a change that would greatly improve racing sport fairness, which currently suffers from an injustice that disadvantages not only athletes, but entire nations rooting for them.

You can read the full article here, and a summary is provided below.

(In internet-lingo, TL;DR is an abbreviation of ‘Too Long; Didn’t Read’. Some people don’t like reading long text, and require a super-short summary.)

Racing Regulations

A typical racing sport involves the referee alerting athletes of the imminent start (“Ready?“), followed by them sounding the starting gun. The time that passes between these events is what we refer to as the ready-start interval, and it is variable. In running (rule 111-8 from UK Athletics) and swimming (rule SW 4.1 from the Fédération Internationale de Natation), the interval depends on how long athletes take to assume position. In speed skating (rule E-255 from the International Skating Union), a random interval between 1 and 1.5 seconds is added to that. In practice, this means there will be around 3.5 to 5.5 seconds between the referee’s “Ready” and the starting shot.

In running and swimming competitions, athletes compete in different heats. In each of those heats, a different ready-start interval will be used. In speed skating competitions, athletes race in pairs, and each pair starts with a different ready-start interval. When all athletes have raced, their times are compared, and the skater with the lowest total time wins. It is very important to realise that this means that every skater starts with different ready-start intervals!

Alerting Effect

Why should we care about how much time there is between “Ready” and start? Well, there is this thing called alerting. If you hear or see a cue that signals an event is about to happen, you will be temporarily alerted. This means that you will be quicker and more accurate to respond when said event actually happens. Now for the crucial bit: there is an optimal interval between the alerting cue and an event. If they are 500 milliseconds (half a second) apart, you will be at your quickest. The longer the interval between cue and event, the slower you will be to respond.

Remember how each heat in swimming and running, and each pair of skaters started with a different ready-start interval? Due to alerting, this means that athletes starting with a long ready-start interval will be disadvantaged compared to athletes starting with a short ready-start interval. Importantly, which athletes are disadvantaged is entirely random!

Olympic Issues

Until now, this has all been theoretical. Based on psychological research into alerting, you would expect some athletes are disadvantaged, but you would also expect that athletes’ training and skills are way more important for their performance. That’s why we decided to actually test our ideas by looking at the 2010 Winter Olympics’ 500 meter speed skating event.

So I spent a few weekends going over the entire mens’ and womens’ events. From the official footage, I took the audio stream. By carefully analysing this in sound software, I could extract the time at which the referee started saying “Ready”, and at which he started to fire his pistol. The difference between these is the ready-start interval. Getting the official finishing times for all races was less of an effort, as those are published online.

After collecting the data, I checked whether the ready-start intervals could predict the race time. Turns out they can, for a bit! In men, the ready-start interval can explain about 5% of the variance in finishing times, and in women it’s a whopping 27%! (Note: that should have been 0%, if the competition was fair.) On average, having an additional second of ready-start interval made athletes 300 (for men) or 700 (for women) milliseconds slower on the whole race. That might not sound like much, but in speed skating that’s the difference between first and fifth place!

The longer a referee waits between “Ready” and “Start”, the later athletes finish!

Potential Solution

What’s currently causing an issue, is the variability of the ready-start interval. Once you take out that, you remove the difference in alerting that gives some athletes an unfair disadvantage. One way in which this variability could be removed, is by introducing an extra step to the starting procedure. Ideally, the referee would ask athletes to go to the start (“Go to the start“), and would then ask them to assume their starting positions (“Get set“). When starting positions have been assumed, the referee should be able to press a button to activate a computerised start. This would signal the athletes to get ready (“Ready“), wait for a fixed amount of time, and then fire the starting shot.

The procedure outlined here would remove any random alerting differences between races, and would thus remove the current bias against athletes starting with longer ready-start intervals.

Conclusion

In racing sports, athletes compete against each other to in different heats to determine who wins the gold, or who advances to the next round. Each race starts with a different interval between the referee’s “Ready” cue and the starting shot. The length of this interval directly affects an athletes alertness: The longer the ready-start interval, the slower an athlete can respond to the starting short. We demonstrate that this is not just a theoretical issue, by showing ready-start intervals are correlated with finishing times in speed skating. On average, an extra second of ready-start interval will result in half a second of extra racing time. That could mean the difference between first and fifth place! An easy solution would be to remove the current variability in ready-start intervals, and improve racing sport fairness.

Reference

• Dalmaijer, E.S., Nijenhuis, B.G., & Van der Stigchel, S. (2015). Life is unfair, and so are racing sports: Some athletes can randomly benefit from alerting effects due to inconsistent starting procedures. Frontiers in Psychology, 6(1618). doi: 10.3389/fpsyg.2015.01618