Reaction time is a measure that is extremely useful for assessing brain health - but only if it is collected accurately.

Reaction time is one of the most frequently collected and depended upon metrics for cognitive assessment – but most methods are extremely inaccurate. The Brain Gauge is the world’s most accurate commercially available device that can deliver the reaction time test.

It seems like there is a constant and steady improvement in technology and computing power. The cell phones that we carry now have exponentially more computational power than computers that we (I speak for the people in my generation) were programming in the late 1970s and early 1980s. However, computational power is not everything. When I first started transitioning our lab to Windows based computers, it quickly became quite apparent that timing the controllers in the lab might present a problem. Even though the new computers allowed us to crunch numbers and analyze data much more rapidly than ever before, we had to pay much more attention to how we timed signals that were delivered by the computer to trigger events during experiments. All of a sudden, we no longer had the ability to time neural events with sub-msec accuracy. The paradox was that the faster computers had more trouble delivering or recording accurately and precisely timed event signals. We really did not give this a lot of thought, as once we became aware of this paradox, we worked around it and went to extremes to avoid the pitfalls of depending on new computer technology for doing precision event timing. However, a recent communication by Richard R. Plant (Behav Res 2016) entitled “A reminder on millisecond timing accuracy and potential replication failure in computer-based psychology experiments: An open letter” directed us to re-examine the issue. In his letter, Dr. Plant points out that there is a “replication crisis” in psychology research that is leading many to question the validity of many experiments that have been published. The article does an excellent job outlining the problems with computer-based psychology experiments, and if you are interested, I highly recommend the article. One of the main points of the article is that these experiments are not using laboratory grade equipment to collect what should be very accurate measures. Commercial grade computers and peripherals are very good at what they are designed to do – but they were not designed for precision timing experiments.

We decided to do a relatively simple experiment to compare the timing accuracy between a laboratory grade research device (the Brain Gauge), a laptop computer and a mobile device. We developed the Brain Gauge as a laboratory research tool, and we wondered how the commodity grade commercial devices would do in comparison. There are numerous software packages that collect and analyze data on various aspects of cognition, and many of them incorporate either reaction time or response time into their analytics. In our simple experiment, we collected the reaction time robotically to either a tactile or visual stimulus in one of four set ups. The first set up was a Brain Gauge Pro, where the tactile stimulus was delivered to one of the probe tips and the response was delivered robotically to the other. The second set up was a Brain Gauge, where the tactile stimulus was delivered to the Brain Gauge and the response was robotically delivered to a computer mouse. In the third experimental set up, a visual stimulus was delivered from the computer monitor and the response was delivered to a computer mouse. In the fourth set up, a visual stimulus was delivered by a mobile device and the response was delivered to the keypad on the mobile device.

Response latencies with associated variability of the 4 conditions are shown below.

The boxplot above compares the accuracy of four different methods of reaction time testing. Systems reliant on a touchscreen (yellow) have the highest value, are the least accurate and have the largest variability. Systems reliant on a USB Mouse (blue) are the second highest. Brain Gauge (orange; tactile testing with a computer mouse) was significantly better than either visual modality although not as accurate as Brain Gauge Pro (grey; tactile testing with dedicated tactile hardware response device).

The graph above is a direct comparison of variability of 4 different reaction time testing methods. Raw data is plotted with offset of median latency subtracted, a technique used by many reaction time assessments to adjust for systematic latencies. All data plotted on the same scale. The visual tasks (Touchscreen and USB Mouse) had significantly greater variabilities than the tactile tasks (Brain Gauge and Brain Gauge Pro), as measured by standard deviation. This figure displays the same data as shown in Figure 1, but plots all data points on the same scale.

The data demonstrate a high degree of variability in the two visual modalities – so high that it would certainly make any reaction time measure questionable and measures of reaction time variability (where controls often score below 10 msec) quite impossible. The tactile reaction times, which use the laboratory grade Brain Gauge device, have relatively low variability.

One thing to keep in mind is that when most online computer testing is done, it is done on a wide range of computers that will have inherent latencies and variabilities for the exact same test. In this experiment, we did not compare the latency of one visual display to another. Different operating systems and different computer peripherals all introduce their own variability, and some of the earliest papers reporting reaction time (in the mid 1800s) were probably more accurate than many of the measures reported today. Unless, of course, the contemporary papers used laboratory equipment, such as the Brain Gauge.