The very first federal admissibility hearing for fMRI lie-detection evidence wrapped up May 14 in a Tennessee court room. The decision, expected in a couple weeks, could have a significant influence on the direction that brain scan evidence takes in the courtroom.
A special session was held to determine whether brain scans that were generated by the company Cephos could be entered as evidence in the federal court case of Lorne Semrau, whom the government has accused of defrauding Medicare and Medicaid.
FMRI brain scan evidence has yet to be admitted for lie detection in court, and this case is the most serious consideration yet of the technique in an American court.
“It’s in some ways a potentially watershed moment,” said Owen Jones, a professor of law and biological sciences at Vanderbilt University, one of the few hearing observers in the nearly empty court room. “I had the sense throughout that, whichever this court decides, this was going to be a significant moment.”
In federal court, the admissibility of scientific evidence is governed by the Daubert standard, first established in the early 1990s. To be entered into the record, scientific evidence has to be empirically testable, subjected to peer review, have a calculable error rate, and be generally accepted by a relevant scientific community. Through a pretrial hearing, a judge decides whether the evidence offered meets these criteria.
In the Semrau case, Steven Laken, CEO of Cephos, is the expert witness whom the defense would use to bring in the brain scan evidence. He testified at the Daubert hearing on May 13 and 14. He was followed by plaintiff’s witnesses’ Marcus Raichle, a neuroscientist at Washington University in St. Louis, and Peter Imrey, a biostatistician at the Cleveland Clinic.
A transcript will likely become available, but in the meantime, Wired.com spoke with Jones, who is also the incoming director of the MacArthur Foundation Law and Neuroscience Project.
Wired.com: What was the tack the defense took in making its case?
Owen Jones: Their basic effort was devoted to demonstrating two things. First, that fMRI technology itself is sound, and second, that this specific application of fMRI to lie detection is sound and scientifically credible. There was much discussion of publications. Through Laken, the defense walked the judge through understanding a bit about how the technology worked and how the test was administered and why some people in the lie-detection community consider this technique to be scientifically valid.
Wired.com: What points did the prosecution focus on in cross-examination?
Jones: They focused on the fact that this would be the first case in which fMRI brain scan evidence like this would be admitted. Reference was made to the recent
Brooklyn case, where it was not admitted. And there was some discussion about the extent to which Cephos stands to gain financially if this technique becomes widely acceptable. One of the things highlighted was the seeming inconsistency between some scans Cephos did of Semrau, some of which suggested that he was lying and some of which suggested that he was telling the truth. There was much discussion on Laken’s basis for discounting the scan session in which the conclusion was that the defendant was lying. Laken discounted the evidence because of the alleged fatigue of the defendant.
There was discussion on cross on general ecological validity, which means the degree to which real-world situations conform to the experiments done in the laboratory. The prosecution pointed out that there was a long duration between the event in question and the scan itself, roughly six to eight years. There was also much discussion about the difference in age between the defendant and the maximum and also median ages of subjects in published research studies. Semrau is 63 or 64, and in prior studies the oldest subjects were 50.
Certainly there was a lot of discussion about the alleged accuracy rates of the technology. And that’s obviously one of the important factors in the Daubert standard. The scientific expert is claiming that they have 100 percent accuracy at finding liars.
There was also discussion about the nature of the questions administered. Some of the questions were short. Some were long. Some were highly detailed. Others were quite general. So, there were questions about the methodology of the test and whether they were sound.
There was also a heavy emphasis on the extent to which the published studies do not have subjects for whom there are real and significant consequences for failing the truth verification tests. That very specifically raises the question about whether the published studies are at all relevant to this particular case.
Wired.com: What was Laken’s response?
Jones: My recollection is that Dr. Laken pointed out, as did his attorney, that no tests are perfect, and it is very difficult to imagine a realistic scenario in which one could study in a scientifically controlled way people who have very large stakes in the outcomes of their testimony. Laken’s argument was, in part, that there is no present reason to believe that the areas of the brain involved in [lie detection] are meaningfully different if one does or does not have high stakes involved in the issue.
Wired.com: In the recent Brooklyn case, an attorney successfully had fMRI evidence excluded arguing that it was the fundamental right of juries to decide the credibility of witnesses, so machine lie detection should always be excluded. Did the lawyers in this case raise that point?
Jones: Well, the issue is not so much whether machine lie detection should always be excluded. It’s whether there’s any sound scientific reason to think that a machine is more reliable at discovering lies than a member of the jury. The legal system tries to let the jury be the arbiter of what is and is not true – and that will still be true, ordinarily, even if the brain-scan evidence is admitted. A jury could choose to disregard it, or adopt the interpretation of the opposing party. The Daubert standard requires the judges to be gatekeepers of the quality of information that reaches the jury. One of things you see here is that the prosecution will analogize the fMRI to discredited polygraphy and the defense will analogize fMRI lie-detection to widely accepted DNA forensic tests.
Wired.com: The prosecution, which is trying to exclude the fMRI evidence, called Marcus Raichle and Peter Imrey, a neuroscientist and statistician, respectively.
Jones: Raichle is an extremely distinguished neuroscientist and is considered to be among the founders of functional brain imaging generally. [I should acknowledge both he and Imrey are colleagues of mine, within the Law and Neuroscience Project.] He testified about the extent to which fMRI had not been adequately tested in real world contexts.
He testified that in his view the peer review and publication prong of Daubert — particularly with respect to the difference between group averaged studies and claims about individual states of turthtelling or lying – was not sufficiently satisfied. And with respect to general acceptance, Raichle offered compelling evidence that although there were consistencies in the findings, they were not necessarily applicable to lie detection in a particular case. The cross examination focused on the extent to which fMRI is generally accepted.
Imrey gave a masterful overview of biostatistics as applied to the lie detection context. And basically encouraged the court to think about the extent to which the relevant question was whether or not this lie detection technique as used by Cephos is sufficiently valid. In his view, it suffered many statistical failings including selection bias, measurement bias, confounding and chance. He emphasized the extent to which the error rate was not sufficiently known for the court to conclude that the testimony was admissible. There was much discussion of the extent to which Dr. Laken may be overinterpreting the statistics
Wired.com: After the prosecution’s witnesses, Laken got recalled on Friday. Were there any fireworks there?
Jones: In much of this part of the testimony, Dr. Laken described why he did not believe his study was vulnerable to the criticisms that Dr. Imrey and Raichle had leveled at it. There were many technical statistical points to which he and Dr. Imrey obviously disagree. For example, he believes that the sample sizes in the relevant studies were enough and that the individual in appropriate circumstances can serve as his own control for both lying and truth telling. In that respect, it’s important to mention that Dr. Semrau had been asked to answer each of the questions in both the affirmative and in the negative.
Wired.com: Now that the hearing is over and we expect a ruling within a couple of weeks, do you have any impressions about who might win the motion? I’m asking explicitly for your opinion here as a professor of both law and biology.
Jones: Both sides presented strong cases and strong testimony. The defense expert was very polished and thorough. The prosecution experts were very knowledgeable and specific. Both sides presented the judge with a great deal of relevant evidence. At the end of the day, the government will probably win the recommendation to exclude this evidence. This is in part because it is probably the case that fMRI lie detection is, though promising, insufficiently sound at present to pass the Daubert admissibility test.
It is also relevant to consider that if this evidence were admitted for this purpose, it would be the first known time in the United States that the jury would hear fMRI lie-detection evidence. The significance of that is that courts tend to move slowly with respect to new technologies.
Wired.com: Does fMRI brain scan evidence getting its first Daubert hearing say anything about the state of the technology?
Jones: The fact that this court chose to hold a Daubert hearing has no precedential value with regard to other courts. What is significant about this case is that whichever way the recommendation comes out, there is likely to be more litigation both in this case and future cases over the admissibility of fMRI lie detection. If it’s admitted, it has some persuasive, though not precedential value for other judges. If it is not admitted the reasons will be specified and those attempting to perform more studies of fMRI lie detection and to have it admitted in the future will have a bit of a roadmap of what to do to increase the chances of getting it in.
Wired.com: Is there anything special about the way the defense is trying to use fMRI in this case?
Jones: One of the things about this case that has gone undernoticed is that even though fMRI lie detection has not yet been admitted, the purposes for which people are seeking to admit it are already rapidly evolving. In this case, the defense is not attempting to introduce fMRI lie detection for purposes of verifying what was at some past time an external state of the world as, for example, when a hypothetical defendant says he was in his house at the time of the alleged murder. That would be a natural context to use lie detection. You’d ask, “Were you home? Are you lying?”
In this case, the defense is taking it to the meta-level. They are using a scan as evidence of a person’s prior state of mind. What’s at issue is whether the defendant knowingly and willfully did what he did. The defense is therefore attempting to offer fMRI to demonstrate his past state of mind. The report actually says, “Doctor Semrau’s brain indicates he is telling the truth in regards to not cheating or defrauding the government.” It means that we’re introducing evidence of the brain’s current assessment of the brain’s former mental state. That’s one of the things that makes it tricky. He’s trying to have his brain testify as to the prior state of his brain.
For fMRI to have already reached that level of complexity in the first case in which there has been a Daubert hearing gives some indication of how much more future litigation there is likely to be in this arena.
