Imagine if lie-detection apps became widely available on everyone’s phones or computers. Merely by processing a video of what you are saying to someone–by analyzing your facial expressions, body movements, auditory details, and the semantic content of your speech—the apps could use data science to attempt to predict if you are lying or telling the truth about who you voted for, your salary, your fidelity, and the list goes on.
“What will that do to society?” asks Ehsan Hoque. “What will it do to our relationships?”
So, even as the University of Rochester computer scientist creates ground-breaking computer technology to detect lies for a host of beneficial medical and public safety purposes, he also wants to help ensure the technology is used constructively—and ethically.
He’ll have the chance to pursue both his goals—developing the technology and establishing a framework for its appropriate use—with a $1 million Early Career Award for Scientists and Engineers (ECASE-ARMY) from the Army Research Office (ARO).
ECASE awards are the highest honor bestowed by the ARO to outstanding scientists and engineers beginning their independent careers. The awards recognize researchers who pursue innovative research at the frontiers of science and technology and who also engage in scientific leadership, education, or community outreach.
“This is a big boost that will help my lab continue to pursue blue-sky research,” says Hoque, an assistant professor of computer science and the Asaro Biggar Family Fellow in Data Science. He is also interim director of the Goergen Institute for Data Science.
“The Army Research Office encourages young scientists to take on ridiculously difficult problems and allow ample intellectual freedom to pursue them,” Hoque says. “It is a wonderful opportunity for young scientists like myself to take on high risks early in their career.”
The technology Hoque hopes to develop with this award builds upon his lab’s recent work on understanding deception based on facial and verbal cues. His Rochester Human-Computer Interaction (ROC-HCI) Lab used data science and created an online crowdsourcing framework called ADDR (Automated Dyadic Data Recorder) to build the largest publicly available deception dataset so far – and to discover, for example, why some smiles are more deceitful than others.
Applications include:
- improved security screening of passengers in airports;
- helping medical practitioners realize when patients are hiding suicidal symptoms;
- new ways for law-enforcement agents to assess risk when engaging in self-defense;
- and helping individuals with autism realize when they are being manipulated or deceived.
Making interrogations more objective
With the new award, Hoque and his lab will develop machine-learning algorithms that could make the interrogations and interviews used in these settings more objective.
A typical interrogation is conducted in distinct phases, starting with general questions to establish rapport.
“How someone answers in the rapport-building phase could influence how the interviewer looks at that person when the important, relevant questions are asked in the subsequent phases, biasing the interviewer’s ability to tell if someone is lying or telling the truth,” Hoque says.
“So, imagine as I’m interviewing you, I have a computer that is helping me out, that can treat each phase as unique and objective,” Hoque says. “We can build algorithms to help quantify some of the nuances that an interviewer might miss–subtle inconsistencies among facial cues, what is said, how it is said.”
At the end of each phase, the algorithm, trained with reinforcement learning, would recommend whether it has “seen” enough to recommend going to the next phase, or whether additional questions should be asked.
The interviewer would still be in charge and still make the decisions, Hoque emphasizes, but “the algorithm is providing independent, quantifiable metrics, so that the interrogators can further quantify their decisions. It adds objectivity and transparency to the interrogation process.”
Greater objectivity could benefit both the interviewer and the person being interviewed, especially in security and law enforcement settings, he says.
Legal and ethical issues of lie-detection technology
But Hoque worries about what could happen when the technology becomes widely available. A possible worst case scenario: Mandatory use of the technology during job interviews.
“We have talked with people who say they would never consider lying on job interviews but would be unwilling to do an interview if a camera was turned on them with lie-detecting technology,” Hoque says. “It’s the comfort factor. So what kind of protection would be available for these people?”
To explore the legal and ethical issues that might arise, his team will identify the ways the technology could be misused and recommend a framework for use aimed at the maximum net gain for society.
Hoque will collaborate with Mark Frank, a professor of communication at the University at Buffalo who has more than 25 years of experience studying deception with US law enforcement agencies.
“I have a fairly well-developed eye for work that would be not just scientifically sound, but also of clear utility, especially in law enforcement and national security settings,” Frank says. “I strongly believe Dr. Hoque is on the right track here to make some practical breakthroughs.”
Technologies that enhance human ability using data science
“Professor Hoque is considered a pioneer in designing and validating AI technologies that can improve human ability,” a Department of Defense fact sheet notes.
For example, as a PhD student at MIT, Hoque developed an app called My Automated Conversation Coach–a 3D character that can automatically play the role of a recruiter, conduct a job interview, and provide feedback. The app provided the first scientific evidence that it is possible for humans to improve their face-to-face interpersonal skills by interacting with a computer.
Hoque has since applied this framework across multiple domains: public speaking, music training, effective conduct of meeting, and end-of-life conversations between cancer patients and their oncologists, among others
His previous honors include a National Science Foundation CAREER award and selection as one of Science News’s 10 Scientists to Watch and MIT Technology Review’s Top 35 Innovators under 35.
He is also an inaugural member of the ACM Future of Computing Academy, established to guide and empower the next generation of computing leaders.
Read more
Using data science to tell which of these people is lying
University researchers are using data science to analyze more than 1 million facial expressions to more accurately detect deception based on a smile.
Conversing with computer may fight fear of public speakingPublic speaking in the number one reported fear of American. The ROCspeak platform is a powerful technology tool to fight the fear.
