This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Deceit often occurs in questionnaire surveys, which leads to the misreporting of data and poor reliability. The purpose of this study is to explore whether eye-tracking could contribute to the detection of deception in questionnaire surveys, and whether the eye behaviors that appeared in instructed lying still exist in spontaneous lying. Two studies were conducted to explore eye movement behaviors in instructed and spontaneous lying conditions. The results showed that pupil size and fixation behaviors are both reliable indicators to detect lies in questionnaire surveys. Blink and saccade behaviors do not seem to predict deception. Deception resulted in increased pupil size, fixation count and duration. Meanwhile, respondents focused on different areas of the questionnaire when lying versus telling the truth. Furthermore, in the actual deception situation, the linear support vector machine (SVM) deception classifier achieved an accuracy of 74.09%. In sum, this study indicates the eye-tracking signatures of lying are not restricted to instructed deception, demonstrates the potential of using eye-tracking to detect deception in questionnaire surveys, and contributes to the questionnaire surveys of sensitive issues.
Keywords: lie detection, eye behavior, questionnaire surveys, deception, eye-trackingQuestionnaire is one of the most widely used tools for data collection due to its wide range of applications, flexibility, speed, and convenience (Taherdoost, 2016). However, there is subjectivity and freedom when filling out questionnaires. Thus, answers to sensitive questions are often distorted (such as evaluations of self or others, substance use, sexual activities, political opinions, unsocial attitudes) (Holtgraves, 2004; Krumpal, 2013). The respondents will go through five stages when answering a self-report: (1) Explain the question. (2) Retrieve information. (3) Generate an opinion. (4) Format a response. (5) Edit the response (Sudman et al., 1997). The effect of social desirability usually operates at the final editing stage (Tourangeau and Rasinski, 1988; Sudman et al., 1997; Holtgraves, 2004). Respondents weigh the benefits and risks of telling the truth. When the risks are higher than the benefits, the respondent will choose to lie (Tourangeau et al., 2000; Walzenbach, 2019). Respondents can exaggerate, minimize, omit, and present themselves in a socially desirable light (Bond and DePaulo, 2006; Rauhut and Krumpal, 2008; Preisendörfer and Wolter, 2014; Walzenbach, 2019). Accordingly, lying in surveys can lead to misreported data and reduce the reliability of the findings. Especially, research on sensitive questions is the most likely area of survey misreporting (Lensvelt-Mulders, 2008; Preisendörfer and Wolter, 2014). Fortunately, the design of questionnaires (such as expressions) can change the sensitivity of questions, which will have a massive impact on people’s responses when filling out the questionnaire (Walzenbach, 2019). Thus, it is essential to identify and modify the questions in the questionnaire that tend to cause lying. Lie detection in questionnaire surveys helps to improve the design of questionnaires before they are published, and to avoid using unreliable results.
Lie detection has been a source of fascination. Even though detecting lies is necessary, the accuracy of human detection of lies is around the chance level, with an average of 54% (Bond and DePaulo, 2006). Deception is usually thought to be correlated with cognitive load. There are three main theoretical perspectives on the relationship between deception and cognitive load. The first theoretical perspective is that lying will experience more complex cognitive processes and bear a higher cognitive load than honesty (Zuckerman et al., 1981; Vrij et al., 2001, 2017; Roma et al., 2018). People will modify the answers that meet social desirability at the response editing stage, and there is more hesitation (Holden et al., 2001; DePaulo et al., 2003). The second theoretical perspective is the opposite (Holden et al., 1985; Leary and Kowalski, 1990). When lying, the respondents do not need to recall accurate information, they directly respond according to social desirability and do not move through the retrieve information stage. The third theoretical perspective suggests that response time depends on the lying schema and the social desirability of the test item (Brunetti et al., 1998; Holden et al., 2001). A previous study conducted a meta-analysis of 26 cognitive lie detection studies with a weighted mean of 74% overall accuracy (Vrij et al., 2017). Whereas, to date, most studies investigating lie detection have focused on face to face communication, such as criminal justice scenarios (Porter and ten Brinke, 2010; Synnott et al., 2015; Vrij and Fisher, 2016) and conversation scenarios (Vrij, 2018; Nahari and Nisin, 2019). The literature investigating the questionnaire surveys without verbal cues is not as rich. Moreover, in the field of lie detection in questionnaire surveys, most studies have only focused on lie detection on personality tests (van Hooft and Born, 2012; Mazza et al., 2020). However, questionnaires cover a wide range of areas, not just limited to personality tests. But up to now, far too little attention has been paid to lie detection in broader questionnaire areas.
Extensive studies about lie detection were limited to simulated scenarios, where participants were instructed to lie. Nevertheless, when instructed to lie, participants’ motivations are low, and they probably do not have any concern with the accuracy and need not fear their behaviors are detected (von Hippel and Trivers, 2011; van Hooft and Born, 2012). For this, several authors have proposed that deception detection studies should be conducted in a more ecological way (Wright et al., 2013; Levine, 2018). As Ganis et al. (2003) and Yin et al. (2016) discussed, there are different patterns of activation while expressing rehearsed or spontaneous lies in fMRI. Furthermore, Delgado-Herrera et al. (2021) performed a meta-analysis of fMRI deception tasks through a review from 2001 to 2019, and the results showed that the tasks with low ecological validity and high ecological validity lead to different areas of brain activation, perhaps because the tasks with high ecological validity are more realistic, and engage a broader network of brain mechanisms. In contrast, the Concealed Information Test results of Geven et al. (2018) showed no significant differences in skin conductance, heart rate, and respiration between spontaneous deception and instructed deception. Ask et al. (2020) found that instructed lies have little effect on human lie-detection performance. Whether the findings of the deception detection for instructed lies can be applied to reality remains controversial. There may be discrepancies between the mental processes of instructed lying and spontaneous lying in real life.
Eye-tracking is often considered an ideal measure for lie detection, as eye behaviors are automatic physiological responses that cannot be consciously controlled (Chen et al., 2013; Gonzales, 2018; Berkovsky et al., 2019). Eye-tracking is an appealing sensor for deception detection in questionnaire surveys, as it does not require direct physical contact (which may disturb the respondents), is easy to use, collects diversified information and can be used in automated screening systems (Cook et al., 2012; Proudfoot et al., 2015; Zi-Han and Xingshan, 2015; Ye et al., 2020). Previous studies showed that eye behaviors reflect people’s cognitive load (Zagermann et al., 2016), emotions (Zheng et al., 2014; Perkhofer and Lehner, 2019; Lim et al., 2020), attention (Lee and Ahn, 2012; Tsai et al., 2012), information processing (Bruneau et al., 2002). High cognitive load usually causes pupil dilation, decreased blink rate, increased saccade velocity and fixation duration (Wang et al., 2014; Zagermann et al., 2016; Einhäuser, 2017; Behroozi et al., 2018; van der Wel and van Steenbergen, 2018; Keskin et al., 2019, 2020). Arousal changes can affect blinks, saccades and fixations (Maffei and Angrilli, 2018), vigilance and fatigue can be detected in saccades, and information process can be predicted in saccades and fixations (Bruneau et al., 2002; Maffei and Angrilli, 2018). Fixation location can indicate the area of current focus (Rudmann et al., 2003). These all help to analyze the mental processes of deception. Furthermore, many studies have applied eye-tracking to detect deception with promising results. Deception changes people’s fixation patterns (Twyman et al., 2014). When lying, the pupil diameter becomes larger due to cognitive load, memory retrieval, vigilance, anxiety, etc. (Twyman et al., 2013; Proudfoot et al., 2016). Vrij et al. (2015) concluded that memory retrieval is greater when lying, so the saccade velocity is higher. George et al. (2017) found that the blink duration and blink count are higher when lying. Webb et al. (2009) suggested that people experience greater arousal when lying, resulting in greater pupil dilation and blink frequency. Borza et al. (2018) analyzed the eye movements to detect deception and obtained an accuracy of 99.3% on the dataset. van Hooft and Born (2012) found that on the personality test, more fixations occurred on the extreme response options when lying, while more fixations occurred on the middle response options when lying honest. They achieved 82.9% lie detection accuracy with eye-tracking. Consequently, eye behaviors attract more attention as psychological and physiological indicators of lying (Bessonova and Oboznov, 2018).
In summary, few studies investigated lie detection in questionnaire surveys, and the mental processes of spontaneous lying may not be identical to that of being instructed to lie. Therefore, this study simulated the scene of evaluating teachers to explore whether the subtle reaction of lying could be identified by eye-tracking in the questionnaire research scenario, and examined whether the changes in eye behaviors during instructed lying can be generalizable to spontaneous lying. In Study 1, the relationship between eye-tracking indicators and deception was initially explored, following the study of van Hooft and Born (2012), the participants were instructed to lie or be honest. We hypothesized that there would be significant differences in eye behaviors between lying and honesty condition in Study 1, which is consistent with the study of van Hooft and Born (2012). However, spontaneous lying in actual situations may cause more diverse mental processes. Consequently, we designed Study 2 to test whether the relationship between eye-tracking indicators and deception is still valid in the actual situation. In Study 2, this study created the motivation for participants to lie, and encouraged participants to lie spontaneously and genuinely. Study 2 investigated the eye behaviors when lying in the actual situation and compared them to the findings of Study 1 to examine if the eye behaviors that appeared in instructed lying still exist in spontaneous lying, and thus identify reliable eye movement indicators for detecting lies. In Study 2, our main hypothesis is that eye-tracking can effectively help to detect deception in questionnaire surveys in realistic situations. The present study has explored whether the eye behaviors in instructed lying can be generalized to reality, found reliable variables for lie detection in the actual situation, and could contribute to understanding the relationship between deception and eye behaviors. Moreover, this study confirmed the potential of eye-tracking in non-verbal lie detection, offered implications for detecting deception in questionnaire surveys.
A scenario was set to ask participants to evaluate their teachers. Chinese students are generally respectful of their teachers and desire to please their parents, teachers, and other people in positions of power (Bear et al., 2014). Chinese cultural expectations of the teacher–student relationship are “well-defined, rigidly hierarchical and authoritarian” (Ho and Ho, 2008). As the old Chinese idiom says, “once my teacher, forever my parents.” Students should respect their teachers as they respect their parents, including showing obedience (Hui et al., 2011). Respect for teachers is a revered virtue in China. Chinese students have high respect for those who provide knowledge and avoid challenging authority (Wei et al., 2015). Meanwhile, when evaluating leaders, students often worry that their teachers can be able to view their evaluations and thus judge them negatively. Hence, most students will choose to make no bad comments in real-name conditions to prevent adverse effects.
Participants were asked to recall a teacher they disliked. Then they were instructed to fill out the questionnaire according to the actual situation and imagine that the evaluation was in real-name condition.
A questionnaire was designed for teacher evaluation. The questionnaire consists of 10 questions, including the evaluation of teaching level and attitude toward the teachers. A five-point scale was used in the study, with negative and positive keywords on either side of the options. Furthermore, this study defined several areas of interest (AOIs). The question text (QT), the extreme negative option (NO), the negative keyword (NK), the extreme positive option (PO), the positive keyword (PK), the extreme options (EO), and the medium options (MO) were defined as boxes of interest. The questionnaire and marked AOIs are shown in Figure 1 .
Example of stimuli. (A) Example of the questionnaire with marked AOIs. (B) Example of the questionnaire after translation into English. AOIs, areas of interest.
An SMI iView X TM RED desktop system with a high spatial resolution (0.005°) and a sampling rate of 500 Hz (1-ms temporal resolution) was used to record the participants’ eye behaviors. The system includes an iView PC. The operator controls the experiment, a 22-in. display screen (pixel resolution 1680 × 1050) to show the experimental stimuli to the participants. And an eye-tracking module was installed under the display screen to track the real-time eye behaviors of the participants.
Thirty one participants, including 18 males and 13 females, were recruited from Sichuan University, aged 20–26 (M = 22.68). All participants were healthy, had normal or corrected-to-normal visions, and had no reported history of neurological or psychiatric disorders. All of them received a small honorarium for their participation.
Firstly, participants were asked to recall a teacher whom they disliked and describe him/her simply. Then, participants were told that they would evaluate the teacher through questionnaires, and their eye behaviors were recorded. Afterward, participants were provided with instructions that directed them to respond honestly or to imagine responding under the condition of real-name evaluation. Each participant was required to answer the questionnaire in the above two situations. The instructions were adapted from previous studies (McFarland and Ryan, 2000; van Hooft and Born, 2012). To eliminate the influence of order, the order of lying and honesty is random. An irrelevant questionnaire would be interspersed between the two responses to eliminate learning effects.
The instruction for encouraging participants to respond honestly is as follows:
You will be presented with ten questions with five response options. Please answer the questions as honestly as possible. Your answers remain confidential and will be used for research purposes only. For this study, we are interested in your honest answers, so please answer the following questions as accurately and honestly as possible.
The instruction for directing participants to imagine evaluation as a real-name situation to respond is as follows:
You will be presented with ten questions with five response options. Please imagine that the teacher you are evaluating can see your answers in real-name. For this study, we are not interested in your honest answers. Instead, for each question, please select the answer you think is more beneficial to you.
After understanding the requirements, Participants sat about 60 cm from the screen. After 2–4 times of eye-tracking calibration, the experimental material was displayed on the screen. The participants were required to respond to complete the evaluation questionnaire. By comparing the differences in the participants’ ratings, this study selected the questions that were rated differently. Afterward, we confirmed with participants whether the differences of rating in each question were caused by lying in the imagined real-name condition. The procedure of the experiment is shown in Figure 2 .
The experiment procedure of Study 1.
