Perceptive Hierarchy of Facial Skin Lesions: An Eye-tracking Study

Marek Jankowski1 and Agnieszka Goroncy2

1Chair of Dermatology and Venereology, Faculty of Medicine in Bydgoszcz, Bydgoszcz and 2Department of Mathematical Statistics and Data Mining, Faculty of Mathematics and Computer Science, Nicolaus Copernicus University in Torun, Torun, Poland

Equal importance is given to every skin lesion in treatment guidelines and severity grading systems for facial lesions. Face recognition studies suggest differentially perceived areas of the human face. The aims of this study were to quantify the visual attention given to facial skin lesions and to explore their hierarchy. Eye movements were tracked in 118 participants who viewed 135 faces with facial skin lesions. The main effect of the image was significant (F[9, 1053]=15.631, p < 0.001, η2=0.118), which implied a difference in the total visual attention between images depicting skin lesions in different aesthetic units. Lesions in the frontal area received the highest area- specific attention, while lesions in the right parotid area had the smallest effect. Using objective computational clustering, 2 very distinct zones of visual attention were identified: the ocular, nasal, perioral, and frontal areas attracted high visual scrutiny and the remaining areas attracted little attention. However, the presence of skin lesions in the high-attention region resulted in a decrease in total visual attention to the face. The main effect of the aesthetic unit was significant (F[15, 1755]=202.178, p < 0.001, η2=0.633). More than 63% of attention-drawing potential of facial skin lesions depends on their anatomical location, which should be considered in disease severity grading and treatment planning.

Key words: acne; eye-tracking; social perception; scar; visual perception.

Accepted Sep 29, 2022; Epub ahead of print Sep 29, 2022

Acta Derm Venereol 2022; 102: adv00799.

DOI: 10.2340/actadv.v102.2514

Corr: Marek Jankowski, Chair of Dermatology and Venereology, Faculty of Medicine in Bydgoszcz, Nicolaus Copernicus University, ul. Sklodowskiej-Curie 9, PL-85-094 Bydgoszcz, Poland. E-mail: marek.jankowski@cm.umk.pl

SIGNIFICANCE

Equal importance of each skin lesion is an unchallenged paradigm of the current treatment guidelines and severity grading systems for facial skin diseases. By tracking the eye movements of observers, this study demonstrates, for the first time, that visual attention to facial skin lesions depends on the latter’s anatomical location. Two distinct zones of high and low visual attention were identified. The attention-drawing potential of facial skin lesions depends on their anatomical locations, which should be considered in disease severity grading and treatment planning.

INTRODUCTION

Skin conditions that affect the face are significantly linked to increased psychosocial burden and have major implications for a patient’s quality of life and self-esteem (1, 2). Therefore it must be ensured that patients with facial skin lesions receive adequate therapy. For several conditions that affect the face, evidence-based therapeutic guidelines were recently published to assist clinicians in making appropriate treatment decisions (3–5). Treatment choices are recommended after taking the disease severity into account.

For acne, one of the commonest dermatological conditions that affects the face, there are at least 25 published severity grading systems based on the type and number of lesions (6–8) with equal importance given to lesions in different anatomical locations. Only in the Global Acne Grading System (GAGS) system, are double values assigned to some anatomical locations (9). This distinction is arbitrary, as the authors did not provide any rationale behind this assignment. Accordingly, neither acne scars nor acne rosacea grading systems take into account the anatomical location (10, 11).

This unchallenged paradigm of equal importance given to each facial skin lesion seems untenable from the perspective of current research on face recognition. Although humans have idiosyncratic and task-specific scan-paths for judging faces, it is well-established that the triangular area between the eyes and mouth receives the highest attention in terms of both frequency and duration of eye fixation (12, 13). This area is also particularly relevant for the decoding of emotional expressions (14, 15). It would be logical to expect that the presence of skin lesions in such high visual attention areas would have a greater effect on how the face is perceived and, by extension, would have more detrimental effects on a patient’s quality of life, which could explain the poor correlation of the current disease severity scores with patients’ quality-of-life scores. However, there is no published study addressing this hypothesis.

Furthermore, conditions such as basal cell or squamous cell cancer have predilections for specific aesthetic units of the face (16, 17); therefore, location-dependent perceptual bias for skin lesions could affect a patient’s self-assessment.

This study aimed to investigate visual recognition of faces with skin lesions and to explore any possible hierarchy in the visual attention received by lesions in various anatomical parts of the face using tracking of eye movements. Based on objective computational methods, we propose a model of facial lesion perception with a 2-tier hierarchy of anatomical locations.

MATERIALS AND METHODS

Participants

This study included 118 adults (75 females) aged between 16 and 62 years (mean (standard deviation [SD] 32.3 (9.71) years) who were blinded to the purpose of the study. All participants were right-handed and had normal or corrected-to-normal vision with no history of neurological disorders. The participants had no medical training.

Stimuli

Frontal colour photographs (The Ethnic Origins of Beauty, www.lesoriginesdelabeaute.com, photographed by Natalia Ivanova) of 5 young women were digitally combined with skin lesions extracted from patients’ images from the authors’ clinical archive. In order to avoid other-race effects, all models were Central European Caucasians. We created a set of 135 photographs that depicted either solitary lesions (surgical scar and melanocytic naevus) or dispersed lesions (nodulocystic acne, acne papules, and acne atrophic scars) in each aesthetic unit of the face. Solitary lesions were presented in a less salient right visual field. For each aesthetic unit, the same lesions were presented in 3 different models to limit the confounding of the models’ facial proportions. All images were of similar size and aligned for midline and interpupillary lines. Examples are shown in Fig. S1.

Procedure

Eye movements were recorded using the recently validated Gazepoint GP3-HD eye-tracker (Gazepoint, Vancouver, Canada) (18). Photographs were presented for 3 s each in life-size and a random order. To ensure that participants focused on the photographs for the duration of the study and to limit the confounding effect of familiarization, each participant was presented with only 36 modified and all 5 original reference images. Consequently, for each modified image, eye fixations were recorded from at least 39 individual participants the reference images were recorded from 118 tests.

Areas of interest representing the aesthetic units of the face (17, 19) were marked with shape rectification secondary to the technical constraints.

Statistical analysis

Statistical analysis included repeated-measures analysis of variance and post-hoc Student’s t-tests with Bonferroni correction, Pearson’s χ2 test of independence, principal component analysis, and k-means clustering algorithm. All the calculations and statistical procedures were performed using R (v4.0.3) and IBM SPSS Statistics (v27) and were considered statistically significant at p-values less than 0.05.

RESULTS

Comparisons of number of fixations

Nasal, frontal, ocular, and labial areas attract significantly more fixations than the other areas (Fig. 1a). The increase in the mean number of fixations for skin lesions was the highest for the frontal area, followed by the nasal, perioral, and left temporal areas (Table I). This study compared the means of averaged fixations between the sexes and did not find significant differences (p = 0.22).

The main effect of the image was significant (F[9, 1053]=15.631, p < 0.001, η2=0.118), which implied a difference in the total visual attention between images of skin lesions within different aesthetic units. In particular, the number of fixations is significantly lower for images of skin lesions in the frontal, nasal, perioral, and mental areas compared with those for healthy-looking faces (Fig. 1b).

The main effect of the aesthetic unit was also significant (F[15, 1755]=202.178, p < 0.001, η2=0.633), which implied that the anatomical location explains more than 63% of the variability in attracting the viewer’s attention.

The differences in fixations were analysed with respect to the image and aesthetic unit simultaneously to investigate the type of interaction effect. The results are summarized in Fig. 2. For mental, bilateral temporal, left infraorbital, and left parotid areas, the number of fixations was significantly higher in images of lesions, while the number of fixations in the remaining aesthetic units was constant. Similarly, lesions in the right parotid and both buccal or zygomatic areas had more effects on fixation to the respective and immediately adjacent areas without affecting the attention to the remaining regions. In a third distinctive group of aesthetic units, which included both ocular, right infraorbital, nasal, labial, and frontal regions, a significantly low number of fixations was noticeable in several images.

Clustering analysis

The data were subject to clustering analysis to identify any objective hierarchy of perception of facial lesions. Since the number of fixations, duration of viewing, and number of revisits are strongly positively correlated (r ≥ 0.68 for healthy-looking and r ≥ 0.63 for lesioned images [p < 0.001]), the principal component analysis and data standardization were performed before applying the k-means clustering algorithm to the raw data for both sets separately.

Two clusters of facial regions that attracted a little and a lot of visual attention were identified in both datasets. There was a significant association between the cluster and the aesthetic unit for healthy-looking (η2[9, n = 9,440]=3,482.6, p < 0.001) as well as lesions (η2[9, n = 84,960]=18,659, p < 0.001). However, the effect sizes for these findings were moderate (Cramer’s V=0.43 and 0.33, respectively) (Fig. 3). The low-attention cluster was predicted to include infraorbital, zygomatic, temporal, buccal, parotid, and mental aesthetic units, while the high-attention cluster included ocular, nasal, labial, and frontal aesthetic units.

Although there was an association between the cluster assignment and the presence of skin lesions in analysed images (η2[8, n = 84,960]=34.969, p < 0.001), the effect size was negligible (Cramer’s V=0.014), which supports the universality of cluster assignment.

DISCUSSION

The frontal, ocular, nasal, and perioral areas in the human face receive a high rate of fixation. Objective algorithmic clusterization included visual dwell time and the number of revisits and corroborated this finding. Furthermore, in contrast to the other areas, attention to these areas is negatively affected by skin lesions elsewhere in the face. Based on these findings, we propose a model of visual attention to faces with skin lesions in 2 major areas (Fig. 4): the aforementioned area that receives high and global attention with high visual scrutiny and is negatively affected by lesions in other areas, and the remaining areas that receive low and local attention with few fixations. Visual attention to the latter areas is increased at the expense of attention to the high-attention areas. Temporal and mental areas are affected only by lesions present in those areas. In contrast, the zygomatic, buccal, and parotid areas form a subcluster, where lesions in 1 area attract attention to all 3 areas. One could argue that the preference for frontal, ocular, nasal, and perioral areas is due to their proximity to the midline and the use of only frontal images in this study. However, low attendance in the equidistant mental and infraorbital areas negates this hypothesis.

The results of this study suggest the perceptional importance of the central features of the face. The frontal and nasal areas have the greatest negative effect on the attention to the eyes and mouth, which are areas required for face identification and emotion-encoding. Clustering results further support the shifting of attention in the presence of facial lesions. As humans fixate longer on the eyes in faces with sad expressions and longer on mouths with happy expressions (15), any shift in attention could affect the decoding of emotions of individuals with skin lesions.

A very significant number of all fixations were allocated to the frontal area and was comparable to that of the ocular regions. It is in contrast with previous studies that emphasized attention to the triangle bound by the eyes and mouth over the remaining “featureless area.” However, we noticed that attention to this region was often not measured even if the total duration of fixation over the “featureless” area (which included the frontal region) was longer than any area of the “central triangle” (20).

Skin lesions in the frontal area attract the most significant attention in the face. However, lesions elsewhere reduce the attention to the frontal region. If we consider that criminals throughout history were branded on the forehead and that the Bible mentions in several passages “seal on the foreheads of the servants of God,” the potential of frontal skin lesions to attract attention was subconsciously obvious even in ancient times.

Attention to the nose remains unaffected by lesions in the adjacent infraorbital regions and is only moderately affected by perioral lesions, while it is grossly decreased for lesions in any other area. The largest loss of attention to the nasal area results from frontal lesions. Although the current study did not include palpebral lesions, diversion of attention to the lesioned ocular area at the expense of primary attention to the nasal area was reported recently (21). Skin lesions in the nasal area attracted significant attention in the current study and were second only to frontal lesions. This corroborates previous reports on the attention-drawing potential of nasal deformities (21).

With respect to visual attention to the ocular region, there was no significant difference if the skin lesions are present in infraorbital, zygomatic, and temporal areas; otherwise, visual attention to the eyes was decreased by skin lesions. This lack of “perceptive penalty” for ocular attention in the presence of zygomatic and temporal lesions does not fit well into our paradigm of high/global and low/local perception dichotomy. One possible explanation is offered by the previously reported human preference for the first fixation just below the eyes during facial recognition (22). Because of the holistic nature of face processing, fixation on these points could allow for processing both ocular and mid-facial features effectively. In addition, facial processing is conveyed largely via mechanisms tuned to horizontal visual structures, which may explain the lack of decrease in attention to the eyes due to temporal lesions (23).

Attention to the perioral area is highly increased by lesions in that area and mental area and slightly increased by lesions in the adjacent buccal area, while lesions elsewhere decrease the attention to this region. Lesions in the frontal and nasal areas result in the most significant loss of attention to this region.

We found that, compared with healthy faces, the total number of fixations over the facial area decreased irrespective of the location of the face skin lesions. This is a novel finding, which we cannot compare with other quantitative studies on facial skin defects as it has not been reported previously. However, this is in agreement with previous qualitative studies, which indicate that total visual attention is at least partially a function of facial attractiveness. A tendency to gaze longer on attractive faces is evident already in newborns (24).

A novel finding of this study is that, using quantitative means, the nose was identified as the most important area where skin lesions result in global visual avoidance despite a local increase in visual scrutiny. Further studies are required to elucidate the implications of this finding. There was lower, but still significant, loss of attention induced by lesions in other “high and global” areas, which further supports our model. We believe that this finding is consistent with the affective events theory and stigma theory. Previous studies have demonstrated that, particularly in dyadic, face-to-face interactions, stigmatized individuals tend to elicit visual avoidance (25, 26). Visual avoidance is believed to reduce the cognitive load. We hypothesize that this “turning of a blind eye” to faces with skin lesions in the central area, which is crucial for emotion-encoding, may be related to previously demonstrated decreased cognitive attention to stigmatized persons (27).

Study limitations

This study has several limitations. Technical restrictions of the eye-tracking software forced us to use orthogonal shapes for the aesthetic units, which could have lowered the differences between the regions evaluated. The use of only frontal images in the current study could have resulted in an underestimation of attention to the extremely lateral areas of the face. Therefore, further studies that include lateral views of the face are required. Due to the predominance of young females in the study group, we could not analyse if the participant’s age and sex affected the perception of skin lesions (28).

The current study population included exclusively Polish nationals, but gaze patterns and face processing are known to be culture-dependent. Western Caucasian observers fixate more on the eyes and mouth, while East Asians fixate preferentially on the nose. Westerners tend to engage analytic perceptual strategies for processing the visual environment, whereas East Asians use holistic perceptual strategies (29). Therefore, the results of the current study may be limited only to Western Caucasian observers. Furthermore, it is known that attention to the ocular and perioral area varies with different emotions; consequently, the use of visual stimuli with only neutral faces limits the relevance of our results in real-life situations. Finally, visual attention to facial features in the current experimental setting is unidirectional. However, in any face-to-face interaction between 2 people, attention is mutually modulated. An individual’s cognitive processing, both self and that focused on others, is affected by the presence of other observers and modulated according to the social norms of eye gaze (30).

The outline of the face has been demonstrated recently to guide fixations to the more salient central facial features (31). The visual stimuli used in this study included skin lesions that did not affect the outline of the face. It would be interesting to examine if this only minimal effect on facial perception would persist with visual stimuli of skin lesions that disrupt the hairline or jawline.

Finally, the pattern of attention to facial features discussed here is an averaged sum of idiosyncratic gaze patterns of individual observers. Gaze patterns vary strikingly between the sexes and individuals; however, they are thought to be stable in time for each individual (32, 33). Further studies are necessary to examine how individual facial gaze patterns are affected by the presence of facial lesions.

The authors have no conflicts of interest to declare.

REFERENCES

1. Do JE, Cho SM, In SI, Lim KY, Lee S, Lee ES. Psychosocial aspects of acne vulgaris: a community-based study with Korean adolescents. Ann Dermatol 2009; 21: 125–129.
2. Tasoula E, Gregoriou S, Chalikias J, Lazarou D, Danopoulou I, Katsambas A, et al. The impact of acne vulgaris on quality of life and psychic health in young adolescents in Greece. Results of a population survey. An Bras Dermatol 2012; 87: 862–869.
3. Hald M, Arendrup MC, Svejgaard EL, Lindskov R, Foged EK, Saunte DM; Danish Society of Dermatology. Evidence-based Danish guidelines for the treatment of Malassezia-related skin diseases. Acta Derm Venereol 2015; 95: 12–19.
4. Nast A, Dréno B, Bettoli V, Bukvic Mokos Z, Degitz K, Dressler C, et al. European evidence-based (S3) guideline for the treatment of acne – update 2016 – short version. J Eur Acad Dermatol Venereol 2016; 30: 1261–1268.
5. Salleras M, Alegre M, Alonso-Usero V, Boixeda P, Domínguez-Silva J, Fernández-Herrera J, et al. Spanish consensus document on the treatment algorithm for rosacea. Actas Dermosifiliogr (Engl Ed) 2019; 110: 533–545.
6. Stathakis V, Kilkenny M, Marks R. Descriptive epidemiology of acne vulgaris in the community. Australas J Dermatol 1997; 38: 115–123.
7. Ramli R, Malik AS, Hani AF, Jamil A. Acne analysis, grading and computational assessment methods: an overview. Skin Res Technol 2012; 18: 1–14.
8. Bernardis E, Shou H, Barbieri JS, McMahon PJ, Perman MJ, Rola LA, et al. Development and initial validation of a multidimensional acne global grading system integrating primary lesions and secondary changes. JAMA Dermatol 2020; 156: 296–302.
9. Doshi A, Zaheer A, Stiller MJ. A comparison of current acne grading systems and proposal of a novel system. Int J Dermatol 1997; 36: 416–418.
10. Clark AK, Saric S, Sivamani RK. Acne scars: how do we grade them? Am J Clin Dermatol 2018; 19: 139–144.
11. Wilkin J, Dahl M, Detmar M, Drake L, Liang MH, Odom R, et al. Standard grading system for rosacea: report of the National Rosacea Society Expert Committee on the classification and staging of rosacea. J Am Acad Dermatol 2004; 50: 907–912.
12. Kanan C, Bseiso DN, Ray NA, Hsiao JH, Cottrell GW. Humans have idiosyncratic and task-specific scanpaths for judging faces. Vision Res. 2015; 108: 67–76.
13. Rogers SL, Speelman CP, Guidetti O, Longmuir M. Using dual eye tracking to uncover personal gaze patterns during social interaction. Sci Rep 2018; 8: 4271.
14. Eisenbarth H, Alpers GW. Happy mouth and sad eyes: scanning emotional facial expressions. Emotion. 2011; 11: 860–865.
15. Calvo MG, Fernández-Martín A, Gutiérrez-García A, Lundqvist D. Selective eye fixations on diagnostic face regions of dynamic emotional expressions: KDEF-dyn database. Sci Rep 2018; 8: 17039.
16. Omodaka T, Minagawa A, Okuyama R. Ultraviolet-related skin cancers distribute differently on the face surface. Bri J Dermatol 2021; 185: 205–207.
17. Choi JH, Kim YJ, Kim H, Nam SH, Choi YW. Distribution of basal cell carcinoma and squamous cell carcinoma by facial esthetic unit. Arch Plast Surg 2013; 40: 387–391.
18. Cuve HC, Stojanov J, Roberts-Gaal X, Catmur C, Bird G. Validation of Gazepoint low-cost eye-tracking and psychophysiology bundle. Behav Res Methods 2022; 54: 1027–1049.
19. Gonzalez-ulloa M: Restoration of the face covering by means of selected skin in regional aesthetic units. Br J Plast Surg 1956; 9: 212.
20. Liao D, Ishii M, Darrach HM, Bater KL, Smith J, Joseph AW, et al. Objectively measuring observer attention in severe thyroid-associated orbitopathy: A 3D study. Laryngoscope 2019; 129: 1250–1254.
21. van Schijndel O, Litschel R, Maal TJ, Bergé SJ, Tasman AJ. Eye tracker based study: Perception of faces with a cleft lip and nose deformity. J Craniomaxillofac Surg 2015; 43: 1620–1625.
22. Hsiao JH, Cottrell G. Two fixations suffice in face recognition. Psychol Sci 2008; 19: 998–1006.
23. Dakin SC, Watt RJ. Biological “bar codes” in human faces. J Vis 2009; 9: 2.1–2.10.
24. Slater A, Von der Schulenburg C, Brown E, Badenoch M, Butterworth G, Parsons S, et al. Newborn infants prefer attractive faces. Infant Behav Dev 1998; 21: 345–354.
25. Blascovich J, Mendes WB, Hunter SB, Lickel B, Kowai-Bell N. Perceiver threat in social interactions with stigmatized others. J Pers Soc Psychol 2001; 80: 253–267.
26. Madera JM. Facial stigmas in dyadic selection interviews: affective and behavioral reactions toward a stigmatized applicant. J Hosp Tourism Res 2016; 40: 456–475.
27. Madera JM, Hebl MR. Discrimination against facially stigmatized applicants in interviews: an eye-tracking and face-to-face investigation. J Appl Psychol 2012; 97: 317–330.
28. Rennels JL, Cummings AJ. Sex differences in facial scanning: similarities and dissimilarities between infants and adults. Int J Behav Dev 2013; 37: 111–117.
29. Blais C, Jack RE, Scheepers C, Fiset D, Caldara R. Culture shapes how we look at faces. PLoS One 2008; 3: e3022.
30. Cañigueral R, Hamilton AFC. The role of eye gaze during natural social interactions in typical and autistic people. Front Psychol 2019; 10: 560.
31. Han NX, Chakravarthula PN, Eckstein MP. Peripheral facial features guiding eye movements and reducing fixational variability. J Vis 2021; 21: 7.
32. Arizpe J, Walsh V, Yovel G, Baker CI. The categories, frequencies, and stability of idiosyncratic eye-movement patterns to faces. Vision Res 2017; 141: 191–203.
33. Coutrot A, Binetti N, Harrison C, Mareschal I, Johnston A. Face exploration dynamics differentiate men and women. J Vis 2016; 16: 16.