Student Scholar Symposium Abstracts and Posters

Document Type

Poster

Publication Date

Spring 5-9-2018

Faculty Advisor(s)

Dr. Vincent Berardi

Abstract

This project served to aid with the development of a research project that aims to analyze the progression of the appearance and context of women’s faces in Time Magazine images from 1960 through 1990. This will be accomplished by using Mechanical Turk to employ crowdsourced labor to extract every image of a face from select issues of Time magazine over the time period of interest. Mechanical Turk will also be used to label the following characteristics of each face: photo/drawing, gaze direction, context (ad/feature/cover), color/monochrome, ethnicity, age, gender, presence of a smile, and image quality. My work focused on the developing the labeling system and exploring the consistency of labelers. Through processing multiple issues from the corpus, I found potentially problematic issues that had not yet been considered such as the presence of masked individuals, images with an overwhelming number of miniature faces, and photos of young children with ambiguous gender and ethnicity. The results of my exploration led to the creation of new categories (e.g. adult versus child) and the establishment of specific criteria to be provided to potential raters to reduce ambiguity in the task. These criteria were included in an instruction set for MTurkers that I created. In addition, I developed an R-script that used Cohen's kappa to analyze interrater reliability across all categories to view the consistency of labelers. The data used was of 3 university students; who individually processed images from the same issue and labeled the faces according to the established protocol. The overall average Cohen’s kappa was .740. The Cohen’s kappa was above .723 for all variables except image quality. Kappa for image quality was .362.  Lastly, I aided in the development of internal checks within the Mechanical Turk system to identify instances of labelers potentially manipulating the system for financial gain.

Comments

Presented at the Spring 2018 Student Research Day at Chapman University.

Download

Included in

Psychology Commons

Share

COinS