Application of Linear Algebra in Infants’ Autism Detection

Project Summary

Dr. Guillermo Sapiro, professor in Pratt School of Engineering at Duke University, conducts ongoing autism research. Using image processing, he attempts to program a computer to detect whether babies (around eight to 14 months of age) display a sign of autism. This very early detection enables doctors to train these babies (when their brain plasticity is high) to behave in ways to counter the behavioral limitations autism imposes, thus allowing these babies to act more normally as they grow up. 

Themes and Categories
Year
Contact
Paul Bendich
bendich@math.duke.edu

Graduate students: Edward Kim, Hyunsoo Kim, and Zhuoqing Chang 

  • Unusual blinking pattern is a possible sign of autism.
  • Goal: Develop a method to detect the eye state (open/closed) from a 30x30 pixel eye image.
  • Method: Use 7,000 training eye images to learn the relationship between each pixel and eye state; then test the system on 3,000 new images.
  • Students learn a method to solve an overdetermined system (using Moore-Penrose pseudoinverse to find the least squares solution).

Summary

One of the behavioral cues Dr. Sapiro looks for is the blinking of the babies’ eyes. This portion of the lecture notes focuses on the creation of an image processing algorithm for the detection of the blinking, in particular for the determination of whether an eye is open or closed, by applying concepts of linear algebra.

The input to the algorithm is a picture of an eye. Each input image consists of 30(width) x30(height)= 900 pixels. Each pixel carries a numerical value in the range of 0 to 255, which represents the combination light intensity detected by the red, blue, and green light receptor channels of a camera. The significance of each pixel varies. The image of an open eye will have two major dark edges around the eyes with a large pupil in the middle. On the other hand, the image of a closed eye will have only one dark edge with no pupil. There is a pattern associated with each of these two categories of images.

The challenge for the algorithm is to determine the pattern from the pixel data with a high probability of success. In our model, this is achieved with the aid of properly chosen weights for the individual pixels. The determination between an open or closed eye is then made in the following simple way. The algorithm multiplies the numerical pixel value by the pixel weight and then sums these products over all pixels. If the result of the summation is positive, the output of the algorithm is the integer ’1’ and the eye is determined to be open. If the sum is negative, the output of the algorithm is the integer ’-1’ and the eye is determined to be closed. 

Read more about this project (PDF).

Related Projects

KC and Patrick led two hands-on data workshops for ENVIRON 335: Drones in Marine Biology, Ecology, and Conservation. These labs were intended to introduce students to examples of how drones are currently being used as a remote sensing tool to monitor marine megafauna and their environments, and how machine learning can be used to efficiently analyze remote sensing datasets. The first lab specifically focused on how drones are being used to collect aerial images of whales to measure changes in body condition to help monitor populations. Students were introduced to the methods for making accurate measurements and then received an opportunity to measure whales themselves. The second lab then introduced analysis methods using computer vision and deep neural networks to detect, count, and measure objects of interest in remote sensing data. This work provided students in the environmental sciences an introduction to new techniques in machine learning and remote sensing that can be powerful multipliers of effort when analyzing large environmental datasets.

This two-week teaching module in an introductory-level undergraduate course invites students to explore the power of Twitter in shaping public discourse. The project supplements the close-reading methods that are central to the humanities with large-scale social media analysis. This exercise challenges students to consider how applying visualization techniques to a dataset too vast for manual apprehension might enable them to identify for granular inspection smaller subsets of data and individual tweets—as well as to determine what factors do not lend themselves to close-reading at all. Employing an original dataset of almost one million tweets focused on the contested 2018 Florida midterm elections, students develop skills in using visualization software, generating research questions, and creating novel visualizations to answer those questions. They then evaluate and compare the affordances of large-scale data analytics with investigation of individual tweets, and draw on their findings to debate the role of social media in shaping public conversations surrounding major national events. This project was developed as a collaboration among the English Department (Emma Davenport and Astrid Giugni), Math Department (Hubert Bray), Duke University Library (Eric Monson), and Trinity Technology Services (Brian Norberg).

Understanding how to generate, analyze, and work with datasets in the humanities is often a difficult task without learning how to code or program. In humanities centered courses, we often privilege close reading or qualitative analysis over other methods of knowing, but by learning some new quantitative techniques we better prepare the students to tackle new forms of reading. This class will work with the data from the HathiTrust to develop ideas for thinking about how large groups and different discourse communities thought of queens of antiquity like Cleopatra and Dido.

Please refer to https://sites.duke.edu/queensofantiquity/ for more information.