Application of Linear Algebra in Infants’ Autism Detection

Project Summary

Dr. Guillermo Sapiro, professor in Pratt School of Engineering at Duke University, conducts ongoing autism research. Using image processing, he attempts to program a computer to detect whether babies (around eight to 14 months of age) display a sign of autism. This very early detection enables doctors to train these babies (when their brain plasticity is high) to behave in ways to counter the behavioral limitations autism imposes, thus allowing these babies to act more normally as they grow up. 

Themes and Categories
Paul Bendich

Graduate students: Edward Kim, Hyunsoo Kim, and Zhuoqing Chang 

  • Unusual blinking pattern is a possible sign of autism.
  • Goal: Develop a method to detect the eye state (open/closed) from a 30x30 pixel eye image.
  • Method: Use 7,000 training eye images to learn the relationship between each pixel and eye state; then test the system on 3,000 new images.
  • Students learn a method to solve an overdetermined system (using Moore-Penrose pseudoinverse to find the least squares solution).


One of the behavioral cues Dr. Sapiro looks for is the blinking of the babies’ eyes. This portion of the lecture notes focuses on the creation of an image processing algorithm for the detection of the blinking, in particular for the determination of whether an eye is open or closed, by applying concepts of linear algebra.

The input to the algorithm is a picture of an eye. Each input image consists of 30(width) x30(height)= 900 pixels. Each pixel carries a numerical value in the range of 0 to 255, which represents the combination light intensity detected by the red, blue, and green light receptor channels of a camera. The significance of each pixel varies. The image of an open eye will have two major dark edges around the eyes with a large pupil in the middle. On the other hand, the image of a closed eye will have only one dark edge with no pupil. There is a pattern associated with each of these two categories of images.

The challenge for the algorithm is to determine the pattern from the pixel data with a high probability of success. In our model, this is achieved with the aid of properly chosen weights for the individual pixels. The determination between an open or closed eye is then made in the following simple way. The algorithm multiplies the numerical pixel value by the pixel weight and then sums these products over all pixels. If the result of the summation is positive, the output of the algorithm is the integer ’1’ and the eye is determined to be open. If the sum is negative, the output of the algorithm is the integer ’-1’ and the eye is determined to be closed. 

Read more about this project (PDF).

Related Projects

In this two-day, virtual data expedition project, students were introduced to the APIM in the context of stress proliferation, linked lives, the spousal relationship, and mental and physical health outcomes.

Stress proliferation is a concept within the stress process paradigm that explains how one person’s stressors can influence others (Thoits 2010). Combining this with the life course principle of linked lives explains that because people are embedded in social networks, stress not only can impact the individual but can also proliferate to people close to them (Elder Jr, Shanahan and Jennings 2015). For example, one spouse’s chronic health condition may lead to stress-provoking strain in the marital relationship, eventually spilling over to affect the other spouse’s mental health. Additionally, because partners share an environment, experiences, and resources (e.g., money and information), as well as exert social control over each other, they can monitor and influence each other’s health and health behaviors. This often leads to health concordance within couples; in other words, because individuals within the couple influence each other’s health and well-being, their health tends to become more similar or more alike (Kiecolt-Glaser and Wilson 2017, Polenick, Renn and Birditt 2018). Thus, a spouse’s current health condition may influence their partner’s future health and spouses may contemporaneously exhibit similar health conditions or behaviors.

However, how spouses influence each other may be patterned by the gender of the spouse with the health condition or exhibiting the health behaviors. Recent evidence suggests that a wife’s health condition may have little influence on her husband’s future health conditions, but that a husband’s health condition will most likely influence his wife’s future health (Kiecolt-Glaser and Wilson 2017).

Fluid mechanics is the study of how fluids (e.g., air, water) move and the forces on them. Scientists and engineers have developed mathematical equations to model the motions of fluid and inertial particles. However, these equations are often computationally expensive, meaning they take a long time for the computer to solve.


To reduce the computation time, we can use machine learning techniques to develop statistical models of fluid behavior. Statistical models do not actually represent the physics of fluids; rather, they learn trends and relationships from the results of previous simulation experiments. Statistical models allow us to leverage the findings of long, expensive simulations to obtain results in a fraction of the time. 


In this project, we provide students with the results of direct numerical simulations (DNS), which took many weeks for the computer to solve. We ask students to use machine learning techniques to develop statistical models of the results of the DNS.

Female baboons occasionally exhibit large swellings on their behinds. Although these ‘sexual swellings’ may evoke disgust from human on-lookers, they provide important information to group members about a female’s reproductive state. To figure out what these sexual swellings mean and whether male baboons notice, we need to look at the data.