Learning to Search More Deeply

Project Summary

Weiyao Wang (Math) and Jennifer Du , along with NCCU Physics majors Jarrett Weathersby and Samuel Watson, spent ten weeks learning about how search engines often provide results which are not representative in terms of race and/or gender. Working closely with entrepreneur Winston Henderson, their goal was to understand how to frame this problem via statistical and machine-learning methodology, as well as to explore potential solutions.

Themes and Categories
Interdisciplinary Inquiry
Data+
Year
2016

Project Results

In order to understand Google's algorithm, the team web-scraped search results and used machine-learning to understand the importance of each feature. They then performed sentiment analysis to quantify public opinions from Twitter and used community-based crawling and seeding to collect information relevant to minority groups.

Download the Executive Summary (PDF)

Faculty Sponsor

Project Manager

Participants

  • Jennifer Du, Duke University Computer Science
  • Weiyao Wang, Duke University Computer Science, Mathematics, and Political Science
  • Jarrett Weathersby, North Carolina Central University Physics
  • Samuel Watson, North Carolina Central University Physics

Disciplines Involved

  • Sociology
  • Anthropology
  • Economics
  • All quantitative STEM

Related People

Related News

Deep Learning + Cultural Fit = Relevance
Monday, October 31, 2016
Learning to Search More Deeply with Data+ Winston Henderson, Duke Alumni Association Board Vice P ...

Related Projects

American Predatory Lending (Year 3)

Alexa Goble (Finance) joined Econ majors Chavez Cheong and Eli Levine in a ten-week exploration of mortgage enforcement actions related to the financial crisis from earlier in this century. Using NLP techniques on mortgage data from Ohio and Massachusetts, the team validated a new experimental approach to understanding the dynamics between state regulatory agencies, mortgage lenders, brokers, and loan originators. This project was a continuation of two previous Data+ projects:

https://bigdata.duke.edu/projects/american-predatory-lending-global-financial-crisis

https://bigdata.duke.edu/projects/american-predatory-lending-and-global-financial-crisis-year-2

 

View the team's project poster here

Watch the team's final presentation on Zoom:

 

Project Lead: Lee Reiners

Project Manager: Malcolm Smith Fraser

Redistricting School Zones in Durham County

Stats/Sociology major Mitchelle Mojekwu joined Neuroscience majors Kassie Hamilton and Zineb Jaidi in a ten-week exploration of data relevant to an upcoming public school zone redistricting in Durham County. Using information acquired from the General Social Survey and the US Census, the team applied modern mathematical and statistical methods for generating proposed redistricting plans, with the aim of providing decision-makers with information they can use to produce school districts that are equitable and reflective of the Durham County student population.

View the team's project poster here

Watch the team's final presentation on Zoom:

 

Faculty Lead: Greg Herschlag

Project Manager: Bernard Coles

 

Agile Waveform Design in the Presence of Opposition: Quantifying Performance

Pryia Juarez (BME/ECE), Jonathan Pilland (ECE/BME), and Matthew Traum (CS/Econ) spent teen weeks analyzing sensor data synthesized by an agile waveform generator. The team used deep reinforcement learning techniques to understand the performance of different synthetic agents representing potential attackers to the sensor system.

 

View the team's project poster here

Watch the team's final presentation on Zoom:

 

Faculty leads: Robert Calderbank, Vahid Tarokh, Ali Pezeshki

Client leads: Dr. Lauren Huie, Dr. Elizabeth Bentley, Dr. Zola Donovan, Dr. Ashley Prater-Bennette, Dr. Erin Trip

Project Manger: Suya Wu

See all Research