Learning to Search More Deeply

Project Summary

Weiyao Wang (Math) and Jennifer Du , along with NCCU Physics majors Jarrett Weathersby and Samuel Watson, spent ten weeks learning about how search engines often provide results which are not representative in terms of race and/or gender. Working closely with entrepreneur Winston Henderson, their goal was to understand how to frame this problem via statistical and machine-learning methodology, as well as to explore potential solutions.

Themes and Categories
Interdisciplinary Inquiry
Data+
Year
2016

Project Results

In order to understand Google's algorithm, the team web-scraped search results and used machine-learning to understand the importance of each feature. They then performed sentiment analysis to quantify public opinions from Twitter and used community-based crawling and seeding to collect information relevant to minority groups.

Download the Executive Summary (PDF)

Faculty Sponsor

Project Manager

  • Michael Lindon, Ph.D candidate, Statistics

Participants

  • Jennifer Du, Duke University Computer Science
  • Weiyao Wang, Duke University Computer Science, Mathematics, and Political Science
  • Jarrett Weathersby, North Carolina Central University Physics
  • Samuel Watson, North Carolina Central University Physics

Disciplines Involved

  • Sociology
  • Anthropology
  • Economics
  • All quantitative STEM

Related People

Related News

Deep Learning + Cultural Fit = Relevance
Monday, October 31, 2016
Learning to Search More Deeply with Data+ Winston Henderson, Duke Alumni Association Board Vice P ...

Related Projects

Farm to Fork (Year 2)

A team of students collaborating with Duke School of Medicine's Root Causes Fresh Produce Program, community members, and physicians throughout the Duke Health network will help integrate data from food deliveries to Duke Health patients with patient health record data and other available data sources to create a dashboard that can analyze, predict, and manage the Root Causes' "Food as Medicine" program. Specific outcomes will contribute to improving the Program's quantitative evaluation of its health impact as well as efficiency and satisfaction for its patients. Students will be assisted with IRB approval and mentorship from faculty and community advisors.

Project Leads: Esko Brummel, Willis Wong

 

Melting Ice, Shifting Krill

A team of students led by researchers at the Duke Marine Lab will explore the changing distribution of krill around the Antarctic Peninsula. Krill are a key prey species in this ecosystem, supporting a number of animals including whales, seals, and penguins, but they are dependent on winter sea ice and may be in trouble as climate change progresses. Using data from acoustic zooplankton surveys, students will create maps and other products to visualize the spatial distribution of krill over the past 20 summers, then create metrics that allow us to quantify the way that krill distribution around the Antarctic Peninsula is changing as the climate shifts and ice melts. These results will be key to our understanding of the impacts of climate change on this polar ecosystem.

 

Project Lead: Douglas Nowacek

Project Manager: Amanda Lohmann

 

Data Science for Community Safety in Durham

A team of students will partner closely with the City of Durham's newly formed Community Safety Department.  The Community Safety Department's mission is to identify, implement, and evaluate new approaches to enhance public safety that may not involve a law enforcement response or the criminal justice system. The student team will (1) analyze and identify geographic and temporal patterns in 911 calls for service, (2) conceptualize and build an abstracted data pipeline and tools that would enrich currently available 911 data with other social, economic, and health-related data, (3) explore associations between areas of high call volume, indicators of mental health distress, and histories of dispossession; and (4) identify methods by which future researchers could examine connections between varied 911 incident responses (e.g. police response, unarmed response, joint police, and mental health response) and life trajectories (e.g. arrest, jail time, hospitalization, unemployment, etc.).

 

Project Lead: Greg Herschlag, Anise Van, City of Durham

Project Manager: Deekshita Saikia

 

See all Research