Learning to Search More Deeply

Project Summary

Weiyao Wang (Math) and Jennifer Du , along with NCCU Physics majors Jarrett Weathersby and Samuel Watson, spent ten weeks learning about how search engines often provide results which are not representative in terms of race and/or gender. Working closely with entrepreneur Winston Henderson, their goal was to understand how to frame this problem via statistical and machine-learning methodology, as well as to explore potential solutions.

Themes and Categories
Interdisciplinary Inquiry
Data+
Year
2016

Project Results

In order to understand Google's algorithm, the team web-scraped search results and used machine-learning to understand the importance of each feature. They then performed sentiment analysis to quantify public opinions from Twitter and used community-based crawling and seeding to collect information relevant to minority groups.

Download the Executive Summary (PDF)

Faculty Sponsor

Project Manager

Participants

  • Jennifer Du, Duke University Computer Science
  • Weiyao Wang, Duke University Computer Science, Mathematics, and Political Science
  • Jarrett Weathersby, North Carolina Central University Physics
  • Samuel Watson, North Carolina Central University Physics

Disciplines Involved

  • Sociology
  • Anthropology
  • Economics
  • All quantitative STEM

Related People

Related News

Deep Learning + Cultural Fit = Relevance
Monday, October 31, 2016
Learning to Search More Deeply with Data+ Winston Henderson, Duke Alumni Association Board Vice P ...

Related Projects

Maximizing Data Communication for Faster Energy Access

Brooke Erikson (Economics/Computer Science), Alejandro Ortega (Math), and Jade Wu (Computer Science) spent ten weeks developing open-source tools for automatic document categorization, PDF table extraction, and data identification. Their motivating application was provided by Power for All’s Platform for Energy Access Knowledge, and they frequently collaborated with professionals from that organization.

Click here to read the Executive Summary

 

Data and the Global Corporate Bond Market

Jake Epstein (Statistics/Economics), Emre Kiziltug (Economics), and Alexander Rubin (Math/Computer Science) spent ten weeks investigating the existence of relative value opportunities in global corporate bond markets. They worked closely with a dataset provided by a leading asset management firm.

Click here for the Executive Summary

Construction Machinery and the Business Cycle

Maksym Kosachevskyy (Economics) and Jaehyun Yoo (Statistics/Economics) spent ten weeks understanding temporal patterns in the used construction machinery market and investigating the relationship between these patterns and macroeconomic trends.

They worked closely with a large dataset provided by MachineryTrader.com, and discussed their findings with analytics professionals from a leading asset management firm.

Click here to read the Executive Summary