Learning to Search More Deeply

Project Summary

Weiyao Wang (Math) and Jennifer Du , along with NCCU Physics majors Jarrett Weathersby and Samuel Watson, spent ten weeks learning about how search engines often provide results which are not representative in terms of race and/or gender. Working closely with entrepreneur Winston Henderson, their goal was to understand how to frame this problem via statistical and machine-learning methodology, as well as to explore potential solutions.

Themes and Categories
Interdisciplinary Inquiry
Data+

Project Results

In order to understand Google's algorithm, the team web-scraped search results and used machine-learning to understand the importance of each feature. They then performed sentiment analysis to quantify public opinions from Twitter and used community-based crawling and seeding to collect information relevant to minority groups.

Download the Executive Summary (PDF)

Faculty Sponsor

Project Manager

Participants

  • Jennifer Du, Duke University Computer Science
  • Weiyao Wang, Duke University Computer Science, Mathematics, and Political Science
  • Jarrett Weathersby, North Carolina Central University Physics
  • Samuel Watson, North Carolina Central University Physics

Disciplines Involved

  • Sociology
  • Anthropology
  • Economics
  • All quantitative STEM

Related People

Related News

Deep Learning + Cultural Fit = Relevance
Monday, October 31, 2016
Learning to Search More Deeply with Data+ Winston Henderson, Duke Alumni Association Board Vice P ...

Related Projects

Maximizing Data Communication for Faster Energy Access

United Nations Sustainable Development Goal 7 calls for universal access to affordable, reliable, sustainable, and modern energy. Researchers and practitioners around the world have responded to this call by producing a wealth of energy access data. While many data gaps still exist, are we capturing the fullest potential from the information and research we do have, and what it tells us about how to accelerate energy access? Power for All’s Platform for Energy Access Knowledge (PEAK) is an interactive knowledge platform designed to automatically curate, organize, and streamline large, growing bodies of data into digestible, sharable, and useable knowledge through automated data capture, indexing, and visualization. A team of students led by Rebekah Shirley will consult with Power for All to creatively visualize PEAK’s library, and to explore machine learning and natural language processing tools that can enable auto-extraction and visualization of data for more effective science communication.

Data and the Global Corporate Bond Market

Are there relative value opportunities in the global corporate bond markets?  
A team of students will work with Professor Emma Rasiel to understand whether an analysis of credit spreads on bonds issued by international firms in multiple countries over time can shed light on potential arbitrage opportunities. The team will have frequent opportunities to interact with analytics professionals at a leading financial advisory and asset management firm.

 

Construction Machinery and the Business Cycle

A team of students will consult with a leading financial advisory and asset management firm that is seeking to understand how big data can shed light on the secondary market for construction machinery. Students will explore a combination of publicly-available datasets that describe the used-machinery market and its potential implications as an indicator for the business cycle. There will be frequent interactions with analytical professionals from the firm.