The data that students see in their statistics courses are often constrained to numeric and tabular data. However, there is an exciting field of data science and statistics known as text analysis. This expedition introduces students to the concept of treating text as data frames of words, and demonstrates how to perform basic analyses on bodies of text using R. Tweets of four Democratic candidates for the 2020 Primary are used as data, and demonstrated text analysis techniques in the expedition include comparisons of word frequencies, log-odds ratios for word usage, and pairwise word correlations.

Mapping History has focused on the categorizing, labelling, digitization, and 3D reconstruction of 16^th & 17^th century maps & atlases of London and Lisbon. Over the course of the summer, the Mapping History team has developed its own unique analytical dataset by painstakingly labelling every element contained within these maps, used python to digitize this dataset, and, now in the projects final stage, has begun the process of reconstructing these historical perspectives in a 3D game engine.

Project Lead: Philip Stern, Ed Triplett

Project Manager: Sam Horewood

View the team's final poster here

Watch the team's final presentation (on Zoom) below:

Our team used artificial intelligence to help Duke University Management Company (DUMAC) operate more efficiently by building a cost optimization tool and analyzing and visualizing venture capital data. In our first project centered around cost optimization, we designed a Python script that suggests optimal cash transfers between prime brokers. In our second project, we utilized web scraping and Tableau to aid DUMAC in understanding the relationship between company age and length of current investments.

Project Lead: Robert McGrail, DUMAC

Project Manager: Yi Wang

Click here to view the team's final summary

Watch the team's final presentation (on Zoom) here:

Predictive Churn Models for Duke Season Ticket Holders and Annual Donors is centered around understanding which annual donors are most likely to churn, i.e. not donate the following year. To solve this problem the team built different models to predict the profiles and timing of donor churn. The team made use of Duke Athletics’ internal data supplemented by external data to build predictive machine learning models.

Project Leads: John Haws, Larry Cleaver

Project Manager: Andrew Carr

Click here to view the team's final project summary

Watch the team's final presentation (on Zoom) here:

Carrying forward the work of a 2019-20 Bass Connections team, our Data+ team has worked to better understand the state of the home mortgage market leading up to the financial crisis. The team has built a more in-depth analysis of North Carolina to understand its different regions. We have also expanded the scope of the analysis developing a quantitative portrait on the state of the mortgage market in Arizona, Florida, Massachusetts, Georgia, and Ohio, creating visualization devices for different mortgage market statistics.

Project Lead:  Lee Reiners

Project Manager: Eric Autry

Click here to view the team's project poster

Watch the project team's final presentation (over Zoom) below:

Our team used years of unanalyzed data in a cloud computing environment to conduct exploratory data analysis using natural language processing techniques, as well as visualizations, for Fleet Management Limited. Through this, and preliminary predictive modelling, we hope to help management decrease the number of preventable incidents as each one costs FML more than $10,000.

Faculty Sponsor: Paul Bendich

Project Manager: Anil Ganti

Client Lead: Shah Irani, Fleet Management Limited

Click here to view the proect team's poster

Watch the team's final presentation (on Zoom) below:

In collaboration with Data and Analytics Practice at OIT, our team has completed a series of critical analyses aiding Duke Facilities Management in further optimizing campus energy usage. Data cleaning tools, imputation techniques, and a variety of time series prediction methods ranging from autoregressive models to deep learning networks have been seamlessly integrated into a single interactive forecasting application allowing collaborators to provide accurate and comprehensive utility usage estimates.

Projects Leads: John Haws, Gagandeep Kaur

Project Manager: Billy Carson

Click here to view the project team's final poster

Watch the team's final presentation (on Zoom) here:

In light of Duke’s reopening amidst the COVID-19 pandemic, this project aims to track the movement of foot traffic in and around Bryan Center by analyzing Wifi log data from all users connected to wireless networks in the center during February 2020. Our team employed Markov Chains, Kernel Density Estimations, and data analysis and visualization tools such as Python and Tableau to create a map of Wifi access points in Bryan Center and a heatmap that visualizes congestion in different floor areas across time. Our goal is to provide Duke OIT and Student Affairs with valuable information on highly congested areas and frequented paths, directing social distancing measures and suggesting alternative paths that can reduce transmission-risk this coming academic year.

Project Leads: John Haws, Mary Thompson, Eric Hope, Sean Dilda

Project Manager: Hunter Klein

Click here to view the project team's poster

Watch the team's final project presentation (on Zoom) here:

The visibility of hate groups such as the Alt-Right became mainstream into contemporary political culture during the Unite the Right Rally in Charlottesville, VA in 2017. This project aims to explore methods to quantify the presence of Latinxs within the Alt-Right, particularly in how they racialize themselves in a space that often spews hate towards Mexicans and other marginalized groups from Latin America. Using data from multiple sources (such as Twitter, Stormfront, and Breitbart), we developed a corpus of tweets, subthreads, and articles, and analyzed this data using basic natural language processing (NLP) techniques.

Project Lead: Cecilia Márquez

Project Manager: Susan Jacobs

Click here to view the team's project summary slides

Watch the team's final presentation (on Zoom) here:

The Protecting American Investors project investigates the evolving structure and content of financial advice from the early 20th century to the birth of the Internet. By converting and cleaning thousands of investment advice columns from historical newspapers and magazines, we assembled a large corpus to address our research questions. Through text analysis methods like topic modeling, we have seen how the business cycle affects the nature of advice, the speed in which different financial innovations were integrated, and how advice differs among various targeted social groups.

Project Lead: Ed Balleisen

Click here to view the team's final project summary slides

Watch the team's final presentation (on Zoom) here:

Between 1935 and 1945, rural electricity access shot up from roughly 10% to 90%. During this time, the Rural Electrification Administration funded an Electric Farm Equipment (EFE) Roadshow as part of its mission to expand electricity access and demand. Digitizing massive amounts of archival data, our team has sought to quantify the effect of the EFE Roadshow on the larger trend of growing residential electricity consumption in rural U.S. towns from 1938 to 1945. We hope that understanding this crucial chapter in our own history will help inform present-day electrification efforts in the developing world.

Project Leads: Victoria Plutshack, Jonathon Free, Robert Fetter

View the team's final project summary here

Watch the team's final presentation (on Zoom) here:

We all need water to survive but how many of us really know where our water comes from? Team 3 has created a functioning website from scratch to give consumers more easily accessible information about who provides their water and how supply looks in relation to the past 30 years.

Project Leads: Megan Mullin, Lauren Patterson

Watch the team's final presentation (on Zoom) here:

Project Manager: Kyle Onda

This project allowed students in BIOL 268D (Mechanisms of Animal Behavior) to explore the relationship between estrogen, female sexual swellings, and male mating success in wild baboons using data from the Amboseli Baboon Research Project. Students learned how to use the popular R packages dplyr and ggplot2 to calculate descriptive statistics about the dataset and perform data visualization to understand and explore patterns in animal mating behavior and sexual signals.

Understanding how to generate, analyze, and work with datasets in the humanities is often a difficult task without learning how to code or program. In humanities centered courses, we often privilege close reading or qualitative analysis over other methods of knowing, but by learning some new quantitative techniques we better prepare the students to tackle new forms of reading. This class will work with the data from the HathiTrust to develop ideas for thinking about how large groups and different discourse communities thought of queens of antiquity like Cleopatra and Dido.

Please refer to https://sites.duke.edu/queensofantiquity/ for more information.

Bernice Meja (Philosophy, Physics), Jessica Yang (Computer Science, ECE), and Tracey Chen (Computer Science, Mechanical Engineering) spent ten weeks building methods for Duke’s Office of Information Technology (OIT) to better understand information arising from “smart” (IoT) devices on campus. Working with data provided by an IoT testbed set up by OIT professionals, the team used a mixture of supervised and unsupervised machine-learning techniques and built a prototype device classifier.

Click here ot read the Executive Summary

Project Lead: Will Brockselsby

Cathy Lee (Statistics) and Jennifer Zheng (Math, Emory University) spent ten weeks building tools to help Duke University Libraries better understand its journal purchasing practice. Using a combination of web-scraping and data-merging algorithms, the team created a dashboard to help library strategists visualize and optimize journal selection.

Click here to read the Executive Summary

Faculty Leads: Angela Zoss, Jeff Kosokoff

Project Manager: Chi Liu

The Middle Passage, the route by which most enslaved persons were brought across the Atlantic to North America, is a critical locus of modern history—yet it has been notoriously difficult to document or memorialize. The ultimate aim of this project is to employ the resources of digital mapping technologies as well as the humanistic methods of history, literature, philosophy, and other disciplines to envision how best to memorialize the enslaved persons who lost their lives between their homelands and North America. To do this, the students combined previously-disparate data and archival sources to discover where on their journeys enslaved persons died. Because of the nature of data itself and the history it represents, the team engaged in on-going conversations about various ways of visualizing its findings, and continuously evaluated the ethics of the data’s provenance and their own methodologies and conclusions. A central goal for the students was to discover what contribution digital data analysis methods could make to the project of remembering itself.

The group worked with two datasets: the Trans-Atlantic Slave Trade Database (www.slavevoyages.org), an SPSS-formatted database currently run out of Emory University, containing data on 36,002 individual slaving expeditions between 1514 and 1866; and the Climatological Database for the World’s Oceans 1750-1850 (CLIWOC) (www.kaggle.com/cwiloc/climate-data-from-ocean-ships), a dataset composed of digitized records from the daily logbooks of ocean vessels, originally funded by the European Union in 2001 for purposes of tracking historical climate change. This second dataset includes 280,280 observational records of daily ship locations, climate data, and other associated information. The team employed archival materials to confirm (and disconfirm) overlaps between the two datasets: the students identified 316 ships bearing the same name across the datasets, of which they confirmed 35 matching slaving voyages.

The students had two central objectives: first, to locate where and why enslaved Africans died along the Middle Passage, and, second, to analyze patterns in the mortality rates. The group found significant patterns in the mortality data in both spatial and temporal terms (full results can be found here). At the same time, the team also examined the ethics of creating visualizations based on data that were recorded by the perpetrators of the slave trade—opening up space for further developments of this project that would include more detailed archival and theoretical work.

Click here to read the Executive Summary

Image credit:

J.M.W. Turner, Slave Ship, 1840, Museum of Fine Arts, Boston (public domain)

Faculty Lead: Charlotte Sussman

Project Manager: Emma Davenport

The aim of this project was to explore how U.S. mass media—particularly newspapers—enlists text and imagery to portray human rights, genocide, and crimes against humanity from World War II until the present. From the Holocaust to Cambodia, from Rwanda to Myanmar, such representation has political consequences. Coined by Raphael Lemkin, a Polish lawyer who fled Hitler’s antisemitism, the term “genocide” was first introduced to the American public in a Washington Post op-ed in 1944. Since its legal codification by the United Nations Convention for the Prevention of Genocide in 1948, the term has circulated, been debated, used to describes events that pre-date it (such as the displacement and genocide of Native People in the Americas), and been shaped by numerous forces—especially the words and images published in newspapers. Alongside the definition of “genocide,” other key concepts, specifically “crimes against humanity,” have attempted to label, and thus name the story, of targeted mass violence. Conversely, the concept of “human rights,” enshrined in the 1948 UN Declaration, seeks to name a presence of rights instead of their absence.

During the summer, the team focused their work on evaluating the language used in Western media to represent instances of genocide and how such language varied based on the location and time period of the conflict. In particular, the team’s efforts centered on Rwanda and Bosnia as important case studies, affording them the chance to compare nearly simultaneous reporting on two well-known genocides. The language used by reporters in these two cases showed distinct polarizations of terminology (for instance, while “slaughter” was much more common than “murder” in discussions of the Rwanda genocide, the inverse was true for Bosnia).

Click here to read the Executive Summary

Faculty Leads: Nora Nunn, Astrid Giugni

Maria Henriquez (Computer Science, Statistics) and Jacob Sumner (Biology) spent ten weeks building tools to help the Michael W. Krzyzewski Human Performance Lab best utilize its data from Duke University student athletes. The team worked with a large collection of athlete strength, balance, and flexibility measurements collected by the lab. They improved the K Lab’s data pipeline, created a predictive model for injury risk, and developed interactive web-based individualized injury risk reports.

Click here to read the Executive Summary

Faculty Lead: Dr. Tim Sell
Project Manager: Brinnae Bent

The students in this project worked on a pervasive question in literary, film, and copyright studies: how do we know when a new work of fiction borrows from an older one? Many times, works are appropriated, rather than straightforwardly adapted, which makes it difficult for human readers to trace. As we continue to remake and repurpose previous texts into new forms that combine hundreds of references to other works (such as Ready Player One), it becomes increasingly laborious to track all the intertextual elements of a single text. While some borrowings are easy to spot, as in the case of Marvel films that are straightforward adaptations of comic book storylines and aesthetics, others are more subtle, as when Disney reinterpreted Hamlet and African oral traditions to create The Lion King. Thousands of new stories are created each day, but how do we know if we are borrowing or appropriating a previous text? Are there works that have adapted previous ones that we have yet to identify?

The students worked with data from over 16.7 million books from Hathitrust, with critical analysis in scholarly articles accessible through JSTOR, and with the topic categories in Wikipedia. The group used Latent Dirichlet Allocation (LDA), a generative model that assumes that all documents are a mixture of topics, to represent key themes and topics as a distribution over words. The students developed a flexible and graduated heuristic for identifying a work as an adaptation; the more pre-selected categories a work fit under, the more likely it was to be marked as an adaptation by their model. Over the summer, the students came to appreciate that all digital humanistic methodologies are contestable and dependent on traditional critical work.

Click here to read the Executive Summary

Faculty Lead: Grant Glass

Andre Wang (Math, Statistics), Michael Xue (Computer Science, ECE), and Ryan Culhane (Computer Science) spent ten weeks exploring the role played by emotion in speech-focused machine-learning. The team used a variety of techniques to build emotion recognition pipelines, and incorporated emotion into generated speech during text-to-speech synthesis.

Click here to read the Executive Summary

Faculty Leads: Vahid Tarokh, Jie Ding

Project Manager: Enmao Diao

This Data Expedition seeks to introduce students to statistical analysis in the field of international development. Students construct a index of wealth/poverty based on asset holdings using four datasets collected under the umbrella of the Living Standards Measurement Survey project at the World Bank. We selected countries to represent different continents with comparable and recent survey data: Bulgaria (2007), Tajikistan (2009), Tanzania (2010-2011), and Panama (2008).

First, we construct an index of wealth based on household assets in the different countries using Principle Components Analysis. Once a poverty index is constructed, students seek to understand what the main drivers of wealth/poverty are in different countries. We include variables for health, education, age, relationship to the household head, and sex. Students then use regression analysis to identify the main drivers of poverty in different countries.

Our aim was to introduce students to the wealth of possibilities that human genotyping and sequencing hold by illustrating firsthand the power of these datasets to identify genetic relatives, using the story of the Golden State Killer’s capture with public genetic databases.

Alec Ashforth (Economics/Math), Brooke Keene (Electrical & Computer Engineering), Vincent Liu (Electrical & Computer Engineering), and Dezmanique Martin (Computer Science) spent ten weeks helping Duke’s Office of Information Technology explore the development of an “e-advisor” app that recommends co-curricular opportunities to students based on a variety of factors. The team used collaborative and content-based filtering to create a recommender-system prototype in R Shiny.

Click here to read the Executive Summary

Lucas Fagan (Computer Science/Public Policy), Caroline Wang (Computer Science/Math), and Ethan Holland (Statistics/Computer Science) spent ten weeks understanding how data science can contribute to fact-checking methodology. Training on audio data from major news stations, they adapted OpenAI methods to develop a pipeline that moves from audio data to an interface that enables users to search for claims related to other claims that had been previously investigated by fact-checking websites.

This project will continue into the academic year via Bass Connections.

Click here to read the Executive Summary.

Kimberly Calero (Public Policy/Biology/Chemistry), Alexandra Diaz (Biology/Linguistics), and Cary Shindell (Environmental Engineering) spent ten weeks analyzing and visualizing data about disparities in Social Determinants of Health. Working with data provided by the MURDOCK Study, the American Community Survey, and the Google Places API, the team built a dataset and visualization tool that will assist the MURDOCK research team in exploring health outcomes in Cabarrus County, NC.

Click here to read the Executive Summary

Nathan Liang (Psychology, Statistics), Sandra Luksic (Philosophy, Political Science),and Alexis Malone (Statistics) began their 10-week project as an open-ended exploration how women are depicted both physically and figuratively in women's magazines, seeking to consider what role magazines play in the imagined and real lives of women.

Click here to read the Executive Summary

Aaron Crouse (Divinity), Mariah Jones (Sociology), Peyton Schafer (Statistics), and Nicholas Simmons (English/Education) spent ten weeks consulting with leadership from the Parents Teacher Association at Glenn Elementary School in Durham. The team set up infrastructure for data collection and visualization that will aid the PTA in forming future strategy.

Click here to read the Executive Summary

Ashley Murray (Chemistry/Math), Brian Glucksman (Global Cultural Studies), and Michelle Gao (Statistics/Economics) spent 10 weeks analyzing how meaning and use of the work “poverty” changed in presidential documents from the 1930s to the present. The students found that American presidential rhetoric about poverty has shifted in measurable ways over time. Presidential rhetoric, however, doesn’t necessarily affect policy change. As Michelle Gao explained, “The statistical methods we used provided another more quantitative way of analyzing the text. The database had around 130,000 documents, which is pretty impossible to read one by one and get all the poverty related documents by brute force. As a result, web-scraping and word filtering provided a more efficient and systematic way of extracting all the valuable information while minimizing human errors.” Through techniques such as linear regression, machine learning, and image analysis, the team effectively analyzed large swaths of textual and visual data. This approach allowed them to zero in on significant documents for closer and more in-depth analysis, paying particular attention to documents by presidents such as Franklin Delano Roosevelt or Lyndon B. Johnson, both leaders in what LBJ famously called “The War on Poverty.”

Click Here for the Executive Summary

Tatanya Bidopia (Psychology, Global Health), Matthew Rose (Computer Science), Joyce Yoo (Public Policy/Psychology) spent ten weeks doing a data-driven investigation of the relationship between mental health training of law enforcement officers and key outcomes such as incarceration, recidivism, and referrals for treatment. They worked closely with the Crisis Intervention Team, and they used jail data provided by the Sheriff’s Office of Durham County.

Click here to read the Executive Summary

ECE majors Mitchell Parekh and Yehan (Morton) Mo, along with IIT student Nikhil Tank, spent ten weeks understanding parking behavior at Duke. They worked closely with the Parking and Transportation Office, as well as with Vice President for Administration Kyle Cavanaugh.

BME major Neel Prabhu, along with CompSci and ECE majors Virginia Cheng and Cheng Lu, spent ten weeks studying how cells from embryos of the common fruit fly move and change in shape during development. They worked with Cell-Sheet-Tracker (CST), an algorithm develped by former Data+ student Roger Zou and faculty lead Carlo Tomasi. This algorithm uses computer vision to model and track a dynamic network of cells using a deformable graph.

Weiyao Wang (Math) and Jennifer Du , along with NCCU Physics majors Jarrett Weathersby and Samuel Watson, spent ten weeks learning about how search engines often provide results which are not representative in terms of race and/or gender. Working closely with entrepreneur Winston Henderson, their goal was to understand how to frame this problem via statistical and machine-learning methodology, as well as to explore potential solutions.

Runliang Li (Math), Qiyuan Pan (Computer Science), and Lei Qian (Masters in Statistics and Economic Modelling) spent ten weeks investigating discrepancies between posted wait times and actual wait times for rides at Disney World. They worked with data provided by TouringPlans.

Building off the work of a 2016 Data+ team, Yu Chen (Economics), Peter Hase (Statistics), and Ziwei Zhao (Mathematics), spent ten weeks working closely with analytical leadership at Duke's Office of University Development. The project goal was to identify distinguishing characteristics of major alumni donors and to model their lifetime giving behavior.

Over ten weeks, Math/CompSci majors Benjamin Chesnut and Frederick Xu joined forces with International Comparative Studies major Katharyn Loweth to understand the myriad academic pathways traveled by undergraduate students at Duke. They focused on data from Mathematics and the Duke Global Health Institute, and worked closely with departmental leadership from both areas.

John Benhart (CompSci, Math) and Esko Brummel (Masters in Bioethics and Science Policy) spent ten weeks analyzing current and potential scholarly collaborations within the community of Duke faculty. They worked closely with the leadership of the Scholars@Duke database.

Over ten weeks, Mathematics/Economics majors Khuong (Lucas) Do and Jason Law joined forces with Analytical Political Economy Masters student Feixiao Chen to analyze the spati-temporal distribution of birth addresses in North Carolina. The goal of the project was to understand how/whether the distributions of different demographic categories (white/black, married/unmarried, etc.) differed, and how these differences connected to a variety of socioeconomic indicators.

Graduate Student: Jacob Coleman, 3rd year Ph.D. student in Statistical Science

Faculty Instructor: Colin Rundel

Class: STA 112, Data Science

Computer Science majors Erin Taylor and Ian Frankenburg, along with Math major Eric Peshkin, spent ten weeks understanding how geometry and topology, in tandem with statistics and machine-learning, can aid in quantifying anomalous behavior in cyber-networks. The team was sponsored by Geometric Data Anaytics, Inc., and used real anonymized Netflow data provided by Duke's Information Technology Security Office.

A virtual reality system to recreate the archaeological experience using data and 3D models from the neolithic site of Çatalhöyük, in Anatolia, Turkey. 

Molly Rosenstein, an Earth and Ocean Sciences major and Tess Harper, an Environmental Science and Spanish major spent ten weeks developing interactive data applications for use in Environmental Science 101, taught by Rebecca Vidra.

The goal of this project is take a large amount of data from the Massive Open Online Courses offered by Duke professors, and produce from it a coherent and compelling data analysis challenge that might then be used for a Duke or nation-wide data analysis competition.

Ethan Levine, Annie Tang, and Brandon Ho spent ten weeks investigating whether personality traits can be used to predict how people make risky decisions. They used a large dataset collected by the lab of Prof. Scott Huettel, and were mentored by graduate students Emma Wu Dowd and Jonathan Winkle.