A Shallow Dive into Deep Sea Data

Project Summary

Large publicly available environmental databases are a tremendous resource for both scientists and the general public interested in climate trends and properties. However, without the programming skills to parse and interpret these massive datasets, significant trends may remain hidden from both scientists and the public. In this data exploration, students, over the course of three hours, accessed two large, publicly available datasets, each with greater than 4 million observations. They learned how to use R and RStudio to effectively organize, visualize and statistically explore trends in deep sea physical oceanography.  

Themes and Categories
Year
2018

Graduate Students: Sarah Solie (Biology) and Arielle Fogel (University Program in Genetics and Genomics, Evolutionary Anthropology)

Faculty Member: Dr. Kate Thomas

Course: Biology 190: Life in the Deep Sea

Students gained experience exploring patterns in multivariate oceanographic data, relevant to their coursework, to answer the following four questions:

  1. How does average temperature and salinity at the surface of the ocean compare to the temperature and salinity at 3000 meters below the surface?
  2. Do the trends observed in question 1 differ across tropical, temperate, and polar climates?
  3. What is the relationship between ocean temperature and salinity across depths ranging continuously from the surface to 5500 meters below sea level?
  4. Do the trends observed in question 2 differ across tropical, temperate, and polar climates?

As students pursued these questions, they were introduced to R, a free software program that provides powerful tools for statistical computing and graphics, and RStudio, an integrated development environment frequently used for easier programming in R. They learned valuable skills for future data analysis, including:

  1. Accessing and downloading two physical oceanography databases (salinity and temperature) from the National Oceanic and Atmospheric Administration (NOAA) and National Oceanographic Data Center (NODC) World Ocean Atlas 2013 - https://www.nodc.noaa.gov/OC5/woa13/woa13data.html
  2. Importing and inspecting a dataset in .csv format in RStudio
  3. Installing and using R packages
  4. Tidying data such that it was interpretable for R analysis
  5. Manipulating data included subsetting, filtering, transforming, and summarizing data
  6. Creating a new categorical variable and assigning values to it based on existing data
  7. Using graphical visualization (see Graphics Created) including:
    1. Boxplots (Figures 1-2)
    2. Scatterplots (Figures 3-6)
  8. Performing statistical tests including:
    1. A two sample t-test
  9. Best practices for data wrangling and analysis (e.g. inspecting data after manipulation, annotating code)

At the end of the exercise, students were provided with additional online resources to continue exploring data with R and RStudio.

The Datasets

Students accessed and explored two massive datasets from the National Oceanic and Atmospheric Administration (NOAA) and National Oceanographic Data Center (NODC) World Ocean Atlas 2013. Specifically, they used the annual temperature statistical mean and the annual salinity statistical mean datasets which contained temperature or salinity observations, respectively, across depth (up to 5500 meters), location (at 1o spatial resolution), and time (1955-2012).

Graphics Created

Temperature by Depth and Climate
Figure 1. Temperature at the surface versus 3000 meters below sea level and its relation to climate
Salinity by Depth and Climate
Figure 2. Salinity at the surface versus 3000 meters below sea level and its relation to climate.
Temperature by depth and climate
Figure 3. Temperature by depth and climate.
Salinity by depth and climate
Figure 4. Salinity by depth and climate.
Average temperature by depth and climate
Figure 5. Average temperature by depth and climate.
Average salinity by depth and climate
Figure 5. Average temperature by depth and climate.

Course Materials

Please see the R Markdown file titled “deep_sea_data.Rmd” as well as the PDF version, which includes figures, titled “deep_sea_data.pdf”.

Student Feedback

“I learned that programming is probably 10% writing out the code and 90% figuring out what went wrong. It is a ton of troubleshooting, and through that troubleshooting is a lot of frustration. However, it was also a lot of fun doing it. Problem solving has always been enjoyable for me, so I had a good time figuring out what I did wrong.”

“It was ... cool learning all of the different manners in which you can analyze data using the program and also compile all of the information—over 4 million data points—into very easy to read graphs that made interpreting the data very simple.”

“I think it was an amazing experience to make 4 million data [points] into [a] few intuitive graphs.”

“Using the skills I learned in these lessons, I can convey a huge group of data that seems chaotic into a series of tables that [are] both easy to see and easy to analyze.”

“I can understand why and how to use the codes with the instruction of the teachers.”

“Coding [in R] made it easier to graph complicated scientific results with many variables that programs like Excel would struggle with.”

Data Sources

  1. Locarnini, R. A., A. V. Mishonov, J. I. Antonov, T. P. Boyer, H. E. Garcia, O. K. Baranova, M. M. Zweng, C. R. Paver, J. R. Reagan, D. R. Johnson, M. Hamilton, and D. Seidov, 2013. World Ocean Atlas 2013, Volume 1: Temperature. S. Levitus, Ed., A. Mishonov Technical Ed.; NOAA Atlas NESDIS 73, 40 pp.
  2. Zweng, M.M, J.R. Reagan, J.I. Antonov, R.A. Locarnini, A.V. Mishonov, T.P. Boyer, H.E. Garcia, O.K. Baranova, D.R. Johnson, D.Seidov, M.M. Biddle, 2013. World Ocean Atlas 2013, Volume 2: Salinity. S. Levitus, Ed., A. Mishonov Technical Ed.; NOAA Atlas NESDIS 74, 39 pp.

Related Projects

A large and growing trove of patient, clinical, and organizational data is collected as a part of the “Help Desk” program at Durham’s Lincoln Community Health Center. Help Desk is a group of student volunteers who connect with patients over the phone and help them navigate to community resources (like food assistance programs, legal aid, or employment centers). Data-driven approaches to identifying service gaps, understanding the patient population, and uncovering unseen trends are important for improving patient health and advocating for the necessity of these resources. Disparities in food security, economic stability, education, neighborhood and physical environment, community and social context, and access to the healthcare system are crucial social determinants of health, which studies indicate account for nearly 70% of all health outcomes.

We led a 75-minute class session for the Marine Mammals course at the Duke University Marine Lab that introduced students to strengths and challenges of using aerial imagery to survey wildlife populations, and the growing use of machine learning to address these "big data" tasks.

Most phenomena that data scientists seek to analyze are either spatially or temporally correlated. Examples of spatial and temporal correlation include political elections, contaminant transfer, disease spread, housing market, and the weather. A question of interest is how to incorporate the spatial correlation information into modeling such phenomena.

 

In this project, we focus on the impact of environmental attributes (such as greenness, tree cover, temperature, etc.) along with other socio-demographics and home characteristics on housing prices by developing a model that takes into account the spatial autocorrelation of the response variable. To this aim, we introduce a test to diagnose spatial autocorrelation and explain how to integrate spatial autocorrelation into a regression model

 

 

In this data exploration, students are provided with data collected from remote sensing, census, and Zillow sources. Students are tasked with conducting a regression analysis of real-estate estimates against environmental amenities and other control variables which may or may not include the spatial autocorrelation information.