Learning to evaluate data sources, understand metadata, and make informed decisions about data quality and ethical implications in science investigations.
Once a problem is framed and a research question is defined, the next step is to ask:
What data can actually help us answer this question, and can we trust it?
In real data investigations, data does not magically appear clean, complete, or perfectly aligned with our goals. According to Lee et al. (2022), considering and gathering data involves understanding what data exists, what data is missing, how it was collected, and whose interests it represents.
This module is where students begin to act like data investigators, not just data users.
By the end of this module, teachers and students will be able to:
Framed Research Question:
What types of electrical wire materials are most commonly associated with residential electrical fire incidents?
All examples in this module connect back to this question.
Before opening any dataset, articulate what data would be useful.
Key questions:
Prompting technique: Zero-shot prompting (best for Understand level tasks)
Example prompt:
“Based on this [insert research question here], list the types of data variables that would be useful for investigating it.”
The LLM helps generate ideas, but teachers and students must decide what is actually feasible and relevant.
Data provenance refers to where the data originates, who collected it, and why it exists. The data provenance can greatly influence the kind of insight generated. Also, there might be bias or some factors embedded in the data that may make it unfit for your task.
Students should investigate:
Fire incident data might come from:
Each source implies different priorities and limitations.
Prompting technique: Chain-of-Thought Prompting (Best for: Analyze / Evaluate tasks)
Example prompt:
“This dataset comes from a municipal fire department.
Step by step, analyze the strengths and limitations of this data source for investigating electrical fire causes.
Consider how the data was collected, what variables are included or missing, and any potential sources of bias.”
Here, the model is asked to evaluate the data source rather than describe it.
Metadata includes information about:
Many people often skip this step, but it’s where misunderstandings begin.
A column labeled WIRE_TYPE might:
Prompting technique: Few-shot prompting (best for Understand / Apply tasks)
Example prompt:
“Here is an example of a well-described dataset column and explanation.
Now help interpret these column names and suggest questions a student should ask about them.”
Few-shot prompting helps the LLM model mirror good interpretive habits.
Not all data represents everyone equally.
Students should ask:
Electrical fires may be underreported in:
Prompting technique: Self-consistency prompting (best for Analyze / Evaluate tasks)
Example prompt:
Take note to use this same prompt 3 different times and compare answers.
“Suggest three different ways this dataset [attach dataset] might be biased.
Then explain which bias is most concerning for this investigation and why.”
This encourages multiple perspectives before settling on one interpretation.
Sometimes the available data is not enough.
Students must decide:
The dataset might list fire incidents but not wire material. That signals a need for:
Prompting technique: Self-consistency prompting (best for Analyze / Create tasks)
Example prompt:
“Given this dataset’s limitations, propose three ways to strengthen the investigation using additional data.”
Once data is selected, students should standardize how it is stored.
Best practices:
FireData_2024_Cleaned.xlsx)Prompting technique: Zero-shot prompting (best for Apply tasks)
Example prompt:
“Suggest a clear file naming convention for this dataset and explain why it supports good data practice.”
If multiple datasets are relevant, students may explore joining them using:
This step reinforces that data analysis decisions affect results.
Prompting technique: Prompt chaining (best for Apply → Analyze tasks)
Example chain:
Prompt 1:
“What variables could be used to join these datasets?”
Prompt 2:
“What problems might arise when joining on this variable?”
In this module, students learn that data is not neutral, complete, or self-explanatory. Considering data carefully—its source, structure, limitations, and ethics—is essential for meaningful science investigations.
LLMs play a valuable role here as thinking partners, helping students ask better questions about data rather than rushing into analysis.
In the next module, we move into exploring and visualizing data, where patterns, variability, and evidence begin to take shape.
Reference:
Lee, V. R., Wilkerson, M. H., & Lanouette, K. (2022). A call for a reimagined science education data literacy. Journal of Research in Science Teaching, 59(6), 1019-1049.
Here are some more articles you might like to read next: