Learning to clean, organize, and transform data through iterative decision-making to prepare it for meaningful scientific analysis.
Real data is almost never ready for analysis. Measurements are missing, values are inconsistent, units don’t match, and datasets include information that doesn’t actually help answer your question.
Data wrangling is the process of cleaning, organizing, and transforming data so it becomes usable for sensemaking. This phase is where you decide what data matters, how it should be structured, and what transformations best support your investigation.
In practice, this is also where most of the work happens.
Data wrangling is not a checklist; it is iterative decision-making. At each step, you:
LLMs can help by explaining options, consequences, and tradeoffs, but you remain responsible for the final choices.
By the end of this phase, you should have:
The following five actions represent the fundamental work of data wrangling. Each requires scientific judgment and intentional decision-making.
Filtering means selecting only the subset of data relevant to your question.
This helps reduce noise and keeps the analysis aligned with the phenomenon you are studying.
Science examples:
LLM support (Zero-shot or Few-shot prompting):
Use the LLM model to suggest reasonable filtering criteria based on your research question, then decide whether they make scientific sense.
Grouping involves organizing data into categories or levels so comparisons are possible.
This is essential for identifying patterns and differences.
Science examples:
LLM support (Few-shot prompting):
Provide examples of how grouping is used in similar investigations and ask the LLM model to suggest groupings, but validate them against your data context.
Cleaning means identifying and deciding how to handle:
There is rarely one “correct” cleaning choice—each decision affects interpretation.
Science examples:
LLM support (Chain-of-thought prompting):
Ask the LLM model to walk through possible cleaning options and their consequences. Use this to compare alternatives, not to outsource judgment.
Transformation involves creating new variables or modifying existing ones to better support analysis.
This step often connects raw measurements to scientific meaning.
Science examples:
LLM support (Prompt chaining):
First prompt: Identify useful transformations.
Second prompt: Explain how each transformation would support the investigation.
Before interpreting patterns, you need a descriptive overview of the dataset.
Common summaries include:
These summaries help you understand variability and typical values.
Science examples:
LLM support (Self-consistency prompting):
Ask the LLM model to generate summaries multiple times or in different formats to check for stability and consistency in interpretation.
Data wrangling is not neutral. Every choice, what to remove, how to group, what to transform, shapes the story the data can tell.
LLMs are most useful here as thinking partners:
They do not decide what is scientifically appropriate, you do.
In the next module, we move from preparing data to analyzing it, where patterns become evidence and claims require justification.
Here are some more articles you might like to read next: