DataOps for Physical AI: Versioning Training Datasets

A robotics team swaps the gripper on a pick-and-place cell. Two weeks later the grasp success rate drops, and the first question is a simple one: which dataset was the deployed model trained on? Nobody knows. The shared drive holds folders named v2, v3, and v3_final, and v3 on Monday is not the same as v3 on Friday.

Teams that build perception for robots, whether in-house or through physical AI training data services, hit this problem sooner than they expect. Dataset volume grows fast, the data changes more often than the code, and the tooling for tracking it is less mature than Git. DataOps practice closes the gap by treating datasets as versioned release artifacts, with the same discipline as application builds.

Why Physical AI Datasets Are Harder to Version

Four properties make them harder than a typical tabular or text dataset:

• They are tied to hardware. A dataset is valid for the sensor setup it represents: resolution, field of view, lens, depth range, noise. Swap the camera and the old data may no longer match what the robot sees.
• They are multimodal and large. RGB, depth, point clouds, and infrared files do not fit in a Git repository. The common approach is to keep data in object storage and track content hashes or pointer files in Git. DVC and Git LFS both work this way.
• Labels are part of the data. Renaming a class or switching annotation format changes the dataset even when no image changed.
• Scenario coverage matters. Two datasets with the same image count can cover very different lighting, clutter, and object sets.

What a Dataset Version Should Record

If a version can’t be reproduced, it can’t be debugged. Each release needs a manifest with these fields:

A manifest can be as plain as this (values are illustrative):

dataset: pick-place-cell-a

version: 2026.10.3

generator:

 scene_config: scenes/cell_a.yaml@9f3c2e1

 seed: 4182

assets: [bin_small@v7, gripper_b@v2]

sensors:

 rgb: {resolution: [1280, 720], fov_deg: 69}

 depth: {range_m: [0.2, 3.0]}

labels: {schema: coco, classes: [part_a, part_b, part_c]}

sources: {synthetic: 48000, real: 1200}

validation_set: real-holdout-2026-09

Run Dataset Changes Through a Pipeline

Treat a dataset change the way you treat a code change. A workable sequence:

1. Define the scenario spec in Git. Object set, lighting ranges, camera positions, label classes.
2. Generate on trigger. A CI job calls the generator, which is easy when generation is exposed through an API.
3. Run automated checks. Validate the label schema, count instances per class, flag class imbalance, find duplicates, and render a sample sheet for a human to scan.
4. Version and store. Write the manifest, compute hashes, assign an immutable dataset ID.
5. Train with the ID logged. Every training run records which dataset version it used.
6. Gate promotion. A model ships only if it passes on the fixed real validation set.

Tracking the Synthetic and Real Mix

Most physical AI datasets combine generated and captured images, and the ratio changes behavior. A model trained mostly on synthetic data tends to handle rare conditions well and miss quirks of the real camera. A model trained mostly on real data shows the opposite pattern. Record the split in the manifest and keep one rule fixed: the validation set is always real, never regenerated, and never overlaps with training. When a new version performs worse, the source mix is the first thing to compare against the previous release.

Triggers That Should Regenerate a Dataset

Not every change needs a new dataset. These do:

• Hardware change. New camera, lens, gripper, or mounting position.
• New object or SKU. The model has never seen it.
• Field failure clusters. Several failures with a shared cause, such as glare on one shelf or a partially hidden label.
• Scheduled refresh. A periodic check for drift between the dataset and current conditions.

The third trigger is the most useful. Each field failure can become a scenario spec: reproduce the condition, generate several thousand labeled variations, and add them to the next release. Recollecting real data for every failure is slow, and some conditions are unsafe or impractical to stage.

Mistakes That Cost the Most Time

• Overwriting folders in place. The old version is gone and the model can’t be reproduced.
• Unrecorded augmentation. If a training script applies random crops or color shifts, the settings belong in the manifest.
• Mixing label schema versions inside one dataset.
• Changing the validation set between runs, then comparing the numbers as if they were equivalent.
• Skipping the dataset ID in the training log. This one creates the gripper problem from the opening.

Start small. A manifest file, a content hash, and a dataset ID in every training run answer the gripper question in minutes instead of days. The rest of the pipeline can be added as the dataset count grows.

John