Data Augmentation

Data augmentation creates transformed training examples that should preserve the target label. It is a robustness tool for domain shift, a core ingredient of self-supervised visual learning, and a label-geometry risk in object detection or instance segmentation.

The augmentation objective

For a transformation $T\sim \mathcal{T}$, training minimizes

$${\mathbb{E}}_{(x,y)}{\mathbb{E}}_T[\ell (f_{\theta }(T(x)),T_y(y))],$$

where $T_y$ transforms labels when needed. A crop or flip must update boxes and masks; mixup creates convex combinations

$$\tilde{x}=\lambda x_i+(1-\lambda )x_j,\tilde{y}=\lambda y_i+(1-\lambda )y_j.$$

Worked example

This snippet horizontally flips an image bounding box and mixes two labels, showing how augmentations must transform labels consistently with pixels.

import numpy as np
 
box = np.array([1, 1, 3, 4])  # xyxy in a width-6 image
W = 6
flipped = np.array([W - box[2], box[1], W - box[0], box[3]])
lam = .3
y1, y2 = np.array([1., 0.]), np.array([0., 1.])
ym = lam * y1 + (1 - lam) * y2
print("original_box_xyxy", box.tolist(), "flipped_box_xyxy", flipped.tolist())
print("mixup_label", np.round(ym, 2).tolist(), "lambda", lam)

Observed output:

original_box_xyxy [1, 1, 3, 4] flipped_box_xyxy [3, 1, 5, 4]
mixup_label [0.3, 0.7] lambda 0.3

The horizontal flip is only correct because the box coordinates are transformed with the image: in a width-6 frame, $x_1=1,x_2=3$ becomes $x_1=3,x_2=5$. The mixup label [0.3, 0.7] likewise preserves the chosen $\lambda =0.3$, so the target changes with the augmented input instead of remaining a hard class.

Caveats

Augmentation teaches invariances. Horizontal flips are wrong for laterality markers, text, and some traffic signs. Strong color jitter can erase medically meaningful intensity. Copy-paste synthetic data can improve rare instances, but unrealistic boundaries may become a shortcut.

References

• Torchvision transforms documentation
• mixup: Beyond Empirical Risk Minimization

Nav

Section — Computer Vision

← Feature Extraction Image Classification →