Object Detection

Object detection predicts both what is present and where it is, usually as class-labeled bounding boxes with confidence scores. It is the box-level task between image classification, which ignores location, and instance segmentation, which predicts object masks.

Boxes and scores

A detector returns a set

$$\{(b_i,c_i,s_i){\}}_{i=1}^m,b_i=(x_1,y_1,x_2,y_2),$$

where $c_i$ is a class and $s_i$ is a confidence score. Two-stage detectors such as Faster R-CNN first propose candidate regions and then classify/refine them; one-stage detectors predict dense boxes directly from feature maps. Most pipelines finish with non-maximum suppression (NMS): sort boxes by score, keep the highest-scoring box, and suppress lower-scoring boxes of the same class whose IoU exceeds a threshold.

Worked example

This snippet computes box IoU and runs non-maximum suppression so overlapping lower-score boxes are removed.

import numpy as np
 
boxes = np.array([[0,0,2,2], [.2,.1,2.2,2.1], [3,0,5,2], [0,3,2,5]], float)
scores = np.array([.9, .75, .8, .3])
 
def box_iou(a, b):
    lo, hi = np.maximum(a[:2], b[:2]), np.minimum(a[2:], b[2:])
    inter = max(0, hi[0]-lo[0]) * max(0, hi[1]-lo[1])
    return inter / (((a[2]-a[0])*(a[3]-a[1])) + ((b[2]-b[0])*(b[3]-b[1])) - inter)
 
keep = []
for i in np.argsort(-scores):
    if all(box_iou(boxes[i], boxes[j]) <= .5 for j in keep):
        keep.append(int(i))
print("pair_iou_0_1", round(box_iou(boxes[0], boxes[1]), 3))
print("kept_indices", keep, "kept_scores", scores[keep].tolist())

Observed output:

pair_iou_0_1 0.747
kept_indices [0, 2, 3] kept_scores [0.9, 0.8, 0.3]

The second box is suppressed because it overlaps the first box too much and has lower confidence. The kept boxes would then be evaluated with detection and segmentation metrics, where AP50 rewards rough object discovery and AP75 tests tighter localization.

Caveats

NMS can remove a real object in crowded scenes when two objects overlap. Small objects disappear when the CNN architecture downsamples too aggressively. Domain-specific variants, such as rotated object detection, change the geometry but still need calibrated scores, duplicate handling, and per-slice evaluation.

References

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Microsoft COCO: Common Objects in Context

Nav

Section — Computer Vision

← Image Classification Rotated Object Detection →

Learning path — Computer vision

← Image Representation Detection and Segmentation Metrics →