Instance Segmentation: Counting Cars from Public CCTV with YOLOv8 seg

Table of Contents

• Why Isn't a Box Enough?

• Guess First

• Semantic vs Object Detection vs Instance Segmentation

• Why Instance Segmentation Matters

• From Mask R-CNN to YOLO-Segmentation

• Building the Model

• Validation Results

• Real-World Test: Public CCTV in Yogyakarta

• Getting the CCTV Stream Link

• Quick Quiz

• Limitations

• Summary

Why Isn't a Box Enough?

Object detection demos look impressive at first glance: blue boxes tracking cars down a street. But picture five cars packed together at a red light. The boxes start overlapping, and it becomes hard to tell exactly where one car ends and the next begins.

That's where instance segmentation comes in. Instead of just "where is the car", it answers "which exact pixels belong to this specific car, as opposed to the one next to it."

Guess First

Before we go further: if three motorcycles and a car are bunched up at an intersection, would plain object detection (boxes) give a clean, precise read, or would it struggle to define where each vehicle actually ends?

(The answer is in the next section.)

Semantic vs Object Detection vs Instance Segmentation

These three get mixed up often, but they solve different problems:

Approach	Output	Distinguishes individual objects?
Semantic Segmentation	Per-pixel label, per class	No — every car becomes one blob of the same color
Object Detection	Bounding box + class	Yes, but only as a box, not the object's real shape
Instance Segmentation	Per-pixel mask, per individual object	Yes, and it follows the object's actual shape

Instance segmentation is essentially the first two combined: detect each object individually (like object detection), then give each one its own pixel-level mask (like semantic segmentation).

Why Instance Segmentation Matters

Back to the "guess first" question: here's the answer, and it doubles as three concrete reasons bounding boxes alone often fall short:

1. Overlapping objects. In dense traffic, bounding boxes overlap and it gets hard to tell where one object's real boundary is. Pixel masks give a precise edge.

2. Objects aren't box-shaped. Vehicles and people rarely form a perfect rectangle from any camera angle. A bounding box always includes extra background space around the object.

3. Area calculations. If the goal is measuring an object's actual pixel area (not just counting it), a bounding box can't give an accurate number because it includes background inside the box.

From Mask R-CNN to YOLO-Segmentation

The "textbook" architecture for instance segmentation is Mask R-CNN — an extension of Faster R-CNN with an added Fully Convolutional Network (FCN) branch dedicated to producing masks, running in parallel with the classification and bounding-box branches.

There's a lighter alternative, though: Mask-YOLO, which pairs a YOLO backbone with a similar FCN branch. That's the foundation for the practical work in this post, using Ultralytics' YOLOv8-seg as a modern implementation of that same YOLO-segmentation idea.

Building the Model

Base model: yolov8s-seg.pt, COCO-pretrained, fine-tuned on a small traffic-CCTV-themed dataset from Roboflow (201 training images, 19 validation, single class: car).

Trained for 30 epochs at 640px resolution. The resulting weights are automatically copied to Google Drive so they survive a Colab disconnect.

Validation Results

These are real numbers from the actual training run, not estimates:

Metric	Box	Mask
Precision	0.860	0.895
Recall	0.867	0.797
mAP50	0.885	0.878
mAP50-95	0.668	0.604

An mAP50 above 0.87 on a training set this small (201 images) shows transfer learning from the COCO-pretrained weights doing real work. The model isn't learning from scratch, just adapting to a new domain.

Real-World Test: Public CCTV in Yogyakarta

The more interesting part: this model wasn't just tested on static images. It was connected directly to a public ATCS traffic CCTV stream in Yogyakarta (the Sugeng Jeroni camera) over HLS, running frame-by-frame inference in real time to count cars.

From one 601-frame test run (~26 seconds) at the final confidence setting:

• Average of 0.7 cars per frame — this varies with real traffic density at the time; other test runs (different confidence settings) recorded averages above 2 cars per frame

• Up to 3 cars detected simultaneously in that run

• Processing speed varied across test runs, roughly 19-78 FPS depending on scene complexity and Colab runtime conditions

The demo video is available in the GitHub repo (link below).

Getting the CCTV Stream Link

A few people asked about this, so it's worth covering here. Public CCTV portals like Yogyakarta's ATCS usually show video through their own web player, not a direct link a script can use. Here's how to grab the raw .m3u8 link:

1. Open the city's public CCTV portal in a browser

2. Click on the camera you want until the video is actually playing

3. Press F12 (open DevTools) → switch to the Network tab → type "m3u8" in the filter box

4. Refresh the page (F5) — a request ending in .m3u8 should show up

5. Right-click it → Copy link address — that's what goes into cv2.VideoCapture()

Every city's portal is different, and the structure sometimes varies too, but the general method is the same.

Quick Quiz

1. What sets instance segmentation apart from semantic segmentation? Answer: Instance segmentation distinguishes between individual objects (car A vs. car B), while semantic segmentation only labels pixels by class without separating individuals.

2. Why is a bounding box less accurate for measuring an object's surface area? Answer: A box always includes background space around the object that isn't actually part of it, so the resulting area isn't precise.

3. What role do the COCO-pretrained weights play in this fine-tuning process? Answer: The model doesn't learn basic visual concepts from scratch — general features (edges, textures, shapes) are already learned from COCO, so a small dataset (201 images) is enough to fine-tune it toward a specific domain (cars from a CCTV angle).

Limitations

A few honest caveats before treating this as a finished solution:

• The training dataset is fairly small (201 images), so the range of vehicle body types, angles, and lighting conditions it covers is limited.

• Performance on cameras with very extreme angles (e.g. very high, zoomed-out vantage points) tends to be weaker than on closer-angle cameras, consistent with that limited training data diversity.

• The recorded output video is encoded at a fixed frame rate, so its playback duration doesn't map 1:1 to the CCTV feed's actual real-time capture speed.

• What was actually verified on the public CCTV feed is frame-level car counts (from detection boxes), not mask quality. There's a real possibility that mask quality degrades faster than detection recall on real-world footage like this (low resolution, compression artifacts, a camera angle far from anything in COCO) — meaning the car counts could look reasonable while the masks underneath are no longer precise. This wasn't systematically measured here.

Fuller technical detail, including additional experiments that were tried, is documented in the GitHub repo's README.

Summary

• Instance segmentation = object detection + semantic segmentation, combined into a per-object pixel mask

• Mask-YOLO (implemented here via YOLOv8-seg) is a lighter alternative to Mask R-CNN

• Fine-tuning from COCO-pretrained weights lets a small dataset (201 images) still reach an mAP50 above 0.87

• The model was tested against real, live public CCTV, not just static images

• Full repo with notebook and demo video: github.com/arielshakaramiro/car-instance-segmentation-yolov8