# Object Detection Metrics: Flood vs Water Detection Use Case - Savonia AMK

> Computer vision technologies are now widely used in daily life and industrial applications.

Computer vision technologies are now widely used in daily life and industrial applications. Especially with the development of new artificial intelligence models, their use has become easier. In object detection projects, different metrics are used to evaluate the performance of the model. Precision, Recall, mAP (Mean Average Precision), and F1-score have varying degrees of importance depending on the requirements of the project. It is important to determine which metrics should be used or considered in projects.

In this article, we will analyze these metrics through a flood vs. puddle detection project and discuss which metric is more critical for different types of projects.

## Key Metrics for Object Detection

When evaluating object detection models, it is crucial to use well-defined metrics to understand their performance. Different metrics highlight various aspects of model behavior, such as accuracy, robustness, and error rates.

### Precision (P)
Precision measures the ratio of correctly predicted positive cases to all positive (true positives and false positives) predictions made by the model.

**Precision = TP / (TP + FP)**

*   **TP (True Positives):** Correctly detected flood/water regions.
*   **FP (False Positives):** Incorrectly detected flood/water regions (false alarms).

It is critical for applications where false alarms/warnings must be minimized. In flood detection, false alarms can lead to unnecessary evacuations, wasted resources, and reduced public trust.

### Recall (R)
Recall measures the proportion of real objects correctly detected by the model.

**Recall = TP / (TP + FN)**

*   **TP (True Positives):** Correctly detected flood/water regions.
*   **FN (False Negatives):** Actual flood/water regions that the model failed to detect.

It is essential for applications where missing detections can have severe consequences. In flood detection, a low recall means the model is failing to detect real flood areas, which can lead to catastrophic outcomes.

### F1-Score
The F1-score provides a balance between precision and recall, ensuring that both false positives and false negatives are considered.

**F1 = 2 * (Precision * Recall) / (Precision + Recall)**

### Mean Average Precision (mAP)
The mAP is a widely used object detection metric that evaluates model performance across different confidence thresholds and object sizes.

*   **mAP@50 (IoU = 0.5):** Measures accuracy when the Intersection over Union (IoU) is at least 50%.
*   **mAP@50-95 (IoU = 0.5 to 0.95):** A stricter evaluation measuring accuracy across multiple IoU thresholds.

## Selecting Metrics by Use Case

Selecting the appropriate metric is determined by the project’s specific objectives:

*   **Flood Detection:** Recall and mAP@50-95 are essential.
*   **Security Camera Face Detection:** Precision is prioritized.
*   **Medical Screening (Cancer Detection):** Recall is critical.
*   **Autonomous Vehicles:** Depend on mAP@50 and F1-Score.
*   **Wildlife Monitoring:** Recall ensures rare animals are detected.
*   **Retail Theft Detection:** Emphasizes precision.
*   **Manufacturing Defect Detection:** Utilizes mAP@50-95.
*   **Weapon Detection:** Requires high recall.

## Implementation: Flood vs. Water Detection

### Object Detection Model
YOLOv8 was chosen for its real-time performance, efficiency, and flexibility.

### Dataset Collection
We combined four datasets from the Roboflow Universe platform, totaling 9,467 images, including background images to enhance robustness and distinguish between general water accumulation and actual flooding.

### Training Configuration
*   **Model:** YOLOv8 (anchor-free detection)
*   **Hyperparameters:** Learning rate: 1e-3, Batch size: 16, Epochs: 100, Optimizer: AdamW
*   **Train/Valid/Test Split:** 70%-20%-10%
*   **Augmentation:** Not applied

## Experimental Results

The best-performing model was trained with background images, 100 epochs, and augmentation disabled.

*   **Flood Precision (90.6%):** Indicates the model reliably detects flood scenarios with minimal false alarms.
*   **Water Recall (72.4%):** Suggests some water surfaces were missed, likely due to environmental factors.

## Conclusion

Our practical implementation using YOLOv8 demonstrated the importance of distinguishing between general water accumulation and actual flooding. While the model performed well in detecting floods with high precision and recall, there is room for improvement in handling complex scenarios. Selecting the right evaluation metrics ensures that object detection models meet the specific needs and challenges of each application.

## References

1. **Roboflow Universe:** [https://universe.roboflow.com](https://universe.roboflow.com)
2. **YOLOv8 Official Documentation:** [https://ultralytics.com/yolov8](https://ultralytics.com/yolov8)

## Authors

*   **Osman Torunoglu**, RDI Specialist, DigiCenter, Savonia-ammattikorkeakoulu Oy
*   **Shahbaz Baig**, RDI Specialist, DigiCenter, Savonia-ammattikorkeakoulu
*   **Rosa Sartjärvi**, RDI Specialist, Water Safety, Savonia-ammattikorkeakoulu
*   **Petri Juntunen**, RDI Specialist, Water Safety, Savonia-ammattikorkeakoulu
*   **Paola Rosales Suazo De Kontro**, RDI Specialist, Hyvinvointi, Savonia-ammattikorkeakoulu