Out-of-Label Hazard Detection for Autonomous Driving: Fusing Optical Flow, Depth, Proximity, and Scene Description

Abstract

In this paper, we address the challenge of improving hazard detection in autonomous driving systems, particularly in scenarios where labeled data is scarce or unavailable. This issue is critical in real-world applications, where diverse and unpre- dictable driving situations make it difficult to label every poten- tial hazard accurately. Recently, the Challenge of Out-of-Label (COOOL) benchmark has been introduced at WACV2025 to promote research on this challenge.To tackle this issue, we present a novel method that integrates a Bootstrapping Language-Image Pretraining (BLIP)-based scenario generation framework with a threshold-based hazard scoring system, thereby enhancing both scenario comprehension and detection accuracy within the benchmark. By incorporating robust driver state logic, bounding box analysis, and BLIP-generated scenario descriptions, our method initially achieves a 40% performance score. Building upon this foundation, we further integrate depth maps and optical flow to improve hazardous object discrimination, resulting in an additional 20% performance improvement. This culminates in a final score of 63% on the public benchmark leaderboard and 50% on the private leaderboard. To foster continued ad- vancements in autonomous driving research, we will make all code and visualization tools publicly available.