AI Eye-Tracking Disabled a Car Mid-Drive

AI Driver Monitoring System Locks Out Driver Over Eye Detection Failure

A vehicle's AI-powered driver monitoring system (DMS) recently made headlines after it reportedly told a driver it 'couldn't see her eyes' — and then automatically disabled the car. The incident, which quickly went viral on social media, has reignited a fierce debate about the role of artificial intelligence in automotive safety and whether these systems are overstepping their boundaries.

The driver was reportedly using an advanced driver-assistance system (ADAS) that requires continuous eye-tracking to verify the operator is paying attention to the road. When the AI system failed to detect her eyes — possibly due to sunglasses, head position, or lighting conditions — the vehicle escalated its warnings and ultimately restricted driving functions, effectively stranding her.

Key Facts at a Glance

• The vehicle's AI-based infrared eye-tracking camera could not locate the driver's eyes during operation
• The system issued escalating warnings before automatically disabling key driving functions
• Similar driver monitoring systems are now standard in vehicles from GM, Ford, BMW, and Mercedes-Benz
• The global DMS market is projected to reach $5.6 billion by 2030, according to Allied Market Research
• Regulatory bodies including the EU and NHTSA are increasingly mandating driver monitoring technology
• The incident has sparked widespread discussion about AI false positives in safety-critical systems

How AI Eye-Tracking Systems Work in Modern Vehicles

Driver monitoring systems use a combination of near-infrared cameras, LED illuminators, and computer vision algorithms to track a driver's gaze, head position, and eyelid movement in real time. The AI model is trained on thousands of facial images to determine whether a driver is looking at the road, appears drowsy, or is distracted.

GM's Super Cruise, Ford's BlueCruise, and BMW's Extended Hands-Free Driving all rely on some version of this technology. These systems typically use infrared light, which can penetrate certain types of sunglasses but struggles with heavily tinted or polarized lenses.

When the system cannot confirm the driver is attentive, it follows a graduated response protocol:

• Stage 1: Visual alert on the dashboard or instrument cluster
• Stage 2: Audible warning chimes or voice prompts
• Stage 3: Haptic feedback such as seat vibration or steering wheel pulses
• Stage 4: Automatic speed reduction and, in some cases, full vehicle disablement
• Stage 5: In the most advanced systems, automatic hazard light activation and emergency stop

This cascading approach is designed to give drivers multiple opportunities to re-engage. But when the AI fundamentally cannot 'see' the driver's eyes — rather than detecting inattention — the system may skip directly to higher alert levels.

Why the System Failed: Common Triggers for False Positives

The incident highlights a well-known limitation in computer vision systems: edge cases. AI models trained on specific datasets can fail when encountering conditions outside their training distribution. For driver monitoring systems, several common scenarios trigger false positives or detection failures.

Polarized sunglasses are among the most frequent culprits. While infrared cameras can see through standard tinted lenses, certain polarization angles block the IR signal entirely, making the driver's eyes invisible to the sensor. Some automakers, including GM, have published lists of compatible eyewear, but most drivers are unaware of this limitation.

Other known triggers include:

• Medical eyewear or thick-framed glasses that obstruct the camera's view
• Facial coverings such as masks or scarves worn in cold weather
• Extreme lighting conditions including low-angle sun glare hitting the cabin
• Non-standard seating positions where shorter or taller drivers sit outside the camera's optimal range
• Certain skin tones and facial features that historically underperform in computer vision models due to training data bias

This last point is particularly concerning. Research from institutions including MIT Media Lab and the National Institute of Standards and Technology (NIST) has repeatedly demonstrated that facial analysis AI systems perform unevenly across demographic groups. A 2019 NIST study found that many commercial facial recognition algorithms had higher error rates for women, older adults, and people with darker skin tones.

The Safety Paradox: When Protection Becomes the Problem

Automakers face an impossible balancing act. Regulators demand that driver monitoring systems be robust enough to prevent distracted driving deaths — which claimed over 3,300 lives in the US in 2022 alone, according to NHTSA data. But making these systems too aggressive creates new risks.

A vehicle that disables itself on a busy highway because it cannot detect a driver's eyes introduces a potentially dangerous situation. Unlike a gradual lane departure warning, a full system disablement can cause confusion, panic, and sudden changes in vehicle behavior that surrounding drivers may not anticipate.

Compared to Tesla's approach, which historically relied on steering wheel torque sensors rather than eye-tracking cameras, camera-based DMS systems represent a more invasive but theoretically more accurate method of monitoring attention. Tesla has since added cabin camera monitoring to its newer vehicles, but its system is generally considered less aggressive in its intervention protocols.

The European Union's General Safety Regulation, which took effect for new vehicle types in July 2024, mandates driver drowsiness and attention warning systems in all new cars. This regulatory push means more vehicles will ship with DMS technology — and more drivers will encounter these edge cases.

Industry Response and Emerging Solutions

Leading DMS suppliers are actively working to reduce false positive rates. Smart Eye, a Swedish company that provides eye-tracking technology to over 15 automakers, has invested heavily in expanding its training datasets to cover a wider range of facial features, eyewear types, and lighting conditions.

Seeing Machines, an Australian DMS provider whose technology is used in GM's Super Cruise, reported in its 2024 earnings that it has processed over 2.5 billion minutes of real-world driving data to improve its algorithms. The company claims its latest generation system achieves a false positive rate below 0.1%, though real-world conditions clearly still produce failures.

Several technical approaches are emerging to address these limitations:

• Multi-modal sensing that combines eye-tracking with steering behavior, pedal inputs, and body posture analysis
• Radar-based driver monitoring that can detect vital signs and micro-movements without relying on optical cameras
• Adaptive AI models that learn individual driver characteristics over time and adjust detection thresholds
• Fallback protocols that maintain vehicle operability while still logging attention concerns

Some researchers advocate for a fundamentally different philosophy: rather than disabling the vehicle, systems should focus on providing graduated assistance while keeping the driver in control.

What This Means for Drivers and the Auto Industry

For everyday drivers, this incident serves as a wake-up call. As vehicles become more reliant on AI co-pilots, understanding how these systems work — and their limitations — becomes essential. Drivers using ADAS features should check their vehicle's documentation for eyewear compatibility and understand the warning escalation process.

For the auto industry, the stakes are enormous. Consumer trust in AI-driven safety features is fragile. A single viral incident of a car 'refusing to drive' can undermine years of marketing and billions of dollars in R&D investment. Automakers need to balance safety mandates with user experience — ensuring systems fail gracefully rather than catastrophically.

The incident also raises important questions about liability. If a driver monitoring system incorrectly disables a vehicle and causes an accident, who bears responsibility? The automaker? The DMS technology supplier? The driver who wore incompatible sunglasses? Current legal frameworks in both the US and EU are still catching up to these scenarios.

Looking Ahead: The Future of AI in Vehicle Safety

The trajectory is clear: AI-based driver monitoring will become more prevalent, not less. The NHTSA is expected to propose updated federal guidelines for driver monitoring systems by late 2025, and China has already mandated DMS for commercial vehicles.

Next-generation systems will likely move beyond simple eye-tracking to holistic driver state assessment, incorporating physiological signals, contextual awareness, and predictive modeling. Companies like Qualcomm and NVIDIA are building dedicated AI processors — such as NVIDIA's DRIVE Orin platform — specifically designed to run these complex models in real time.

The ultimate goal is a system that understands not just where a driver is looking, but whether they are cognitively engaged and capable of taking control. Achieving that level of understanding will require significantly more sophisticated AI — and a much more thoughtful approach to what happens when the system is uncertain.

For now, drivers may want to keep a pair of AI-friendly glasses in the glove box. Just in case.