LumenLens Multimodal Consistency Verification Engine: Detecting When AI Contradicts Itself Across Vision, Audio, and Text

LumenLens Multimodal Consistency Verification Engine: Detecting When AI Contradicts Itself Across Vision, Audio, and Text

The University of Toronto's AI Lab has published LumenLens, an open-source tool for detecting cross-modal factual conflicts when multimodal large models process images, audio, and text simultaneously, in the latest issue of ICLR. The paper reports that across 24,000 test cases on seven mainstream multimodal models including GPT-Vision, Gemini Ultra-Pro, and Claude 4.5, an average of 17.3 percent of responses contained self-contradictions.

Multimodal large models have made significant progress over the past two years, but researchers have become aware of a neglected problem: when a model simultaneously receives video frames, narration audio, and a text description, its judgment about the same fact can diverge across modalities. For example, in a video showing flames burning, the model might correctly identify the visual signal as "fire" while simultaneously taking audio narration that says "water" as the correct answer.

LumenLens's core idea is to treat consistency as an independently trainable detection task, not a byproduct of the model. The system has three independent components: a fact extractor that pulls out discrete fact triples from each modality; a consistency comparator that scores the semantic relationship between triples; and an explanation generator that produces a human-readable diagnostic report whenever a conflict is detected.

The fact extractor is built on a specially trained lightweight model. Trained on a dataset of 12 million human-annotated multimodal triples, it learns to identify object relationships from images, event descriptions from audio, and explicit statements from text. Triples use a simplified event notation such as "[object A, action B, object C]."

The consistency comparator uses an embedding space trained with contrastive learning. Triples are mapped into this space; semantically supporting triples land close together, while contradictory triples are pushed apart. The system learns this "distance-as-conflict" geometry from human-annotated conflict samples during training.

The reported evaluation covered 24,000 multimodal question-answering samples. Across all models, Claude 4.5 had the highest consistency score at 89.4 out of 100, while one Chinese open-source model scored only 61.2. Notably, consistency scores did not correlate strongly with conventional visual-question-answering accuracy: some models above 90 percent VQA accuracy scored below 75 on cross-modal consistency.

On applications, LumenLens has been piloted in autonomous driving, assisted medical diagnosis, and media content moderation. Self-driving company Aurora Robotics has integrated LumenLens as the final validation step in its perception stack. When LumenLens flags a consistency conflict, the vehicle switches to conservative driving mode and requests remote assistance. Over a six-month pilot, the mechanism helped avoid 14 potential traffic incidents.

On the media side, the Canadian Broadcasting Corporation (CBC) has deployed LumenLens in its news fact-checking workflow. When a reporter cites composite material containing video, audio, and a text description, LumenLens automatically generates a consistency report for the editor's reference. Patricia O'Connell, CBC's director of news standards, said the tool has significantly reduced the risk of errors when quoting multimodal material.

The University of Toronto team has open-sourced LumenLens on GitHub under the MIT license. Lead researcher Professor Wei Li said multimodal consistency is one of the most important credibility problems facing today's AI systems. Single-modality accuracy alone no longer fully describes a model's quality; cross-modal consistency must be added as a dimension.