ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space

📄 Abstract

Multimodal Large Language Models (MLLMs) have achieved remarkable progress but still struggle with complex visual reasoning tasks requiring multi-step perception and logical deduction. While explicit visual generation incurs prohibitive computational costs, existing latent approaches often rely on external experts or lack rigorous cognitive logic.

In this paper, we introduce ProLaViT (Progressive Latent Visual Thought), a framework empowering MLLMs to perform structured visual derivation in the continuous latent space. Unlike works dependent on heterogeneous external models, ProLaViT leverages an endogenous self-distillation mechanism, utilizing the model's own visual encoder to supervise latent thoughts. To facilitate this, we construct a scalable programmatic synthesis pipeline enabling the model to internalize algorithmic precision without inference-time tools.

We design two reasoning paradigms: (1) Coarse-to-Fine Causal Chain for spatial tasks, guiding attention from global context to local targets; (2) Dialectical Reasoning Chain for logical tasks, incorporating counterfactual thinking for verification. Furthermore, we propose a Distance-Weighted Diversity Loss to impose topology-aware constraints, preventing feature degeneration by enforcing semantic distinctiveness.

🎯 Key Results: Extensive experiments demonstrate that ProLaViT outperforms baselines on vision-centric benchmarks, achieving superior accuracy and interpretability with high efficiency.

💡 Introduction

Figure 1. Comparison of multimodal reasoning paradigms. Top: Textual CoT suffers from a modality gap, failing to accurately ground spatial relationships. Middle: One-step Latent Prediction attempts to identify the target region in a single leap but suffers from precision loss. Bottom: ProLaViT (Ours) decomposes the task into a progressive latent chain (Locate → Focus → Isolate), enabling precise target isolation and robust reasoning.

Effective visual reasoning requires neither expensive pixel generation nor unstructured latent guessing, but rather Progressive Visual Derivation within a structured latent space. ProLaViT decomposes complex visual tasks into a structured chain of latent thoughts, mimicking human cognitive processes following a Locate → Focus → Isolate causal chain to progressively refine visual attention.

A key challenge in training such multi-step latent models is the absence of ground truth for intermediate images. ProLaViT employs an Endogenous Self-Distillation mechanism, leveraging the MLLM's own frozen vision encoder to extract features from programmatically synthesized auxiliary images, effectively avoiding the domain gaps and pipeline complexity introduced by external dependencies.

🔬 Method

Figure 2. Overview of ProLaViT. The framework generates progressive latent thoughts (green tokens) supervised by an Endogenous Self-Distillation mechanism (Right), where latent states are aligned with visual features from synthesized auxiliary images via L_MSE. To prevent representation degeneration, a Distance-Weighted Diversity Loss (L_div, Left) penalizes excessive similarity between reasoning steps, enforcing topological distinctiveness in the latent space.

🌟 Key Features

Progressive Latent Visual Thought

Decomposes complex visual reasoning into a structured chain of K=4 intermediate reasoning steps with two paradigms: Coarse-to-Fine Causal Chain and Dialectical Reasoning Chain.

Endogenous Self-Distillation

Leverages the MLLM's own frozen vision encoder as teacher. No external vision experts needed, eliminating domain gaps and pipeline complexity.

Distance-Weighted Diversity Loss

Margin-based loss with causal-distance weighting to prevent latent collapse. Steps with larger causal distance are enforced to be more distinct.

Programmatic Data Synthesis

Scalable pipeline generating rigorous visual trajectories (grid, bbox, crop, segmentation) for training without manual annotation.

Multi-Stage Curriculum Learning

Four-phase training: Anchor Initialization → Knowledge Distillation → Chain Evolution → Global Integration.

High Efficiency

Only 1.21x inference latency and +12% TFLOPs compared to base model, while achieving higher accuracy than explicit tool-use methods.

🧠 Reasoning Paradigms

Figure 3. Illustration of the two reasoning paradigms in ProLaViT. Top: The Coarse-to-Fine Causal Chain for spatial tasks progressively narrows attention from global context to specific targets (Locate → Focus → Isolate). Bottom: The Dialectical Reasoning Chain for logical tasks employs a trial-and-error approach (Hypothesize → Critique → Verify) to validate visual consistency.

📊 Experimental Results

Performance on Visual Reasoning Benchmarks

Method	MMVP	VisPuzzle	VStar	ChartQA	BLINK	CV-Bench	BLINK-J	Avg.
Qwen2.5-VL (Base)	77.33	34.75	76.44	78.45	54.49	73.61	59.33	66.00
SFT	77.00	67.50	78.01	76.74	55.60	77.36	49.33	69.86
CoT SFT	77.66	69.75	75.91	75.83	56.54	64.55	68.66	69.18
CoVT	78.33	35.75	79.05	24.99	57.49	75.00	66.00	61.45
LVR	77.30	35.00	76.43	64.86	53.02	76.55	57.33	64.63
One-step Latent Pred.	79.33	72.00	78.01	63.02	56.28	76.10	66.00	70.85
ProLaViT (Full)	79.00	74.25	80.11	78.99	57.23	77.66	76.00	75.11

All values are reported in accuracy (%). ProLaViT achieves the highest average accuracy of 75.11%.

Latent Thought Similarity Visualization

Figure 4. Visualization of latent thought similarity. We plot the cosine similarity heatmaps between the token representations of different reasoning steps (z₁ to z₄). (a) No Diversity Loss: The chain suffers from latent collapse, where subsequent steps become indistinguishable. (b) Margin-Based Formulation: Enforces distinctiveness equally across all pairs, potentially disrupting semantic continuity. (c) Distance-Weighted Loss (Ours): Respects the hierarchical structure, where adjacent steps retain moderate similarity while distant steps are enforced to be distinct.

Efficiency Comparison

Method	VStar	BLINK-J	Latency	TFLOPs
Qwen2.5-VL (Base)	76.44	59.33	1.00×	1.00
Explicit Tool-Use SFT	79.60	74.00	3.40×	2.17
ProLaViT (Ours)	80.11	76.00	1.21×	1.12

ProLaViT achieves higher accuracy with significantly lower latency and compute overhead compared to explicit tool-use.

📚 Citation

@misc{li2026prolavitlearningprogressivelatent,
      title={ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space}, 
      author={Peiming Li and Yifan Wang and Xiaotian Zhang and Zhiyuan Hu and Shiyu Li and Zheng Wei and Yang Tang},
      year={2026},
      eprint={2607.02907},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.02907}, 
}