Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration

TY - CHAP

T1 - Soft Reasoning: Navigating Solution Spaces in Large Language Models through Controlled Embedding Exploration

AU - Zhu, Qinglin Zhu

AU - Zhao, Runcong

AU - Yan, Hanqi

AU - He, Yulan

AU - Chen, Yudong

AU - Gui, Lin

PY - 2025

Y1 - 2025

N2 - Large Language Models (LLMs) struggle with complex reasoning due to limited diversity and inefficient search. We propose Soft Reasoning, an embedding-based search framework that optimises the embedding of the first token to guide generation. It combines (1) embedding perturbation for controlled exploration and (2) Bayesian optimisation to refine embeddings via a verifier-guided objective, balancing exploration and exploitation. This approach improves reasoning accuracy and coherence while avoiding reliance on heuristic search. Experiments demonstrate superior correctness with minimal computation, making it a scalable, model-agnostic solution.

AB - Large Language Models (LLMs) struggle with complex reasoning due to limited diversity and inefficient search. We propose Soft Reasoning, an embedding-based search framework that optimises the embedding of the first token to guide generation. It combines (1) embedding perturbation for controlled exploration and (2) Bayesian optimisation to refine embeddings via a verifier-guided objective, balancing exploration and exploitation. This approach improves reasoning accuracy and coherence while avoiding reliance on heuristic search. Experiments demonstrate superior correctness with minimal computation, making it a scalable, model-agnostic solution.

M3 - Conference paper

BT - Forty-second International Conference on Machine Learning. ICML 2025.

PB - PMLR

T2 - 2025 International Conference on Machine Learning

Y2 - 13 July 2025

ER -