A Spatial State Representation for Value-Aware Learning

Authors

  • Kyle Hall Author
  • David Baker Author
  • Kenneth Duncan Author

Abstract

Value-aware training focuses on errors consequential to the policy, but spatial-reasoning failures can invalidate the state representation itself. Evaluation should check relevant relations and support in the states the policy uses. Independent action outcomes connect those checks to value. Average prediction loss and attention visualization cannot independently certify the complete grounded decision. This focused synthesis develops an empirical comparison that preserves the terminal action, observation process, and total verification cost. Its proposed tests are prospective; no new simulation run, model-training result, or intervention effect is reported.

References

  1. Z. Qiao, J. Lyu, B. Lyu, Y. Shu, S. Gao, and S. Qiu, “Model-Based Offline RL via Robust Value-Aware Model Learning with Implicitly Differentiable Adaptive Weighting,” Proceedings of the 2026 International Conference on Learning Representations, 2026.
  2. S. Chen, T. Zhu, R. Zhou, J. Zhang, S. Gao, J. Niebles, M. Geva, J. He, J. Wu, and M. Li, “Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas,” Proceedings of the 42nd International Conference on Machine Learning, 2025.
  3. Tan, Wei, Yuanhao Li, and Wenkai Liang. “Evo-CuRL: Curriculum-Aware Reinforcement Learning over Code Lineage Graphs for Software Engineering Reasoning.” Proceedings of the 2026 International Conference on Multimedia Retrieval, ACM, 2026, pp. 1327-1335. https://doi.org/10.1145/3805622.3810701.
  4. Kumar, Aviral, Aurick Zhou, George Tucker, and Sergey Levine. “Conservative Q-Learning for Offline Reinforcement Learning.” Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 1179–1191. https://arxiv.org/abs/2006.04779.
  5. Lewis, Patrick, et al. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 9459–9474. https://arxiv.org/abs/2005.11401.
  6. Geifman, Yonatan, and Ran El-Yaniv. “Selective Classification for Deep Neural Networks.” Advances in Neural Information Processing Systems, vol. 30, 2017. https://arxiv.org/abs/1705.08500.
  7. Ovadia, Yaniv, et al. “Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift.” Advances in Neural Information Processing Systems, vol. 32, 2019. https://arxiv.org/abs/1906.02530.
  8. Bertsimas, Dimitris, and Melvyn Sim. “The Price of Robustness.” Operations Research, vol. 52, no. 1, 2004, pp. 35–53. https://doi.org/10.1287/opre.1030.0065.
  9. Rasmussen, Carl Edward, and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006. https://gaussianprocess.org/gpml/.
  10. Hu, Edward J., et al. “LoRA: Low-Rank Adaptation of Large Language Models.” International Conference on Learning Representations, 2022. https://openreview.net/forum?id=nZeVKeeFYf9.
  11. Guo, Chuan, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. “On Calibration of Modern Neural Networks.” Proceedings of the 34th International Conference on Machine Learning, PMLR, vol. 70, 2017, pp. 1321–1330. https://proceedings.mlr.press/v70/guo17a.html.

Downloads

Published

2025-06-11

Versions

Issue

Vol. 1 No. 6 (2025): June 2025

Section

Articles