Data-Driven Decision-Making Algorithms

Last updated on Jul 17, 2025
Generated by Microsoft Designer

A main research direction for the AIR-DREAM Lab is to develop high-performance, robust, generalizable, and real-world deployable data-driven decision-making algorithms. We are specifically interested in offline policy learning methods, such as offline reinforcement learning (RL), offline imitation learning (IL), and offline planning, which enable a simulation-free and low-cost solution to address many real-world problems.

Our current research focus includes:

Sample-efficient / high-generalization offline RL / IL / planning algorithms

Foundation models for decision-making

Safe offline RL algorithms

Hybrid RL that combines offline and online policy learning

Offline policy learning under imperfect reward

Feedback-efficient RLHF

Algorithms
Xianyuan Zhan
Xianyuan Zhan
Faculty Member

Publications

Efficient Robotic Policy Learning via Latent Space Backward Planning
Current robotic planning methods often rely on predicting multi-frame images with full pixel details. While this fine-grained approach …
PDF Cite Code Project Project Website
Efficient Robotic Policy Learning via Latent Space Backward Planning
Universal Actions for Enhanced Embodied Foundation Models
Training on diverse, internet-scale data is a key factor in the success of recent large foundation models. Yet, using the same recipe …
PDF Cite Code Project Project Website
Universal Actions for Enhanced Embodied Foundation Models
H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps
Solving real-world complex tasks using reinforcement learning (RL) without high-fidelity simulation environments or large amounts of …
PDF Cite Code Project Project Website
Robo-MUTUAL: Robotic Multimodal Task Specification via Unimodal Learning
Multimodal task specification is essential for enhanced robotic performance, where Cross-modality Alignment enables the robot to …
PDF Cite Code Project Project Website
Data Center Cooling System Optimization Using Offline Reinforcement Learning
The recent advances in information technology and artificial intelligence have fueled a rapid expansion of the data center (DC) …
PDF Cite Project Project Website
Data Center Cooling System Optimization Using Offline Reinforcement Learning
Diffusion-Based Planning for Autonomous Driving with Flexible Guidance
Achieving human-like driving behaviors in complex open-world environments is a critical challenge in autonomous driving. Contemporary …
PDF Cite Code Project Project Website
Diffusion-Based Planning for Autonomous Driving with Flexible Guidance
Skill Expansion and Composition in Parameter Space
Humans excel at reusing prior knowledge to address new challenges and developing skills while solving problems. This paradigm becomes …
PDF Cite Code Project Project Website
Skill Expansion and Composition in Parameter Space
Are Expressive Models Truly Necessary for Offline RL?
Among various branches of offline reinforcement learning (RL) methods, goal-conditioned supervised learning (GCSL) has gained …
PDF Cite Code Project Project
Are Expressive Models Truly Necessary for Offline RL?
Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learning
One important property of DIstribution Correction Estimation (DICE) methods is that the solution is the optimal stationary distribution …
PDF Cite Code Project Website
Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learning
Instruction-Guided Visual Masking
Instruction following is crucial in contemporary LLM. However, when extended to multimodal setting, it often suffers from misalignment …
PDF Cite Code Project Project Website
Instruction-Guided Visual Masking
Bidirectional-Reachable Hierarchical Reinforcement Learning with Mutually Responsive Policies
Hierarchical reinforcement learning (HRL) addresses complex long-horizon tasks by skillfully decomposing them into subgoals. Therefore, …
PDF Cite Code Project
OMPO: A Unified Framework for RL under Policy and Dynamics Shifts
Training reinforcement learning policies using environment interaction data collected from varying policies or dynamics presents a …
PDF Cite Project
Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL
Off-policy reinforcement learning (RL) has achieved notable success in tackling many complex real-world tasks, by leveraging previously …
PDF Cite Project
DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning
Multimodal pretraining has emerged as an effective strategy for the trinity of goals of representation learning in autonomous robots: …
PDF Cite Code Project Project Website
Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic
Learning high-quality Q-value functions plays a key role in the success of many modern off-policy deep reinforcement learning (RL) …
PDF Cite Project Website
A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents
The burgeoning fields of robot learning and embodied AI have triggered an increasing demand for large quantities of data. However, …
PDF Cite Code Project
A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents
Query-Policy Misalignment in Preference-Based Reinforcement Learning
Preference-based reinforcement learning (PbRL) provides a natural way to align RL agents’ behavior with human desired outcomes, but is …
PDF Cite Project
Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update
In this study, we investigate the DIstribution Correction Estimation (DICE) methods, an important line of work in offline reinforcement …
PDF Cite Project
Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model
Safe offline reinforcement learning is a promising way to bypass risky online interactions towards safe policy learning. Most existing …
PDF Cite Code Project Website
Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model
FlexSSL : A Generic and Efficient Framework for Semi-Supervised Learning
Semi-supervised learning holds great promise for many real-world applications, due to its ability to leverage both unlabeled and …
PDF Cite Project
A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning
The optimization of traffic signal control (TSC) is critical for an efficient transportation system. In recent years, reinforcement …
PDF Cite Project Project
Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL
Offline reinforcement learning (RL) offers an appealing approach to real-world tasks by learning policies from pre-collected datasets …
PDF Cite Project Project
Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularization
Offline reinforcement learning (RL) has received considerable attention in recent years due to its attractive capability of learning …
PDF Cite Project
PROTO: Iterative Policy Regularized Offline-to-Online Reinforcement Learning
Offline-to-online reinforcement learning (RL), by combining the benefits of offline pretraining and online finetuning, promises …
PDF Cite Project
Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization
Based on the IVR framework, we further propose two practical algorithms, Sparse Q-learning (SQL) and Exponential Q-learning (EQL), which adopt the same value regularization used in existing works, but in a complete in-sample manner.
PDF Cite Code Project
Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization
When Data Geometry Meets Deep Function: Generalizing Offline Reinforcement Learning
DOGE marries dataset geometry with deep function approximators in offline RL, and enables exploitation in generalizable OOD areas rather than strictly constraining policy within data distribution.
PDF Cite Code Project
When Data Geometry Meets Deep Function: Generalizing Offline Reinforcement Learning
An Efficient Multi-Agent Optimization Approach for Coordinated Massive MIMO Beamforming
Beamforming plays an important role in 5G Massive Multiple-Input Multiple-Output (MMIMO) communications. Optimizing beamforming …
PDF Cite Project Project
Mind the Gap: Offline Policy Optimization for Imperfect Rewards
This paper proposes an offline policy optimization approach for imperfect rewards. Abstract: Reward function is essential in …
PDF Cite Project
Offline Multi-Agent Reinforcement Learning with Coupled Value Factorization
Offline reinforcement learning (RL) that learns policies from offline datasets without environment interaction has received …
PDF Cite Project
When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning
H2O introduces a dynamics-aware policy evaluation scheme, which adaptively penalizes the Q function learning on simulated state-action pairs with large dynamics gaps, while also simultaneously allowing learning from a fixed real-world dataset.
PDF Cite Code Project Project
When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning
A Policy-Guided Imitation Approach for Offline Reinforcement Learning
Offline reinforcement learning (RL) methods can generally be categorized into two types: RL-based and Imitation-based. RL-based methods …
PDF Cite Code Project
Discriminator-Guided Model-Based Offline Imitation Learning
Offline imitation learning (IL) is a powerful method to solve decision-making problems from expert demonstrations without reward …
PDF Cite Project Project
Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations
We study the problem of offline Imitation Learning (IL) where an agent aims to learn an optimal expert behavior policy without …
PDF Cite Code Project
Model-Based Offline Planning with Trajectory Pruning
Offline reinforcement learning (RL) enables learning policies using pre-collected datasets without environment interaction, which …
PDF Cite Code Project
Constraints Penalized Q-Learning for Safe Offline Reinforcement Learning
We study the problem of safe offline reinforcement learning (RL), the goal is to learn a policy that maximizes long-term reward while …
PDF Cite Project
DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning
Optimizing the combustion efficiency of a thermal power generating unit (TPGU) is a highly challenging and critical task in the energy …
PDF Cite Project Project