Data-Driven Decision-Making Algorithms

Last updated on Jun 25, 2024
Generated by Microsoft Designer

A main research direction for the AIR-DREAM Lab is to develop high-performance, robust, generalizable, and real-world deployable data-driven decision-making algorithms. We are specifically interested in offline policy learning methods, such as offline reinforcement learning (RL), offline imitation learning (IL), and offline planning, which enable a simulation-free and low-cost solution to address many real-world problems.

Our current research focus include:

  • Sample-efficient / high-generalization offline RL / IL / planning algorithms
  • Foundation models for decision-making
  • Safe offline RL algorithms
  • Hybrid RL that combines offline and online policy learning
  • Offline policy learning under imperfect reward
  • Feedback-efficient RLHF
Algorithms
Xianyuan Zhan
Xianyuan Zhan
Faculty Member

Publications

Are Expressive Models Truly Necessary for Offline RL?
Among various branches of offline reinforcement learning (RL) methods, goal-conditioned supervised learning (GCSL) has gained …
PDF Cite Code Project Project
Are Expressive Models Truly Necessary for Offline RL?
Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learning
One important property of DIstribution Correction Estimation (DICE) methods is that the solution is the optimal stationary distribution …
PDF Cite Code Project Website
Diffusion-DICE: In-Sample Diffusion Guidance for Offline Reinforcement Learning
Instruction-Guided Visual Masking
Instruction following is crucial in contemporary LLM. However, when extended to multimodal setting, it often suffers from misalignment …
PDF Cite Code Project Project Website
Instruction-Guided Visual Masking
Bidirectional-Reachable Hierarchical Reinforcement Learning with Mutually Responsive Policies
Hierarchical reinforcement learning (HRL) addresses complex long-horizon tasks by skillfully decomposing them into subgoals. Therefore, …
PDF Cite Project
OMPO: A Unified Framework for RL under Policy and Dynamics Shifts
Training reinforcement learning policies using environment interaction data collected from varying policies or dynamics presents a …
PDF Cite Project
Offline-Boosted Actor-Critic: Adaptively Blending Optimal Historical Behaviors in Deep Off-Policy RL
Off-policy reinforcement learning (RL) has achieved notable success in tackling many complex real-world tasks, by leveraging previously …
PDF Cite Project
DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning
Multimodal pretraining has emerged as an effective strategy for the trinity of goals of representation learning in autonomous robots: …
PDF Cite Code Project Project Website
Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Critic
Learning high-quality Q-value functions plays a key role in the success of many modern off-policy deep reinforcement learning (RL) …
PDF Cite Code Project
A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents
The burgeoning fields of robot learning and embodied AI have triggered an increasing demand for large quantities of data. However, …
PDF Cite Code Project
A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents
Query-Policy Misalignment in Preference-Based Reinforcement Learning
Preference-based reinforcement learning (PbRL) provides a natural way to align RL agents’ behavior with human desired outcomes, but is …
PDF Cite Project
Revealing the Mystery of Distribution Correction Estimation via Orthogonal-gradient Update
In this study, we investigate the DIstribution Correction Estimation (DICE) methods, an important line of work in offline reinforcement …
PDF Cite Project
Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model
Safe offline reinforcement learning is a promising way to bypass risky online interactions towards safe policy learning. Most existing …
PDF Cite Code Project Website
Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model
FlexSSL : A Generic and Efficient Framework for Semi-Supervised Learning
Semi-supervised learning holds great promise for many real-world applications, due to its ability to leverage both unlabeled and …
PDF Cite Project
A Fully Data-Driven Approach for Realistic Traffic Signal Control Using Offline Reinforcement Learning
The optimization of traffic signal control (TSC) is critical for an efficient transportation system. In recent years, reinforcement …
PDF Cite Project Project
H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps
Solving real-world complex tasks using reinforcement learning (RL) without high-fidelity simulation environments or large amounts of …
PDF Cite Project Project
Look Beneath the Surface: Exploiting Fundamental Symmetry for Sample-Efficient Offline RL
Offline reinforcement learning (RL) offers an appealing approach to real-world tasks by learning policies from pre-collected datasets …
PDF Cite Project Project
Offline Multi-Agent Reinforcement Learning with Implicit Global-to-Local Value Regularization
Offline reinforcement learning (RL) has received considerable attention in recent years due to its attractive capability of learning …
PDF Cite Project
PROTO: Iterative Policy Regularized Offline-to-Online Reinforcement Learning
Offline-to-online reinforcement learning (RL), by combining the benefits of offline pretraining and online finetuning, promises …
PDF Cite Project
Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization
Based on the IVR framework, we further propose two practical algorithms, Sparse Q-learning (SQL) and Exponential Q-learning (EQL), which adopt the same value regularization used in existing works, but in a complete in-sample manner.
PDF Cite Code Project
Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularization
When Data Geometry Meets Deep Function: Generalizing Offline Reinforcement Learning
DOGE marries dataset geometry with deep function approximators in offline RL, and enables exploitation in generalizable OOD areas rather than strictly constraining policy within data distribution.
PDF Cite Code Project
When Data Geometry Meets Deep Function: Generalizing Offline Reinforcement Learning
An Efficient Multi-Agent Optimization Approach for Coordinated Massive MIMO Beamforming
Beamforming plays an important role in 5G Massive Multiple-Input Multiple-Output (MMIMO) communications. Optimizing beamforming …
PDF Cite Project Project
Mind the Gap: Offline Policy Optimization for Imperfect Rewards
This paper proposes an offline policy optimization approach for imperfect rewards. Abstract: Reward function is essential in …
PDF Cite Project
Offline Multi-Agent Reinforcement Learning with Coupled Value Factorization
Offline reinforcement learning (RL) that learns policies from offline datasets without environment interaction has received …
PDF Cite Project
When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning
H2O introduces a dynamics-aware policy evaluation scheme, which adaptively penalizes the Q function learning on simulated state-action pairs with large dynamics gaps, while also simultaneously allowing learning from a fixed real-world dataset.
PDF Cite Code Project Project
When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learning
A Policy-Guided Imitation Approach for Offline Reinforcement Learning
Offline reinforcement learning (RL) methods can generally be categorized into two types: RL-based and Imitation-based. RL-based methods …
PDF Cite Code Project
Discriminator-Guided Model-Based Offline Imitation Learning
Offline imitation learning (IL) is a powerful method to solve decision-making problems from expert demonstrations without reward …
PDF Cite Project Project
Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations
We study the problem of offline Imitation Learning (IL) where an agent aims to learn an optimal expert behavior policy without …
PDF Cite Code Project
Model-Based Offline Planning with Trajectory Pruning
Offline reinforcement learning (RL) enables learning policies using pre-collected datasets without environment interaction, which …
PDF Cite Code Project
Constraints Penalized Q-Learning for Safe Offline Reinforcement Learning
We study the problem of safe offline reinforcement learning (RL), the goal is to learn a policy that maximizes long-term reward while …
PDF Cite Project
DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning
Optimizing the combustion efficiency of a thermal power generating unit (TPGU) is a highly challenging and critical task in the energy …
PDF Cite Project Project