Artificial Intelligence Value Alignment via Inverse Reinforcement Learning

Abstract

Value alignment, one of the Artificial Intelligence (AI) Alignment problems, pertains to ensuring that AI systems adhere to human values. These intricate problems lack definitive solutions, and significant research has been conducted to address it. Nevertheless, substantial progress is still required to effectively tackle the AI Alignment problems. Current trajectories in AI development, particularly in the realm of Deep Learning and RLHF, pose significant existential risks due to potential misalignments in AI objectives and human values. In the present study, our objective is to address the AI value alignment problem through the utilization of Inverse Reinforcement Learning (IRL). The central idea revolves around employing the IRL framework to acquire a reward function from an expert who exhibits behaviour consistent with human values. Subsequently, the AI system will mimic the expert’s actions, thereby aligning its behaviour with human values in a verifiable way.