Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits
Date:
This paper present SECOND THOUGHTS, a new learning paradigm that enables language models (LMs) to re-align with human values.
By modeling the chain-of-edits between value-unaligned and value-aligned text, with LM fine-tuning and additional refinement through reinforcement learning.
The generated editing steps also offer better interpretability and ease for interactive error correction.
Leave a Comment