11+ hour, 20+ min ago  (345+ words) training adapts a language model so its outputs match human values and preferences. The classic RLHF pipeline " collect human rankings, train a reward model, run PPO " is expensive and notoriously unstable. Ludwig provides a family of modern preference learning trainers…...