AI Governance Engineer
GitHub

Reinforcement learning from human feedback (RLHF)

By Updated

A way to align a pre-trained model: supervised fine-tuning on human demonstrations, then reinforcement learning against a reward model trained on human rankings of outputs 1. The raters' instructions and the reward model become governed artefacts, because they shape what the model refuses and prefers.

Developed in
ch. 11, By learning paradigm
Chapters
ch. 11, AI Defined
Contrast with
Fine-tuning
Source
1 numbered reference, listed below

Where it is used

The term is not used under this name in running prose; the sections listed under "Developed in" treat it.

Reinforcement learning from human feedback (RLHF): related terms and patternsThe term at the centre; related terms on solid edges, the terms it is contrasted with on dashed edges, and the patterns that use it most in the colour of their layer.RelatedContrastReinforcement learning from humanfeedbackRelated terms (6)Machine learning · RelatedMachine learningReinforcement learning · RelatedReinforcement learningReward hacking · RelatedReward hackingSelf-supervised learning · RelatedSelf-supervised learningSupervised learning · RelatedSupervised learningUnsupervised learning · RelatedUnsupervised learningContrast with (1)Fine-tuning · ContrastFine-tuning
Reinforcement learning from human feedback (RLHF): related terms and patternsThe term at the centre; related terms on solid edges, the terms it is contrasted with on dashed edges, and the patterns that use it most in the colour of their layer.RelatedContrastRelated terms (6)Machine learning · RelatedMachine learningReinforcement learning · RelatedReinforcement learningReward hacking · RelatedReward hackingSelf-supervised learning · RelatedSelf-supervised learningSupervised learning · RelatedSupervised learningUnsupervised learning · RelatedUnsupervised learningContrast with (1)Fine-tuning · ContrastFine-tuningReinforcementlearning fromhuman feedback
The term's neighbourhood Terms related to this one on solid edges, terms it is often confused with on dashed edges, and the patterns that use it most in the colour of their layer. Follow a dashed edge to check the distinction before you use either term.
Neighbourhood as a table
Neighbourhood of Reinforcement learning from human feedback (RLHF)
FamilyItemRelation
Related termsMachine learningRelated
Related termsReinforcement learningRelated
Related termsReward hackingRelated
Related termsSelf-supervised learningRelated
Related termsSupervised learningRelated
Related termsUnsupervised learningRelated
Contrast withFine-tuningContrast
  • Machine learning

    The branch of AI in which a system improves at a task by learning patterns from data rather than by following rules people wrote.

  • Reinforcement learning

    Learning to maximise a reward signal through trial and feedback.

  • Reward hacking

    A system finding an unintended way to maximise its reward or objective without doing what its designers meant.

  • Self-supervised learning

    Learning by predicting parts of the input itself, such as the next token, over large corpora; the AI Act's definition of a general-purpose model names…

  • Supervised learning

    Learning from labelled examples.

  • Unsupervised learning

    Learning structure (clusters, anomalies) from data without labels.

Sources

  1. [1] "Training language models to follow instructions with human feedback" (Ouyang et al.; supervised fine-tuning on demonstrations, then reinforcement learning from human feedback on ranked outputs; arXiv 2203.02155). arXiv. 2022-03-04. https://arxiv.org/abs/2203.02155 (verified: primary)

Definitions of legal terms paraphrase the cited text, which governs. Dated statements are as of .

Cite this term

García Aibar, J. (2026). Reinforcement learning from human feedback (RLHF). In AI Governance Engineering: The Thesis & Body of Knowledge (v0.5.0), Glossary. https://doi.org/10.5281/zenodo.22956197. https://aigovernanceengineer.com/glossary/reinforcement-learning-from-human-feedback-rlhf. CC BY 4.0

BibTeX

@misc{aige2026reinforcementlearningfromhumanfeedbackrlhf,
  author  = {Jorge García Aibar},
  title   = {{Reinforcement learning from human feedback (RLHF)}},
  note    = {Glossary, AI Governance Engineering: The Thesis \& Body of Knowledge, version 0.5.0},
  year    = {2026},
  doi     = {10.5281/zenodo.22956197},
  url     = {https://aigovernanceengineer.com/glossary/reinforcement-learning-from-human-feedback-rlhf}
}