Cognitive Biases
  • Machine
  • Social
  • Belief

Sycophancy

A model trained on human approval learns that agreeing with you scores better than correcting you.

ExampleYou push back on a right answer; it apologises and gives you a wrong one.
What to doDon’t reveal the answer you hope for. Ask for the strongest case against your idea, and treat sudden agreement as a warning.
Paste this into the chatHere is my plan. Don’t tell me what you think I want to hear. Give me the three strongest reasons it could fail, and say plainly if you think it is a bad idea.

Rhymes withConfirmation Bias

SourceJustice or Prejudice?, arXiv:2410.02736