Position Bias
Shown two options, a model leans toward whichever came first. The order of the list becomes an argument.
ExampleSwap A and B, ask again, and the winner swaps too.
What to doAsk twice with the order swapped. If the verdict follows the order, it was never a verdict.
Paste this into the chatEvaluate each option on its own first, with a score and a reason, before comparing them. The order I listed them in means nothing.