Additional results for HPS Reward, in-distribution prompts

"a horse washing the dishes"
VADER (Ours)
ModelScope
DDPO
DPO
"a bear riding a bike"
VADER (Ours)
ModelScope
DDPO
DPO
"a goat playing chess"
VADER (Ours)
ModelScope
DDPO
DPO
"a kangaroo playing chess"
VADER (Ours)
ModelScope
DDPO
DPO
"a duck riding a bike"
VADER (Ours)
ModelScope
DDPO
DPO
"a gorilla riding a bike"
VADER (Ours)
ModelScope
DDPO
DPO
"a lion riding a bike"
VADER (Ours)
ModelScope
DDPO
DPO
"a goat washing the dishes"
VADER (Ours)
ModelScope
DDPO
DPO
"a cat riding a bike"
VADER (Ours)
ModelScope
DDPO
DPO
"a kangaroo washing the dishes"
VADER (Ours)
ModelScope
DDPO
DPO
"a sheep playing chess"
VADER (Ours)
ModelScope
DDPO
DPO
"a duck washing the dishes"
VADER (Ours)
ModelScope
DDPO
DPO
"a gorilla playing chess"
VADER (Ours)
ModelScope
DDPO
DPO
"a pig washing the dishes"
VADER (Ours)
ModelScope
DDPO
DPO
"a lizard playing chess"
VADER (Ours)
ModelScope
DDPO
DPO