Additional results for HPS Reward, in-distribution prompts

"a bear playing chess"
VADER (Ours)
ModelScope
DDPO
DPO
"a goat washing the dishes"
VADER (Ours)
ModelScope
DDPO
DPO
"a lion playing chess"
VADER (Ours)
ModelScope
DDPO
DPO
"a horse washing the dishes"
VADER (Ours)
ModelScope
DDPO
DPO
"a monkey washing the dishes"
VADER (Ours)
ModelScope
DDPO
DPO
"a duck riding a bike"
VADER (Ours)
ModelScope
DDPO
DPO
"a tiger riding a bike"
VADER (Ours)
ModelScope
DDPO
DPO
"a deer playing chess"
VADER (Ours)
ModelScope
DDPO
DPO
"a racoon riding a bike"
VADER (Ours)
ModelScope
DDPO
DPO
"a bear washing the dishes"
VADER (Ours)
ModelScope
DDPO
DPO
"a chicken riding a bike"
VADER (Ours)
ModelScope
DDPO
DPO
"a gorilla playing chess"
VADER (Ours)
ModelScope
DDPO
DPO
"a gorilla riding a bike"
VADER (Ours)
ModelScope
DDPO
DPO
"a camel playing chess"
VADER (Ours)
ModelScope
DDPO
DPO
"a dog washing the dishes"
VADER (Ours)
ModelScope
DDPO
DPO