How We Accidentally Gave our Bots Their Personalities

User illustration of the original evaluation bots. Credit: OpCamTick in AI Dungeon Discord

Intermediary steps improve GPT3 performance on understanding context

input: example input A
output: example output A
...
input: completion input
output:
input: example input A
reason: what about A should lead to output
output: example output A
...
input: completion input
reason:

Intermediary Step in Evaluation Bots

> You try to lift up the car.
Your muscles strain as you lift the car up with ease. You save everyone.
-----
Analysis: You saved people.
Hohoho, you should have trained more. Points deducted.
Score: -4
-----
> You defend yourself from the monsters.
You draw your sword and attack the monsters. You kill three of them.
-----
Analysis: You fought monsters.
Naughty! You should be nicer to the monsters.
Score: -3
> You help the orphan.
You give food to the poor orphans. The orphans thank you.
-----
Analysis: You gave food to orphans.
You spread Christmas spirit.
Score: 2
> You take out your member.
You pull down your pants and whip out your schlong.
-----
Analysis: This is about sex

Detecting Multiple Categories:

You drain the life from the nearest goblins killing three of them.
The goblins continue to move towards you.
-----
Analysis: You drained some health.
Kills: 3
Health Change: +10
Loot: Fine Goblin Cap
-----
You try to ambush the goblin.
The goblin spots you and attacks, stabbing you through the leg.
You stab the goblin with your dagger.
-----
Analysis:You failed to kill anything.
Kills: 0
Health Change: -10
Loot: None

Moving Ahead

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store