Introducing AI Dungeon Translate

Latitude Team
4 min readApr 1, 2021

--

AI Dungeon players can now translate their stories into emojis by just clicking a button.

Latitude is proud to announce AI Dungeon Translate, a novel text-to-emoji tool, is available for a limited time to all AI Dungeon players. We built it after conducting extensive analysis in which we found that while many people use emojis in their day-to-day life (e.g. texting, emails), there’s a distinct lack of emojis in long form text. We believe that AI Dungeon Translate can help bridge what we are now calling the ‘emoji gap’.

Update 6 April: We took down the emoji feature which ran from April 1-April 5. Expect to see the emojis back in the future! We saw a lot of cool emoji completions while it was up.

You can access AI Dungeon Translate in two ways: first, inside of any AI Dungeon story there’s a new icon in the actions bar which you can click for translation.

AI Dungeon game screenshot. The user’s action is, “You run away.” The game responded: “You pick up another vase and use it to smash in the face of yet another attacker. You run to the balcony and jump over the side. You must escape. (New line here). You are not a hero.” Below is the toolbar: Flag, World Info, Pin, Edit, Undo, Redo, Restore, Retry, Help, and at the end is a new tool called Spell with a smiley face icon. The Spell icon is circled to highlight the new AI Dungeon Translate feature.
Selecting AI Dungeon Translate from the Action Icons

A few seconds later, the newly translated version will pop-up in a modal!

A screenshot of an AI Dungeon game showing the results of using the new AI Dungeon Translate feature. It is a modal with the header titled: “Hmm…”, repeating the game’s response: “You pick up another vase and use it to smash in the face of yet another attacker. You run to the balcony and jump over the side. You must escape. (New line here). You are not a hero.” Below is a new line showing five emojis, “Translation: wilted flower, face with monocle, face screaming in fear, world map, person runni
AI Dungeon Translate Example Output

Secondly, in the `Play` and `Edit` modes you can click on any piece of text and translate it directly from there.

AI Dungeon game screenshot showing results of using the new AI Dungeon Translate feature. It is a modal with the header titled: “Hmm…”, repeating the game’s response: “You pick up another vase and use it to smash in the face of yet another attacker. You run to the balcony and jump over the side. You must escape. (New line here). You are not a hero.” Below is a new line showing five emojis, “Translation: wilted flower, face with monocle, face screaming in fear, world map, person running.”
AI Dungeon Translate from Edit

It can even translate into complex sentences relatively well. Yeet!

AI Dungeon game screenshot showing results of using the new AI Dungeon Translate feature. It is a modal repeating the game’s response: “You turn to head back in and then pause. Why should you? The king has a dozen men. He doesn’t need you. You could run and start a new life, maybe somewhere without war and violence. ” Below is a new line showing nine emojis, “Translation: top hat, prince, artist palette, person: beard, door, stop sign, globe showing Asia-Australia, triangular flag, prohibited.”
AI Dungeon Translate Example Output

How it Works

This feature builds on prior work by Andrew Mayne showing that OpenAI’s GPT-3 could be used to summarize movie plots. We realized how emojis can profoundly summarize content and decided to give it a glow up representing novel text with emojis as the clearly next logical step here.

Well, the problem with translating to emojis is it’s rather hard to figure out where each emoji should go. You can train the model by giving it a few examples of direct naïve translation which (works successfully in many instances like with English → French) to try for a few-shot attempt, but this doesn’t work so well for emojis as we found out from our first attempt:

[Few-shots] English: I’m late for work
Emoji: ⏰ 🏃 💻
English: I’m so full from lunch I might throw up
Emoji: 🤢 🍤
English: You turn to head back in and then pause. Why should you? The king has a dozen men. He doesn’t need you. You could run and start a new life, maybe somewhere without war and violence.Emoji: [Out] ‘💰 👣 🙏 💰 🛑’

Notice how the few-shots leaked into the output and it didn’t really know what to do? We can get a bit of a performance boost by including an intermediary step that re-words the initial content to what it’ll need for the final output.

[Fewshots]: English: I’m late for work
Meaning: An alarm clock for time and a person running with a briefcase
Emoji: ⏰ 🏃 💻
English: I’m so full from lunch I might throw up
Meaning: A person with a full belly and a puke
Emoji: 🤢 🍤
English: You turn to head back in and then pause. Why should you? The king has a dozen men. He doesn’t need you. You could run and start a new life, maybe somewhere without war and violence.
Meaning: [Out] ‘A king, a man, a dozen men, a head, a pause, a back, a new life
Emoji: 🤴 🤴 🤴 🤴 🤴’

While it got better by adding a king, it was a bit extra repeating it five times and still didn’t capture what the longer sentence meant. We realized it needed a little help to figure out how to map emojis to its meaning, so we had GPT-3 generate the emojis in the intermediary step as well.

[Fewshots] English: I’m late for work
Meaning: An alarm clock (⏰) and a person (🏃) running with a briefcase (💻)
Emoji: ⏰ 🏃 💻
English: I’m so full from lunch I might throw up
Meaning: A puke (🤢) and food (🍤)
Emoji: 🤢 🍤
English: You turn to head back in and then pause. Why should you? The king has a dozen men. He doesn’t need you. You could run and start a new life, maybe somewhere without war and violence.
Meaning: [Out]’A crown (🎩), a man (🧔), a dozen (🎨), a man (🧔), a stop (🛑), a new life (🌏), a war (🏹), a violence (🔪)
Emoji: 🎩 🧔 🎨 🛑 🌏 🏹 🔪’

Low key this is a neat example of how important the prompt is for GPT-3: if the prompt doesn’t focus the model to generate the information it needs in a format that it’s able to parse, it can lead to nonsensical outputs. With a bit of toying around we’re finally vibing with some really useful results! We think this methodology can be applied to multiple other GPT-3 generation tasks and we’re excited to see where it takes us!

And for our friends who prefer emoji-based communication: 👀📝😅🖥️🤔🥺👉👈👁️👄👁️🧠 🤖🥰🎉✨

--

--