![]() Probably, human utterances represent greater perplexity at the level of the phrase, to give dialogue greater surprise and variety. Meena optimizes how to drop in the right word, but its phrases overall tend to be way too obvious, leading to a kind of bland, general quality about the way it engages. One objective might be to optimize for what may be an utterance-level perplexity. It's possible some future objectives for Meena could create more interesting exchanges. The most interesting parts of human exchange are not information exchange, in general. They have agendas, they have goals, they use rhetorical devices. Humans don't tend to speak only as an exchange of information or as wordplay. ![]() But it also quickly becomes superficial and tiresome. Wordplay is interesting up to a point, and it feels like it reflects intelligence, in some fashion. Patterns of language in the Meena form are highly associative at a word level, which makes Meena's best examples those of wordplay. Meena is recreating a distribution of language that is startlingly accurate, but also merely a recreation of information, which is boring. In a sense, that's quite amazing, but that's an objective that is also a problem. The authors find low perplexity, the X axis, corresponds with a human measure of dialogue, "sensibleness and specificity average," or "SSA," on the Y axis. Google's Meena chatbot scores low on "perplexity," which is good, meaning it has less of a hard time finding the right word. It is simply ingesting a giant gob of social media and reflecting it, but with very specific choices at each word. Nor is it subject to constraints that would shape its perplexity computation thematically. Unlike prior efforts at chatbots, Meena isn't being given help to narrow the perplexity through cues about domain of speech. That's an astounding accomplishment for which Adiwardana and colleagues can be very proud. Meena's best performance achieves a state of the art perplexity of 10.2, meaning it has narrowed all the tens of thousands of words in the English language that could be in any one position in a sentence to just 10 likely words. The objective function, in this case, was what's known as "perplexity." Perplexity represents how many words the Meena program finds probable as the next word in any sentence. Adiwardana et al.Īnd those shortcomings in Meena point to the profound challenge at the heart of Meena, even as good as it is. Meena's finest moments are when its word association turns into humorous wordplay, such as in this exchange with a human. Meena's expressions tend to cooperate with the theme, which may be desirable in a sense but also leads to interactions that are trivial and dull. There's nothing especially surprising or interesting in the above exchange between Meena and the human. That's something that happens in an encounter where two humans have a subtext, such as to impress or challenge one another, as is the case between actors Billy Crystal and Meg Ryan in the scene. But it shows qualities of human interaction that highlight what is missing in all of Meena's interactions.įor one thing, the responses don't simply continue the last utterance, they can take the discussion in a surprising next direction. ![]() The example of the dialogue from When Harry Met Sally, by a master writer Nora Ephron, is not natural human speech. Sadly, the humans were not asked by Adiwardana and colleagues to rate conversations for "interestingness," because this and other exchanges in the sample are incredibly dull, like the worst text message exchange you've ever peeked at.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |