Echoes in GenAI generations

  • Nebojsa Jojic, Microsoft

In our recent PNAS paper we demonstrate that large language models produce little variation in generated narratives. Compared to those generations, a human-written narrative is usually Sui Generis, i.e. one of a kind. Or as we’d say in ML and statistics, human writing is in the tails of the distribution of the content LLMs generate. We introduced the Sui Generis (SG) score which can be used to evaluate distinctiveness of written text, whether it was written by a human or by a machine. SG scores may find its uses both in model improvement and in assistive tools (e.g. helping you to sound less like a GPT). As LLMs exhibit increasingly useful abilities to compare and refine ideas, and occasionally add to them, good writing in the future will likely still require human-led, possibly collaborative effort, but greatly assisted by AI.

Read our paper

Watch Next