Text Generation

Text generation, or natural language generation, is the task of automatically producing text in human language from an input, which may be a prompt, structured data or another text. It is the counterpart of language understanding: while the latter starts from a text to extract its meaning, generation starts from an intention or a context and decides how to express it in words.

How it works today

Today's systems are based on the Transformer architecture and generate text autoregressively: they predict the next token—a word or a fragment—one at a time, conditioned on everything before. At each step, the model produces a probability distribution over the next token, and one must decide which to choose. Strategies range from greedy decoding (always taking the most probable) to sampling methods that control diversity: temperature, which flattens or concentrates the distribution, and top-k and top-p (nucleus) sampling, which restrict the choice to the most probable candidates.

What it is for

Text generation is behind summarization, translation, question answering, chatbots, assisted writing, code generation and the drafting of reports from data.

How it is evaluated, and its risks

It is measured with automatic metrics such as BLEU, ROUGE or perplexity, though all have limits—they correlate poorly with human judgment and do not capture coherence well—so human evaluation remains the reference. And the risks deserve naming: hallucinations (fluent but false or unsupported statements), biases inherited from the data, and the potential for disinformation at scale.