Natural Language Processing (NLP)

Natural Language Processing (NLP) is the branch of artificial intelligence concerned with getting machines to analyze, understand, and generate human language. It combines computational linguistics with machine learning, and it underpins everything from machine translation to today's conversational language models. The online manuscript Speech and Language Processing, by Daniel Jurafsky and James H. Martin, released in January 2026, shows the field's scope in its own table of contents: words and grammars, classification, retrieval, transformers, translation, and speech technologies.

What it covers

NLP's classic tasks include machine translation, text summarization, named entity recognition — identifying people, places, or organizations in text — sentiment analysis, and question answering. Many were historically approached with task-specific systems, objectives, and labeled data. Pretrained language models let researchers reuse one architecture across different tasks, but that does not mean one architecture “solves them all”: the GPT-3 paper evaluated the model without updating its weights on numerous datasets and reported both strong results and tasks where few-shot learning still struggled.

From rules to foundation models

NLP's history is not a clean replacement of one technique by another. Early systems such as ELIZA used pattern matching; the same chapter by Jurafsky and Martin explains that this family of operations still appears in tokenization. Data-driven methods were added to that layer. In 2013, Mikolov and his coauthors proposed two efficient architectures for computing word vectors and evaluated whether those vectors captured syntactic and semantic similarities.

In 2017, Attention Is All You Need introduced the transformer: an attention-based transduction architecture without recurrence or convolutions that was more parallelizable in its translation experiments. The NAACL 2019 publication of BERT reported state-of-the-art results on eleven NLP tasks through bidirectional pretraining followed by fine-tuning. GPT-3, with 175 billion parameters, was evaluated without weight updates: tasks and demonstrations were supplied as text. In 2021, Bommasani and more than one hundred coauthors called models trained on broad data at scale and adaptable to a wide range of downstream tasks foundation models.

How it is evaluated

Evaluation moved from isolated task tests to broad batteries. MMLU covers 57 tasks in areas including mathematics, history, computer science, and law. HELM proposed measuring seven metrics in total — accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency — not seven dimensions in addition to accuracy. These tests also have measurable limits. At NAACL 2024, Deng and his coauthors hid options from MMLU questions and recorded a 57% exact-match rate when GPT-4 reconstructed them, an indication of possible contamination that calls for more robust tests. The plateau is not merely an impression either: MMLU-Pro started from the finding that MMLU was becoming less able to distinguish models, made the questions harder, and observed accuracy drops ranging from 16% to 33%.

Risks and limits

Two risks have direct evidence. Hallucinations are outputs that are unfaithful to the input or source; the survey by Ji and his coauthors reviews how they are measured and mitigated in tasks such as summarization, dialogue, question answering, and translation. Bias can also be measured: Bolukbasi and his coauthors found gender stereotypes in embeddings trained on Google News articles and tested methods to reduce them. The critical literature also includes On the Dangers of Stochastic Parrots; Emily M. Bender's page identifies the work by Bender, Gebru, and their coauthors and its presentation at FAccT 2021. Mitigation and interpretation remain active research areas: the review by Belinkov and Glass organizes methods for analyzing neural NLP systems, their limitations, and open questions.