The Civic Node

David Rozado

3 min read

Overview

David Rozado is an academic researcher (PLOS ONE 2024 publication; affiliated with the Manhattan Institute as commentator; previously published peer-reviewed work on word-embedding bias in PLOS ONE 2020) whose research program documents and quantifies political bias in AI systems, particularly large language models. Author of the most-cited contemporary peer-reviewed study on LLM political tilt: The political preferences of LLMs (PLOS ONE 19(7), July 2024). Operates the public DepolarizingGPT project (depolarizinggpt.org) demonstrating that political alignment can be moved via supervised fine-tuning.

Key Facts

  • PLOS ONE study (July 2024): The political preferences of LLMs — Rozado — administered 11 political-orientation tests to 24 conversational LLMs across ~12 organizations; found consistent left-of-center bias
  • 2,640 total test administrations (11 tests × 10 retakes × 24 models); 96,240 individual question/answer items
  • Methodology: Automated stance detection via gpt-3.5-turbo (93% human-rater agreement, κ=0.91 for conversational models)
  • Demonstrated SFT malleability: Created LeftWingGPT, RightWingGPT, DepolarizingGPT with 14K-34K text snippets each (7-17M tokens) — modest fine-tuning corpus capable of shifting political alignment significantly
  • Funding: Institute for Cultural Evolution (ICE think tank); Steve McIntosh consulted on DepolarizingGPT data — both disclosed
  • Earlier work: Wide range screening of algorithmic bias in word embedding models using large sentiment lexicons (PLOS ONE 2020); The Political Biases of ChatGPT (Social Sciences 2023); The Self-Perception and Political Biases of ChatGPT (Information 2024)
  • Has published commentary in Manhattan Institute outlets on AI political bias as policy concern

Newsletter Relevance

Rozado’s research is the single most-cited peer-reviewed source on LLM political bias — anyone arguing that contemporary AI systems share a political tilt cites his data. The empirical findings are independently verifiable and methodologically sound; his interpretation (LLMs as politically captured by progressive cultural elites) carries motivated framing from his broader research project.

For newsletter use, this is the source to cite for the existence and direction of LLM political bias. For interpretation, Rozado’s own framing should be contextualized — distinguishing data from frame is appropriate journalistic practice.

Connects to broader wiki themes: AI as political-information infrastructure; the parallel between social-media algorithm bias debates and emerging LLM alignment debates; the political project to constrain LLM outputs that mirrors the project to constrain social-media content.

Connections

Source Appearances

Open Questions

  • The hypothesis that ChatGPT-generated synthetic data has propagated original ChatGPT political bias across the broader LLM ecosystem — is this empirically testable beyond Rozado’s speculation?
  • Whether Rozado’s methodology generalizes beyond multiple-choice political tests to open-ended generation — the latter is closer to actual LLM use cases.
  • How the funding-conflict disclosure (ICE think tank with explicit “depolarizing” political project) interacts with the appearance of methodological neutrality — appropriate to disclose, but raises framing-vs-data questions.

Graph View

Backlinks