Documenting Trofinetide Treatment Phenotypes in Rett Syndrome with a Dual-LLM Consensus Pipeline

The 30-second take

• What: A dual-LLM consensus pipeline extracts 18 treatment-response phenotypes from 26,763 notes on 108 Texas Children’s Hospital patients with sustained trofinetide exposure; a frozen diarrhea-tuned prompt transferred to 17 other phenotypes, beating regex by 47 points on diarrhea and matching experts on 92 of 102 spot checks.
• Abundance angle: today, real-world monitoring of a newly approved rare-disease drug is scarce chart-review labor. Config-only LLM consensus is a step toward cheaper default phenotype extraction if presence labels remain this reliable (mid-horizon: clinic validation and regulation).
• Who should care: Rare-disease informatics groups, Rett clinics on trofinetide, and RWE teams tired of keyword charts that cannot tell severe active diarrhea from a mention.

What the paper actually did

Trofinetide, described as approved in 2023 as the first Rett syndrome treatment, still lacks scalable real-world monitoring because chart review is slow. The authors built a dual-LLM consensus pipeline to extract 18 treatment-response phenotypes — 4 adverse effects, 13 Rett clinical features, and medication refusal — from clinical notes.

They analyzed 26,763 notes from 108 patients with confirmed sustained trofinetide exposure at Texas Children’s Hospital. Notes are keyword-filtered, classified by two LLMs that must agree, then aggregated into a patient-by-timepoint matrix. Architecture and prompt were chosen on a diarrhea development set, then frozen and applied to the other 17 phenotypes by swapping only the phenotype definition (an out-of-sample test). Prompt comparison uses 183 expert-labeled cells.

A prompt requiring diarrhea to be active and severe performed best (84.2 ± 0.0% accuracy; F1 = 0.674), 47 percentage points above regular-expression matching (p < 0.0001). Experts judging six pipeline calls per phenotype (three positive, three negative) agreed with 90.2% (92/102). The 13 Rett features are manifestations of the disorder, not drug adverse effects; gross and fine motor impairment, language/communication, and stereotypy were documented present in 92–100% of patients at every visit. The pipeline records documented presence, not direction of change.

What makes this disruptive

The scarce capability is turning messy clinic prose into a phenotype matrix without a new labeled dataset per symptom. Freezing a diarrhea-tuned prompt and swapping definitions for 17 other phenotypes is a strong “configuration, not retraining” claim if the 90.2% expert spot-check holds.

Beating regex by 47 points on a prompt that demands active and severe diarrhea shows why keywords fail: notes mention diarrhea constantly without meaning a current severe event. Dual consensus is the reliability knob.

This documents presence at visits, explicitly not improvement or worsening — a limitation they are honest about, and one users will still try to over-interpret.

Why it matters (outside the lab)

Abundance lens: watching how a rare-disease therapy lands in real clinics is still artisanal chart review. If dual-LLM consensus becomes default informatics, more of that monitoring can be ordinary operations rather than a grant-sized abstraction project.

Near-term: Rett programs can extract presence matrices with human audit. Medium-term: regulators, site shift, and phenotype drift decide whether this is a default RWE tool. No claim that trofinetide “works” from this paper’s abstract.

Validation, not a calendar, is the gate.

Limitations & open questions

Single health system (Texas Children’s), 108 patients. Expert review is six calls per phenotype (102 total), not a full gold chart. Diarrhea prompt was tuned on a development set; transfer to 17 phenotypes is encouraging but not a full per-phenotype trial in the abstract.

Presence ≠ change; 92–100% documentation of core Rett features every visit may be ceiling effects, not treatment response. Preprint ≠ EHR product. Abundance is not automatic: better extraction does not expand drug access on a date.