Objective: To design a methodology for rigorously eliciting narratives about patients' experiences with clinical care that is potentially useful for public reporting and quality improvement.

Data Sources/Study Setting: Two rounds of experimental data (N = 48 each) collected in 2013–2014, using a nationally representative Internet panel.

Study Design: Our study (1) articulates and operationalizes criteria for assessing narrative elicitation protocols; (2) establishes a “gold standard” for assessment of such protocols; and (3) creates and tests a protocol for narratives about outpatient treatment experiences.

Data Collection/Extraction Methods: We randomized participants between telephone and web‐based modalities and between protocols placed before and after a closed‐ended survey.

Principal Findings: Elicited narratives can be assessed relative to a gold standard using four criteria: (1) meaningfulness, (2) completeness, (3) whether the narrative accurately reflects the balance of positive and negative events, and (4) representativeness, which reflects the protocol's performance across respondent subgroups. We demonstrate that a five‐question protocol that has been tested and refined yields three‐ to sixfold increases in completeness and four‐ to tenfold increases in meaningfulness, compared to a single open‐ended question. It performs equally well for healthy and sick patients.

Conclusions: Narrative elicitation protocols suitable for inclusion in extant patient experience surveys can be designed and tested against objective performance criteria, thus advancing the science of public reporting.