Aphasia can be caused by a stroke, brain injury or dementia. It is defined as a language disorder that impacts the domains of speaking, understanding, reading and writing. Given the impact on quality of life and conversation, there is a particular need for a valid, brief, practical and reliable measure of natural conversation. The Basic Outcome Measure Protocol for Aphasia (BOMPA) was specifically designed to bring together three easy-to-administer evaluations including a self-rating of quality of life by the person with aphasia, as well as a speech and language therapist’s (SLTs) rating of aphasia impairment, and ability to participate in a scripted conversation. This article reports on the development of BOMPA, as well as on a reliability study evaluating inter-rater reliability of SLT ratings of aphasia severity and ability to participate in conversation. Twenty SLTs viewed a set of 12 specially created and randomly ordered videos of BOMPA administration to six individuals with aphasia in both ‘without support’ and ‘with support’ conditions. Inter-rater reliability was determined to be moderate-to-strong. This first evaluation of BOMPA is limited by the relatively small number of participants. But BOMPA may be a worthwhile addition to the measures currently available to SLTs.