Empirical studies on neurovascular diseases demand valid and reliable diagnostic assessments. Using 13 standardized case summaries on cerebral ischemic events, interrater agreement was tested in this study. Eighty-one diagnosticians gave their diagnostic assessments on the basis of three different classification systems (ICD-9, draft of ICD-10, German classification of neurological diseases; CSGN). As a result, the interrater reliabilities showed marked differences between the three systems, the chance-corrected agreements ĸdiffered from 0.38 (ICD-9) to 0.72 (ICD-10). The data analysis demonstrated that the neurovascular diseases were only partly covered by the different diagnostic systems.