Dr. Batalden suggests that more information than was provided in our article is needed to understand how to improve QI interventions. We agree completely. A good evaluation study should first assess whether an intervention was successful (1). When there is no effect, a good study should provide information about how well the program was implemented, as well as information on potential moderating effects. Annals articles must be concise, and we could not include all of these details in our article. However, we did assess virtually all of the factors mentioned by Dr. Batalden. We conducted detailed assessments of the almost 1500 change initiatives attempted. We also surveyed clinicians and medical directors at each of the clinics both before and after the intervention to assess these factors, and we made site visits to a sample of clinics to collect qualitative data that might shed light on why the intervention was not more successful. Before publishing our evaluation, we conducted preliminary analyses of all of that information to assure ourselves that we were not misrepresenting the results or missing a critical determinant of success. We hope that subsequent analyses will improve our understanding of why the intervention described was not more successful.

Dr. Agins and Mr. Steinbock are concerned that our evaluation might have been contaminated by the participation of control clinics in the HIVQUAL Project or other QI efforts. We had the same concern and asked each study clinic detailed questions about its participation in QI activities both before and after the collaborative. We did not find much evidence of ongoing QI activities in the control clinics. The relatively small improvement in both the intervention and control sites suggests that our lack of results was not due to large improvements in the control clinics.

We did assess whether there were larger effects in the population of focus in the 11 clinics (out of 44 total) that restricted their efforts to a subset of the clinic population. In models that compared improvement in the focus population with improvement in control clinics, only the increase in the number of visits in 3 or 4 quarters was significant (P = 0.04).

We hope that future studies continue to address these types of issues to enhance our understanding of the interventions that will lead to the greatest improvements.