Australia: Identifying complaint PRONE medical practitioners: Mitigating or multiplying risk?

Medical malpractice insurers and health complaint commissions are all too familiar with the detrimental impacts of so called "frequent flyer" clinicians. Arguably, a history of complaints against individual clinicians suggests a failure to implement (or successfully implement) strategic intervention to prevent the recurrence of claims.

Intervention tends to be reactive following a complaint or claim. From the clinician's perspective, intervention actions are likely to be resisted or simply ignored due to time constraints and other factors.

Regulatory bodies, complaint commissions, insurers and the like would undoubtedly welcome the use of a tool to accurately assess the future risk of complaint against individual clinicians, allowing for proactive steps to be taken where necessary.

The PRONE score research

A project by Matthew Spittal of the University of Melbourne and Marie Bismark and David Studdert of Stanford University, recently published in BMJ Quality & Safety, sought to develop a relatively simple and reliable algorithm that can estimate the future risk of complaints being brought against practitioners as a means for facilitating proactive intervention.

The study was carried out with the assistance of Australian health service commissions from all states (except South Australia) and used a data set of 13,849 complaints made against 8,424 doctors over a 12-year period (from 2000 to 2011). The study found that:

• Sixty percent of complaints were related to clinical aspects of care with the most common being treatment (39%), diagnosis (16%) and medications (8%).
• Approximately 20% of complaints were related to communication issues such as the attitude or manner of doctors (13%) and the quality or amount of information provided (6%).
• Nearly 50% of doctors complained about were general practitioners and 15% were surgeons.
• Seventy-nine percent of doctors complained about were male.
• Eighty percent of doctors complained about were aged between 35 to 65 years.
• For doctors who were the subject of more than one complaint, on average, 398 days elapsed between the first complaint and the next.

From this, an algorithm was developed to produce a PRONE (Predicted Risk of New Event) score indicating the likelihood of a future complaint against an individual clinician using only the following four variables:

• the doctor's speciality
• whether the doctor was male or female
• number of previous complaints, and
• time since the last complaint.

The PRONE scores produced for clinicians using these four variables alone were reported as having performed well in predicting subsequent complaints (determined by reviewing the historical data supplied by the health commissions).

PRONE score limitations

The study concludes that the PRONE score exhibited strong predictive properties and has considerable potential to determine the likelihood that doctors named in complaints will be the subject of future complaints. However, the following limitations were noted:

• Variables not incorporated into the algorithm can also be used to predict complaints, including characteristics of individual patients and doctors, the doctor-patient relationship and the environment where the doctor works. Factors such as these affect patient satisfaction and resulting complaints, but are difficult to measure and to include into a tool intended for routine use.
• Exposure to complaint risk arising from the volume of patients treated by an individual clinician or the type of procedures performed was not incorporated.
• The extent to which the PRONE tool might actually be applied is unknown. For example, it is suggested that use in a limited data catchment setting (such as a single hospital) may undermine the accuracy of risk prediction.

The study also points out that even if the PRONE score is adopted and used effectively, it is insufficient on its own to improve the quality and safety of care and should be considered as a front-end strategy for subsequent intervention to take place.

Potential benefits and risks

A relatively simple, effective tool such as the PRONE score would be of interest to regulatory and complaint bodies, as well as medical malpractice insurers involved with risk prevention and public safety in the healthcare industry.

Arguably, the methodology might be best applied to large data sources, such as those held by long-term medical malpractice insurers, complaint commissions or large healthcare facilities. Generating PRONE scores from a review of claim/complaint data may complement other risk analysis methodologies already used.

However, the PRONE score recipient would then need to determine what steps, if any, are to be taken for those clinicians with a score predicting a high likelihood of future complaints. If active intervention was considered appropriate, options might include:

• recommending or requiring targeted "Continuing Professional Education" sessions to be undertaken
• increasing insurance premiums or adding specific preconditions, endorsements or exclusions to the subject policies, or
• implementing conditions of practice, if warranted.

Questions relating to public safety might arise if this tool is put into practice. For example, what are the obligations of an insurer, health service commission or hospital to disseminate high PRONE scores to other organisations so that appropriate precautions are taken to reduce risks to public health and safety?

It is worth considering the implications of a plaintiff obtaining PRONE score data for an individual clinician via disclosure (or some other means) during litigation. Questions would undoubtedly be asked of any facility or employer that was aware of an elevated PRONE score for the defendant clinician.

It is also conceivable that a mid to high PRONE score held by a clinician after a single prior complaint might alter the way in which a complaint agency responds to a further (as yet unsubstantiated) claim against that clinician.

If this methodology were adopted, it seems inevitable that the gathering and use of complaint data and the accuracy of resulting PRONE scores would be challenged by some clinicians, particularly if those scores were used to place restrictions on practice.

What does this mean for insurers?

Medical malpractice insurers are acutely aware of the cost of defending claims against clinicians, including the limited opportunity to recover costs when a claim is successfully defended.

A tool that identifies and quantifies the risk of future complaints against clinicians and thereby creates an opportunity for intervention would undoubtedly be welcomed. However, such analysis and intervention at the individual clinician level could prove to be difficult to manage, very costly and not without additional risks.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.