Vascular Web Logo

SVS  SVS Foundation

 Hospital Rankings Can Be Affected by Adjustments in Statistical Reliability

December 29, 2010   Contact: Sue Crosson-Knutson   312-334-2311

CHICAGO - New research published in the January issue of the Journal of Vascular Surgery® explores the impact that adjustments in statistical reliability can have on hospital quality rankings for vascular surgery. A total of 14,569 patients who underwent common vascular procedures (open and endovascular abdominal aortic aneurysm repair, carotid endarterectomy, lower extremity bypass and aorto-femoral bypass) were identified through the 2007 National Surgical Quality Improvement Project (NSQIP). For each of the hospitals in the study, a ratio of observed to expected mortality (O-E ratio) using standard NSQIP techniques was calculated. 
“We first estimated the extent to which hospital variations in mortality can be attributed to statistical noise (unexplained variations in a sample), patient factors and true difference in quality,” said Nicholas H. Osborne, MD, MS, coauthor and research fellow in the University of Michigan Department of Surgery in Ann Arbor.
The study revealed that estimates of hospital mortality are more reliable at high volume hospitals compared to low volume hospitals. The estimated reliability ranged from 0 percent for the lowest volume hospitals to 55 percent for one hospital that performed more than 350 cases annually.

After stratifying hospitals into quartiles of volume, the proportion of the mortality due to statistical noise ranged from 94 percent in hospitals with the lowest volume to 64 percent with the highest number of cases. The proportion of the mortality due to “signal,” which could represent the fraction of mortality attributable to quality, was estimated to range from 3 percent in hospitals with the lowest caseload to 18 percent in hospitals with the highest caseload.

To determine how much traditional techniques result in misclassifications of hospitals, estimates were then adjusted for statistical noise using empirical Bayes methods and rankings were compared based on the standard O-E to rankings after the reliability adjustment.

“The 172 hospitals reported an average adjusted mortality rate of 2.4 percent for the five procedures, varying from 0 percent to 17 percent,” said Dr. Osborne. He added after adjusting for statistical noise using reliability adjustment, hospital mortality was greatly diminished, varying only from 1.7 percent to 4.1 percent.

“This adjustment for reliability had a dramatic effect on hospital rankings,” said Dr. Osborne. “Overall, 43 percent of hospitals were reclassified into either a higher or lower quartile of performance using traditional methods of risk adjustment. After adjusting for statistical noise, 51 percent all hospitals in the best quartile of performance according to traditional O-E ratios were not classified in the best quartile, and 26 percent of hospitals in the worst quartile were no longer classified as such.”

“Reliability adjustment should be standard for comparing hospital quality,” said Dr. Osborne. “Our findings show that adjusting mortality for reliability reduces statistical noise and provides more stable estimates of hospital quality.”
# # #

About Journal of Vascular Surgery®

Journal of Vascular Surgery® provides vascular, cardiothoracic and general surgeons with the most recent information in vascular surgery. Original, peer-reviewed articles cover clinical and experimental studies, noninvasive diagnostic techniques, processes and vascular substitutes, microvascular surgical techniques, angiography and endovascular management. Special issues publish papers presented at the annual meeting of the Journal's sponsoring society, the Society for Vascular Surgery®. Visit the Journal Web site at

About the Society for Vascular Surgery®

The Society for Vascular Surgery® (SVS) is a not-for-profit professional medical society, composed primarily of vascular surgeons, that seeks to advance excellence and innovation in vascular health through education, advocacy, research, and public awareness. SVS is the national advocate for 3,370 specialty-trained vascular surgeons and other medical professionals who are dedicated to the prevention and cure of vascular disease. Visit its Web site at® and follow SVS on Twitter, Facebook, Linkedin, and YouTube.

Contact Us

Society for Vascular Surgery
633 North Saint Clair Street, 22nd Floor | Chicago, IL 60611
Phone: 312-334-2300 | 800-258-7188
Fax: 312-334-2320

Follow Us


VascularWeb® is the prime source for all vascular health and disease information, and is presented by the Society for Vascular Surgery®. Its members are vascular surgeons, specialists, and vascular health professionals who are specialty-trained in all treatments for vascular disease including medical management, non-invasive procedures, and surgery.