1000 Journal of Food Protection, Vol. 85, No. 7, 2022, Pages 1000-1007 https://doi.org/10.43 15/JFP-22-007 Published 2022 by the International Association for Food Protection This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Research Paper Foodborne Outbreak Rates Associated with Restaurant Inspection Grading and Posting at the Point of Service: Evaluation Using National Foodborne Outbreak Surveillance Data THUY N. KIM@htps:/orcid.org/0000-0001-6470-7046,'* LAURA WILDEY,? BRIGETTE GLEASON,* JULIA BLESER,* MELANIE J. FIRESTONE@ hiips://orcid.org/0000-0003-2244-3729,! GINA BARE,? JESSE BLISS,* DANIEL DEWEY-MATTIA,? HARLAN STUEVEN,° LAURA BROWN,° DAVID DYJACK,? AnD CRAIG W. HEDBERG! !University of Minnesota School of Public Health, Division of Environmental Health Sciences, 420 Delaware Street S.E., MMC 807, Minneapolis, Minnesota 55455; *National Environmental Health Association, 720 South Colorado Boulevard, 1000N, Denver, Colorado 80246; *National Center for Emerging and Zoonotic Infectious Diseases, Centers for Disease Control and Prevention, 1600 Clifton Road, Atlanta, Georgia 30329; National Network of Public Health Institutes, 1300 Connecticut Avenue N.W., no. 150, Washington, DC 20036; *Dining Safety Alliance, 200 Union Boulevard, Suite 200, Lakewood, Colorado 80228; and °National Center for Environmental Health, Centers for Disease Control and Prevention, 4770 Buford Highway N.E., MS F58, Atlanta, Georgia 30341, USA MS 22-007: Received 11 January 2022/Accepted 14 February 2022/Published Online 17 February 2022 ABSTRACT A previously conducted national survey of restaurant inspection programs associated the practice of disclosing inspection results to consumers at the restaurant point of service (POS) with fewer foodborne outbreaks. We used data from the national Foodborne Disease Outbreak Surveillance System (FDOSS) to assess the reproducibility of the survey results. Programs that participated in the survey accounted for approximately 23% of the single-state foodborne illness outbreaks in restaurant settings reported to FDOSS during 2016 to 2018. Agencies that disclosed inspection results at the POS reported fewer outbreaks (mean = 0.29 outbreaks per 1,000 establishments) than those that disclosed results online (0.7) or not at all (1.0). Having any grading method for inspections was associated with fewer reported outbreaks than having no grading method. Agencies that used letter grades had the lowest numbers of outbreaks per 1,000 establishments. There was a positive association (correlation coefficient, R* = 0.29) between the mean number of foodborne illness complaints per 1,000 establishments, per the survey, and the mean number of restaurant outbreaks reported to FDOSS (R? = 0.29). This association was stronger for bacterial toxin-mediated outbreaks (R* = 0.35) than for norovirus (R? = 0.10) or Salmonella (R* = 0.01) outbreaks. Our cross-sectional study findings are consistent with previous observations that linked the practice of posting graded inspection results at the POS with reduced occurrence of foodborne illnesses and outbreaks associated with restaurants. Support for foodborne illness surveillance programs and food regulatory activities at local health agencies is foundational for food safety systems coordinated at state and federal levels. HIGHLIGHTS ¢ Jurisdictions with point-of-service disclosure reported fewer outbreaks. * Grading used in inspections was associated with fewer outbreaks than no grading. ¢ Foodborne illness complaints may lead to increased outbreak detection and reporting. Key words: Foodborne illness; Foodborne outbreak; Inspection results; Public disclosure, Restaurant inspection; Restaurant inspection grading It is estimated that known foodborne pathogens are incident in which two or more people become ill from the responsible for 9.4 million illnesses annually in the United same contaminated food or drink (J); sporadic cases are States (2, 19). Depending on the pathogen, <1 to 10% of — jJInesses that have not been identified to be part of an cases are known to be associated with a recognized outbreak (3). Nevertheless, outbreak investigations provide key information on the food, pathogens, and settings associated with foodborne illness. An outbreak is defined by the Centers for Disease Control and Prevention (CDC) as an outbreak. Restaurants are an important setting for both outbreak-associated and sporadic (non—outbreak-associated) foodborne illness in the United States (2, 1/3). The percentage of foodborne illness outbreaks attributed to restaurant settings increased from a mean of 41% for the * Author for correspondence. Tel: 612-503-9277; E-mail: period 1967 to 1997 (/4) to a mean of 61% for the period kim00977@umn.edu. 2009 to 2015 (8). 2202 Ainr 6} Uo Jasn UOqUaAaLd pUe JOUOD eseasiq 40) s1ajUBD Aq JPd'Q001-2-S8-2606-P76 L/6S78Z0E/000 L/Z/Se/yPd-a|on1e/dy[/Woo'ssasdualje"ueIpuaW/:dyY Woy papeojuMog J. Food Prot., Vol. 85, No. 7 In recognition of the important role that restaurants play in prevention of foodborne illness and outbreaks, studies have identified model practices for agencies that inspect restaurants for compliance with food safety regulations. Study findings suggest that disclosing inspec- tion results at the point of service (POS) (1.e., at the establishment) using some form of grading (letter grade, color, numerical score, emoji, etc.) is associated with improved public health outcomes (5, 9, 20, 23). The evidence gathered by these efforts suggests that such disclosure yielded improved inspection scores (5), improved sanitary conditions (23), decreased incidence of Salmonella infection (9), and decreased hospitalizations due to foodborne illness (20). The results of these studies strongly suggest that the actions of restaurant inspection programs play an important role in reducing foodborne illness transmitted in restaurant settings. In 2021, a national survey of restaurant inspection programs found that disclosure at the POS was associated with fewer foodborne illness outbreaks reported per 1,000 licensed food establishments. Survey methods were previ- ously described (/5). Briefly, the survey was disseminated to a total of 790 restaurant inspection agencies at two times: 7 January 2020 and 3 March 2020 (/5). A third dissemination of the same survey occurred on 2 November 2020. Although not included in the original study results, these data were included in the analysis for this study. The net total number of agencies responding to the survey was 165. Of these, 140 respondents represented local agencies, whereas the remain- der represented state or territorial agencies (/5). This survey captured various restaurant inspection agency characteristics across the United States, including estimates of complaints received and use of methods of grading, inspection results disclosure, and inspection violation schemes. It also captured counts of foodborne illness outbreaks, sporadic illness cases, and foodborne illness complaints. Survey recipients represented inspection agencies that disclosed inspection results online and those enrolled in the U.S. Food and Drug Administration (FDA) Voluntary National Retail Food Regulatory Program Standards program (Retail Program Standards). This program helps food regulatory programs meet the widely recognized Voluntary National Retail Food Regulatory Program Standards (2/). The FDA Food Code is a model set of science-based, comprehensive food safety guidelines that provides the technical and legal basis for local, state, tribal, and federal food codes that regulate retail food service in the United States (22). A limitation of the survey-reported data was the lack of important details on the etiologic agent (e.g., bacterial or viral pathogen) and setting of these outbreaks (/5). We sought to address these gaps by using data routinely reported by state public health agencies to the CDC through the Foodborne Disease Outbreak Surveillance System (FDOSS). FDOSS is a national, passive surveillance system that collects information on enteric and nonenteric food- borne outbreaks, including information on the number of cases, case outcomes, dates of illness onset, implicated foods, and locations of food preparation (1). The objective of our present study was to use FDOSS outbreak data to EVALUATION OF RESTAURANT GRADING SURVEY RESULTS 1001 compare the number of outbreaks per 1,000 licensed restaurants by restaurant inspection grading and disclosure practices conducted by agencies responding to the initial survey (15). MATERIALS AND METHODS We used results from the previously conducted national survey of regulatory restaurant inspection agencies at state, county, city, district, and territorial levels as a baseline for this study (15). We limited analyses to local agencies representing city, county, or district jurisdictions (7 = 140), hereafter referred to as “agencies.” The decision to focus on local agencies is supported by the tendency of restaurant inspection programs to operate at the local government level (/5). The agencies were drawn from 34 states representing all regions of the country (median = three agencies per state, range = | to 14). This current study used the following data from the original survey: jurisdiction of the survey respondents, number of licensed restaurants, number of com- plaints received from 2016 to 2018, method of inspection grading, and method of public disclosure of inspection results. As with inspection practices, inspection terminology can vary by agency. We defined public disclosure as the act of voluntarily and preemptively publicizing some or all inspection data to the public (e.g., posting at the restaurant or online). This study also defined grading method as the act of applying an ordinal ranking system to inspection results (e.g., numerical scores or letter grades). Disclosure at the POS is inclusive of any type of display of inspection results on the restaurant premises, regardless of font size or location. Complaints are reports to public health of possible foodborne illness from the public, including individuals or groups of individuals (7). We obtained foodborne outbreak data for our analysis from FDOSS; data also contained associated details about etiology and food preparation location. We applied the following inclusion criteria to the FDOSS data extracted on 18 November 2019: the primary mode of transmission was foodborne; the outbreak report was finalized; date of first illness was between | January 2016 and 31 December 2018; the number of estimated primary illnesses was greater than one; the exposure location was within the jurisdiction of an agency that participated in our survey; and the location where food was prepared was a restaurant setting—including sit- down dining, buffet, fast food, or other or unknown restaurant type. We linked the FDOSS data to the survey data by jurisdiction, identified by the reporting agency. An outbreak was attributed to a regulatory agency if the agency’s jurisdiction was listed in FDOSS as the location in which the exposure occurred. Outbreaks in which exposure occurred in multiple counties were assigned to agencies based on the listed exposure locations. If a multicounty outbreak had exposure locations in jurisdictions for multiple agencies, each outbreak was counted once for each agency. Multistate outbreaks were excluded from analysis. Some counties contain city agencies that conduct inspections independently of the county agency. These incidences were identified by comparing the survey-reported population served by the county agency with the U.S. Census Bureau estimates of population for the jurisdiction. Using this method, city-level exposure data were used to identify and assign outbreak counts to the appropriate agency for four outbreaks. Outbreaks for which multiple pathogens were identified were counted only once in the outbreak total but were counted for each pathogen for pathogen-specific analyses. We grouped FDOSS restaurant outbreaks by etiology. Outbreaks in FDOSS with the suspected etiology of “other- 7202 Ainr 6} UO JasN UOHUBAaIq PUB [OJUOD aseasiq JO} s19}UB_ Aq Jpd'Q001-2-G8-2606-P 6 | /6S78Z0€/0001/2/S84Pd-a}oI12/dj[/Woo'ssaidualje"uelpiewu//:dy}Y Woy pepeo|umMoq 1002 KIM ET AL. TABLE 1. Etiological distribution of outbreaks in restaurant settings reported to the FDOSS for agencies participating in the restaurant grading project survey compared with all other jurisdictions, 2016 to 2018° J. Food Prot., Vol. 85, No. 7 TABLE 2. Number and mean annual rate of outbreaks in restaurant settings reported to the FDOSS by disclosure methods and grading methods for agencies participating in the restaurant grading project survey, 2016 to 2018" Restaurant outbreaks for Restaurant outbreaks in survey group agencies, all other jurisdictions, n (%) (n = 381) n (%) (n = 1,257) Bacterial toxin 36 (9) 109 (9) Bacillus 11 23 Clostridium 10 44 Staphylococcus 6 37 Unspecified 9 5 Campylobacter 10 (3) 30 (2) Ciguatoxin 0 (0) 3 (0.2) Cryptosporidium 1 (0.3) 2 (0.2) Cyclospora 4 (1) 23 (2) Escherichia 6 (2) 18 (1) Hepatitis 1 (0.3) 9 (0.7) Norovirus 177 (46)? 489 (39) Salmonella 48 (13) 125 (10) Sapovirus 2 (0.5) 5 (0.4) Scombroid toxin 4 (1) 27 (2) Shigella 0 (0) 5 (0.4) Vibrio 39 (10)? 18 (1) Multiple etiologies 6 (2) 22 (2) Unknown etiology 47 (12)? 372 (30) “ FDOSS, Foodborne Disease Outbreak Surveillance System. ° Proportion of outbreaks significantly different between survey group and all other jurisdictions. Norovirus (RR = 1.14; 95% CI = 1.01, 1.29) and Vibrio (RR = 2.94; 95% CI = 1.99, 4.35) were more frequently reported by agencies in the survey group, whereas unknown etiologies (RR = 0.48; 95% CI = 0.36, 0.66) were less frequently reported. bacterium” were reviewed; most were attributed to an unspecified bacterial toxin based on details provided by the reporting agency. These counts were then combined with Bacillus cereus, Clostrid- ium perfringens, and Staphylococcus aureus and collectively referred to as “bacterial toxin—mediated.” The proportions of outbreaks by etiology were compared between agencies that participated in the restaurant grading project survey (survey group) and all other agencies reporting to FDOSS. This comparison between the two groups enumerated the contributions of the survey group in the context of the overall national outbreak surveillance data for the study period. We calculated mean and median values for rates to identify trends in outcomes based on each category of grading method, disclosure method, and inspection violation scheme. Mean rates for the survey group and all other agencies were compared using t¢ tests, and P values were reported based on unequal variance assumptions. The level of significance was set at a = 0.05. Analysis was conducted using SAS 9.4 (SAS Institute, Cary, NC). Scatterplots and R* values were obtained using Microsoft Excel (Microsoft, Redmond, WA) to assess the relationship between the mean number of complaints reported and the mean number of outbreaks by etiology. RESULTS There were 2,608 single-state foodborne outbreaks reported to FDOSS during 2016 to 2018, with 1,638 attributed to food prepared in a restaurant setting. Of these, Outbreaks per No. of 1,000 restaurants No. of | outbreaks in. }=——W—————_ agencies restaurants Mean(SD) Median Disclosure methods Point of service 8 24 0.29 (0.2) 0.3 Online 36 226 0.70 (0.7) 0.4 None 11 72 1.0 (1.0) 0.5 Grading methods Letter grade 42 310 0.57 (0.7) 0.3 Numerical score 19 148 0.69 (0.7) 0.4 None 12 89 0.96 (0.9) 0.7 Other 16 138 0.76 (0.8) 0.4 “ FDOSS, Foodborne Disease Outbreak Surveillance System. outbreaks in the survey group jurisdictions accounted for 23% (n= 381), and all other jurisdictions accounted for the remaining 77% (n = 1,257). Outbreak numbers and etiology by group. The proportion of outbreaks in restaurant settings was signifi- cantly higher among agencies in the survey group compared with all other agencies (relative risk [RR] = 1.10, 95% confidence interval [CI] = 1.03, 1.17; Table 1). The most common etiologies reported to FDOSS in restaurant settings from the survey group were norovirus (177 [46%] outbreaks), Salmonella (48 [13%] outbreaks), Vibrio spp. (39 [10%] outbreaks), and bacterial toxin—mediated (36 [9%] outbreaks) (Table 1). The etiology was unknown for 47 outbreaks (12%) (Table 1). The proportions of restaurant setting outbreaks attributed to norovirus (RR = 1.14; 95% CI= 1.01, 1.29) and Vibrio spp. (RR = 2.94; 95% CI= 1.99, 4.35) were significantly higher among the survey group, whereas the proportion of unknown outbreaks was signif- icantly lower (RR = 0.48; 95% CI = 0.36, 0.64) among the survey group compared with all other agencies. Outbreak rates by inspection disclosure and grad- ing methods. There was a pattern of lower mean annual number of outbreaks per 1,000 licensed restaurants for agencies in the survey group that disclosed inspection results at the POS compared with agencies that either disclosed online or did not disclose (means: 0.29 POS versus 0.70 online, 1.0 did not disclose) (Table 2). A similar pattern was also seen for inspection grading methods; agencies with any form of grading method had a lower mean annual number of outbreaks per 1,000 licensed restaurants than agencies with no grading method (means: 0.57 letter grade, 0.69 numerical score versus 0.96 no grading method). Comparison of POS and online disclosure methods. Inspection disclosure methods varied across agencies within states. For example, in 10 states that had six or more 7202 Ainr 6} UO Jasn UOHUBAaIq PUB [OUD aseasiq JO} Ss19}UB_ Aq Jpd'Q00}-2-G8-2606-P 6 | /6S78Z0€/0001/2/S84Pd-a}oy12/dj[/Woo'ssaidualje"uelpiewu//:dy}Y Woy pepeo|umMoq J. Food Prot., Vol. 85, No. 7 TABLE 3. Mean annual rate of outbreaks in restaurant settings reported to the FDOSS by POS disclosure versus online without POS disclosure for agencies participating in the restaurant grading project survey, 2016 to 2018* Outbreaks per 1,000 restaurants (n = 202) Mean (SD) Median P value? Disclosure method 0.002 POS 0.3 (0.2) 0.3 Online without POS 0.8 (0.7) 0.5 “FDOSS, Foodborne Disease Outbreak Surveillance System; POS, point of service. ° P value for comparison of means. agencies included in the survey, in only two states did all of the agencies in the state use the same practices for disclosing inspection results. Of the 28 agencies that disclosed at the POS according to the survey, 24 (86%) also disclosed online. However, there were fewer outbreaks reported by agencies that disclosed at the POS, compared with agencies that disclosed online without POS disclosure (0.3 POS versus 0.8 online, P = 0.002) (Table 3). Complaint rates by restaurant outbreak etiologies reported to FDOSS. There was a positive association (correlation coefficient, R*? = 0.29) between the mean number of complaints per 1,000 licensed restaurants per year reported to FDOSS and the mean number of restaurant outbreaks per year reported to FDOSS (R? = 0.29; Fig. 1). When reported restaurant outbreaks were stratified by etiology, there was a positive association between the mean number of complaints and the mean number of norovirus outbreaks in restaurants reported to FDOSS (R? = 0.10; Fig. 2), and a positive association for bacterial toxin—mediated restaurant outbreaks (R* = 0.35; F ig. 3). Conversely, there was no meaningful trend for Salmonella (R* = 0.01; Fig. 4), suggesting that Salmonella outbreaks are not associated with foodborne illness complaints. DISCUSSION Relevance to practice. Our findings were consistent with previous survey (/5) results that showed that the disclosure of graded inspection results at the POS was associated with fewer outbreaks reported to FDOSS. These results provide further support for recommendations (/5) to post graded restaurant inspection results at the POS by demonstrating that agencies that used some grading system had lower mean numbers of FDOSS restaurant outbreaks per 1,000 establishments than did agencies that did not post graded inspection results. Agencies that used letter grades had the lowest mean and median numbers of FDOSS restaurant outbreaks per 1,000 licensed restaurants, al- though the study had limited power to distinguish among the grading methods. Restaurant inspections are a measure of how well a restaurant adheres to food safety guidelines that prevent foodborne illness. The finding that posting graded inspec- EVALUATION OF RESTAURANT GRADING SURVEY RESULTS 1003 tion results at the POS was associated with fewer outbreaks occurring in restaurants based on FDOSS data is consistent with hypotheses that consumers use this information to guide their dining decisions (10, //, 23). Because access to this information is important to consumers, a favorable score may attract more consumers, whereas a less favorable score may provide food operators with additional incentive to improve their food safety performance. Disclosure of inspection results at the POS allows this measure of food safety performance to be readily available and interpretable to consumers at a location where many dining decisions are made. Distribution of outbreaks. The higher proportion of outbreaks reported by the survey group suggests that these agencies were more likely to report restaurant-associated outbreaks and were more likely to report outbreaks due to norovirus but were less likely to report outbreaks of unknown etiology than all other agencies. This suggests that agencies in the survey group were better at determining the outbreak setting and etiology of the outbreaks they investigated. The relative effectiveness of agencies in the survey group to detect and investigate outbreaks adds further support for the credibility of findings within this group regarding differences in outbreak reporting based on inspection grading and disclosure practices. Usefulness of consumer complaints. In addition to our findings regarding inspection reporting, the results of this study support the importance of agencies having a mechanism to receive foodborne illness complaints. Our finding of a positive correlation between the number of complaints received per 1,000 licensed restaurants and the number of restaurant outbreaks reported to FDOSS means that the ability to receive and investigate foodborne illness complaints may be an important predictor of the ability of the agency to detect foodborne outbreaks. In particular, the positive associations between complaints and restau- rant outbreaks of bacterial toxin—mediated and norovirus outbreaks reflects the reliance on complaint-based sur- veillance to detect these outbreaks with short incubation periods. It is primarily through complaint-based surveil- lance systems that these types of outbreaks, and others with short incubation periods, are detected by public health agencies, thereby underscoring the need for continued complaint-based surveillance systems (7). In contrast, Salmonella-associated outbreaks are detected primarily through pathogen-specific surveillance; this supports the finding of no effect between the occurrence of complaints and outbreaks of Salmonella, which has a longer incubation period than toxin-mediated pathogens (4, 17). Complaint-based surveillance is one of the two main methods of foodborne outbreak detection in the United States (7). Although this study does not assume that having the ability to receive complaints is indicative of the existence of a complaint system, it is notable that 81% of local health departments have a complaint-based surveil- lance system (/6) and approximately 75% of all foodborne 7202 Ainr 6} UO JASN UOHUBAaIq PUB [OUD aseasiq JO} s19}UB_ Aq Jpd'Q001-2-G8-2606-P 6 | /6S78Z0€/0001/2/S84Pd-a}oy12/dj[/Woo'ssaidualje"uelpLew//:dy}Y Woy pepeo|umoq 1004 KIM ET AL. 35 _—_ 3) Qa n 3 bol) .— £ g w 2.5 @ o © 2 ae ® -“ cw £¢ - x 8 is > oo Vero ve @ ave 9** 2" o | wc 3 1 | ®e ———_--_ a 8 6 = ae = jj x ggeast SC ioe Letts © 0.5 2 % e = 2 @e* e e 0 &e ” 0 50 100 J. Food Prot., Vol. 85, No. 7 150 200 250 Mean no. complaints per 1,000 restaurants FIGURE 1. Mean annual number of outbreaks in restaurant settings per 1,000 restaurants reported to the Foodborne Disease Outbreak Surveillance System (FDOSS) and the mean number of survey-reported complaints per 1,000 restaurants per year for agencies (@) participating in the restaurant grading project survey, 2016 to 2018. outbreaks are detected through complaint systems (6). The usefulness of complaints to detect outbreaks has been demonstrated by multiple studies (/2, /6-/8, 25). A survey of local health departments identified a positive correlation between outbreak and complaint rates per population served; agencies that received more complaints detected more outbreaks (/6). An analysis of the Florida 2.5 1:5 Mean no. norovirus outbreaks in restuarant settings per 1,000 restaurants 0 50 100 Department of Health’s complaint and outbreak reporting system found that 56% of foodborne outbreaks were identified through complaints (/8). Likewise, complaints led to detection of 80% of foodborne outbreaks in Rhode Island (25) and 79% of confirmed foodborne outbreaks in Minnesota (/7). Not only can complaints be used to detect outbreaks, but they can also help identify specific R? = 0.1033 vooure* 150 200 250 Mean no. complaints per 1,000 restaurants FIGURE 2. Mean annual number of norovirus outbreaks in restaurant settings per 1,000 restaurants reported to the Foodborne Disease Outbreak Surveillance System (FDOSS) and the mean number of survey-reported complaints per 1,000 restaurants per year for agencies (®) participating in the restaurant grading project survey, 2016 to 2018. 7202 Ainr 6} UO JasN UOHUBASIq PUB [OUD aseasiq JO} S19}UB_ Aq Jpd'Q001-2-G8-2606-P 6 | /6S78Z0€/0001/2/S84Pd-a}oy12/dj[/Woo'ssaidualje"ueIpewu//:dy}Y Woy pepeojumMoq J. Food Prot., Vol. 85, No. 7 0.9 © 2 408 2 = e 28 ap BE og se” oo vor 0.5 Es €a 8 % 04 AAA coe ee er £3 = @ oe § 8 03 wa 0 5 eo oo 2] 5 02 eee Om | eer* c+ : £201 ) ) 50 100 EVALUATION OF RESTAURANT GRADING SURVEY RESULTS 1005 ry deg 150 200 250 Mean no. complaints per 1,000 restaurants FIGURE 3. Mean annual number of bacterial toxin—mediated outbreaks in restaurant settings per 1,000 restaurants reported to the Foodborne Disease Outbreak Surveillance System (FDOSS) and the mean number of survey-reported complaints per 1,000 restaurants per year for agencies (@) participating in the restaurant grading project survey, 2016 to 2018. indicators of risk. For example, a study of consumer complaints in Washington, DC, found that complaints were significantly correlated with cited inspection violations of improper holding temperatures and contaminated equip- ment (/2). These studies highlight the usefulness of consumer complaints and underscore the need for 0.9 w pore) £ 5 08 wu uw eS Ss 0.7 I . 8 2 06 £f = 2805 5S } = Oo 3 =) 0.4 ® sa : — — 2 203 S ® & e ° ° e A 02 |- ee a ee ‘aa ee a ae : © Ss 01 ° eo @ > ° °° 0 ; - 100 complaint-based surveillance in foodborne outbreak de- tection for pathogens with short incubation periods. Strengths and limitations. Strengths of this study include the use of national data (FDOSS) through a well- established outbreak surveillance system to validate out- 150 200 250 Mean no. complaints per 1,000 restaurants FIGURE 4. Mean annual number of Salmonella outbreaks in restaurant settings per 1,000 restaurants reported to the Foodborne Disease Outbreak Surveillance System (FDOSS) and the mean number of survey-reported complaints per 1,000 restaurants per year for agencies (®@) participating in the restaurant grading project survey, 2016 to 2018. 7202 Ainr 6} UO JasN UOHUBAaIq PUB [OUD aseasiq JO) S19]UB_ Aq Jpd'Q00}-2-G8-2606- P76 | /6S78Z0€/0001/2/S84Pd-a}oI12/dj[/Woo'ssaidualje"uelpiew//:dyY Woy pepeo|umMoq 1006 KIM ET AL. break counts reported via survey. The surveyed agencies accounted for nearly one-quarter of restaurant setting outbreaks reported to FDOSS. This study did not adjust for potential confounders such as jurisdiction size, geo- graphic region, state-level food program inspection and reporting requirements, funding, and staffing of the inspection agency. These factors may have affected an agency’s ability to investigate consumer complaints, detect outbreaks, and subsequently report them to FDOSS. However, there did not appear to be an association between jurisdiction size and reported outbreak rate (R? < 0.01). In most states there was considerable variation among agencies with respect to restaurant grading and disclosure practices. As noted above, the higher proportion of outbreaks attributable to norovirus and lower proportion of outbreaks with unknown etiology among the surveyed agencies may reflect that they had a better capacity to investigate foodborne illness outbreaks than did agencies that did not respond to the survey. There are inherent limitations to the use of FDOSS data. First, because the FDOSS database is dynamic, agencies are permitted to submit, update, or delete reports at any time. Data used in the analysis for this study were pulled at one point in time; therefore, previous and future analyses using FDOSS data extracted in a similar fashion may produce slightly different results. Second, outbreak counts are reflective of those that were able to be detected. Not all outbreaks are identified by public health agencies, and as noted previously, the majority of foodborne illnesses are not a part of recognized outbreaks. It is unknown how well the etiologies and locations implicated in outbreaks reflect those of sporadic foodborne illnesses, i.e., illnesses not associated with outbreaks. Limitations related to using the survey methods described include the use of a convenience sample of agencies that were enrolled in the Retail Program Standards program, which limited the representativeness of these results to enrollees. Agencies that enroll in this voluntary program may differ from those that choose not to enroll; however, because most (98%) of the agencies participating in the study were participants in the Retail Program Standards program, participation in the Retail Program Standards program is unlikely to bias the findings with respect to the main effect measures. Due to the inquiry of data from multiple time points (survey results during 2019 to 2020 and outbreak data during 2016 to 2018), survey responses may not be truly reflective of practices during the time the outbreaks occurred. A consumer’s propensity to file a foodborne illness complaint involving a restaurant is influenced by a variety of factors, including poverty status. Unpublished work studying the association of foodborne illness and inspection report data in Hennepin County, MN, found that census blocks with high poverty levels were associated with fewer foodborne illness complaints (OR = 0.31; 95% CI: 0.13 to 0.73) (24). Nevertheless, underlying poverty status in the survey group was not deemed an important confounder in our analysis. Because the ability to detect outbreaks in restaurants heavily relies on complaint-based surveillance, J. Food Prot., Vol. 85, No. 7 any biasing effect that poverty status may have on consumer propensity to file a complaint would also be reflected in the number of outbreaks. There are also different kinds of complaints that can be received about a restaurant: those that relate specifically to foodborne illness and those that relate to specific good retail practice violations. Although our study did not differentiate between the two types, it is plausible that the occurrence of violations may be an indicator of food safety practices that could lead to foodborne illness in the future. Although this was a cross-sectional study that cannot control for the effects of policy changes within inspection programs, our associations are consistent with studies in Los Angeles County (20) and New York City (9) that demonstrated reductions in the occurrence of foodborne illnesses after implementation of posting of inspection grades at the POS. This study assessed the impact of the presence of disclosure at the POS, rather than the specific manners (e.g., location, font size) by which it occurred. If additional evidence were needed to encourage local food regulatory agencies to adopt a practice of grading and posting inspection results at the POS, then a randomized community-control trial could be considered as a next step. Policy implications. Surveys of public health agencies that are validated by national surveillance data can be powerful tools to identify model practices that contribute to prevention of foodborne outbreaks and illnesses. Particu- larly, our cross-sectional study findings are consistent with previous observations that linked the practice of posting graded inspection results at the POS with reduced occurrence of foodborne illnesses and outbreaks associated with restaurants. Other food regulatory practices, such as maintaining a robust foodborne illness complaint system, may improve foodborne illness surveillance, outbreak detection, and response. Improving foodborne illness and outbreak surveillance is a prerequisite for improving and measuring the effectiveness of our food safety systems. Support for foodborne illness surveillance programs and food regulatory activities at local health agencies is foundational for food safety systems coordinated at state and federal levels. ACKNOWLEDGMENTS This study was funded through cooperative agreement 6NU380T000300 between the Centers for Disease Control and Prevention (CDC) and the National Environmental Health Association (NEHA). The findings and conclusions are solely the responsibility of the authors and do not necessarily represent the official views of CDC and NEHA. Additional support was provided by the Global Food Ventures MnDRIVE Fellowship at the University of Minnesota. REFERENCES 1. Centers for Disease Control and Prevention. 2018. Foodborne Disease Outbreak Surveillance System. U.S. Department of Health and Human Services, Atlanta. Available at: https://www.cdc.gov/ fdoss/index.html. Accessed 30 August 2021. 2. Centers for Disease Control and Prevention. 2019. Surveillance for foodborne disease outbreaks, United States, 2017: annual report. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, Atlanta. Available at: https://www.cdc.gov/ 7202 Ainr 6} UO Jasn UOHUBAaIq PUB [OUOD aseasiq JO} S19}UB_ Aq Jpd'Q00}-2-G8-2606-P 76 | /6S78Z0€/0001/2/S84Pd-a}oI12/dj[/Woo'ssaidualje"uelpieww//:dy}y Woy pepeo|umMoq J. Food Prot., Vol. 85, No. 7 10. 11. 12. 13. 14. fdoss/pdf/2017_FoodBorneOutbreaks_508.pdf. Accessed 30 Decem- ber 2020. Centers for Disease Control and Prevention. 2021. FoodNet Fast: pathogen surveillance tool. U.S. Department of Health and Human Services, Atlanta. Available at: http://wwn.cdc.gov/foodnetfast. Accessed 11 August 2021. Chai, S. J., W. Gu, K. A. O’Connor, L. C. Richardson, and R. V. Tauxe. 2019. Incubation periods of enteric illnesses in foodborne outbreaks, United States, 1998-2013. Epidemiol. Infect. 147:e285— e292. Choi, J., and R. Scharff. 2017. Effect of a publicly accessible disclosure system on food safety inspection scores in retail and food service establishments. J. Food Prot. 80:1188-1192. Council to Improve Foodborne Outbreak Response. 2013. Develop- ment of target ranges for selected performance measures in the CIFOR guidelines. Council of State and Territorial Epidemiologists, Atlanta. Council to Improve Foodborne Outbreak Response. 2019. Foodborne illness surveillance and outbreak detection, p. 67-85. Jn Guidelines for foodborne disease outbreak response, 3rd ed. Council of State and Territorial Epidemiologists, Atlanta. Dewey-Mattia, D., K. Manikonda, A. J. Hall, M. E. Wise, and S. J. Crowe. 2018. Surveillance for foodborne disease outbreaks—United States, 2009-2015. Morb. Mortal. Wkly. Rep. Surveill. 67:1—-11. Firestone, M. J., and C. W. Hedberg. 2018. Restaurant inspection letter grades and Salmonella infections, New York, New York, USA. Emerg. Infect. Dis. 24:2164—-2168. Firestone, M. J., and C. W. Hedberg. 2020. Consumer interest and preferred formats for disclosure of restaurant inspection results, Minnesota 2019. J. Food Prot. 83:4715-4721. Fung, A., M. Graham, and D. Weil. 2007. Full disclosure: the perils and promise of transparency. Cambridge University Press, Cam- bridge, UK. Jemaneh, T. A., M. Minelli, A. Farinde, and E. Paluch. 2018. Relationship between priority violations, foodborne illness, and patron complaints in Washington, DC, restaurants (2013-2015). J. Environ. Health 80:14-19. Jones, T. F., and F. J. Angulo. 2006. Eating in restaurants: a risk factor for foodborne disease? Clin. Infect. Dis. 43:1324—1328. Jones, T. F., and J. Yackley. 2018. Foodborne disease outbreaks in the United States: a historical overview. Foodborne Pathog. Dis. 15:11— 15. EVALUATION OF RESTAURANT GRADING SURVEY RESULTS 15. 16. 17. 18. 19. 20. 21, 22. 23. 24. 25. 1007 Kim, T. N., M. J. Firestone, N. DeJarnett, L. Wildey, J. C. Bliss, D. T. Dyjack, J. Edwards, H. Stueven, and C. W. Hedberg. 2021. Disclosing inspection results at point-of-service: affect of character- istics of food establishment inspection programs on foodborne illness outcomes. J. Environ. Health 83:8-13. Li, J.. G. H. Shah, and C. Hedberg. 2011. Complaint-based surveillance for foodborne illness in the United States: a survey of local health departments. J. Food Prot. 74:432-437. Li, J., K. Smith, D. Kaehler, K. Everstine, J. Rounds, and C. Hedberg. 2010. Evaluation of a statewide foodborne illness complaint surveillance system in Minnesota, 2000 through 2006. J. Food Prot. 73:2059-2064. Li, X., A. C. Sapp, N. Singh, L. Matthias, C. Bailey, J. A. DeMent, and A. H. Havelaar. 2020. Detecting foodborne disease outbreaks in Florida through consumer complaints. J. Food Prot. 83:1877-1888. Scallan, E., R. M. Hoekstra, F. J. Angulo, R. V. Tauxe, M. A. Widdowson, S. Roy, J. L. Jones, and P. M. Griffin. 2011. Foodborne illness acquired in the United States—major pathogens. Emerg. Infect. Dis. 17:7. Simon, P. A., P. Leslie, G. Run, G. Z. Jin, R. Reporter, A. Aguirre, and J. E. Fielding. 2005. Impact of restaurant hygiene grade cards on foodborne-disease hospitalizations in Los Angeles County. J. Environ. Health 67:32-36. U.S. Food and Drug Administration. 2019. Voluntary national retail food regulatory program standards—November 2019. Available at: https://www.fda.gov/food/voluntary-national-retail-food-regulatory- program-standards/voluntary-national-retail-food-regulatory- program-standards-november-2019. Accessed 26 July 2021. U.S. Food and Drug Administration. 2019. FDA Food Code. Available at: https://www.fda.gov/food/retail-food-protection/fda- food-code. Accessed 26 July 2021. Wong, M. R., W. McKelvey, K. Ito, C. Schiff, J. B. Jacobson, and D. Kass. 2015. Impact of a letter-grade program on restaurant sanitary conditions and diner behavior in New York City. Am. J. Public Health 105:e8 1-e87. Young, H. (University of Minnesota). 2019. Personal communica- tion. Yousaf, A., B. Viveiros, and G. Caron. 2019. Rhode Island Department of Health foodborne illness complaint system: a descriptive and performance analysis. J. Food Prot. 82:1568-1574. 7202 Ainr 6} UO JasN UOHUBAaIq PUB [OUOD aseasiq JO) S19}UB_ Aq Jpd'Q00-2-G8-2606-P 6 | /6S78Z0€/0001/2/S84Pd-a}oy12/dj[/Woo'ssaidualje"uelpHewu//:dy}Y Woy pepeo|umMoq