CDC-Data-2025/attachments/JEH1-2.21-Feature-Disclosing-Inspection-Results_djvu.txt

at Point-of-Service: Affect 


= 


sc Bid 


Programs on Foodborne 
IIIness Outcomes 


ADVANCEMENT OF THE SCIENCE 


Disclosing Inspection Results 


of Characteristics of Food 
Establishment Inspection 


A b St ra ct The significant proportion of foodborne illnesses 
attributed to restaurants highlights the importance of food establishment 
inspections. The objectives of this cross-sectional study were to characterize 
local inspection programs and evaluate the effects of programmatic 
characteristics, such as active public disclosure of inspection results, on 
select operational and foodborne illness outcomes. Between January 7 
and April 6, 2020, an online 36-question survey was administered to 790 
government-run food establishment inspection programs at state and 
local levels. Of 149 survey respondents, 127 (85%) represented local food 
establishment inspection agencies. Agencies that disclosed at the point-of- 
service reported fewer mean numbers of re-inspections by 15%, foodborne 
illness complaints by 38%, outbreaks by 55% (p = .03), and Salmonella 
cases by 12% than agencies that disclosed online only. Agencies that used 
some type of grading method for inspection results reported fewer mean 
numbers of re-inspections by 37%, complaints by 22%, outbreaks by 61%, 
and Salmonella cases by 25% than agencies that did not grade inspections. 
Programmatic characteristics appear to be associated with foodborne illness 
outcomes. These results warrant future research to improve the effectiveness 


of food establishment inspection programs. 


Introduction 

Approximately 51% of each consumer dollar 
dedicated to food spending in 2019 was spent 
in the food service industry, specifically in 
restaurants, compared with just 25% in 1955 
(National Restaurant Association, 2020). 
Coincidentally, there is growing evidence that 
restaurants are an important source of spo- 
radic and outbreak-associated foodborne dis- 


8 Volume 83 ¢ Number 6 


ease in the U.S. Jones & Angulo, 2006). In 
2017, there were 841 foodborne illness out- 
breaks resulting in 14,481 illnesses, 827 hos- 
pitalizations, 20 deaths, and 14 food recalls 
in the U.S., including Puerto Rico and Wash- 
ington, DC (Centers for Disease Control and 
Prevention [CDC], 2019). 

Among the illnesses and outbreaks for 
which a single location was identified, 44% 


Thuy N. Kim, MPH, CFOI 
Melanie J. Firestone, MPH, PhD 
University of Minnesota 

School of Public Health 


Natasha DeJarnett, MPH, PhD 
Laura Wildey, CP-FS 

Jesse C. Bliss, MPH 

David T. Dyjack, DrPH, CIH 
National Environmental 

Health Association 


Jennifer Edwards, PhD 
National Network of 
Public Health Institutes 


Harlan Stueven, MD 
Dining Safety Alliance 


Craig W. Hedberg, PhD 
University of Minnesota 
School of Public Health 


and 64%, respectively, were attributed to 
foods prepared in a restaurant setting (CDC, 
2019). The rise in expenditure on foods 
eaten away from the home and the sig- 
nificant proportion of foodborne illnesses 
attributed to restaurants have highlighted 
the importance of food establishment 
inspections, as they could flag the existence 
of food safety hazards and mitigate their 
public health impact. 

Public disclosure of inspection results 
from food establishments enables consum- 
ers to make informed decisions about where 
they choose to eat (Fung et al., 2007). Con- 
sumer priority of hygienic food preparation 
practices, in turn, incentivizes food estab- 
lishments to improve hygiene practices—a 
proxy for better sanitary conditions—within 
their facility. Improved and maintained san- 
itary conditions, theoretically, lead to fewer 
foodborne illnesses. From a programmatic 
standpoint, however, disclosure of inspec- 
tion results can create more work for the 
environmental health workforce tasked 
with putting the information into a present- 
able format. In a survey of the environmen- 
tal health workforce, 76% of workers sur- 
veyed indicated working in food safety and 
protection programs; however, 17% of all 
respondents performed public health duties 
outside of environmental health, and of 
those, 37% spent >50% of their time work- 
ing in nonenvironmental health programs 
(Gerding et al., 2019). 

The value of actively disclosing inspec- 
tion results to the public has been dem- 


Vr... 1 
Summary Statistics for Local Agency Respondents (n = 124) 
a aaa Sa 
Active disclosure 82 (66) 
Active disclosure methods 
Online 75 (91) 
Point-of-service 24 (29) 
Other 4 (5) 
No active disclosure 42 (27) 
Grading methods 
Numerical score 53 (43) 
Letter grade 20 (16) 
Other 34 (27) 
No grading 30 (24) 
Inspection violation schemes (n =75) 
P-PF-C 24 (32) 
C/NC 21 (28) 
RF-GRP* 23 (31) 
P-PF-C 10 (43) 
C/NC 4 (17) 
Major/minor 3 (13) 
Other 7 (9) 
P-PF-C = Priority-Priority Foundations-Core; C/NC = Critical/Noncritical; RF-GRP = Risk Factor-Good Retail Practices. 
*Of the 23 agencies that indicated using RF-GRP, 6 agencies used RF-GRP only. The other 17 agencies used RF-GRP in 
combination with the other schemes listed below. 


onstrated in several settings throughout 
the U.S. The debate about the best mode 
to convey inspection results to the public, 
however, is still ongoing. A study of people 
at the Minnesota State Fair found increased 
interest in public access to inspection 
results. Furthermore, fairgoers expressed 
interest in disclosure methods of posting 
online and at the point-of-service, that is, at 
a food establishment (Firestone & Hedberg, 
2020). For local inspection agencies that 
disclose inspection results, the most com- 
mon method is through online disclosure 
only, typically accessed via departmental 
websites. Drawbacks of this method include 
difficulty in navigating these websites and 
lengthy reports that are confusing to the 
general public. Moreover, this method might 
not be accessible to those who are most vul- 
nerable to foodborne illness, such as older 
adults (Fleetwood, 2019). 


Disclosure at the point-of-service elimi- 
nates a barrier to using inspection data 
in the decision-making process, as this 
approach does not require a person to have 
online access to check a website for inspec- 
tion results. With the introduction of pub- 
lic disclosure by means of a color-coded 
inspection sticker placed at or near restau- 
rant entrances, Columbus Public Health 
(Ohio), saw inspection scores improve by 
1.14 points out of a possible 100 points 
(Choi & Scharff, 2017). In New York City, 
New York, implementation of public disclo- 
sure at the point-of-service in the form of 
letter grades was associated with improve- 
ments in sanitary conditions (Wong et al., 
2015) and a 5.3% decrease in Salmonella 
cases per year (Firestone & Hedberg, 2018). 
Furthermore, in Los Angeles County, Cali- 
fornia, public disclosure of letter grades at 
the establishment led to a 13% decline in 


hospitalizations due to foodborne illness 
(Simon et al., 2005). 

While the act of disclosure is important, 
what information is disclosed and how the 
public interprets it is also important. Famil- 
iarity with the symbols used to represent 
inspection results lends to easier interpreta- 
tion by the general public. Grading practices 
can include letter grading and/or numerical 
grading, similar to most grading methods in 
a school system (e.g., A, B, C grades or 100%, 
90%, 80%) or other ordinal methods (e.g., 
stoplight colors, emoticons). 

During inspections, a labeling system is 
used to classify different types of violations 
and convey severity of the violations. These 
violation schemes often correlate with the 
version of the Food and Drug Administration 
(FDA) Food Code an agency has adopted and 
can be used in combination at the agency's 
discretion. For example, in Food Code ver- 
sions before 2009, violations that were more 
likely “to contribute to food contamination, 
illness, or environmental health hazard” were 
classified as critical. In 2009, FDA revised 
the Food Code to distinguish critical items 
as priority if the item includes a quantifiable 
measure to show control (e.g., cooking), or 
priority foundation if the item requires the 
purposeful incorporation of specific actions 
(e.g., training) (Food and Drug Administra- 
tion [FDA], 2015). The categorization of risk 
factor or good retail practices corresponds to 
the organization of the FDA Food Establish- 
ment Inspection Report. 

Current inspection practices and methods 
of disclosure vary widely across jurisdictions 
in the U.S. and present unique challenges to 
evaluating program effectiveness. The objec- 
tives of this cross-sectional study were to 1) 
characterize local inspection programs and 
2) evaluate the effects of programmatic char- 
acteristics, such as active public disclosure 
methods, on select operational and food- 
borne illness outcomes. 


Methods 

An online 36-question survey was adminis- 
tered via Qualtrics to 790 government-run 
food establishment inspection programs at 
state, county, city, district, and territorial lev- 
els. Recipients were chosen based on avail- 
ability of program inspection data online or 
participation in FDAs Voluntary National 
Retail Food Regulatory Program Standards 


January/February 2021 e Journal of Environmental Health 9 


ADVANCEMENT OF THE SCIENCE 


TABLE 2 


Mean, Standard Deviation, and Median Estimates for Outcomes by Disclosure Methods, Grading Methods, 
and Inspection Violation Schemes 


Average # of Re- 


Average # of 


Average # of 


Average # of Salmonella 


Inspections/Establishment/ Complaints/1,000 Outbreaks/1,000 Cases/100,000 Population 
Year Establishments/Year Establishments/Year Served/Year 
(m= 109) (nm = 100) (n= 101) (n= 48) 
Mean (SD) Median Mean (SD) Median Mean (SD) Median Mean (SD) Median 
Disclosure methods 
Online 0.40 (0.55) 0.24 44.2 (49.6) 27.3 1.7 (2.4) 0.84 14.4 (7.2) 14.0 
Point-of-service 0.35 (0.46) 0.17 30.3 (45.3) 22.2 0.9 (1.4) 0.25 12.9 (6.5) 14.0 
None 0.53 (0.46) 0.50 31.3 (36.0) 18.5 7.0 (24) 0.00 9.9 (9.9) 6.7 
Other* 0.36 (0.43) 0.17 74.5 (86.4) 42.7 3.7 (4.7) 2.39 - - 
Grading methods 
Numerical score 0.32 (0.37) 0.17 40.6 (54.6) 22.2 3.0 (10.7) 0.35 12.4 (6.8) 13.6 
Letter grade 0.31 (0.48) 0.13 34.9 (41.7) 24.6 1.3 (1.6) 0.71 13.0 (7.0) 14.2 
None 0.59 (0.64) 0.50 49.1 (49.2) 29.2 6.5 (25.0) 0.82 15.9 (12.2) 13.1 
Other 0.46 (0.57) 0.27 36.4 (35.0) 27.6 1.9 (2.7) 0.95 12.0 (6.2) 12.7 
Inspection violation schemes 
P-PF-C 0.39 (0.45) 0.18 47.2 (53.3) 29.0 1.5 (1.7) 0.95 15.7 (7.4) 16.4 
C/NC 0.38 (0.49) 0.25 48.7 (45.2) 42.7 1.1 (1.4) 0.85 12.7 (8.8) 13.1 
RF-GRP 0.32 (0.39) 0.17 38.1 (51.1) 22.8 2.4 (2.2) 1.97 16.8 (8.1) eal 
Other 0.29 (0.37) 0.19 57.9 (73.9)* 11.8% 0.77 (0.78)* 0.62* 10.9 (7.8)* 11.7* 
P-PF-C = Priority-Priority Foundations-Core; C/NC = Critical/Noncritical; RF-GRP = Risk Factor-Good Retail Practices. 
*Contains data from <5 respondents. 


(Retail Program Standards). The Retail Pro- 
gram Standards provide recommendations 
aimed at facilitating inspections that are more 
effective and implementing foodborne illness 
prevention strategies. Enrollees in this pro- 
gram intend to actively use these standards as 
a tool to assess and improve their regulatory 
programs (FDA, 2019). 

We administered the survey in two rounds. 
The first round consisted of 151 recipients 
whose inspection data were publicly available 
online, resulting in a 40% response rate (n = 
60 respondents). The second round included 
639 recipients who participated in the Retail 
Program Standards, resulting in a response 
rate of 19% (n = 122 respondents). Via the 
survey, we obtained information on general 
program characteristics such as size of popu- 
lation served; number of routine inspections 
conducted; number of licensed establish- 
ments within the inspection jurisdiction; and 


10 


Volume 83 ¢ Number 6 


operational characteristics such as public dis- 
closure method, grading method, and FDA 
Food Code version in use. 

The time period for the survey was chosen to 
match the availability of inspection data from 
the agencies. Three geographically diverse 
local inspection agencies piloted the survey 
to ensure appropriateness and relevancy of 
questions and answer choices. The data col- 
lection period was January 7—April 6, 2020. 
We paused data collection in April due to the 
COVID-19 pandemic response taking prec- 
edent at state and local health departments. 

We categorized inspection agencies into 
two main types, state and local. A state 
agency was defined as an inspection program 
that oversees the inspection of food establish- 
ments at the state government level, includ- 
ing U.S. territories and Washington, DC. A 
local agency differs in that the oversight of 
the inspection programs is at the county, city, 


city-county, or district government level. 
One survey respondent represented a univer- 
sity and thus was excluded from this analy- 
sis, as there could be significant policy dif- 
ferences between government agencies and 
universities. Local agencies were the primary 
focus of this analysis, as most food establish- 
ment inspection programs are operated at the 
local government level. 

Four operational and foodborne illness out- 
comes were calculated as rates from a combi- 
nation of variables obtained from the survey 
and expressed as an average number of: 

1. Re-inspections/establishment/year (calcu- 
lated as the quotient of average number 
of re-inspections and number of licensed 
food establishments within the jurisdiction 
of the agency). 

2. Foodborne illness complaints/1,000 licensed 
food establishments/year (2016-2018; most 
recent years included in data set). 


TABLE 3 


Mean, Standard Deviation, and Median Estimates for Outcomes by Point-of-Service (POS) Disclosure 


Versus Online (no POS) Disclosure 


Average # of Re-Inspections/ 


Establishment/Year 


(n= 71) 


Average # of 
Complaints/1,000 
Establishments/Year 
(n = 62) 


Average # of 
Outbreaks/1,000 
Establishments/Year 
(n= 63) 


Average # of Salmonella 
Cases/100,000 Population 
Served/Year 
(n= 31) 


p-value 65 .16 44 
POS disclosure 
Mean (SD) 0.35 (0.46) 30.3 (45.3) 0.92 (1.4) 11.7 (6.6) 
Median 0.17 22.17 0.25 12.5 
Online (no POS) disclosure 
Mean (SD) 0.41 (0.57) 48.6 (50.0) 2.04 (2.69) 13.3 (8.5) 
Median 0.24 29.0 0.95 12.7 


3. Foodborne outbreaks/1,000 licensed food 
establishments/year (2016-2018). 

4. Salmonella cases reported/100,000 popula- 
tion served/year (2016-2018). 

In addition to the survey data, we were 
able to obtain some Salmonella case counts 
using departmental websites for jurisdictions 
that reported these data online. 

For the purposes of this study, active dis- 
closure was defined as agencies that volun- 
tarily and preemptively publicize some or 
all inspection data to the public. Inspection 
violation scheme was not included in the 
survey, but was determined by searching 
online for inspection data from the respond- 
ing agencies. 

Predictors were classified into three 
categories: 

1. Disclosure method consisting of online, 
point-of-service, no disclosure, and other 
disclosure methods. 

2. Grading method consisting of numerical 
score, letter grade, no grading, and other 
grading methods. 

3. Inspection violation scheme used for rou- 
tine inspections consisting of subcatego- 
ries Priority-Priority Foundations-Core; 
Critical/Noncritical; Risk Factor-Good 
Retail Practices; and other schemes. 

The Risk Factor-Good Retail Practices sub- 
category relates to the inspection report form 
and therefore can be used in combination 
with other violation schemes. The mean and 
median values of outcomes for each combi- 
nation of schemes were assessed in addition 


to the nonmutually exclusive scheme catego- 
ries previously stated. One respondent used 
a combination of three schemes: Risk Factor- 
Good Retail Practices, Critical/Noncritical, 
and Red/Blue. Of note, Red/Blue is similar 
and is sometimes used in reference to Criti- 
cal/Noncritical; therefore, this respondent's 
jurisdiction was included in the Risk Factor- 
Good Retail Practices and Critical/Noncriti- 
cal scheme combination. 

Mean and median values were calculated 
to identify trends in outcomes based on each 
subcategory. The means were compared 
using t-tests; p-values were reported assum- 
ing unequal variance. The analysis was con- 
ducted using SAS 9.4m6 University Edition. 
Linear regression was used to determine 
associations between the outcome variables 
reported by the local responding agencies. 
The level of statistical significance was set at 
o = .05. 


Results 

Of the 149 survey respondents, 127 (85%) 
represented a local food establishment 
inspection agency. More than one half of 
agencies (66%) actively disclosed inspection 
scores to the public and most (91%) did so 
by posting online; only some (30%) posted 
at the point-of-service. Approximately 43% 
of the agencies used numerical scores as 
a grading method, 24% used no grading 
method, and 16% used letter grades (Table 
1). Frequently used inspection violation 
schemes included Priority-Priority Foun- 


dations-Core (32%) and Critical/Noncriti- 
cal (28%). The scheme Risk Factor-Good 
Retail Practices (31%) was used in combina- 
tion with other violation schemes. Of the 23 
agencies that used Risk Factor-Good Retail 
Practices with another scheme, 43% used 
Priority-Priority Foundations-Core, 22% 
used Critical/Noncritical, and 13% used 
Major/Minor schemes. Violation schemes 
for 53 respondents could not be determined 
using online searching. 

Agencies disclosing at the point-of-service 
had lower mean values for all outcome mea- 
sures than did agencies disclosing online 
(Table 2). Of the 24 agencies disclosing 
inspection results at the point-of-service, 
however, 21 (88%) also disclosed inspection 
results online (Table 1). Due to this overlap, 
we made further comparisons of agencies dis- 
closing at the point-of-service and agencies 
disclosing online only (Table 3). Agencies 
that disclosed inspection results at the point- 
of-service reported fewer mean numbers of 
re-inspections by 15%, complaints by 38%, 
outbreaks by 55% (p = .03), and Salmonella 
cases by 12% than did agencies that disclosed 
online only. 

Agencies that used some type of grading 
method for inspection results reported fewer 
mean numbers of re-inspections by 37%, 
complaints by 22%, outbreaks by 61%, and 
Salmonella cases by 25% than did agencies 
that did not grade inspection results. Agen- 
cies using letter grades had lower mean values 
for complaints by 14% and outbreaks by 43% 


January/February 2021 e Journal of Environmental Health 11 


ADVANCEMENT OF THE SCIENCE 


TABLE 4 


Linear Regression Comparisons of Outcomes 


Average # of Average # of 


Complaints/1,000 


Average # of 
Outbreaks/1,000 


Average # of Salmonella 
Cases/100,000 Population 


Re-Inspections/ 


Establishment/Year Establishments/Year Establishments/Year Served/Year 
Parameter | p-Value | # Parameter | p-Value| # Parameter | p-Value| # Parameter | p-Value| # 
Estimate Estimate Estimate Estimate 
(SE) (SE) (SE) (SE) 
Average # of re-inspections/ - 11.49 306 91 | 0.943 (3.44) 784 92 | -0.18 (3.21) .956 44 
establishment/year (11.16) 
Average # of 0.001 .306 91 - 0.058 .079 93 | 0.06 (0.031) .051 48 
complaints/1,000 (0.000995) (0.033) 
establishments/year 
Average # of outbreaks/1,000 | 0.00089 78 92 0.579 .079 93 - 0.40 (0.50) A3 47 
establishments/year (0.00323) (0.326) 
Average # of Salmonella -0.00042 96 44 1.305 .051 48 0.035 43 47 - 
cases/100,000 population (0.0074) (0.652) (0.044) 
served/year 


than agencies using numerical scores, but 5% 
more Salmonella cases (Table 2). Almost one 
third of agencies, however, using numerical 
scores also used letter grades (Table 1). 

Agencies that used a Critical/Noncritical 
violation scheme reported 3% more mean 
complaints but 3% fewer mean re-inspec- 
tions, 27% fewer outbreaks, and 19% fewer 
Salmonella cases than those using Priority- 
Priority Foundations-Core schemes. Agencies 
that used Risk Factor-Good Retail Practices 
schemes tended to have fewer re-inspections 
and complaints but more outbreaks and Sal- 
monella cases than did agencies not using 
these schemes (Table 2). Although most of 
these findings are not statistically different 
from each other, the overall pattern of results 
is noteworthy. 

Regarding associations between outcome 
measures, we observed an almost statistically 
significant relationship between reported 
number of complaints/1,000 establishments/ 
year and number of Salmonella cases/100,000 
population/year. Every unit of increase in 
reported Salmonella cases/100,000 popula- 
tion/year was associated with an increase in 
1.03 complaints/1,000 establishments (p = 
.051) (Table 4). 


Discussion 

The trends observed in this study comple- 
ment the existing literature that supports 
the value of transparency in the disclosure of 


12 Volume 83 * Number 6 


food establishment inspection data. Disclo- 
sure at the point-of-service was associated 
with fewer mean numbers of re-inspections, 
complaints, outbreaks, and Salmonella cases 
than disclosure online only, with a signifi- 
cant difference (p = .03) in the number 
of outbreaks between the two disclosure 
methods. These findings are consistent with 
previous studies in New York City and Los 
Angeles that demonstrated benefits to dis- 
closure at the point-of-service. In this study, 
disclosure at the point-of-service included 
posting of inspection results inside and 
outside of the food establishment. It was 
not the goal of this study to parse the out- 
comes resulting from disclosures of inspec- 
tion results posted inside or outside of food 
establishments. Future studies might be 
warranted to evaluate the effectiveness of 
the nuance of disclosure location at food 
establishments. 

Letter grading methods were associated 
with fewer complaints and outbreaks than 
numerical scoring methods but both meth- 
ods had better outcomes than for inspections 
in the absence of a grading system. The Criti- 
cal/Noncritical inspection violation scheme 
was associated with fewer outbreaks and Sal- 
monella cases than Priority-Priority Founda- 
tions-Core or Risk Factor-Good Retail Prac- 
tices schemes. These results suggest that how 
local agencies conduct and score food estab- 
lishment inspections and disclose results 


to the public likely affect the success of the 
programs to control and prevent foodborne 
illnesses and food safety hazards. 

A strength of this study is that use of the 
Retail Program Standards listserv allowed 
for direct contact and survey dissemination 
to managers or primary contacts of food 
establishment inspection programs. The use 
of this listserv also enabled access to a wide 
geographic range of potential respondents, 
as this program includes agencies from 
all 50 states and Washington, DC, as well 
as five U.S. territories: American Samoa, 
Guam, Northern Mariana Islands, Puerto 
Rico, and the Virgin Islands. Additionally, 
given the variations in inspection practices, 
many survey questions included an open- 
text option for “Other” answers that were 
not listed as potential answer choices. This 
feature allowed for the capture of unique or 
less common practices. 

There are several limitations to this 
study. First, the presence of selection bias 
cannot be understated given the use of a 
convenience sample of survey recipients 
and online recruitment, which limits the 
representativeness of the results to those 
who participated in the FDA Retail Food 
Program. Second, Salmonella cases were 
self-reported. Many inspection agencies do 
not track the number of Salmonella cases, 
as that is typically the duty of epidemiol- 
ogy divisions. As such, the number of cases 


reported by survey respondents might not 
reflect true case counts. Third, missing data 
and an abbreviated collection period weak- 
ened the survey data analysis; the data col- 
lection period was truncated by local and 
state health departments needing to focus 
on the COVID-19 pandemic response. This 
necessity limited the ability to obtain miss- 
ing data points and limited the ability of 
agencies to respond. Fourth, the survey did 
not collect information about the number 
and types of triggers for re-inspection of 
an establishment, which vary across agen- 
cies. A potential confounder might be the 
size of the inspection agency or the num- 
ber of inspectors, as agencies with more 
inspectors or more aggressive practices 
could potentially be able to conduct more 
re-inspections or to detect more violations, 
illnesses, and outbreaks than smaller agen- 
cies. Fifth, the survey did not allow for cap- 
ture of programmatic changes that occurred 
between 2016 and 2018 (e.g., if a jurisdic- 
tion updated its food code during this time). 


"SS 


References 


Although most findings were not statisti- 
cally significant on an individual basis due to 
limitations in sample size, the overall pattern 
of results supports and enhances the existing 
literature on the performance of food estab- 
lishment inspection programs. For example, 
for every unit increase in complaints, there 
was a corresponding increase in the number 
of re-inspections. There was a similar rela- 
tionship with reported foodborne outbreaks. 
Future research should include a larger num- 
ber of agencies by a factor of 2 or 3 to clarify 
several of these relationships. 


Conclusion 

Overall, characteristics of food establishment 
inspection programs appear to be associated 
with foodborne illness and outcomes. These 
results warrant future research efforts to 
improve the effectiveness of these programs. 
This study suggests that agencies that disclose 
at the point-of-service reported 55% fewer 
average number of outbreaks compared with 
those using online disclosure only. Similarly, 


applying a grading scheme as a summary 
measure of inspection results was associated 
with improved foodborne illness outcomes. 
Policy makers should consider these findings 
when evaluating program effectiveness mea- 
sures and when considering changes to exist- 
ing food inspection programs. “39M 


Acknowledgements: This study was funded 
through cooperative agreement 6NU380T 
000300 between the Centers for Disease Con- 
trol and Prevention (CDC) and the National 
Environmental Health Association (NEHA). 
The findings and conclusions are solely the 
responsibility of the authors and do not nec- 
essarily represent the official views of CDC 
and NEHA. 


Corresponding Author: Thuy N. Kim, Divi- 
sion of Environmental Health Sciences, Uni- 
versity of Minnesota School of Public Health, 
420 Delaware Street SE, MMC 807, Minne- 
apolis, MN 55455. 

E-mail: kim00977@umn.edu. 


Centers for Disease Control and Prevention. (2019). Surveillance 
for foodborne disease outbreaks, United States, 2017: Annual report. 
Atlanta, GA: U.S. Department of Health and Human Services, 
Centers for Disease Control and Prevention. https://www.cdc.gov/ 
fdoss/pdf/2017_FoodBorneOutbreaks_508.pdf 

Choi, J., & Scharff, R. (2017). Effect of a publicly accessible disclo- 
sure system on food safety inspection scores in retail and food ser- 
vice establishments. Journal of Food Protection, 80(7), 1188-1192. 

Firestone, MJ., & Hedberg, C.W. (2018). Restaurant inspection let- 
ter grades and Salmonella infections, New York, New York, USA. 
Emerging Infectious Diseases, 24(12), 2164-2168. 

Firestone, MJ., @ Hedberg, C.W. (2020). Consumer interest and 
preferred formats for disclosure of restaurant inspection results, 
Minnesota 2019. Journal of Food Protection, 83(4), 715-721. 

Fleetwood, J. (2019). Scores on doors: Restaurant hygiene ratings 
and public health policy. Journal of Public Health Policy, 40(4), 
410-422. 

Food and Drug Administration. (2015). FDA procedures for standard- 
ization of retail food safety inspection officers: Procedures manual 
updated to the 2013 FDA Food Code and the supplement to the 
2013 Food Code. College Park, MD: Author. https://www.fda.gov/ 
media/94681/download 

Food and Drug Administration. (2019). FDA issues 2019 Voluntary 
National Retail Food Regulatory Program Standards. https://www. 


fda.gov/food/cfsan-constituent-updates/fda-issues-2019-volun 
tary-national-retail-food-regulatory-program-standards 

Fung, A., Graham, M., & Weil, D. (2007). Full disclosure: The perils 
and promise of transparency. Cambridge, UK: Cambridge Univer- 
sity Press. 

Gerding, J.A., Landeen, E., Kelly, K.R., Whitehead, S., Sarisky, J., & 
Brooks, B.W. (2019). Uncovering environmental health: An initial 
assessment of the profession’s health department workforce and 
practice. Journal of Environmental Health, 81(10), 24-33. 

Jones, T.E, & Angulo, FJ. (2006). Eating in restaurants: A risk fac- 
tor for foodborne disease? Clinical Infectious Diseases, 43(10), 
1324-1328. 

National Restaurant Association. (2020). National statistics: Res- 
taurant industry facts at a glance. https://restaurant.org/research/ 
restaurant-statistics/restaurant-industry-facts-at-a-glance 

Simon, PA., Leslie, P, Run, G., Jin, G.Z., Reporter, R., Aguirre, A., & 
Fielding, J.E. (2005). Impact of restaurant hygiene grade cards on 
foodborne-disease hospitalizations in Los Angeles County. Journal 
of Environmental Health, 67(7), 32-36. 

Wong, M.R., McKelvey, W., Ito, K., Schiff, C., Jacobson, J.B., & Kass, 
D. (2015). Impact of a letter-grade program on restaurant sanitary 
conditions and diner behavior in New York City. American Journal 


of Public Health, 105(3), e81-e87. 


Copyright 2021, National Environmental Health Association (www.neha.org) 


January/February 2021 e Journal of Environmental Health 13