ISSN: 2378315X BBIJ
Biometrics & Biostatistics International Journal
Research Article
Volume 2 Issue 2  2014
Combinational Effects of Clinical Area and Healthcare Workers’ Job Type on the Safety Culture in Hospitals
HeonJae Jeong^{1}*, Su Mi Jung^{2}, Eun Ae An^{3}, So Yeon Kim^{4} and Byung Joo Song^{4}
^{1}Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University, USA
^{2}Data Analytics center, Kangwon National University, Korea
^{3}Leadership Development Division, The Catholic Education Foundation, Korea
^{4}Performance Improvement Team, Seoul St. Mary’s Hospital, The Catholic University of Korea, Korea
Received: March 04, 2015  Published: March 17, 2015
*Corresponding author: HeonJae Jeong, Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University, 624 North Broadway, Rm. 455, Baltimore, MD, 21205, USA, Tel: 4109555315; Fax: 4109556959; Email:
Citation: Jeong HJ, Jung SM, An EA, Kim SY, Song BJ (2014) Combinational Effects of Clinical Area and Healthcare Workers’ Job Type on the Safety Culture in Hospitals. Biom Biostat Int J 2(2): 00024. DOI: 10.15406/bbij.2015.02.00024
Abstract
As patient safety is taking center stage in medical care, many resources have been invested to improve safety. Patient safety culture has been one of the most important constituents of safety and, therefore, has been measured with various instruments. One fundamental challenge is that a healthcare worker’s safety attitude can be shaped by multiple factors such as job type and clinical area. Due to the complex organizational and cultural structure in a hospital, such multidimensional dynamics are not thoroughly understood. This study used the crossed random effects model to assess combinational effects of clinical area and job type and calculate the safety attitudes score of each combination with the empirical Bayes (EB) method. We used the Korean version of the Safety Attitudes Questionnaire (SAQK), which consists of six domains: teamwork climate (TC), safety culture (SC), job satisfaction (JS), stress recognition (SR), perception of management (PM) and working conditions (WC). Each clinical area and job type independently affects SAQK scores of all six domains. For TC, SC, SR, and PM, there were combinational effects of clinical area and job type on SAQK scores (e.g., a certain job type in a certain clinical area showed higher or lower SAQK scores than another clinical area, or the other way around). We also applied the EB method to achieve more accurate combinationlevel estimates when the variance in SAQK score of the combination was large, which is a common situation in hospitals with multiple clinical areas and job types.
Keywords: Safety culture; Safety Attitudes Questionnaire; Patient safety; Variance components model; Crossed random effects model; Random effect; Empirical Bayes method
Abbreviations
SAQ: Safety Attitudes Questionnaire; TC: Teamwork Climate; SC: Safety Climate, JS: Job Satisfaction, SR: Stress Recognition, PM: Perception of Management; WC: Working Conditions; EB: Empirical Bayes; LR : LikelihoodRatio; HCWs: Healthcare Workers
Introduction
This article is the second in a series of articles on methodology to analyze patient safety culture among healthcare workers (HCWs). In the previous article, we developed and validated the Korean version of the Safety Attitude Questionnaire (SAQK) and proposed a method to estimate SAQK scores for clinical area more accurately than traditional SAQ analysis methods [1]. Here, we expand the scope to devise methods to explore combinational effects of different characteristics, clinical area and job type.
In any industry with the risk of accidents, safety culture and climate among the workers are considered the most important drivers for safety improvement initiatives; this environment provides momentum for workers to adhere to the safety rules and better collaborate to build a highly reliable organization [25]. Health care is no exception and many resources have been invested to improve and manage safety culture. The first and foremost step was to measure the culture, and thus various instruments were developed to do so [68]. Among those instruments, the Safety Attitudes Questionnaire (SAQ) is one of the most popular and thoroughly validated instruments, consisting of six distinct domains: teamwork climate (TC), safety climate (SC), job satisfaction (JS), stress recognition (SR), perception of management (PM), and working conditions (WC) [9]. Since it was developed, SAQ has been translated into various languages and used in many countries [1015]; recently, the SAQK was developed to enable international comparison of safety profiles between Korea and other countries [1].
To date, the SAQ has been used mostly to show clinical areaspecific scores and compare them across hospital (crosssectional) or temporal changes [16]. Though less frequently used, job typespecific scores have also been of interest in a few studies [14]. Both approaches are meaningful in that clinical area naturally determines the culture in which an HCW is submerged, and job type also can influence safety attitudes since it reflects how one was trained and of what professional society one is a member, which can influence the power gradient across different job types [17]. However, few studies have explored the effects of those two factors simultaneously, especially their interaction in influencing safety culture. The difficulties arise because most healthcare organizations have multiple, sometimes tens of, clinical areas and various job types. Traditional models cannot address the complexity between clinical area and job type, which might easily exceed hundreds of combinations. In addition, huge variation exists in the number of HCWs in each combination, which makes the dataset highly unbalanced, frequently with unequal variances. Indeed, many combinations have only one or two HCWs and, thus, para meterizing the effects of all these combinations is neither practical nor efficient.
Therefore, we applied the crossed random effects model with an interaction term between clinical area and job type. With this model, we can effectively control the issues of too many combinations, as well as the problem from a heavily unbalanced data structure [1821].
Another advantage of using the random effects model is that it allows for applying the empirical Bayes (EB) method to obtain more accurate estimates of SAQ scores from each clinical area and job type combination [1]. The EB method operates by letting areas and job types with large variance that is typically caused by small sample size borrow information from areas and job types with small variance [2124]. The theoretical background of Bayes method is to use prior distribution as a starting point, and update the distribution with the actual observations. For example, if we want to obtain a Bayesian estimate of one domain score of SAQK in a certain cluster (e.g., clinical area and job type), we first assume a prior distribution of the cluster level random effect across the hospital, and apply the specific information from the cluster to the prior distribution. Thus, Bayesian estimates are natural compromise between prior distribution over clusters and clusterspecific scores, and larger the variance of a clusterspecific score, the more weight is assigned to the prior distribution. Therefore, choosing an appropriate distribution is probably one of the most important steps for the Bayesian estimation process, requiring much experience from a researcher [25]. On the other hand, EB method does not force one to set a prior distribution; rather, it utilizes data of interest to generate prior distribution of the cluster random effect first, and pretends that data has not been used yet [21]. With prior distribution obtained, the same steps described above can be used to obtain more accurate clusterspecific estimates. This is why this method is called the empirical Bayes. In this particular study, EB not only provided accurate estimates, but also served as a convenient tool to estimate the SAQ score of each combination [1].
Therefore, this study was conducted with three goals:
 Test whether SAQ scores vary significantly over clinical area and job type,
 If so, test whether there is a combinational effect of clinical area and job type on SAQ scores, and
 Apply the EB method to obtain the SAQ score of each combination.
Methods
Because this study shares the dataset from the previous SAQK series article [1], the detailed process of survey development and validation is not described in this article. However, information regarding survey respondents and basic data structure is essential to describe this study, and therefore they are depicted in the first parts of the methods and results sections.
Administering the Korean Version of Safety Attitude Questionnaire
To measure safety culture by clinical area and job type, we utilized the Korean version of the Safety Attitude Questionnaire. The SAQK was developed and validated with permission of the original SAQ developers and composed of 34 items in 6 domains TC (5 items), SC (6 items), JS (5 items), SR (4 items), PM (10 items), and WC (4 items) and demographic information, including the respondent’s clinical area and job type.
The SAQK was administered anonymously to healthcare workers in a large metropolitan hospital in Seoul from October 2013 through November 2013. Two options of modality were offered for respondents’ convenience in administering the SAQK: paper and pencil survey and electronic survey via the hospital’s intranet.
A 5point Like rt scale (1 = Disagree Strongly, 2 = Disagree Slightly, 3 = Neutral, 4 = Agree Slightly, 5 = Agree Strongly) was used to measure SAQK items and the scores were then converted into 0 to 100 scales as the original SAQ developers recommended. Domain scores were obtained for each respondent by calculating the arithmetic mean score of each domain.
Individual Effect of Clinical Area and Job Type on SAQ Scores
The following modelbuilding steps are applied to each of the six SAQK domains because differences across domain scores are outside the scope of this study. In this section, we explored whether each of the clinical area and job type variables explains the variance of SAQK scores with statistical significance. First, we began with a simplest model for a domain score, ${y}_{k}$
of person $k$
.
${y}_{k}=\beta +{\epsilon}_{k}$
$\beta $
is the overall hospital mean of the SAQK scores of all survey participants. ${\epsilon}_{k}$
is the random deviation of the SAQK score of a person $k$
from the overall mean $\beta $
.
Here, ${\epsilon}_{k}$
is assumed to follow a normal distribution with a mean of 0 and variance ${\varphi}^{2}$
. Then we developed a variance components model by adding clinical area effects and obtained the following model to describe the SAQK score, ${y}_{ik}$
of person $\kappa $
, working in clinical area $i$
,
${y}_{ik}=\beta +{\zeta}_{i}+{\epsilon}_{ik}$
where $\beta $
is the overall mean of SAQ scores for a certain domain and ${\zeta}_{i}$
is a random deviation of the mean SAQ score for clinical area $i$
from $\beta $
. The random effect of clinical areas ${\zeta}_{i},$
is assumed to be normally distributed, having a mean of 0 and variance ${\tau}_{1}{}^{2}$
, and to be independent over clinical areas. Therefore, ${\epsilon}_{i}{}_{k}$
is the random deviation of ${y}_{ik}$
from the mean of clinical area $i$
, that is, the residual within the clinical area. ${\epsilon}_{i}{}_{k}$
is also assumed to follow a normal distribution with a mean of 0 and variance ${\sigma}^{2}$
and be independent of both clinical areas and survey respondents [21].
We then developed another variance components model with job type $j$
as the random effect instead of clinical area $i$
,
${y}_{jk}=\beta +{\eta}_{j}+{\epsilon}_{jk}$
where all the variables and their assumptions are equivalent to those of model 1.2, except that they address job type rather than clinical area. To be specific, ${\eta}_{j}$
, the main variable of interest; random deviation of the mean SAQ scores of job type $j$
from $\beta $
is assumed to follow a normal distribution with 0 mean and ${\tau}_{2}{}^{2}$
variance.
At this point, we tested whether clinical area and job type significantly influence the SAQ domain scores. For clinical area, the likelihoodratio (LR) test was conducted to test the null hypothesis, H0: ${\tau}_{1}{}^{2}$
= 0, by comparing model 1.2 to model 1.1. For job type, the null hypothesis, H0: ${\tau}_{2}{}^{2}$
= 0, with the LR test, was used to compare model 1.3 to model 1.1. Since ${\tau}_{1}{}^{2}$
and ${\tau}_{2}{}^{2}$
cannot have negative values, the pvalues obtained from the LR tests had to be divided by 2 [21]. All pvalues in this article are those that were derived earlier.
Combinational Effects of Clinical Area and Job Type on SAQ Scores
We built a crossed random effects model for each SAQ domain. First, we developed a model that includes both clinical area random effect and job type random effect. In this model, neither clinical area nor job type is nested in the other. In other words, we did not assume any hierarchy between clinical area and job type, and thus the following model is an additive crossed random effects model:
${y}_{ijk}=\beta +{\zeta}_{i}+{\eta}_{j}+{\epsilon}_{ijk}$
where ${y}_{ijk}$
is the SAQ domain score of a person $k$
, whose job type was $j$
, working in clinical area $i$
. Again, $\beta $
is the overall mean of the SAQ domain score of all respondents, and ${\zeta}_{i}$
and ${\eta}_{j}$
are the random deviation of the mean SAQ scores of clinical area $i$
and job type $j$
around $\beta $
, respectively. As in models 1.2 and 1.3, ${\zeta}_{i}$
and ${\eta}_{j}$
are assumed to follow normal distributions with a mean of 0 and variance of ${\tau}_{1}^{2}$
and ${\tau}_{2}^{2}$
, respectively. Therefore, ${\epsilon}_{ijk}$
is the random deviation of ${y}_{ijk}$
from the sum of $\beta $
, ${\zeta}_{i}$
and ${\eta}_{j}$
, and is assumed to be normally distributed with a mean of 0 and variance ${\sigma}^{2}$
. Here, ${\sigma}^{2}$
reflects not only the variability among HCWs with a certain job type working in the same clinical area, but also interactions between clinical area and job type, that is, deviations of the mean SAQ scores for the combinations of clinical area and job type from the means that are implied in the additive manner of the random effects [21].
Then, we added the random interaction term ${\kappa}_{ij}$
. Here, the interaction means that the SAQ scores of clinical area and job type are correlated; for example, nurses in a cardiac intensive care unit responded more positively to SAQ than nurses in other clinical areas. The rationale of having this variable was that several combinations of clinical area and job type included multiple HCWs. Since ${\kappa}_{ij}$
can have different values for different combinations of clinical area $i$
and job type $j$
, this model actually relaxed the additive assumption of model 2.1 and allowed us to quantify the combinational effects [21]:
${y}_{ijk}=\beta +{\zeta}_{i}+{\eta}_{j}+{\kappa}_{ij}+{\epsilon}_{ijk}$
where all the variables and their assumptions are the same as in model 2.1. The random interaction term, ${\kappa}_{ij}$
follows normal distribution with a 0 mean and variance, ${\tau}_{3}{}^{2}$
. ${\kappa}_{ij}$
is assumed to be independent of the other random terms, ${\zeta}_{i}$
, and ${\eta}_{j}$
over combinations of clinical area and job type [21].
Through an LR test comparing model 2.2 to model 2.1, we can test the null hypothesis H0: ${\tau}_{3}{}^{2}$
= 0. If the LR test rejects the null hypothesis, then there were combinational effects of clinical area and job type on SAQ scores. If not, plugging in the interaction term does not add any value, and therefore we would just return to model 2.1. The pvalues of the LR test results in this article were adjusted to test on the boundary of parameter space, as indicated in an earlier section [21].
CombinationSpecific Empirical Bayes Estimation of SAQ Scores
From model 2.2, we used the EB method to achieve the estimations of random effects, ${\zeta}_{i}$
, ${\eta}_{j}$
and ${\kappa}_{ij}$
for each combination and, by summing them, we obtained combinationspecific random deviation from the overall mean SAQ domain score. If the interaction term was not statistically significant, we returned to model 2.1 and used only ${\zeta}_{i}$
and ${\eta}_{j}$
to obtain the deviation. By adding the overall mean, $\beta $
, to the random deviation, we obtained the SAQ score of each combination.
For all the analyses, Stata 13.1 (Stata Corp, College Station, Texas) was used.
Results
Characteristics of Respondents
A total of 1,381 questionnaires was returned. After excluding those missing clinical area and job type variables, 1,142 questionnaires were analyzed. As depicted in Table 1, 73.7% of the survey respondents was female. Regarding work experience, HCWs with 510 years of experience formed the largest portion (25.4%) of the respondents, followed by HCWs with 34 years of experience (21.8%). There were 16 job types among the respondents. Nurses were 53.3% and physicians were the next largest group. Among physicians, residents accounted for 14.5%, followed by senior physicians (9.9%), clinical instructors (5.6%), and interns (3.1%). Additional personnel included radiology technologists (4.5%), medical laboratory technologists (1.8%), and clinical supporting staff (3.5%). Since almost all nurses in Korea are female, nurses comprised more than half of the respondents.
Characteristics 
N 
% 
Gender 


Male 
300 
26.3 
Female 
842 
73.7 
Work years 


Less than 6 months 
77 
6.7 
7 ~ 11 months 
122 
10.7 
1 ~ 2years 
193 
16.9 
3 ~ 4 years 
249 
21.8 
5 ~ 10 years 
290 
25.4 
11 ~ 20 years 
150 
13.1 
Over 21 years 
61 
5.3 
Job type 


Nurses 
609 
53.3 
Residents 
166 
14.5 
Senior physicians 
113 
9.9 
Clinical instructors 
64 
5.6 
Radiologic technologists 
51 
4.5 
Clinical supporting staff 
40 
3.5 
Interns 
35 
3.1 
Medical laboratory technologists 
20 
1.8 
Pharmacists 
10 
0.9 
Dental hygienists 
10 
0.9 
Physical therapists 
6 
0.5 
Administration 
5 
0.4 
Hospital officers 
4 
0.4 
Other 
4 
0.4 
Nutritionists 
3 
0.3 
Medical record officers 
2 
0.2 
Total 
1,142 
100.0 
Table 1: Characteristics of Respondents.
Although not reflected in Table 1, the hospital had 72 clinical areas. To briefly introduce the dispersion of HCWs by clinical area, we summarized the data in stemandleaf plots, as seen in Figure 1. The numbers of HCWs varied substantially over the 72 clinical areas, ranging from 2 to 53 HCWs. We do not show specific names of clinical areas because they were not of interest in this study.
Figure 1: Stemandleaf plots of the number of respondents by clinical area.
Table 2 depicts the combinations of clinical area and job type. To illustrate, 12 clinical areas had only one job type working, 12 clinical areas had two different job types, and 19 clinical areas had three job types working. Two areas had nine job types, which was the largest number of job types in one unit. There were 243 different combinations between clinical area and job type. Although not shown in Table 2, the largest combination had 34 HCWs.
Number of job types
in a clinical area (a) 
Number of
clinical areas (b) 
Number of clinical area and job type combinations (a x b) 
1 
12 
12 
2 
12 
24 
3 
19 
57 
4 
14 
56 
5 
5 
25 
6 
6 
36 
7 
1 
7 
8 
1 
8 
9 
2 
18 
Total 
72 
243 
Table 2: Number of clinical area and job type combinations.
Internal Consistency and Construct Validity
We calculated Cronbach’s alpha to check internal consistency, which was 0.836 for TC, 0.841 for SC, 0.907 for JS, 0.734 for SR, 0.928 for PM, and 0.758 for WC. The results of confirmatory factor analysis suggested good model fit [26]: goodness of fit index (0.911), adjusted goodness of fit index (0.894), normed fit index (0.924), comparative fit index (0.944), and root mean square error of approximation (0.044).
Effects of Clinical Area and Job Type on SAQK Domain Scores
Table 3 contains the results from the first three models. For model 1.1, the naked model without random effects, the overall means ($\beta $
) and standard deviations ( $\varphi $
) of the six SAQK domain scores are listed. For model 1.2, the overall means ( $\beta $
), standard deviations ( ${\tau}_{1}$
) of the clinical area random effect around the overall mean, and standard deviation ( $\sigma $
) of residuals around clinical areaspecific mean for six SAQK domains are listed. Model 1.3 has the same structure as model 1.2, except for having a standard deviation ( ${\tau}_{2}$
) of job type, instead of clinical area, random effects. LR test results are also described as superscripts beside random effects standard deviations.
The means ( $\beta $
s) of the three models were similar, ranging from 55.23 (WC in model 1.1) to 68.84 (SR in model 1.2). Note that the mean of JS was much different between the model with job type random effect (65.15 in model 1.3) and the other two models (58.55 in model 1.1 and 59.52 in model 1.2). For both model 1.2 and 1.3, all the random effects for the six domains were statistically significant from LR tests. JS showed the largest standard deviation of clinical area random effects, ${\tau}_{1}$
, at 7.84, and SR showed the smallest ${\tau}_{1}$
at 3.29. The standard deviation of job type random effects, ${\tau}_{2}$
, was the largest in JS (6.54) and the smallest in PM (1.54).

TC 
SC 
JS 
SR 
PM 
WC 
Model 1.1 
$\beta $

64.86 
65.32 
58.55 
68.66 
61.39 
55.23 
$\varphi $

17.60 
16.47 
21.26 
18.51 
17.23 
16.70 
Model 1.2 
$\beta $

65.60 
65.65 
59.52 
68.84 
61.78 
55.65 
${\tau}_{1}$

5.35* 
3.74* 
7.84* 
3.29* 
3.79* 
3.88* 
${\tau}_{2}$

16.85 
16.03 
20.00 
18.19 
16.78 
16.27 
Model 1.3 
$\beta $

66.99 
66.20 
65.15 
65.22 
61.20 
56.10 
${\tau}_{2}$

3.23* 
2.88* 
6.54* 
4.10* 
1.54* 
1.95* 
$\sigma $

17.31 
16.33 
20.27 
18.10 
17.16 
16.60 
Table 3: SAQK domain scores and variance components model parameters.
* : p<0.05 from the LR tests comparing the model to model 1.1
Table 4 describes parameters from the crossed random effects models. The means ( $\beta $
s) of the two models were similar. The main interest of these crossed models was the standard deviation ( ${\tau}_{3}$
) of the random interaction term in model 2.2. For the TC, SC, SR, and PM domains, the random interactions were statistically significant, suggesting that certain clinical areas were related to the SAQ domain scores of certain job types, or the other way around. The JS and WC domains showed the smallest standard deviations ( ${\tau}_{3}$
) of the random interaction term (JS: 2.95 and WC: 2.42) and these were not statistically significant.

TC 
SC 
JS 
SR 
PM 
WC 
Model 2.1 
$\beta $

67.59 
66.50 
64.72 
65.43 
61.46 
56.23 
${\tau}_{1}$

3.47 
3.97 
5.20 
2.86 
3.97 
3.57 
${\tau}_{2}$

4.73 
3.50 
6.15 
4.06 
1.88 
1.66 
$\sigma $
σσ 
16.67 
15.83 
19.70 
17.87 
16.68 
16.25 
Model 2.2 
$\beta $

67.62 
66.37 
64.84 
65.39 
61.50 
56.24 
${\tau}_{1}$

3.64 
2.63 
4.60 
1.07 
3.16 
3.20 
${\tau}_{2}$

3.41 
2.86 
6.21 
3.94 
1.59 
1.69 
${\tau}_{3}$

4.96* 
4.50* 
2.95 
5.52* 
3.79* 
2.42 
$\sigma $

16.30 
15.54 
19.61 
17.37 
16.46 
16.46 
Table 4: Crossed random effects model parameters.
* : p<0.05 from the LR tests comparing model 2.2 to model 2.1
Empirical Bayes Estimation of CombinationSpecific SAQK Scores
To obtain estimates of SAQK scores specific to clinical area and job type combinations, we used the empirical Bayes method. For the TC, SC, SR, and PM domains, model 2.2 was used since the random interaction term significantly affected SAQK scores. For the JS and WC domains where interaction was not significant in LR tests, we returned to model 2.1. Since showing specific names of clinical area and job type was not the purpose of this study, we depicted the combinationspecific SAQ domain scores without identifying clinical area or job type in Figure 2. The yaxis on the left side is the SAQK domain score and the right side is the deviation of the combinationspecific SAQK score from the domain means score.
Figure 2: Combinationspecific SAQK score (EB) and its deviation from the mean for all six SAQK domains.
Note: x axis: combination of clinical area and job type; y axis (left): SAQK domain score; y axis (right): deviation of SAQK score from mean.
Discussion
The primary aim of this study was to examine whether effects in SAQK scores arise from different combinations of clinical area and job type. To pursue this aim, we applied a crossed random effects model, known as the twoway error components model, with a random interaction term and tested whether the interaction existed for each of the six SAQK domains.
One might wonder why we did not use ordinary least squares (OLS) regressions or the fixed effects (FE) model on existing combinations of clinical area and job typethis particular study had 243 combinationsrather than applying random effects. Actually, OLS and FE models are computationally simple, and they might provide more intuitive combinationspecific scores. However, those models cannot statistically prove or disprove whether clinical area and job type affect the SAQK scores in conjunction. That is, we had to parameterize the dispersion of SAQK scores over clinical area and job type, and then we could decide whether there was an interaction between them.
In model 2.1, we had two random effects, clinical area and job type, with the assumption that they were not correlated; that is why we call it an additive model. This was a strong assumption, and anyone who has experience in HCWs’ working environment might say the assumption cannot hold. Actually, that was the strategy of this study: If there is correlation between them, the estimators of their random effects might be incorrect, and the correlation is reflected in the residuals of the model. Then we added the interaction term that will take up the correlation absorbed in the residual. If there was significant interaction, the latter model yielded better likelihood. If there was no significant interaction, then we discarded the interaction term and returned to the additive model.
Also, as a byproduct of this random effects approach, we could easily calculate EB estimates of SAQ scores for each of the 243 clinical areajob type combinations. The EB predictor is the best linear unbiased predictor (BLUP) and is particularly useful in the hospital of this study where the SAQK was administered. A total of 1,142 HCWs responded to the survey, which means that each combination had only around 4.7 HWCs on average. Considering the largest combination had 34 HCWs, many combinations contained fewer than 4.7 HWCs, even 1 or 2. For those small combinations, simple calculation of the combinationspecific mean does not provide much information, especially for those who determine resource allocation for safety improvement programs in a hospital. EB methods can improve the accuracy of combinationspecific estimates by allowing small combinations to borrow strengths from other combinations that have larger sample size and small standard deviation [21,23,24,27].
Other model structures could have multiple random effects variables, such as clinical area job type. For example, we could build a threelevel hierarchical model where job type is nested under clinical area, or the other way around. Though we explored those models, we returned to the current crossed random effects model. Clinical area is most obviously the key clustering variable regarding safety culture [1] since HCWs in one area work together and communicate every day. Job type plays a role in shaping culture as much as physical work area; there are many training and education programs and sessions for specific job types, letting them share similar attitudes on safety. In this case, structuring clinical area and job type in a matrix format would be a more reasonable approach and modeling such matrix as a statistical equation is the crossed random effects model.
With regard to data structure, note that 609 nurses comprised more than half of the survey respondents. Unlike physician groups, such as residents and interns, which were measured with higher resolution, all 609 nurses were measured as one group. This was because there were only a few nurses at a higher position and almost all the nurses perform the same work. Though their actual roles might vary and this could affect their SAQK scores, such difference can be captured in the clinical area variable or the interaction term, which is the rationale of this study. With respect to analysis, the only concern arising from a huge nurses group is that the number of clusters could be small, which could lead to difficulty in assigning value to the random parameter. However, in this particular study, 16 different job types existed and, therefore, no significant problem arose in plugging in the job type as a random effect [21].
The overall means from all models were similar for all domains, with the exception of the JS domain. In model 1.1, the simplest model without random effects, and model 1.2, the model with clinical area as the random effect, the mean was 58.55 and 59.52, respectively. However, the rest of the models, all of which contain job type as random effects, the means were around 65. This is because the overall mean is estimated inversely proportional to cluster variances in the random effects model. If a small cluster has small withincluster variance compared to betweencluster variance, then the cluster can have a weight similar to the large cluster. Thus, compared to the ordinary least squares model, the random effects model is likely to put more weight on small clusters [21]. In this study, job type might have caused such effect in the JS domain. Though this phenomenon is not the focus of this study, it is worthy of examination in a future study, especially regarding SAQ score differences across domains.
We used the LR test as the primary method to test the significance of random effects for each model. Basically, the LR test compares one model to another, and it is especially efficient when a model is built on top of another, which is the case of this study. However, note that several inferential methods can be applied other than the LR test, such as pseudo quasilikelihood, also known as joint maximization methods [28,29]. Especially when multiple covariates are added, stepbystep comparison with LR would not be efficient. Future studies including various respondent characteristics and areaspecific or job typespecific information should take advantage of such other inferential strategies.
Although we built crossed random effects models by carefully addressing both mathematical challenges and realworld situations, there were some issues that the structure of data in this study could not clearly handle. First and foremost, it is difficult to establish whether an HCW is working in more than one clinical area. To illustrate, if a surgeon is working in both an operating room and a surgical intensive care unit, to which clinical area should his or her responses be assigned? To date, we have forced these respondents to choose one clinical area and regarded that respondent as working in a single clinical area, even thought is not true. The methodology used in this study can be applied to resolve the issue by asking respondents to identify their primary clinical area and their secondary clinical area. Then we can apply the crossed random effects model with those two areas exactly as we did with one clinical area and one job type. In that situation, the surgeon’s response can be thought of as shaped by the safety attitudes of both the operating room and the intensive care unit. With this approach, we can develop a much more detailed map that shows the topography of safety culture.
Conclusion
Following up the previous SAQK article that investigated the effects of clinical areas on SAQK score distribution [1], we examined how job type influenced the SAQK in this study. We showed that not only did clinical area and job type affect SAQK scores independently, but they also interact and affect HCWs’ safety attitudes. In addition to the results of this study, the methodology that we devised can help healthcare organizations better understand their safety culture, on which they can develop more effective patient safety improvement programs. We hope this study assists in hospitals’ relentless endeavor to save lives.
References
 HeonJae Jeong, Su Mi Jung, Eun Ae An, So Yeon Kim, Hyeyeon Yoon, et al. (2015) Development of the Safety Attitudes Questionnaire  Korean Version (SAQK) and Its Novel Analysis Methods for Safety Managers. Biom Biostat Int 2(1): 00020.
 Zhang H, Wiegmann DA, Von Thaden TL, Sharma G, Mitchell AA (2002) Safety Culture: A Concept in Chaos? Proceedings of the Human Factors and Ergonomics Society Annual Meeting 46(15): 14041408.
 Chassin MR, Loeb JM (2013) Highâ€Reliability Health Care: Getting There from Here. Milbank Q 91(3): 459490.
 Von Thaden TL, Wiegmann DA, Mitchell AA, Sharma G, Zhang H (2003) Safety culture in a regional airline: Results from a commercial aviation safety survey. Paper presented at: 12th International Symposium on Aviation Psychology, Dayton, OH, 2003.
 Reason J (2000) Safety paradoxes and safety culture. Injury Control and Safety Promotion 7(1): 314.
 Morello RT, Lowthian JA, Barker AL, McGinnes R, Dunt D, et al. (2013) Strategies for improving patient safety culture in hospitals: a systematic review. BMJ Qual Saf 22(1): 1118.
 Etchegaray JM, Thomas EJ (2012) Comparing two safety culture surveys: safety attitudes questionnaire and hospital survey on patient safety. BMJ Qual Saf 21(6): 490498.
 Colla J, Bracken A, Kinney L, Weeks W (2005) Measuring patient safety climate: a review of surveys. Qual Saf Health Care 14(5): 364366.
 Sexton J, Helmreich R, Neilands T, Rowan K, Vella K, et al. (2006) The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res 6(1): 44.
 De Carvalho RE, Cassiani SH (2012) Crosscultural adaptation of the Safety Attitudes QuestionnaireShort Form 2006 for Brazil. Rev Lat Am Enfermagem 20(3): 575582.
 Zimmermann N, Kung K, Sereika SM, Engberg S, Sexton B, et al. (2013) Assessing the safety attitudes questionnaire (SAQ), German language version in Swiss university hospitalsa validation study. BMC Health Serv Res 13(1): 347.
 NordenHagg A, Sexton JB, KalvemarkSporrong S, Ring L, KettisLindblad A (2010) Assessing Safety Culture in Pharmacies: The psychometric validation of the Safety Attitudes Questionnaire (SAQ) in a national sample of community pharmacies in Sweden. BMC Clin Pharmacol 10(1): 8.
 Deilkas ET, Hofoss D (2008) Psychometric properties of the Norwegian version of the Safety Attitudes Questionnaire (SAQ), generic version (short form 2006). BMC Health Serv Res 8(1): 191.
 Lee WC, Wung HY, Liao HH, Lo CM, Chang FL, et al. (2010) Hospital safety culture in Taiwan: a nationwide survey using Chinese version safety attitude questionnaire. BMC Health Serv Res 10(1): 234.
 Raftopoulos V, Pavlakis A (2013) Safety climate in 5 intensive care units: A nationwide hospital survey using the GreekCypriot version of the Safety Attitudes Questionnaire. J Crit Care 28(1): 5161.
 Watts BV, Percarpio K, West P, Mills PD (2010) Use of the safety attitudes questionnaire as a measure in patient safety improvement. J Patient Saf 6(4): 206209.
 Jeong HJ, Pham JC, Kim M, Engineer C, Pronovost PJ (2012) Major culturalcompatibility complex: considerations on crosscultural dissemination of patient safety programmes. BMJ Qual Saf 21(7): 612615.
 Raudenbush SW (1993) A crossed random effects model for unbalanced data with applications in crosssectional and longitudinal research. Journal of Educational Statistics 18(4): 321349.
 Baayen RH, Davidson DJ, Bates DM (2008) Mixedeffects modeling with crossed random effects for subjects and items. Journal of memory and language 59(4): 390412.
 McCulloch CE, Neuhaus JM (2001) Generalized linear mixed models. Wiley Online Library.
 RabeHesketh S, Skrondal A (2008) Multilevel and Longitudinal Modeling Using Stata. (2nd Edn) College Station, Stata Press, Texas, USA.
 Militino A, Ugarte M, Goicoa T (2007) A BLUP synthetic versus an EBLUP estimator: an empirical study of a small area estimation problem. Journal of Applied Statistics 34(2): 153165.
 Berger J (1983) Parametric Empirical Bayes Inference: Theory and Applications: Comment. Journal of the American Statistical Association 78(381): 5557.
 Carlin BP, Louis TA (1997) Bayes and empirical Bayes methods for data analysis. Statistics and Computing 7(2): 153154.
 Lynch SM (2007) Introduction to applied Bayesian statistics and estimation for social scientists. Springer Science & Business Media.
 Hair JF, Tatham RL, Anderson RE, Black W (2006) Multivariate data analysis. (6th Edn), Pearson Prentice Hall Upper Saddle River, New Jersey, USA.
 Morris CN (1983) Parametric empirical Bayes inference: theory and applications. Journal of the American Statistical Association 78(381): 4755.
 Breslow NE, Clayton DG (1993) Approximate inference in generalized linear mixed models. Journal of the American Statistical Association 88(421): 925.
 Bellio R, Varin C (2005) A pairwise likelihood approach to generalized linear models with crossed random effects. Statistical Modelling 5(3): 217227.

