Evaluation of User Satisfaction in Using the Emergency Department Information System (EDIS) from User’s Perspective ^†

Abstract

The service industry depends on Internet development that always changes. The changing service environment provides a range of opportunities to develop new information technology to communicate with customers and add new channels to reach all service providers. Hospital administration pays attention to improving medical service quality and using information systems for clinical care. A nursing information system is the most important part of the whole medical care information system. The aim of this study is to explore the key factors of user interface (UI) and the function of the emergency department information system (EDIS) from the viewpoint of the user’s operation. The questionnaire design is based on literature reviews and invited academic and industry scholars to conduct a pre-test for the validity of the content. The staff of a regional teaching hospital in central Taiwan were taken as the research subjects The study was based on the information system success model proposed by DeLong and McLean in 2003. In order to understand the use of information systems in hospitals, the TAM (technology acceptance model) is used to investigate the factors that influence the acceptance of new technology. A survey was conducted on the employees who are using the EDIS at present. There are four categories of participants: attending physicians, nurses, specialist nurses, and secretaries/registers. The results indicate that “user satisfaction” affects the success of the EDIS. For the question, “emergency medical information system has more benefits than traditional paper”, respondents gave good marks. For the question, “the stability of the emergency medical information system is very satisfactory“ was given the lowest. The aspects of the information system success model for the terms “system quality”, “information quality”, “service quality”, etc. were analyzed. User satisfaction is positively related, although user education is different. The reference data allow hospital administrators to consider the needs of users while developing or conversing about new systems for the future.

1. Introduction

The measurement of success in an introduced system relies on many indicators. User satisfaction is an important indicator of information system success. The introduction of the emergency information system in clinical practice not only simplifies the work and the procedure of clinical care for medical personnel, but it also improves work productivity, reduces paperwork and workload, provides real-time information, and even increases the accuracy of medical orders; thus, it effectively boosts the level of satisfaction in patients and medical staff.

The four aspects of this study are as follows:

• To investigate the impact of the emergency information system on users.
• To investigate the overall impact of user satisfaction on the company.
• To review the relationship between the impact of emergency information systems and the impact on user satisfaction including their influences.
• To find out the factors of users’ relationships with the emergency information system.

2. Related Studies

2.1. Influential Factors of Satisfaction to the Emergency Information System

Based on the individual case study by Zhang, she found cognitive awareness of system characteristics, quality of informative product, cognition of user environment, user knowledge, and professionalism of the information personnel were the relevant factors for affecting the user satisfaction of an information system [1]. In 2005, Smith, Smith, Krugman, and Oman suggested a focus group of nursing staff for improvement, so that users’ attitudes could be turned in a more positive direction [2].

2.2. Information System Success Model (ISSM)

To assess the effectiveness of an information system, the most frequently discussed model of system quality and user satisfaction is the information system success model (ISSM). DeLone and McLean proposed an information system success model (ISSM) in 1992 based on an empirical study which includes system quality (for its usability, usefulness, and portability), information quality (for accuracy, timeliness, and integrity), service quality (for personnel properly understanding, helping, and solving users’ problems), user willingness and satisfaction (for the level of assistance by the system in completing an individual task), and net benefit (to see if the system has improved work performance and saved decision time) [3].

DeLone and McLean further modified their proposed information model in accordance with the advancement of information technology and electronic commerce in 2003 by adding a new construct of service quality and suggesting “user intention” as the representative aspect. Moreover, “net benefits” would replace and combine “individual impact” and “organizational impact” into one single construct by representing the impact of the information system in terms of its overall influence or benefit [4].

2.3. Technology Acceptance Model (TAM)

The Technology Acceptance Model (TAM) is a currently popular approach to studying the acceptance of certain technologies by users, such as evaluating the acceptance of a newly introduced medical technology by medical caregivers. It is by far the most common research model in foreign studies of technology acceptance [5]. Individuals will affect one’s attitude (A) toward an event and such attitudes determine the cognitive behavioral control. TAM can be used in the domestic medical industry. Similarly, perceived usefulness (PU) and ease of use (PEOU) continue to serve as a directing force on the user’s will (AU) and intention (BI). The order of influence is, first, the perceived usefulness, then followed by perceived ease of use and satisfaction.

3. Methods

3.1. Research Framework

In this study, 130 medical staff in a teaching hospital in the central region of Taiwan were the subjects. The subjects worked in four different departments, used the emergency information system, and received the questionnaire. They were, respectively, attending physicians, nurses, nurse practitioners, registrars, and cashier who actually use the system and these questionnaires would hopefully gather information for the purpose of this study. The initial draft of the questionnaire was discussed, designed, and revised by various experts and professors of different fields advised on the expert validity. The pre-test was first completed by nurses in the emergency department who would answer the questionnaire as a pilot test before the actual official survey began. In order to examine the effect of the success of the information system in the emergency room, a research model is developed in Figure 1.

3.2. Research Hypothesis

Based on the purpose of study and statistics in the literature, we propose:

Hypothesis 1.

System quality positively affects the information system success model and user satisfaction.

Hypothesis 2.

Information quality positively affects the information system success model and user satisfaction.

Hypothesis 3.

Service quality positively affects the information system success model and user satisfaction.

Hypothesis 4.

Net benefits positively affect the information system success model and user satisfaction.

Hypothesis 5.

Subjective usability positively affects the technology acceptance model and user satisfaction.

Hypothesis 6.

Subjective ease of use positively affects the technology acceptance model and user satisfaction.

Hypothesis 7.

User attitude positively affects the technology acceptance model and user satisfaction.

Hypothesis 8.

User intention positively affects the technology acceptance model and user satisfaction.

Hypothesis 9.

Actual usage positively affects the technology acceptance model and user satisfaction.

A total of valid surveys were collected after removing the survey lack of information, not match or empty. The SPSS software version 20 (IBM Corp. Released 2011. IBM SPSS Statistics for Windows, Version 20.0. Armonk, NY, USA: IBM Corp.) is used to perform the analysis.

4. Data Analysis

In this survey, we only considered participants who had experience in using the emergency department information system (EDIS). For the verification of reliability analysis and substantive relationship, the method of correlation variance analysis and multiple regression analysis were adopted, with the results explained as follows.

4.1. Reliability Analysis

In the pilot study and according to George and Mallery (2003), the Cronbach’s alpha (α) coefficient is considered as the most frequently used estimate of internal consistency reliability [6]. An α value above 0.7 (α ≥ 0.7) is acceptable. Accordingly, the inter-item reliability for each of the multi-item variables were assessed. The results are given in

Table 1. Reliability testing results.

Construct	Factors	(α)	Qualified
ISSM	System Quality	0.812	Yes
	Information Quality	0.701	Yes
	Service Quality	0.892	Yes
	User Satisfaction	0.811	Yes
	Net Benefits	0.901	Yes
TAM	PU	0.901	Yes
	PEOU	0.966	Yes
	A	0.811	Yes
	BI	0.798	Yes
	AU	0.829	Yes

. They indicate that the Cronbach’s alpha coefficient is high in all scales, ranging from the lowest of 0.701 (Information Quality) to the highest of 0.966 (Perceived ease of use). This confirms that the measurement scales of the constructs are stable and consistent in Table 1.

4.2. The Correlation between the Various Aspects of the Information System Success Model and User Satisfaction

For the verification of substantive relationship, the method of correlation variance analysis and multiple regression analysis were adopted, with the results explained as follows. The research model and hypotheses were examined as follows “

Table 2. Correlation variance analysis of various aspects of the information system success model and user satisfaction (H1~H4).

Aspect	Average	Standard Deviation	Variance	F-Value	Significance	Hypothetical Result
User satisfaction	3.73	0.692	0.479
System quality	3.62	0.628	0.394	13.292	0.000 ***	Valid
Information quality	3.76	0.709	0.502	8.510	0.000 ***	Valid
Service quality	3.52	0.809	0.654	8.516	0.000 ***	Valid
Net benefits	3.98	0.669	0.448	13.295	0.000 ***	Valid

*** p < 0.01

,

Table 3. Regression analysis of various aspects of the information system success model and user satisfaction (H1~H4).

	User Satisfaction
	Unstandardized Coefficient		Standardized Coefficient	T-Value	Significance	Hypothetical Result
Aspect	Regression Coefficient	Standard Deviation
System quality	0.348	0.192	0.315	4.327	0.000 ***	Valid
Information quality	0.118	0.066	0.121	1.798	0.004 ***	Valid
Service quality	0.189	0.056	0.221	3.386	0.001 ***	Valid
Net benefits	0.368	0.069	0.356	5.318	0.000 ***	Valid
	R2 = 0.799, Adjust R2 = 0.792

*** p < 0.01.

,

Table 4. Correlation of the variance analysis of various aspects of the information system success model and user satisfaction (H5~H9).

Aspect	Mean	Standard Deviation	Variance	F	Significance	Hypothetical Results
User satisfaction	3.73	0.692	0.479
PU	4.09	0.672	0.452	7.897	0.000 ***	Valid
PEOU	3.88	0.815	0.664	8.420	0.000 ***	Valid
A	4.01	0.666	0.443	8.155	0.000 ***	Valid
BI	4.29	0.685	0.470	4.765	0.000 ***	Valid
AU	4.00	0.698	0.487	7.760	0.000 ***	Valid

*** p < 0.01.

and

Table 5. Regression analysis of each aspect of the technology acceptance model and user satisfaction (H5~H9).

	User Satisfaction
	Unstandardized Coefficient		Standardized Coefficient	T-Value	Significance	Hypothetical Result
Aspect	Regression Coefficient	Standard Deviation
Subjective usability	0.167	0.119	0.162	1.401	0.164 *	Invalid
Subjective ease of use	0.203	0.077	0.239	2.650	0.009 **	Invalid
User attitude	0.197	0.122	0.190	1.621	0.108 *	Invalid
User intention	−0.014	0.091	−0.013	−0.149	0.882 *	Invalid
Actual usage	0.325	0.083	0.328	3.899	0.000 ***	Valid
	R2 = 0.648, Adjust R2 = 0.632

*** p < 0.01 ** p < 0.05 * p < 0.1

”.

5. Conclusions and Suggestion

5.1. Conclusions

This study analyzed the different functions of the emergency information system under different categorizations used by the users of different occupations to investigate more in depth the actual needs of various users. It was found that users of different job positions showed significant differences in “system quality”, “information quality”, “service quality”, “user satisfaction”, and “actual usage”, where doctors were less contemptous in every aspect than other medical administrative staff. Most medical staff require higher education, with at least a college degree or above, but in terms of the education level, users showed no significant difference in factorial aspects of the information system success model and the technology acceptance model.

5.2. Suggestions

• Improve the management’s knowledge and support of information innovation technology;
• Provide user-friendly system interfaces;
• Information establishment must conform with users’ working habits and external cooperation of information technology.

5.3. Limitations and the Future

Although the authors expected a more rigorous and comprehensive research framework and procedure because of the time limit, the questionnaire design, and sampling constraint, this study still had many limitations and issues. The limitations and possible directions for the futures will be discussed as follows.

5.3.1. Improve the Research Tool

This research adopted the questionnaire as the tool which has its primary structure based on the information system success model proposed by DeLone and McLean (2003). Information quality, system quality and service quality were the three constructs used to understand the correlation with “user satisfaction”, while “technology acceptance” was the intervening variable. The goal was to evaluate the impact of system quality, information quality and service quality on user satisfaction when the user’s technology acceptance was considered, even though it has been pretested and verified for reliability. However, the question of whether the content of the questionnaire encompassed all aspects of the emergency information system would require further investigation.

5.3.2. Limited Research Subjects

The study was based on subjects from a district city teaching hospital in the central region of Taiwan. In the future, subjects should be included from other medical institutions with emergency information systems throughout the country in order to provide a more complete insight into the effective improvement of nursing staff’s acceptance and satisfaction of the emergency information system.

5.3.3. Addition of Qualitative and Quantitative Research Methods

The study primarily used a survey questionnaire as the main approach. Thus, when inferring the research results, it was limited due to incomplete data and a lack of inmdepth quantitative data. Therefore, the combination of both quantitative and qualitative approaches can be adopted to increase to a more in depth qualitative data collection process and to increase understanding, such as when explaining users’ attitudes, strengths vs. weaknesses, and operational difficulties of the emergency information system, which would provide a more complete picture of system users. By incorporating the actual feeling and thoughts of users in the questionnaire, as well as a way to convert information into a quantitative model, it would allow a reflection of the actual situation.