An Integrated Key Performance Measurement for Manufacturing Operations Management

Abstract

This paper proposes a comprehensive production performance measurement framework and illustrates the method to evaluate the performance and guide practitioners to make further improvement. The development comprises four steps. (1) Performance indicators derived from business excellence models are enumerated to provide the performance model: 74 indicators, which can be classified in terms of their characteristics, are identified in six criteria. (2) A multiple criteria decision-making approach based on the analytic hierarchical and network processes, which determine the weights of the criteria and indicators, is applied. In addition, this study introduced additional formulas to derive the final performance values. (3) A performance measurement framework that integrates the measurement and result analysis processes is implemented. (4) The proposed framework is verified through a case study. The results of the case study show that the proposed framework identifies the gaps and discrepancies among the management levels, enabling the determination of means for continuous improvement.

1. Introduction

1.1. Background

The ability to measure operational performance is an important factor for competing enterprises in the global market. Performance measurement helps in the evaluation of the long-term effects of outputs for improving competitiveness and decision-making power. A company’s competitiveness and profits are reduced by a consistent continuation of subpar performance, as this eventually leads to a failure to meet customer need. In this overall perspective, using performance measurement to understand the company’s circumstances is critical to gaining the success and sustaining competitiveness in the era of constantly changing markets [1].

To measure this performance, well-extracted performance indicators are important sources for the effective performance measurement. It is generally believed that inspecting all the processes of the company environment yields well-extracted indicators that can increase the chances for success [2]. Evaluation of well-extracted indicators provides advantages to a company in the form of defects corrections. Therefore, gathering indicators should be obtained regarding financial or non-financial measures. Before the 1990s, financial measurements such as firm revenue, market share, and return on investment, were the main methods for evaluating performance [3]. However, some shortcomings to this method have been discovered, i.e., the fact that it is easy to concoct and falsify financial measures. Moreover, especially in the manufacturing domain, it is easier for managers to focus on reducing cost. This causes a deterioration of quality and disturbs long term improvement. For this reason, simultaneous consideration of corporate and manufacturing strategy is imperative to having a well-formulated performance measurement. For example, production quality is considered one of the main subjects in smart manufacturing [4,5]. Various performance measurement methods for the evaluation of manufacturing performance were adopted in theoretical and empirical studies [3]. Ref. [6] developed a management tool of Balanced Scorecard (BSC), which answered the call for a multi-source performance management system that used both non-financial and financial strategic indicators. From the BSC, many methods, such as supply chain operations reference (SCOR) and business excellence models, have been derived. The Baldridge, Deming and European Foundation for Quality Management (EFQM) model are the most well-known and commonly used models throughout the world [7]. Many organizations have adopted business excellence models because of the realization that these models promote the adoption of the best practices and tools that best fit a successful, quality strategy [8]. Additionally, these models have also been established as the holistic approach to organizational performance measurement. This study investigated the business excellence models and Appendix A presents the extracted indicators.

In general, production performance measurement is regarded as a Multiple Criteria Decision Making (MCDM) problem, because manufacturing performance cannot be evaluated using only one criterion; rather, it has many criteria in addition to complex concepts [9]. There are various MCDM techniques for evaluating multiple criteria, e.g., data envelopment analysis (DEA), the analytic hierarchical process (AHP), the analytic network process (ANP) and the technique for order of preference by similarity to ideal solution (TOPSIS). Among them, AHP and ANP are proposed by [8] first. Since, Refs. [10,11,12,13] use AHP for medical decision-making problems. Refs. [14,15,16] use AHP to improve the performance of flexible manufacturing cells, and [17,18,19] adapt AHP for selecting and evaluating supply chains. Moreover, Refs. [20,21,22] use ANP for online banking, wireless sensor network and urban planning, and brand making, respectively.

As manufacturing environments have become more complex than ever before, manufacturing-related indicators are increasing, which requires categorizing many indicators according to established criteria. In this study, we apply the AHP and ANP methods that categorized many indicators considering the relationships among indicators for evaluation of manufacturing operations management in a real-world company in Korea.

1.2. Research Statement

The general objective of this study is the development of a framework for performance measurement using an MCDM method (hybrid AHP/ANP approach) shown in Figure 1. This hybrid approach determines the weights of the criteria and performance indicators according to the relationship between each element. To organize the list of performance indicators, we investigate business excellence models and categorize them into six criteria. Furthermore, we develop a performance measurement framework that includes an assessment stage and a continuous improvement stage. The four contributions of the study are as follows:

• We list non-financial performance indicators based on the three business excellence models: Baldridge, EFQM and Deming. One of the core contributions of this work is the categorization of production related indicators into six criteria.
• We use of an MCDM approach to define the weights that state the importance of each criterion and indicator according to pairwise comparisons. We develop an additional formulation to derive the performance value based on the results of the MCDM method.
• We specify the process of inducing continuous improvement, which includes a method to analyze the assessment results showing the performance measurement gaps among the three management levels, i.e., strategic, tactical and operational level.
• We present the results of a case study that show the developed framework; additionally, we use the case study output to verify the proposed framework process.

2. Problem Description

2.1. Designing the Structure of the Indicators

Several studies have suggested many criteria for use as the indicators, e.g., cost, delivery dependence, flexibility, quality, utilization, time and efficiency.

Table 1. Summarize literature review of the accepted criteria and applied method.

Author	Criteria	Method
Yu and Hu [3]	Productivity, Production Amount, Production Cost, Inventory Amount, Quality cost	Fuzzy TOPSIS
Lin et al. [22]	Perceived usefulness, Perceived ease of use, Risk, Trust, Satisfaction	ANP
Ordoobadi [23]	Cost, Flexibility, Productivity, Quality, Reliability, Human Resource Impact	ANP
Abdel-Maksoud [24]	Product Quality, Customer Satisfaction, On-time delivery, Employee morale, Efficiency and utilization	Framework
Yang et al. [25]	Cost, Delivery Dependence, Employee, Flexibility, Quality, Utilization	AHP/ANP
Yurdakul [26]	Dependability, Time, Flexibility, Quality, Cost	AHP
Sun [27]	Manufacturing, Supply chain, Innovation, Financial, Service quality, Human Resource	AHP/Fuzzy TOPSIS
Pagone et al. [28]	Cost, Quality, Environmental Sustainability, Time	Fuzzy TOPSIS

illustrates the selection of criteria and methods for measuring their production performance. There are some limitations of the related model in that it just introduces the list of criteria sorts the preference criteria, without deriving differences in criteria preference among employees

Conventional performance measurement studies are based on criteria strongly related with production performance, and they characterize information solely on production management aspects. However, production performance is complex in that worker productivity, production line environment status and efforts to improve strongly influence the results of performance. The main differentiators for this study are ‘Overall Waste’, ‘Human Resource’ and ‘Continuous Improvement’. These criteria enable the measurement to cover current production management aspects such as lean manufacturing and Toyota Production System. The criteria comprise six components that cover manufacturing performance, and

Table 2. Characteristics of accepted criteria and included indicators.

Criteria	Characteristics	Indicators
Production	This covers all of the manufacturing processes that take place on the shop floor. It considers the production system’s effectiveness and measures the manufacturing process efficiency.	9
Quality	This is a measure of the reliability of a company’s control system, and it shows the trust level from the raw material to the finished product.	18
Flexibility	Companies try to match the variable nature of customer’s needs such that the product mix and production volumes need to be varied. For this reason, this criterion measures the manufacturing system’s flexibility performance.	5
Overall Waste	This is excerpted from the Toyota Production System, which was mainly presented by Ohno Taiichi. It measures the expenses of the laboring force, time and other resources due to the inefficiency of the process or glut of inventory.	10
Human Resource	This is an imperative resource on the shop floor, even though automation facilities decrease its importance. It comprises individual performance measurement indicators as well as organizational performance measurement indicators.	19
Continuous Improvement	This is an ongoing effort to improve existing manufacturing processes and resources. Companies should encourage continuous improvement to promote manufacturing efficiency. It measures the organization wide approach to improvement activities.	13

explains the criteria characteristics and presents the number of indicators used in this study (Appendix A provides additional details).

2.2. AHP/ANP Method

2.2.1. Fundamentals of the AHP/ANP Method

AHP decomposes the unstructured problem into a hierarchical structure, such as a decision tree. Decision elements at each hierarchical level are compared through the pair-wise method and assigned relative scales. The detailed process of AHP comprises three steps [29]:

(1)

The problem is decomposed into a hierarchical structure. The top level can be decomposed into some criteria and its lower level elements which cannot decompose furthermore.

(2)

Pair-wise comparison is performed. The pair-wise comparison indicates the perception of which element has more weight than the other.

(3)

The maximum eigenvalues and eigenvectors for the relative weights of the elements are derived. Then, the consistency property of the matrix must be checked to verify reliability of the decision making.

The Consistency Index (CI) and Consistency Ratio (CR), which illustrate the reliability, are defined as follows:

$$CI=\frac{{\lambda }_{\max }-n}{n-1}, CR=\frac{CI}{RI}$$

(1)

where λ_max is the maximum eigenvalue, n is the number of elements and the random index (RI) is the average consistency index of randomly generated pair-wise comparison indicators (

Table 3. Random Index (RI).

Attributes	2	3	4	5	6	7	8	9	10
RI	0	0.58	0.90	1.12	1.24	1.32	1.41	1.45	1.49

).

Using RI, the CR can be calculated, and this value shows whether the evaluations are sufficiently consistent. In general, if CR > 0.1, the decision-making structure is reliably consistent. If CR < 0.1, then it is inconsistent, and the evaluation procedure must be repeated until the CR satisfies the threshold.

The ANP method is an extension of the AHP that generalizes the AHP by replacing the hierarchy with a network that is composed of different clusters (groups of elements). The ANP has unique characteristics in that it assumes a mutual interaction relationship that exist between the clusters and contains feedback loops among the sub-elements [23]. Between the clusters and their elements different relationships (inner or outer dependence, feedback) exist as shown in Figure 2. The ANP still involves the hierarchical representation of relationships, but it does not require as strict a hierarchical structure as the AHP [30]. The ANP approach is capable of handling interdependence among elements by obtaining the composite weights through the development of a ‘super matrix’ which is similar to Markov chains [31].

2.2.2. Applying the Indicators with the AHP and ANP Method

The AHP has been criticized because the decision problems are structured in a hierarchical manner. Some decision-making problems cannot be structured hierarchically because they involve the interaction and dependence of higher-level elements on lower-level elements [29]. The ANP also has some limits, e.g., it cannot perform comparisons between elements at the same hierarchical level, but it can be used for network structured decision-making problem [32]. For this reason, a hybridization of AHP and ANP was developed to overcome the complementary shortcomings. [30] used the AHP to evaluate various factors for different product mixes and then used the ANP to analyze more complex interrelationships among the decision levels and attributes. [33] use the AHP and ANP to determine the weight of each criterion when generating the performance model for a decision-making problem regarding the location of a facility.

In this study, the AHP and ANP are utilized to determine the weights of the criteria and indicators and the structure of the performance measurement is a categorization of 74 indicators into six criteria. A few of the 74 indicators have mutual interactions because some circumstances cannot be reflected by a single indicator. In addition, assuming relationships between the criteria and indicators with the hierarchical structure is problematic in that all indicators do not have a mutual relationship and their relationships vary depending on varying circumstances. For this reason, when forming the structure of the performance measurement, we should simultaneously include their mutual interactions and independent relationships. Figure 3 presents the structure of the performance measurement.

For simplification, we assume that the indicators have a hierarchical structure in accordance with the criteria. For this reason, the AHP should be utilized to assess indicators. The absence of the mutual interactions of the indicators are corrected by applying the ANP.

3. A Multi-Stage Framework

The conceptual framework involves five stages and has three main sections: performance measuring using MCDM, the analysis of gaps among different levels, and the alignment process with corporate and manufacturing strategy. Figure 4 shows the overall process of the framework and the following subsections explain the purpose and functions of each stage.

3.1. Stage 1: Extracting Indicators Based on Production Types

The starting point of the framework begins with the selection of the production type: Make to Stock (MTS), Make to Order (MTO) and Engineer to Order (ETO). This selection identifies the context of the manufacturing circumstance and presents the basis of the related indicators. Once the production type is identified, it is reflected in the indicators list in Stage 2. Each product type has weighted indicators that adequately reflect the production circumstance as shown in

Table 4. Weighted indicators considering production type.

Make to Stock	Make to Order	Engineer to Order
- Waste of inventory - Rate of standardization	- Variation of process - Rate of on-time delivery - Rate of process comprehension	- Level of stock - Flexibility of human resource - Rate of on-time delivery - Rate of process comprehension

. Defined weighted indicators play an important role as additional performance factors when Stage 5 is progressing.

The output of this stage presents each management level with a list of indicators for the subsequent stages. The main challenge is identifying the indicators that reflect the company’s circumstance. Additionally, the indicators must be extracted in a balanced approach

3.2. Stage 2: Selecting Management Levels

In general, there are three hierarchical levels in manufacturing components: strategic level, tactical level and operational level. These levels are distinguished by their duties, position, job tenure, etc. The strategic level consists of the board of directors, president, vice-president and other members of the chief executives. They are responsible for controlling and overseeing the entire organization’s goals. The tactical level consists of the department managers related to production. They plan dispatch production, arrange production scheduling in detail, and supervise the shop floor. Lastly, the operational level consists of production workers on machines. These workers perform the production activities in manufacturing facilities including the equipment, inventory and storage areas. At this stage, the stakeholders who execute the assessment are selected.

3.3. Stage 3: Verification of Extracted Indicators

Throughout this stage, each management level checks the indicators in the list, and the outputs of this stage will be used in the performance assessment. This stage also monitors whether the previous stages are being satisfactorily performed such that the desired output can be attained. To meet the objectives, this stage is complemented with three different indicator lists, because each management level might choose different indicators based on their needs. If inadequate selections are identified, the process returns to the Stage 1. This process should be repeated until dissatisfaction has converged to a small value. In general, it is appropriate to perform the repeated process one to three times because relative improvement of the reliability value decreases as the number of iterations increases.

3.4. Stage 4: Assessment

This stage includes performing questionnaires based on the extracted indicators to make a grade, and the AHP/ANP analysis is applied to deduce the final results. If the basic value is drawn through the AHP/ANP analysis, the priority value among criteria, the priority value among indicators and the measured value of the overall production performance are derived with additional formulae. During this stage, the level of performance is calculated and the level of inconsistency among the management levels regarding each indicator and criterion can be measured. If the level of inconsistency is proven to be too big, Stage 5 is executed without the preceding stages. Figure 5 shows a flow chart of the assessment process.

Step 1 includes the survey, the AHP/ANP analysis, and the calculation of an index through an additional formula. The priority extracted indicators are estimated through pair-wise comparisons made by interested parties in the strategic, tactical, and operational levels using indicators acquired from the previous stage. In addition, the scores of the indicators and the criteria are rated on a point scale where 1 is poor, 3 is normal and 5 is excellent. Consequently, the mean estimated by each interested party is allotted as a Score. The Criteria Weight (CW), which is a rating of the importance of each criterion is determined through the ANP using pair-wise comparison. The Indicator Weight (IW), which is a value of the importance of each indicator, is determined through the AHP. Using CW and IW, the final outputs of performance measurement, criteria priority and final performance value, can be derived. The notation for the measurements of indices in this study are listed as follows:

• i    Interested party index (i = 1, 2, 3),
• j    Criteria index (j = 1, 2, …, 6),
• k    Indicator index (k = 1, 2, ..., m),
• m    Number of extracted indicators,
• α    Weight by production environment,
• TWC_ij    Score of the jth criterion from the ith interested party (criteria priority),
• IW_ijk    Weight of the kth indicator of the jth criterion from the ith interested party (AHP),
• Score_ijk    Score of the kth indicator allotment of the jth criterion from the ith interested party,
• OP_i    Overall point of the ith interested party and
• CW_ij    Weight of the jth criterion from the ith interested party (ANP).

The TWC_ij and OP_i can be calculated by using the following equations, and these results are to be analyzed to induce continuous improvement.

$$TW{C}_{ij}={\displaystyle \sum }_{j=1}^6{\displaystyle \sum }_{k=1}^m\alpha * I{W}_{ijk}* Scor{e}_{ijk}$$

(2)

$$O{P}_i={\displaystyle \sum }_{j=1}^{6}TW{C}_{ij}* C{W}_{ij}$$

(3)

The criteria priority (TWC_ij) determined through Equation (2) indicates which criterion has more importance than other criteria. The value α is used to allot greater importance to the priority value of a given criterion. This value can be estimates as α = IW + 1, because IW has an importance weight score that is always under 1. Equation (3) is used to acquire the final outcome of the measurement. Through this value, the performance level of the current production scheme can be identified at each management level.

Step 2 is the evaluation of the inconsistency via analysis of the values derived from Step 1. The notations for the inconsistency and their definitions are listed as follows:

• l_ik    Lower value of the score of the kth indicator of the jth criterion given by each management level,
• u_jk    Upper value of the score of the kth indicator of the jth criterion given by each management level,
• m_jk     Difference between highest value (u_jk) and lowest value (l_jk),
• $\overline{l_{jk}}$    Minimum value of the lower values existing in the common section and
• $\overline{m_{jk}}$     Maximum value of the difference value (m_jk) existing in the common section.

Based on these notations, we can identify two issues: inconsistency and discrepancy for inducing continuous improvement. First, the inconsistency, also called the gap of indicators among management levels, can be analyzed. One of the most significant characteristics of this framework is the ability to select indicators to consider through consultation with interested parties. Different selections of indicators mean that the part to which importance is attached in considering a manufacturing strategy differs depending on each management level. The difference in the selection, expressed as a gap, becomes the factor that may be the reason for an inefficiency of production. Here, unless the selected indicator was extracted by specific management level, the value is 0 or l_ik = 0. If the minimum value of all indicators is zero, then $\overline{l_{jk}}$ = 0. A gap is judged to be the most serious inconsistency.

Second, large discrepancies in each management level are judged. A large discrepancy suggests that there are huge strategy gaps among the management levels resulting in lower productivity. The upper values (u_jk) and the lower values (l_jk) of specific indicators for a specific criterion given in each management level are measured. In general, if the difference between them (m_jk) is more than 0.2, the criterion must be analyzed. In order to assign an order of appropriate measurement, priority should be decided. First, the m_jk values should be sorted in a descending order then the indicators with the maximum value, that is, $\overline{m_{jk}}$ should be sent to next stage.

3.5. Inducing Continuous Improvement

Second, large discrepancies in each management level are judged. A large discrepancy suggests that there are huge strategy gaps among the management levels resulting in lower productivity. The upper values (u_jk) and the lower values (l_jk) of specific indicators for a specific criterion given in each management levels are measured. In general, if the difference between them (m_jk) is more than 0.2, the criterion must be analyzed. In order to assign an order of appropriate measurement, priority should be decided. First, the m_jk values should be sorted in a descending order than the indicators with the maximum value, that is, $\overline{m_{jk}}$ should be sent to the next stage.

The output of the analysis of inconsistency in Step 2 of the previous stage may be analyzed to drive continuous improvement. An example of a noticeable table of Gap and Discrepancy is shown in

Table 5. Example of a noticeable table of Gap and Discrepancy.

	Indicators	Relevancy and Difference Value
		Strategic	Tactical	Operational
Gap	Excess in stock rate	○	×	○
	Rate of delegation of authority	○	×	×
	Cpk (Process capability index)	○	○	○
Discrepancy	Standardized rate	0.44
	Turnover rate	0.29
	Setup Time	0.23

. The ‘o’ symbol means that the specific management level chose the indicator, whereas the ‘×’ mark means that it was not be selected.

First, if a gap occurs, it is considered that there is a serious disagreement among management levels regarding the corporate and manufacturing strategy. In this case, the interested parties among the management levels discuss their opinions on the indicators for which gaps had occurred. In addition, they discuss the importance of the specific indicators so that there will not be any gaps in the future, and they agree on whether to even continue extracting indicators in the future.

Second, if a discrepancy occurs, it is necessary to reconcile the difference between the approaches of a strategy among management levels. After indicators with a large m_jk value are established, it is important for the interested parties to determine the current status and understand the degree of resource commitment regarding the related process. Through this, the difference in perspectives on the measurement of the performance factor among management levels is analyzed. It is necessary for each management level to resolve this by presenting opinions regarding the indicators.

4. Application Case Study and Discussion

4.1. Selecting Production Type and Extracting Indicators

In order to verify the utilization of this framework, we have conducted a case study of a steel manufacturer. This company is the primary vendor of POSCO, which is the largest steel company in the Republic of Korea. The company consists of 120 workers, 70 of which were working on production lines. Six strategic managers, five tactical managers and five operational workers were invited to assess the performance. The company utilizes an MTO production strategy owing to its collaboration with a large steel company. As mentioned in Section 3.4, the indicators (34 strategic, 37 tactical and 21 operational) were presented in a survey through consultation with each management level.

4.2. Assessment

4.2.1. Step 1: Questionnaires and AHP/ANP Application

We performed a pair-wise comparison survey for the criteria and indicators to apply the AHP/ANP analysis. Because the score allotment for criteria assumes the mutual interaction relationship, it was analyzed by using the ANP.

Table 6. Priority weights of the criteria after analytic network process (ANP).

Criteria	Criteria Weight (CW)
	Strategic Level	Tactical Level	Operational Level
Production	0.079	0.087	0.089
Quality	0.207	0.209	0.177
Flexibility	0.026	0.059	0.077
Waste	0.061	0.040	0.047
Human Resource	0.406	0.375	0.300
Continuous Improvement	0.220	0.230	0.313
Consistency	0.17365	0.17665	0.10464

presents the score allotment.

The calculated consistency of each management levels was in the range of 0.1 to 0.2, afforming the reliability of the data. First, it was found through the data that there is a similar priority of criteria between the strategic and tactical levels, but that of the operational level was somewhat different. From this fact, we could identify that there was a difference for priority of criteria between the office level and the plant level. Then, the AHP was applied when allotting a score to an indicator. To acquire a result, each stakeholder provided a pair-wise comparison for all indicators and allotted a score (1, 3, 5) for each indictor. The allotted scores show the averages of indicator scores and the IW means the results of AHP.

Table 7. Allotment table of scores of indicators in a criterion of quality after analytic hierarchical process (AHP).

Indicator	Indicator Weight (IW)			Allotted Scores
	Strategic	Tactical	Operational	Strategic	Tactical	Operational
Rate of process change by failure	0.025	×	×	4.5	×	×
On-time delivery compliance	0.452	0.151	0.526	4.5	4.3	4.0
Final product pass rate	0.139	0.552	0.176	5.0	4.3	3.5
Production schedule compliance rate	0.203	×	0.125	4.5	×	4.0
Correction measure	0.075	0.085	0.105	4.5	3.6	3.0
Inventory inspection	0.063	0.049	0.032	2.5	2.3	3.0
Rework rate	×	×	0.036	×	×	3.5
Load levelling status	0.042	0.163	0	2.5	2.3	×
Consistency	0.16448	0.11840	0.20269

contains the scores of indicators in the criterion of quality.

The consistency also shows reliability and the ‘×’ mark in Table 7 means that the specific management level did not choose the indicator, specified as a gap in the analysis of inconsistency. This means that gap analysis is required to fix the problem.

Next, the outcome scores of the final production performance were calculated using Equation (2) and they are shown in

Table 8. Allotment of total weight criteria (TWC) of each criterion.

Criteria	Total Weight Criteria (TWC)
	Strategic Level	Tactical Level	Operational Level
Production	3.658	4.126	3.303
Quality	4.355	3.817	3.257
Flexibility	3.264	3.126	3.703
Waste	3.789	3.561	3.148
Human Resource	3.352	3.708	1.980
Continuous Improvement	3.210	3.139	3.135

.

If a value history is available, this value allows for analysis of the current status of production performance via comparison to previous values. Moreover, it becomes possible to compare individual management levels by using this final outcome score value. Using Equation (3), the overall production performance score can be derived: Strategic is 3.375, Tactical is 3.596, and Operational is 2.878. Because the operational level score was lower than scores of other management levels, the indicators allotted to the lower scores must be checked, and their causes must be identified.

4.2.2. Step 2: Gap and Discrepancy Analysis for Inducing Continuous Improvement

In this case study, it was found that a gap had occurred in eight of the indicators: four in ‘Quality’, two in ‘Overall waste’, one in ‘Human resources’ and one in ‘Continuous improvement’. The results of Gap analysis were shown in

Table 9. Table of Gap analysis results.

Criteria	Indicator	Strategic	Tactical	Operational
Quality	Rate of process change by failure	○	×	×
	Production schedule compliance rate	○	×	○
	Inventory difference rate	○	○	×
	Standardization rate	○	○	×
Overall waste	Waste of standby & allowance	×	○	×
	Waste of unnecessary motions	○	×	○
Human resources	Turnover rate	○	○	×
Continuous improvement	Frequency of production innovation activity	○	○	×

.

From discussions with management levels regarding the outcome of the gap, the indicator that seemed to most significantly contribute to the gap analysis was the ‘Frequency of production innovation activity’. The strategic and tactical levels selected this indicator, but the operational level did not because of the resistance to innovation the work site. The strategic and tactical levels wanted to heighten productivity through innovation activities, but the operational level was not interested in innovation because there were fewer advantages. For this, a long-term plan for innovation was established.

Next, from the analysis of the discrepancy indicator extracted with IW, it was determined that there were differences in the priority of waste by inventory, standardization rate, Cpk, etc. Figure 6 shows the details of the discrepancy.

4.3. Discussion of Application Study

From the discrepancy analysis, it was found that there were great differences in the indicators in the production and quality criteria. This means that different priorities existed during work site production depending on each management level, so the problem would be found through consultation. The following text describes consultation feedback. The indicators that contained the most significant results were those of the ‘Waste by stock’ and ‘Standardization rate’. The value of ‘Waste by stock’ was high in the tactical level yet low in the strategic level. Investigating this indicator, it was determined that stock management was not performed well at the site and many expenses were paid according to the stock. On the site, stocks and products were stored in outdoor yards (in addition to indoor storage), and the managers in the tactical level felt a need for the expansion of the plant; in contrast, the stakeholders at the strategic level had yet to even consider expansion. Next, the value of ‘standardization rate’ indicator was high for the operational level yet low for the tactical level. Investigating this indicator, it was determined that the site was interested in increasing efficiency through standardization, but the tactical level had a lower priority for the standardization. Through this observation, the tactical level could understand the desire for standardization at the site and develop the standardization methods for the future.

As a result of the case study, it was found that there the significance of indicators varied. It seems that this was primarily caused by varying interest in the priority of the work to be completed at each level. In order to overcome this limitation, it is necessary to identify the problems through consultation among the management levels for the indicators with high discrepancies. Among the 34 indicators, the 10 indicators that were prioritized at all management levels are as follows; input of materials; Waste by accident; Delivery observation rate; Understanding process rate; Manufacturing cycle time; Waste by stock; Pass rate of finished goods; C_pk; Innovation of production process; Straight to process rate.

5. Conclusions

In this study, we investigated business excellence models for extracting related indicators. Based on the extracted indicators, we built a performance measurement structure for applying a hybrid AHP/ANP. A five-stage framework of performance measurement was proposed. It includes estimating the weight between hierarchical consistent indices through applying the hybrid AHP/ANP in the assessment stage and extracting implications to be applied to the site in the diagnostic stage. A case study was conducted to verify the proposed framework. This research will be very helpful to managers who want to evaluate their performance.

As the market environment has complex factors, the evaluation of performance is widely accepted as an MCDM problem. Especially in the manufacturing industry, which has complete distinctive management levels, lessening the gap and discrepancy among management levels is very important to get the company’s competitiveness. Our proposed framework affords a steppingstone for identifying those issues. Our proposed model can be used by small- and medium-size companies to measure the performance completely, because building structure of indicators and doing pair-wise comparison to each management level requires time and is cost consuming. Moreover, defining the relations among indicators is ambiguous as company size is bigger. Our proposed model fits the manufacturing industry, models (Baldridge, EFQM, Deming, etc.) from which we extracted the criterion are specialized in manufacturing. Moreover, a characteristic of our research, dividing the management level into three, is easily applied into manufacturing industry.

This study has four aspects to be improved in future research. First, if more performance indicators are selected, the assessment process will be rather time consuming. Thus, there is a challenge to neglect unimportant indicators before assessment. Developing the knowledge management system is helpful in this issue. Second, more case studies could verify and strengthen the proposed framework. Further case studies are recommended to analyze the framework’s usefulness and difference between companies or industries. Third, as many states of the art technologies, such as the Internet of Things (IoT), are being applied into the real sites, new indicators should be developed to reflect their performance. Feedback from professionals would definitely enrich the list of indicators. Finally, as environmental factors become important in manufacturing, it is necessary to establish environmental criteria.