Quantitative Evaluation of High-Tech Industry Policies Based on the PMC-Index Model: A Case Study of China’s Beijing-Tianjin-Hebei Region

Abstract

High-tech industrial agglomeration plays a significant role in regional sustainable development. Local governments have issued many industrial policies to accelerate the development of high-tech industries in China. Evaluating high-tech industry policies from the perspective of regional industrial synergy can prevent problems in policy implementation and promote the industrial synergy in a region. For this purpose, taking China’s Beijing-Tianjin-Hebei (BTH) region as a case, we evaluate seven policies governing the high-tech industry in this region by using the approach which integrates the policy modeling consistency index (PMC-Index) model and text mining. We propose an evaluation system with consideration of regional industrial synergy, which is based on the PMC-Index model. The results show that the lowest PMC-Index value of the seven policies is 5.30, the highest is 8.17, and the average is 6.67. Among the policies, four are of excellent or perfect grade and relatively comprehensive; three are of acceptable grade and relatively insufficient. The overall designs of the high-tech industrial policies are reasonable but there is still much room for improvement. According to the average scores of the main indicators, the policies function relatively poorly in terms of policy release agency, policy timeliness, policy type and policy receptor. The optimizations for the shortcomings of each policy are also suggested. This study may not only provide some enlightenment to policymakers, but also provide a supplement for the policy evaluation field.

1. Introduction

The high-tech industry has the advantages of high technology, high efficiency, low energy consumption and low emission [1]. If high-tech industrial agglomeration is formed in a region, it will give play to a knowledge spillover effect, reduce the technology acquisition cost of enterprises, and accordingly, improve the level of low-carbon technology [2,3,4]. In China, the high-tech industry refers to the industry with relatively high research and development (R&D) investment intensity in national economy sectors, mainly including the manufacture of aircrafts, computer, electronic and communication, and medicines [5]. Nowadays, the development of most other industries has become inseparable from the support of high-tech industries [6]. In particular, the wide application of new-generation technologies including the internet of things (IoT), artificial intelligence (AI) and cloud computing has had a profound impact on industries with greater pollution, such as transportation, logistics and manufacturing [7,8,9]. Production efficiency has been greatly improved, and the damage to the environment has also been controlled [10]. Technological upgrading and industrial transformation promote the sustainable development of various industries in a region [11]. Therefore, the high-tech industry has attracted the attention of local governments in China. The governments are actively attracting more high-tech enterprises to gather and are also trying to promote the growth of high-tech industries in the region [12]. Industrial policy is a government behavior for protecting, supporting, and adjusting the industrial environment for economic development [13]. This can ameliorate market failure caused by ineffective resource allocation and ultimately optimize the industrial structure [14,15,16,17]. Consequently, governments have attached great importance to the policies governing the high-tech industry in many regions of China [18,19].

With the acceleration of China’s urban development process, individual cities have increasingly come to rely on combinations of infrastructure, transportation, and information networks to form urban agglomerations [20,21]. The urban agglomeration has become the new carrier for high-tech innovation and economic growth, and the construction of urban agglomeration with regional economic integration as the core has become a general trend [22]. However, the inter-city development of high-tech industry is yet extremely imbalanced in many regions [23,24]. There is a huge difference between actual effects and stated goals of the policies [25,26]. Scientific evaluation of high-tech industry policies can ensure the scientific deployment and maintenance of policies and the timeliness of policy adjustment for the long-term and sustainable development of high-tech industries [27]. In order to better promote the development of the high-tech industry in a region, it is necessary to evaluate and optimize high-tech industry policies from the perspective of regional coordination.

Previous studies on high-tech industry policies have focused on the impact of a certain type of policy or the descriptive analysis of a policy text’s content at a macro level, lacking a systematic evaluation of single policy texts and an evaluation from the perspective of regional coordination. To bridge the gap, this study selects the BTH urban agglomeration as a case and uses the PMC-Index model and text mining to implement an evaluation for high-tech industry policies issued in this region. The contributions of the present work mainly include the following points. Firstly, high-tech industry policies are analyzed according to text mining technology and the PMC-Index model, from the perspective of regionally-coordinated development. Secondly, an evaluation indicator system of high-tech industry policies is proposed based on the PMC-Index model. Finally, seven high-tech industrial policies of the BTH region are isolated as samples for evaluating their respective advantages and disadvantages, and the suggestions for improvement are given.

The remainder of this paper is structured as follows. Section 2 is a literature review. Section 3 introduces the study case, policy samples, and methodology. Section 4 presents and analyses the research results. Section 5 concludes the paper and summarizes the limitations.

2. Literature Review

The literature on the evaluation of high-tech industry policies mainly involves two streams of literature: the impact of policies and the policies themselves.

Scholars have mostly studied the impact of policies on different fields and used a variety of econometric methods for estimating impact assessments with the help of quasi-natural experiment designs. The common methods have included propensity score matching (PSM), difference in differences (DID) and regression discontinuity (RD) [28]. Bruhn and McKenzie [29] investigated the impacts of the In-Tech program in Poland on science–industry collaboration, research and innovation, and product commercialization using the RD method. Xu and Liu [30] discussed the impact of the high-tech identification policy on corporate innovation by applying the PSM approach. Liu, et al. [31] used the Heckman two-step method and the PSM-DID method to study the influence of the high-tech enterprise certification policy on firm innovation in China. Evaluating the impact of policies is beneficial to the formulation and improvement of policies. It is equally important to evaluate the scientificity and rationality of policies themselves.

Early research related to the evaluation of policy text content has focused on qualitative analysis. Most of these policy evaluations have involved text interpretation [32]. With the deepening of public policy research, policy evaluation has become increasingly comprehensive in scope and gradually focused on the combination of qualitative and quantitative research [33]. The mainstream paradigm is based on the policy instruments theory proposed by Rothwell and Zegveld [34], dividing policy instruments into supply type, environment type and demand type, and combining them with text mining and bibliometrics methods, such as policy coding, quantitative scale tables, complex network and co-word analysis. Based on Chinese central government documents, Zhou, et al. [27] used the methods of co-word analysis and social network analysis to study the high-tech industry policy in four dimensions: themes, objects, key process along industry chain, and measures. Zhi, et al. [35] investigated the evolution of China’s PV industry policy from the perspective of policy instruments, and their results show that the policies had a tendency from supply-side to demand-side. Yang, et al. [36] built a research framework based on bibliometrics for exploring Chinese information technology policy. However, the literature has mostly discussed high-tech industry policies from a macro perspective, and lacks a systematic evaluation of the single policy for high-tech industry policies. In addition, previous research methods have mostly relied more on qualitative analysis and human judgment, with strong subjectivity.

In 2010, scholars began to adopt the PMC-Index model proposed by Ruiz Estrada [37], which offered a quantitative evaluation method for the single policy text. At present, this method has been mainly applied in cultivated land protection policy [38], pork industry policy [39], green development policy [40], waste separation management policy [41] and employment promotion policy [42]. There is no big difference in the methodology of these studies, and most have applied the PMC-Index model with integration of text mining methods. In view of the application in these policy fields, this study introduces the PMC-Index model into the field of high-tech industrial policy to fill the gap. The PMC-Index model has two main advantages [37,38,39]: (1) it can quantitatively evaluate each single policy from multiple dimensions and distinguish the internal consistency of the specific policies in a relatively objective way; and (2) the strengths or weaknesses of a policy text can also be directly observed through the PMC-Index and PMC-Surface. Furthermore, this study establishes an evaluation index system for high-tech industrial policies and brings the measurement of regional industry synergy into the policy evaluation model.

3. Materials and Methods

3.1. Case Description

The Beijing-Tianjin-Hebei (BTH) region is an urban agglomeration and also one of the most dynamic and advanced regions in China [43]. It includes two provincial municipalities (Beijing, Tianjin) and 11 cities in Hebei Province (Figure 1). According to the raw data of the China Statistical Yearbook, the following is calculated regarding this region. It covers an area of about 216,000 square kilometers, accounting for 2.3% of China’s total. By the end of 2019, there were 110 million permanent residents in the region, accounting for 8.1% of China’s total, and the region’s GDP was 8.5 trillion-yuan, accounting for 8.6% of China’s total. The BTH region is located in the northeastern part of the North China Great Plain, bordering on the Bohai Sea. The terrain of this area inclines from the Northwest to Southeast, including various types of landforms.

Promoting the coordinated development of the BTH is a major national strategy in China, and its core is to orderly relieve “Beijing’s non-capital functions”. Accelerating the collaborative innovation and the synergy of high-tech industries is an important task of regional coordinated development [44,45]. Unfortunately, the development of high-tech industry in the three places is unbalanced, and the level of Beijing is much higher than that of Tianjin, with Hebei reaching the lowest level [46]. The Chinese “Fourteenth Five-Year Plan”, issued in 2021, was proposed to “accelerate the coordinated development of the BTH” and “promote the deep integration of the BTH industry and innovation chains”. Due to a lack of effective transformation and docking mechanisms for innovation, Beijing’s technological output often flows across Tianjin and Hebei to China’s southeast coastal areas [47]. The BTH region should form an effective industry chain layout around key regional industries.

3.2. Policy Samples

The high-tech industry policies were all issued by the local governments in the BTH region, and the policy texts can be retrieved in the database “PKULAW” (PKULAW is the earliest and largest database of laws and regulations in China, including a large number of published official laws and regulations since the founding of the People’s Republic of China). To ensure that the policies were effective, the texts were screened one by one according to the following criteria: (1) the title, document number, printing time and other relevant information are clear and complete; (2) areas intersecting with high-tech industries (e.g., technology-based small- and medium-sized enterprises and strategic emerging industries) are removed for the sake of clarity; (3) although some policy texts are not marked “high-tech industry”, they are included as their contents are sufficiently relevant; (4) policy types are considered to include local planning, opinions, notices, and other guidance and normative documents; and (5) policies are taken covering the period from 2010 to 2020. Mainly considering the process of coordinated development in the BTH region and the evolution characteristics of regional high-tech industrial policies, 2010 was selected as the starting year and the year 2020, before the beginning of this study, was selected as the end time. Finally, seven policies were found suitable according to these criteria and selected as the evaluation objectives (

Table 1. Policy samples for the PMC-Index model.

Label	Policy Name	Release Agency	Release Time
P1	Notice of Beijing Municipal Commission of Development and Reform on printing and distributing “Beijing high-tech industry development plan during the 13th Five-Year Plan Period”	Beijing Municipal Commission of Development and Reform	15 November 2016
P2	Implementation opinions of the General Office of Tianjin Municipal People’s Government on accelerating the development of high-tech enterprises in Tianjin	General Office of Tianjin Municipal People’s Government	10 July 2015
P3	Implementation opinions of the General Office of Hebei Provincial People’s Government on accelerating the cultivation and development of high-tech service industry	General Office of Hebei Provincial People’s Government	27 July 2012
P4	Notice of Tianjin Science and Technology Bureau and Binhai New Area Government on printing and distributing specific measures to promote high quality development of Tianjin Binhai High-tech Industrial Development Zone	Tianjin Science and Technology Bureau and Binhai New Area Government	4 September 2020
P5	Opinions of the General Office of Cangzhou Municipal People’s Government on promoting healthy and rapid development of pharmaceutical industry in our city	General office of Cangzhou Municipal People’s Government, Hebei Province	30 November 2017
P6	Xingtai high-tech enterprises cultivation plan	Xingtai Science and Technology Bureau	14 March 2020
P7	Opinions on promoting the development of Zhongguancun’s high-tech enterprises	Zhongguancun Science Park Management Committee	27 March 2010

).

3.3. Establishment of the PMC-Index Model

Ruiz Estrada proposed the PMC-Index model, which is supplied with data from text mining technology and uses a binary system to balance the effects of various influencing factors [37]. Balanced variables facilitate linear data fusion for various indicators to enhance accuracy and prevent subjective judgment. It is a policy measurement model that facilitates scientific and quantitative evaluations. Ruiz Estrada asserts that such modeling is an empirical endeavor supported by a variety of theories and quantitative or qualitative elements related to the causes and effects of social policies [37]. The PMC-Index model is based on an “Omnia Mobilis” assumption, where every entity on the globe is moving and interrelated [37]. All relevant variables should be incorporated into the model to the greatest extent possible under this assumption, and no relevant variables can be deleted [38]. This assumption is contrary to the idea that certain conditions or variables are static or interchangeable with others. The PMC-Index and PMC-Surface can show the internal consistency of policies and reveal the advantages and disadvantages of any policy [39,40]. The PMC-Index model can be implemented in four steps (Figure 2).

3.3.1. Variable Selection

Variable selection facilitates the deep processing of intelligent data as a fundamental step in the policy evaluation using the PMC-Index model. With the use of ROSTCM (a software for content mining and analysis), the content of the policy text samples was segmented and the vocabulary was sorted in descending order of frequency. After eliminating redundant and interferential phrasing such as “nation”, “society”, “high-tech”, “development” and “promote”, the 60 most frequent and relevant words were selected for further analysis and reference. (

Table 2. Statistical frequency of effective words from the policy texts.

Vocabulary	Frequency	Vocabulary	Frequency	Vocabulary	Frequency
Enterprise	119	Talent	16	Financing	8
Management	113	Conversion	15	Loan	8
Technology	92	Resources	15	Patent	7
Innovation	60	Start a business	15	Agglomeration	7
Service	55	Research	15	Demonstration area	7
Capital	39	Cultivation	15	Launch a pilot project	7
Science and technology	35	Application	14	Administration	7
Research and development	30	Market	14	Planning	7
Mechanism	27	Subsidy	13	Strategy	6
Industrialization	26	Approval	11	Opinion	6
Investment	21	Government	10	Research institute	6
Organization	21	Introduction	10	Industry chain	6
Achievements	20	Basics	9	Tax revenue	6
Affirmation	18	Area for development	9	Incubator project	6
Advance	18	Environment	9	Public	6
Property rights	18	Regulations	9	Declare	6
Knowledge	18	College	8	Scientific research	6
Platform	18	Project	8	Finance	6
Engineering	17	Exemption	8	Personnel	5
Encourage	17	Insurance	8	Industry park	5

).

Furthermore, a semantic network graph of the high-frequency theme words was constructed with the help of Gephi (a software for complex network analysis), as shown in Figure 3. The thickness of the line in the figure indicates the strength of the connection between words. The thicker the line is, the stronger the correlation is. In the network graph, the high-frequency theme words in the high-tech industry policy are combined in the form of network, and the structural relationship between the words is intuitively reflected in the form of image, which will provide an important reference for the variable setting in the construction of the PMC-Index model. According to the position of words in the network, the words can be divided into four groups, and the words in the central position have denser connections with the other words. As the first group, “Enterprise” is at the center of the network, and this word has the highest density of connection with the other words, indicating that it is the core theme word of the policy texts. “Innovation”, “Management”, “Organization”, “Technology”, “College”, and “Agglomeration” are close to the central position, composing the second group, and they have strong correlations with the first group, which indicates that the high-tech industrial policies emphasize the technology innovation of enterprises, organization and management of enterprises, cooperation between enterprises and colleges, and enterprise agglomeration. “Property rights”, “Government”, “Tax revenue”, “Service”, “Project”, “Advance”, “Platform”, “Research and development”, “Investment”, “Cultivation”, and “Industry park”. According to the connections between the third and the above groups, it is indicated that the policy texts also pay attention to enterprise technology investment, enterprise property, the construction of industrial parks and government tax preferences. The remaining words belong to the fourth group, which is at the edge of the network, and it is indicated that talent introduction, capital investment, industry chain management and innovation environment are relatively important in the high-tech industrial policies.

The evaluation system was established according to the classical framework of the PMC-Index model. Firstly, nine main-variables need to be selected., the following main-variables policy type (X₁), policy timeliness (X₂), incentive measures (X₃), policy function (X₄), implementation mechanism (X₅), policy operability (X₆), policy release agency (X₈), and policy receptor (X₉) were selected with reference to the universal standard variables of the related research as listed in

Table 3. Variable selection of the PMC-Index model evaluation.

Main-Variables	Sub-Variables	Reference
Policy type (X1)	Predicting (X1:1); Proposal (X1:2); Describing (X1:3); Supervision (X1:4); Diagnosis (X1:5); Guidance (X1:6)	Yang et al. [48]
Policy timeliness (X2)	Long-term (X2:1); Medium-term (X2:2); Short-term (X2:3); Current year (X2:4)	Kuang et al. [38]
Incentive measures (X3)	Talent introduction and training (X3:1); Capital investment (X3:2); Tax preference (X3:3); Construction of industrial park (X3:4); Intellectual property protection (X3:5); Financial measures (X3:6); Public service (X3:7)	Dai et al. [40]
Policy function (X4)	Technological innovation (X4:1); Government procurement (X4:2); Achievement transformation (X4:3); Institutional constraints (X4:4); Normative guidance (X4:5)	Dai et al. [40]
Implementation mechanism (X5)	Organizational leadership (X5:1); Working mechanism (X5:2); Department collaboration (X5:3)	Li et al. [39]
Policy operability (X6)	Sufficient basis (X6:1); Clear objective (X6:2); Scientific plan (X6:3); Clear responsibility (X6:4)	Yang et al. [48]
Regional industry synergy (X7)	Industry chain (X7:1); Innovation chain (X7:2)	Cao et al. [45]
Policy release agency (X8)	Party committee 1 (X8:1); Government (X8:2); Development and reform commission 2 (X8:3); Government functional department (X8:4)	Wu et al. [49]
Policy receptor (X9)	Local government (X9:1); Industry park (X9:2); Enterprise (X9:3); College or university (X9:4); Scientific research unit (X9:5)	Yang et al. [48]

¹ This refers to the local committees at all levels and grassroots committees of the Communist Party of China. ² This is a government department responsible for overall planning and coordinating national economic and social development, and comprehensively coordinating economic restructuring.

, and the one main variable regional industry synergy (X₇) was selected according to the characteristics of the BTH’s coordinated development. Secondly, the high-tech industrial policy also has specific measures with different dimensions under each main-variable. Hence, each main variable is further decomposed into several sub-variables according to the text mining. For example, incentive measures (X₃) involve talent introduction and training (X_3:1), capital investment (X_3:2), tax preference (X_3:3), construction of industrial park (X_3:4), intellectual property protection (X_3:5), financial measures (X_3:6), and public service (X_3:7). Meanwhile, some of the sub-variables are appropriately improved according to the literature (Table 3). Finally, a PMC-Index model evaluation indicator system is constructed of nine main variables and 40 sub-variables (Table 3). The main variables are encoded by X₁ to X₉ and the sub-variables are encoded by X_1:1 to X_9:5.

Table 4. Evaluation standards for the sub-variables.

Variables		Evaluation Standards
X1	X1:1	Whether there is a pre-judgment on the direction and prospect of the industry in the policy
	X1:2	Whether there are countermeasures for industrial development in the policy
	X1:3	Whether there is a description of the current situation of industrial development in the policy
	X1:4	Whether there is supervision over the process of industrial development in the policy
	X1:5	Whether there is a diagnosis of the problems in industrial development in the policy
	X1:6	Whether there is a guiding goal for industrial development in the policy
X2	X2:1	Whether a 7 year or longer plan is involved in the policy
	X2:2	Whether a 4–6-year plan is involved in the policy
	X2:3	Whether a 2–3-year plan is involved in the policy
	X2:4	Whether a current year plan is involved in the policy
X3	X3:1	Whether it involves incentive measures for talent introduction and training
	X3:2	Whether it involves incentive measures for capital investment
	X3:3	Whether it involves incentive measures for tax preference
	X3:4	Whether it involves incentive measures for the construction of an industrial park
	X3:5	Whether it involves incentive measures for intellectual property protection
	X3:6	Whether it involves incentive measures for finance
	X3:7	Whether it involves incentive measures for public service
X4	X4:1	Whether the policy function involves the technological innovation
	X4:2	Whether the policy function involves the government procurement
	X4:3	Whether the policy function involves the achievement transformation
	X4:4	Whether the policy function involves the institutional constraints
	X4:5	Whether the policy function involves the normative guidance
X5	X5:1	Whether there is a leading organization for the policy implementation mechanism
	X5:2	Whether there is a working mechanism for the policy implementation
	X5:3	Whether it involves the negotiation and communication between departments and between superiors and subordinates
X6	X6:1	Whether the policy basis is sufficient
	X6:2	Whether the policy objective is clear
	X6:3	Whether the policy plan is scientific
	X6:4	Whether the division of responsibilities of relevant departments is clear
X7	X7:1	Whether it involves the construction of a regional industry chain
	X7:2	Whether it involves the construction of a regional innovation chain
X8	X8:1	Whether the policy release agencies involve the local party committee
	X8:2	Whether the policy release agencies involve the local government
	X8:3	Whether the policy release agencies involve the Development and Reform Commission
	X8:4	Whether the policy release agencies involve the government functional department
X9	X9:1	Whether the policy receptors involve local governments
	X9:2	Whether the policy receptors involve industry parks
	X9:3	Whether the policy receptors involve enterprises
	X9:4	Whether the policy receptors involve the college or university
	X9:5	Whether the policy receptors involve the scientific research unit

shows the specific evaluation standard for the sub-variables. The content of each policy text is analyzed according to the sub-variable evaluation standard. If positive, the value is “1”; otherwise, the value is “0”.

3.3.2. Building a Multi-Input-Output Table

As shown in

Table 5. Multi-input-output of each high-tech industrial policy.

Main-Variables	X1	X2	X3	X4	X5	X6	X7	X8	X9
Sub-variables	X1:1	X2:1	X3:1	X4:1	X5:1	X6:1	X7:1	X8:1	X9:1
	X1:2	X2:2	X3:2	X4:2	X5:2	X6:2	X7:2	X8:2	X9:2
	X1:3	X2:3	X3:3	X4:3	X5:3	X6:3		X8:3	X9:3
	X1:4	X2:4	X3:4	X4:4		X6:4		X8:4	X9:4
	X1:5		X3:5	X4:5					X9:5
	X1:6		X3:6
			X3:7

, a multi-input-output table is established to quantify the values of the main variables and provide a data analysis framework for the following specific calculation of the PMC-Index model. Based on the “Omnia Mobilis” assumption described above, the influence of every variable is balanced as much as possible by setting each main variable or sub-variable with the same weight [37].

3.3.3. PMC-Index Measurement

The values of sub-variables are set according to Formulas (1) and (2), and the values of the main variables are calculated by Formula (3). The PMC-Index is calculated by Formula (4).

$$X~N\left[0,1\right]$$

(1)

$$X=\left\{XR:\left[0~1\right]\right\}$$

(2)

$$X_t={\displaystyle \sum }_{j=1}^n\frac{x_{tj}}{T\left(x_{tj}\right)}$$

(3)

$$\mathrm{PMC}-\mathrm{Index}={\displaystyle \sum }_{t=1}^m{X}_t$$

(4)

where, t is the ordinal number of main variables, t = 1, 2, 3, …, m. j is the ordinal number of sub-variables, j = 1, 2, 3, …, n. T is the number of the total sub-variables under the corresponding main-variable in analysis.

According to the existing research [37,38,39,48], each of the policies is graded from “Unacceptable” to “Perfect” based on the PMC-Index (

Table 6. Evaluation grade for policies based on the PMC-Index.

PMC-Index	9~8	7.99~6	5.99~4	3.99~0
Evaluation grades	Perfect	Excellent	Acceptable	Unacceptable

).

3.3.4. PMC-Surface Construction

A PMC-Surface shows the value of PMC-Index in the form of a graph, which can display the advantages and disadvantages of the high-tech industrial policy more intuitively. The main variable is introduced into the PMC-matrix based on the variable score; the PMC-Surface is then calculated according to Formula (5).

$$\mathrm{PMC}-\mathrm{Surface}=\left[\begin{array}{ccc}{X}_1& X_2& X_3\\ X_4& X_5& X_6\\ X_7& X_8& X_9\end{array}\right]$$

(5)

4. Results and Analysis

4.1. The PMC-Index of the Policies

A multi-input-output table of the seven policies was established (Appendix A,

Table A1. Multi-input-output of seven high-tech industrial policies of the BTH region.

		P1	P2	P3	P4	P5	P6	P7
X1	X1:1	1	0	1	1	1	0	0
	X1:2	1	1	1	1	1	1	1
	X1:3	1	1	1	0	0	0	0
	X1:4	1	1	1	1	1	1	1
	X1:5	1	1	0	0	0	0	0
	X1:6	1	1	1	1	1	1	1
X2	X2:1	0	0	1	1	0	0	1
	X2:2	1	1	1	1	1	0	1
	X2:3	1	1	1	1	0	0	1
	X2:4	1	1	0	1	0	1	1
X3	X3:1	1	1	1	1	1	0	1
	X3:2	1	1	1	1	1	1	1
	X3:3	1	1	1	1	1	1	1
	X3:4	1	0	1	1	1	0	1
	X3:5	1	1	1	1	1	1	1
	X3:6	1	0	1	1	1	1	0
	X3:7	1	0	1	1	0	0	1
X4	X4:1	1	1	1	1	1	1	1
	X4:2	1	0	1	1	0	0	1
	X4:3	1	0	1	1	1	1	0
	X4:4	1	1	1	1	1	1	1
	X4:5	1	1	1	1	1	1	1
X5	X5:1	1	1	1	1	1	1	1
	X5:2	1	1	1	1	1	0	1
	X5:3	0	1	0	1	1	0	1
X6	X6:1	1	1	1	1	1	1	1
	X6:2	1	1	1	1	1	1	1
	X6:3	1	1	1	1	1	1	1
	X6:4	0	1	1	1	1	1	0
X7	X7:1	1	0	1	1	1	1	0
	X7:2	1	1	1	1	1	1	0
X8	X8:1	0	0	0	0	0	0	0
	X8:2	0	1	1	1	1	1	0
	X8:3	1	0	0	0	0	0	0
	X8:4	0	0	0	1	0	0	1
X9	X9:1	1	1	1	1	1	1	1
	X9:2	1	0	0	1	1	0	1
	X9:3	1	1	1	1	1	1	1
	X9:4	1	0	1	1	1	0	0
	X9:5	1	0	1	1	1	1	0

), and the PMC-Index values and grades were calculated, with the results shown in

Table 7. PMC-Index values and grades of seven high-tech industrial policies of the BTH region.

	P1	P2	P3	P4	P5	P6	P7	Mean
X1	1.00	0.83	0.83	0.67	0.67	0.50	0.50	0.71
X2	0.75	0.75	0.50	1.00	0.25	0.25	1.00	0.68
X3	1.00	0.57	1.00	1.00	0.86	0.57	0.86	0.84
X4	1.00	0.60	1.00	1.00	0.80	0.80	0.80	0.86
X5	0.67	1.00	0.67	1.00	1.00	0.33	1.00	0.81
X6	0.75	1.00	1.00	1.00	1.00	1.00	0.75	0.93
X7	1.00	0.50	1.00	1.00	1.00	1.00	0.00	0.79
X8	0.25	0.25	0.25	0.50	0.25	0.25	0.25	0.29
X9	1.00	0.40	0.80	1.00	1.00	0.60	0.60	0.77
PMC-Index	7.42	5.90	7.05	8.17	6.82	5.30	5.76	6.67
Evaluation grades	Excellent	Acceptable	Excellent	Perfect	Excellent	Acceptable	Acceptable	Excellent

. On the whole, the high-tech industry policies in the BTH region are excellent on average, because the mean value of the PMC-Index is 6.67. The ranking of the seven policies based on the PMC-Index is P4 > P1 > P3 > P5 > P2 > P7 > P6. According to the grade division, among the seven policies, one is perfect and three are excellent and three are acceptable.

4.2. The PMC-Surface of the Policies

The PMC-Surface displays the quantitative evaluation results of the PMC-Index in a three-dimensional and intuitive way. In the curve graph, the convex or forward part indicates that the score of the corresponding variable is relatively high, while the concave or backward part indicates that the score of the corresponding variable is low. We established the PMC-Matrix and the PMC-Surface graphs and selected P4 with the highest PMC-Index and P6 with the lowest PMC-Index as examples for analysis in this paper, as shown in Figure 4 and Figure 5. The surface of P4 is relatively smooth, compared with P6, and is in the high score range according to color. Therefore, P4 is judged to have a strong performance, with its performance in various indicators more balanced, while P6 has a great weakness.

4.3. Analysis for Each of the Seven Policies

In this study, the evaluations of each policy can be analyzed in detail and the order of policy improvement can be judged according to the following two principles [48]: (1) the first priority is given to the main variable whose value is lower than the mean and the variable with the larger difference has priority; and (2) the second priority is given to the main variable whose value is lower than the full mark but higher than the mean and the variable with the larger difference has priority. A larger difference in the priority indicates poorer policy performance (and thus a priority for optimization). Certainly, the approach is not absolute, but can be adjusted according to the specific situation.

P4 gains the first ranking with a PMC-Index value of 8.17, and its grade is perfect. Among the nine main-variables, except for X₁ (policy type) and X₈ (policy issuing agency), the other variables get the full score, and only X₁ scores lower than average, which indicates that this policy design is relatively reasonable and scientific, and that policy dimensions have been considered comprehensively. As a policy jointly issued by Tianjin Science and Technology Bureau and Binhai New Area government, P4 is the only one jointly issued by two publishing agencies among the seven and thus has a higher score than the others. The optimization of X₈ and X₁ across the order X₈–X₁ (from X₈ to X₁) would enhance the effectiveness of this policy.

P1 gains the second ranking with a PMC-Index value of 7.42, and its grade is excellent. P1 has a loss of scores in the variables of X₂ (policy timeliness), X₅ (implementation mechanism), X₆ (policy operability) and X₈ (policy release agency), among which the scores of X₅, X₆ and X₈ are lower than the average. In particular, because of the lack of inter-departmental collaboration and clear working mechanism, the scores of X₅ and X₆ are 0.14 and 0.18 lower than the average, respectively, and the difference value is much higher than that of X₈. Therefore, we can focus on the improvement of P1 from the two indicators X₅ and X₆ and the optimization path of the policy is X₆–X₅–X₈–X₂.

P3 gains the third ranking with a PMC-Index value of 7.05, and its grade is excellent. Seven main variables are higher than or equal to the mean. The variable value of X₅ (implementation mechanism) is 0.14 lower than average, while the value of X₈ (policy release agency) is 0.04 lower than average. However, among the seven main-variables, the difference values of X₁ (policy type), X₂ (policy timeliness), and X₉ (policy receptor) are 0.17, 0.25, and 0.20 respectively. Thus, the optimization path of this policy is X₅–X₈–X₂–X₉–X₁.

P5 gains the fourth ranking with a PMC-Index value of 6.82, and its grade is excellent. The scores of X₁ (policy type), X₂ (policy timeliness), X₄ (policy function), and X₈ (policy release agency) are lower than the average with the differences of 0.05, 0.43, 0.06, and 0.04, respectively. Among them, the difference between the X₂ value and the mean is the largest, and the X₂ value of P5 is also the lowest among the seven policies. X₃ (incentive measures) also needs to be improved because it is not full score. The improvement path for P5 is X₂–X₄–X₁–X₈–X₃.

P2 gains the fifth ranking with a PMC-Index value of 5.90, and its grade is acceptable. The values of X₃ (incentive measures), X₄ (policy function), X₇ (regional industry synergy), X₈ (policy release agency), and X₉ (policy receptor) are below average and the differences from the average are 0.27, 0.26, 0.29, 0.04, and 0.37, respectively. P2 has a low value of X₉ (policy receptor) because it only covers the government and enterprises. X₁ (policy type) and X₂ (policy effectiveness) are 0.17 and 0.25 lower than the full score respectively. Thus, the optimization path of the policy is X₉–X₃–X₄–X₇–X₈–X₂–X₁.

P7 gains the sixth ranking with a PMC-Index value of 5.76, and its grade is acceptable. The scores of X₁ (policy type), X₄ (policy function), X₆ (policy operability), X₇ (regional industry synergy), X₈ (policy release agency), and X₉ (policy receptor) are below average with differences of 0.21, 0.06, 0.18, 0.79, 0.04, and 0.17, respectively. The policy does not involve the content of the BTH regional industry synergy, so it is the only policy with X₇ score of 0. The best performance of the policy lies in X₂ (policy timeliness) and X₅ (implementation mechanism), which get the full score. The improvement path for P7 is X₇–X₁–X₆–X₉–X₄–X₈–X₃.

P6 gains the last ranking with a PMC-Index value of 5.30, and its grade is acceptable. X₆ (policy operability) and X₇ (regional industry synergy) get full score and only the two variables are not lower than the mean. X₅ (implementation mechanism) has a value of 0.33, and it is the variable with the largest difference (0.48) between its score and the mean value among all main-variables. In term of the implementation mechanism, P6 only involves the leading organization of policy implementation while failing to include the working mechanism and interdepartmental collaboration. The differences between the other main-variables and their mean value are 0.21, 0.43, 0.27, 0.06, 0.04, and 0.17, respectively. Therefore, the improvement path of P6 is X₅–X₂–X₃–X₁–X₉–X₄–X₈.

4.4. Comparative Analysis

The characteristics and problems of high-tech industry policies in the BTH region are further clearly expressed in this study by performing a horizontal comparative analysis of the seven policies based on the multi-input-output table (Table 7).

X₁ (policy type) has a mean value of 0.71, ranking seventh among the nine main-variables. This variable average is low, so that it is an area that needs to be especially strengthened. The policy type determines whether a policy has features such as predicting, proposal, describing, supervision, diagnosis, and guidance. Except for P1, which is relatively comprehensive, the other policies are somewhat deficient, mainly because of the lack of diagnosis and describing content.

X₂ (policy timelines) has a mean value of 0.68 with ranking eighth of all the main variables. Therefore, it is also a deficiency that needs to be urgently improved. Policy timelines mainly involve the combination of long-term, medium-term, short-term, and current year objectives. Except for P4 and P7, which score full marks in this variable and are comprehensively involved in goal setting, the other policies lose points mainly due to the lack of long-term goals.

The mean value of X₃ (incentive measures) is 0.84, ranking third among the nine main-variables. To promote the implementation of policy objectives, a series of incentive measures should be usually formulated in the content when policies are introduced [40]. The incentive measures of the high-tech industry policy mainly include the following categories: talent introduction and training, capital investment, tax preference, construction of industrial park, intellectual property protection and financial measures. Among these policies, P1, P3, and P4 all involve the above incentive measures. The other policies generally lack one or two types of incentive measures and need to supplement public service and financial measures.

The mean value of X₄ (policy function) is 0.86, ranking second of the nine main variables. Policy function is the inherent attribute of policy, which refers to the role and expected effect that the policy can play in the implementation process [40]. High-tech industrial policies mainly involve technological innovation, government procurement, achievement transformation and institutional constraints and normative guidance. Only P1, P3, and P4 contain the above functions, and the other policies do not involve government procurement or achievement transformation.

X₅ (implementation mechanism) has a mean value of 0.81, ranking fourth in the main variables. The implementation mechanism is very important for one policy, and the high-tech industrial policy is no exception. It mainly involves organizational leadership, working mechanism and departmental collaboration [39]. P1, P3, and P6 policies do not earn full marks on X₅, with the main reason for point loss being that none of the three policies involved department collaboration.

X₆ (policy operability) has a mean value of 0.93, ranking first of the nine main variables, which shows that the high-tech industrial policies of the BTH region are relatively comprehensive in terms of sufficient basis, clear objective, scientific plan, and clear responsibility. Except for P1 and P7, which lack clear responsibility, the other policies gain full marks on X₆.

The mean value of X₇ (regional industry synergy) was 0.79, ranking fifth among the main variables. As a system of regional economic development, urban agglomeration constitutes a region under the current innovation network [19]. Policymakers face the challenge of promoting deep coordination in the industry chain and innovation chain in the BTH region [44]. P2 and P7 scored below average while the other policies earned full marks, which indicates that policymakers pay attention to the coordination of regional industrial chain or innovation chain in the BTH region.

X₈ (policy release agency) is the main variable with the worst performance, and its mean value is only 0.29. P4 earned 0.5, and the other policies all earned only 0.25. Generally speaking, the joint issuance of policies by multiple departments is conducive to the comprehensiveness of a policy [49]. P2, P3, P5, and P6 are issued by the government, P1 and P7 are issued by the government functional department, but P4 is jointly issued by the government and the government functional department.

The average value of X₉ (policy receptor) is 0.77, ranking sixth among the main variables. P1, P4, and P5 all get full marks, and P1, P3, P4, and P5 score higher than the average, indicating that the orientation of these policy receptors is relatively clear. P2, P6, and P7 score lower than the average, and P2 earns the lowest value, which is only 0.4. The receivers of high-tech industrial policies generally include local government, industry park, enterprise, college or university and scientific research unit. However, some policies for high-tech industry in the BTH region need to supplement these receivers.

5. Conclusions and Discussion

In this study, with the BTH urban agglomeration as a case, a combination of the PMC-Index model and text mining was introduced to implement an evaluation for the high-tech industry policies based on the policy texts. It is found that the policies in the case are generally reasonable but there is still much room for improvement, and the shortcomings of each policy were analyzed and improved according to the multi-input-output table. The consistency and completeness of policy texts is a significant factor influencing the degree of achievement of policy objectives [38]. However, the existing literature focuses on the policy impact [30,31], and few scholars have comprehensively evaluated different policy texts in the field of high-tech industry policies. This study suggests the advantages of the PMC-Index model on evaluating this aspect of high-tech industry policies, though the PMC-Index model has been maturely used in other public policy areas [38,39,40,41,42]. Furthermore, the BTH region, similar to other urban agglomerations around China, faces the challenge of promoting the regionally-coordinated development and low carbon development at present [50,51]. In this view, it is also important to evaluate the consistency of high-tech industry policies in a region.

The main conclusions can be summarized as follows. Firstly, the design of high-tech industry policies in the BTH region is reasonable in general. The PMC-Index values of the seven policies range from 5.30 to 8.17 with an average of 6.67. Secondly, the high-tech industry policy levels of Beijing, Tianjin, and Hebei are relatively balanced. Among the two policies in Beijing, one is excellent and the other is acceptable; among the two policies of Tianjin, one is perfect and the other is acceptable; and among the three policies of Hebei, two are excellent and one is acceptable. Thirdly, the overall performance of these polices on regional industry synergy (X₇) is good, because all policies except for P2 and P7 have full scores, which shows that the BTH region attaches importance to the coordinated development of the high-tech industries. Fourthly, there is still much room for improvement in the high-tech industry policies of the BTH region and the performances of the policies in terms of policy release agency, policy timeliness, policy type and policy receiver are comparatively weak. Specific improvements can be made with reference to the evaluation standards and the multi-input-output table. The values of these variables may provide a workable foundation for policy improvements.

Because it is difficult to effectively measure or quantify the policy, high-tech industry policy evaluation is a complex work in the field of economics and management. This study develops the evaluation method and index system from the perspective of policy text evaluation and regional collaborative development. At the same time, this research provides a theoretical reference for policymakers to formulate or improve high-tech industrial policies. However, with the change of policy environment, some original evaluation standards might not adapt to the new environment. Hence, the policy cannot be simply evaluated according to this method, and stakeholders also need to take comprehensive factors into consideration including different backgrounds, times and social needs. In addition, there are still some limitations in this study. The limitations of this study are mainly reflected in the choice of variables. In terms of variable setting, some universal standard variables are not only referred to, but some non-standard variables are also set according to this research problem and the characteristics of specific policy texts, which is subjective to some extent. To reduce the subjectivity, future researchers can comprehensively use a variety of text analysis techniques to mine the characteristics of policy texts. Although the data of this study are only from the BTH region, the research method is also applicable to other regions. In addition, regionally-coordinated development is advocated by many regions. Therefore, the high-tech industrial policies of different regions can be selected for comparison in the future.