Research on Evaluation Method of Digital Project Cloud Model Considering Weight Sensitivity

Abstract

Digitization is a driving force for social development and corporate innovation. Digital projects have become an indispensable part of the sustainable development of enterprises. However, due to the imperfect decision-making system of digital projects and the lack of experience of traditional enterprises’ digital projects, the decision-making of digital projects is an unavoidable challenge in the digital transformation of enterprises. For the digital project decision of the STATE GRID Corporation of China, this paper conducts a sensitivity analysis of digital project evaluation index weights based on cloud model theory, on top of historical successful project experience to support digital project decision-making. Firstly, this paper establishes a comprehensive evaluation index system for digitalization projects from five aspects: economic efficiency, interconnection, intelligent management, value release, and development innovation. The coefficient of variation method is used for index screening, and the weight intervals are formed by four subjective and objective assignment methods. Then, the LSOM model is established to generate the weight values in the interval, and, finally, the sensitivity of digital project comprehensive evaluation indexes is analyzed based on the cloud model to select the most robust index weights for project evaluation and choose the optimal project. The feasibility of the proposed method is verified by arithmetic examples.

1. Introduction

A broad and profound digital revolution is coming in this era. Digital transformation is playing an increasingly prominent role in various application areas, such as service providers [1], smart cities [2], public sector services [3], healthcare and infectious diseases [4,5], and marketing [6], among others. At the national level in China, digitization is receiving increasing attention. The development of digitalization will be the future direction of the country for some time. During China’s 13th Five-Year Plan, companies have made significant progress in information technology development. The 19th Party Congress proposed to build a strong network, a digital China and a smart society, and to promote the deep integration of the Internet, big data, artificial intelligence, and the real economy. China’s 14th Five-Year Plan proposes to “accelerate digital development and build a digital China”. Digital transformation has gradually become an inevitable choice for enterprises to survive, create core competitiveness, and achieve sustainable development. Power grid enterprises follow the social development trend and implement big data strategy. However, compared with digital native enterprises, the digital transformation of traditional enterprises is difficult [7]. The choice of digital project solution issues a challenge to traditional enterprises, such as grid enterprises. If the wrong choice is made, it will slow down the digital transformation of the enterprise and make the enterprise suffer from the digital wave. Therefore, in order to reduce the negative impact of program decision errors and avoid irreparable damage, grid enterprises need to establish a comprehensive and appropriate digital project program selection evaluation index system. Based on the experience of implemented digitalization projects, a sensitivity analysis of the index weights has been conducted to determine reasonable evaluation index weights, and, finally, according to the results, to determine a robust long-term quality digitalization project.

The power grid system is an important aspect of infrastructure for national development. Its construction scale and quality will be directly related to people’s livelihood and economic development [8]. Along with the continuous development of digital technology in China, the connection between digitalization and the electric power network is gradually deepening, and the importance of digitalization projects in power grid enterprises is becoming increasingly apparent. At present, in the face of the complex application of multiple scenarios of digital construction, multi-dimensional comprehensive evaluation, intelligent identification of target characteristics, quantitative control of dynamic evaluation, and other technical and economic evaluation needs, the traditional post-evaluation methods carried out manually cannot realize the requirements of online automated intelligent evaluation and cannot adapt to the new situation of digital construction evaluation needs. There is an urgent need to carry out research for the evaluation of grid digitalization construction project programs. Considering various factors, such as digital system and cross-domain data reuse, an index system for evaluating the economic benefits of enterprise digitization is established. An index screening and weight sensitivity analysis model is constructed to solve the practical problems of quantitative evaluation faced in the process of digital transformation and construction of enterprises. Therefore, this paper aims to establish a cloud-model-based evaluation and selection decision method for the digitalization project scheme of power grid enterprises. First, this paper uses CiteSpace software to visualize and analyze the literature on enterprise digitization published in the CNKI platform and draw a knowledge map. We explore the research hotspots and research trends of enterprise digitization and explore the development of digitization project evaluation indexes and evaluation methods, as well as cloud models. Secondly, based on the existing focus and research results, a comprehensive evaluation index system for digitalization projects is established from five aspects: economic efficiency, interconnection, intelligent management, value release, and development innovation. In addition, for the digitalization project program, the coefficient of variation method is first used to filter on the basis of the established index system. Then, the LSOM (linear stochastic optimized model) is used to generate evaluation index weights in the range. Then, a sensitivity analysis is performed on the evaluation index weights based on the cloud model to discern the most universal evaluation index weights. The obtained weights are used to decide the robust and long-term quality digitalization projects to promote the digital transformation of grid enterprises. Finally, the calculation example shows that the method is feasible and has some practical significance for the comprehensive evaluation of digitalization projects.

2. Literature Review

2.1. Enterprise Digitization

After the 1990s, the new technological revolution, represented by digital technology, has advanced by leaps and bounds. Informatization, digitization, and intelligence have advanced rapidly, and the leading position of big data has gradually come to the fore. Especially in recent years, the global economy has entered a new period of deep adjustment and structural change, and the digital economy has developed rapidly, providing tremendous energy for economic recovery and social progress. Countries have introduced corresponding policies to promote the transformation of traditional industries to digitalization. As the research object of this paper is the digitization project proposal comparison of STATE GRID Corporation of China, the CNKI database is adopted for the visual analysis of the literature. CNKI, the China National Knowledge Infrastructure, is a professional knowledge database and a leading digital publishing platform in the world. It is a professional website dedicated to providing knowledge and intelligence services for various industries at home and abroad. Since there are few studies related to the digitization of power grid enterprises, this paper uses the keywords of enterprises and digitization to visualize and analyze 3151 documents about the digitization of enterprises published in the CNKI platform.

As can be seen from Figure 1, the number of articles issued from 2002 to 2010 was relatively small, basically below one hundred. Then, 2011–2014 saw a more moderate and steady increase in the number of articles issued, but there was a small decline from 2015 to 2017. Further, 2018 and 2019 saw the number of articles issued usher in a rebound, and 2020 and 2021 represented a significant increase, reaching more than eight hundred articles at the end of 2021. The number shows that research on enterprise digitization can still become a major hot direction in the future.

Keywords are the embodiment of a paper’s core ideas and keenly summarize the topic of the paper. Therefore, this paper analyzes the keywords of enterprise digitization literature to explore the research hotspots and research trends in this field. Figure 2 shows the effect of keyword clustering using CiteSpace. CiteSpace’s computing results show that a total of 385 nodes and 848 connected lines were generated for the keyword analysis graph of enterprise digital literature. The size of the nodes in the graph is determined by the frequency of keyword occurrences and represents the research hotness in the field. From the high-frequency keywords in the field of enterprise digitalization derived from the statistical analysis, we can see that the keywords appearing more frequently are: digital transformation (487), digital economy (366), big data (90), enterprise digital transformation (89), intelligent manufacturing (71), circulation industry (69), digital manufacturing (64), and pillar one (63). According to the information in Figure 1, most of the research on enterprise digitalization can be grouped under the themes of digital transformation, manufacturing development, and big data.

(1)

Digital Transformation

The purpose of digital transformation in traditional industries is: to use digital technology to crack the problems in the development of enterprises and industries, to redefine and redesign products and services, and to achieve business transformation, innovation, and growth [9]. In practice, digital transformation has a great impact on the digitalization process, innovation model, and value creation of enterprises. The authors of Ref. [10] pointed out that realizing the digital transformation of traditional manufacturing enterprises to smart manufacturing requires the application of new technologies, such as big data, cloud computing, block chain, and Internet of things, to enhance the technological innovation capability of enterprises. The authors of Ref. [11] found that the intelligent development of enterprises will form the substitution of advanced machinery and equipment for low-end labor, while the application of digital technology will increase the labor demand for highly educated labor and optimize the human capital structure of enterprises. Digital transformation for a wide range of enterprises is conducive to promoting the continuous growth of the digital economy.

(2)

Manufacturing Development

With the rapid development of information technology, the traditional manufacturing industry is also innovating and integrating with it. The digitalization of the manufacturing industry also occupies an important position in the new round of global industrial change. The authors of Ref. [12] specified that, with the development of a new generation of information technology, the deep integration of manufacturing and digital technology has given rise to a new model of networked collaborative manufacturing. The authors of Ref. [13] found that, in the era of the digital economy, the application of digital technology significantly improves the efficiency of production tools, which can significantly shorten machine overhaul time, downtime, and switching time between processes, reduce operation and maintenance costs and inventory costs, and significantly improve production efficiency. The authors of Ref. [14] proposed that manufacturing enterprises should strengthen the assessment and diagnosis of digital transformation in the process of promoting digital transformation, periodically carry out assessments to comprehensively grasp the current situation of digital transformation of enterprises, and optimize and adjust digital transformation strategies and measures. Digital transformation is an important way to transform the development path of the Chinese manufacturing industry and promote the high-quality development of the manufacturing industry.

(3)

Big Data

With the advent of the data era, big data has become a new driving force for high-quality economic development. Through big data management, enterprises can better understand the business environment and customer needs. In the digital economy context, opportunity-awareness is the ability of organizations to continuously scan the external turbulent environment brought about by digital technology, perceive the latest digital trends and opportunities, integrate and integrate data information, and form digital thinking to accurately predict and quickly respond to customer needs [15]. The authors of Ref. [16] clarified that, to a certain extent, the value of big data does not only lie in the data resource itself but more in the driving effect of data combined with specific process practices. The authors of Ref. [17] reveal the mechanism of action of dynamic capabilities to stimulate data-driven effects for digital transformation and draw conclusions with certain theoretical value and practical insights. Big data has created opportunities for the development of enterprises, and enterprises still need to translate big data into enterprise performance.

The compilation of emergent research keywords helps to understand the hot spots of research in the literature at a certain stage. As can be seen in Figure 3, the top three are digital manufacturing (4.02), theoretical framework (3.83), and key technology (3.62). Among them, “theoretical framework” has been around for the longest time and has been a key point in the academic community from 2002 to 2017. The shortest keyword is “digital manufacturing”, which lasted from 2002 to 2013. The keyword “key technology” appeared in 2004 and lasted for only 2 years, while “digital factory” appeared in 2014 and lasted for a shorter period. After 2019, terms such as “digital finance”, “high quality development”, and “transformation” and the emergence of terms such as “upgrading” are closely related to the sharp increase in the number of publications after 2019, as shown in Figure 1, where scholars realize the value of digitalization as the Internet and digital processing became more powerful. Coupled with policy support in recent years, research on enterprise digitization has gradually deepened in the dimensions of digital finance, high-quality development, and transformation and upgrading.

2.2. Digital Evaluation

Enterprise digitization is a new development stage for which fewer digitization projects have been evaluated. However, enterprise informatization and enterprise digitization are two development stages that are intertwined. There are many coexisting and shared things at the technical and methodological levels that can be learned from each other. Most of the existing studies focus on the construction of an evaluation system and innovation of evaluation methods for enterprise digital transformation. In the early stage, scholars constructed the evaluation system of information technology from the perspective of competitive advantage [18], strategic objectives [19], etc. With the increasing emphasis on the digital capabilities of companies, scholars’ attention began to shift. The authors of Ref. [20] developed a set of key readiness indicators (KRI) methodology within a self-assessment framework tailored to the needs of SMEs to deepen the interpretation of the overall digital level of companies in specific areas of intervention. The authors of Ref. [21] constructed an evaluation index system around two dimensions: productive capacity and enterprise system. The authors of Ref. [22] used a four-level approach to assess the degree of digital utilization of SMEs in terms of digital awareness, digital demand, digital collaboration, and digital transformation. In terms of evaluation methods, the existing studies include assessment through partial least squares structural equation modeling [23], exploration of transformation path framework based on the dynamic capability perspective [24], assessment using synergy and substitution effects [25], and visual analysis of KPI matrix [26]. It is worth noting that there are more studies on models of digital transformation maturity, and models of digital maturity studies have been published in the literature [27,28] and by the German National Academy of Science and Engineering [29]. These studies provide a solid theoretical basis for the establishment of the evaluation index system of enterprise digitalization projects in this paper.

2.3. Cloud Model

Cloud modeling is a very suitable tool for uncertain scenarios and analysis of sensitivity. Existing studies have applied cloud models to linguistic information and uncertain linguistic information environments [30,31], uncertain reasoning problems [32], virtual business partner selection [33], and agricultural surface source pollution sensitivity level evaluation [34]. In terms of project evaluation, the authors of Ref. [35] used a cloud model approach to represent evaluation information in order to convert between qualitative information and quantitative values. The authors of Ref. [36] proposed a cloud-model-based evaluation method for water resources evaluation problems. The authors of Ref. [37] proposed a state evaluation algorithm based on combined assignment and cloud model. The authors of [38] introduced a cloud model for sensitivity analysis. The authors of Ref. [39] applied the cloud generator to obtain the index affiliation of economic, social, and ecological environmental benefits. The performance of fitness on improving cloud models was further verified by post-evaluation of benefits in dykes. The authors of Ref. [40] also showed the cloud map of a post-evaluation of wastewater treatment in Enshi through the cloud model, which visualized the evaluation value of the project in a concrete way. Besides, cloud models have also been used in indicator assignment [41]. As for the power-related aspects, there are currently the analysis of the sensitivity to the probability of component failure [42], the preliminary selection of substation sites for distribution networks [43], and the assessment of the reasonableness of the environmental sensitivity of typical regional falling lightning pregnancy [44]. However, there is still a gap in the analysis of sensitivity of comprehensive evaluation indexes for digitalization projects of power enterprises, which has not been explored using the cloud model. Using the advantages of uncertainty and fuzziness of cloud model for sensitivity analysis of comprehensive evaluation indexes of digital projects is a problem worthy of research.

3. Comprehensive Evaluation Index System of Digital Projects

The current evaluation index system in enterprise digitalization is generally the evaluation index system of enterprise digital transformation or the evaluation index system of enterprise digital-level assessment. The research on the comprehensive evaluation of the digitalization project program is still in the initial stage, and a unified evaluation index system has not been formed yet. This paper establishes an evaluation index system based on existing literature studies and the STATE GRID Corporation of China’s understanding of digital transformation. The idea is shown in Figure 4.

The digital transformation of the STATE GRID Corporation of China is based on the following three points: (1) digitalization is an inevitable choice to adapt to the trend of the energy revolution and the digital revolution; (2) digitalization is an inherent requirement to enhance management and improve services; (3) digitalization is a powerful engine to cultivate new opportunities and new growth points. Based on this, the STATE GRID Corporation of China for digitalization project program focus on the evaluation direction can be from these three aspects. The first point lies in the interconnection, that digitalization is based on accurate data collection, efficient transmission, and safe and reliable utilization, which cannot be separated from the support of network, platform, and other software and hardware infrastructure, as well as that the data can be reused in the field, which is also the embodiment of its connectivity. The second point lies in the intelligent management, digital project operation, and enhancing the effect to a large extent, which affects whether it can achieve a more excellent use of value, and data system security is also an important guarantee for management. The third point is the value release; big data is a “rich mine” and there is a huge value mining potential based on the digital project national grid that can actively develop platform business, data products business, and efforts to provide customers with multiple services.

Besides, it was found in the process of literature review that economy, efficiency, and prospect are usually considered in the construction of an evaluation index system. Economy and efficiency are in conflict with each other, so the evaluation of the digitalization project should not focus on economic efficiency or efficiency improvement, but economic efficiency is the best goal. In the digital project prospect, the degree of fit with the STATE GRID Corporation of China development direction and innovation ability is the key assessment direction of whether the digital project can be developed for a long time. Therefore, the development of innovation should also be included in the evaluation scope of the digital project program.

In summary, this paper takes the objectives reached by the digitalization project of power grid enterprises as the basis and follows the principles of comprehensiveness, scientificity, purposefulness, independence, and combination of quantitative and qualitative. The comprehensive evaluation index system is divided into five primary evaluation indexes of economic efficiency, interconnection, intelligent management, value release, and development innovation and has corresponding secondary evaluation indexes, totaling 22 secondary indexes. Regarding the setting of the second-level indicators and the calculation methods shown, they are improved on the basis of the research on enterprise digitization literature, fitting the nature of digitization project programs, and not exactly the same as the results of existing studies. Among them, ${\mathit{a}}_{\mathbf{3}}$, ${\mathit{a}}_{\mathbf{5}}$, ${\mathit{a}}_{\mathbf{9}}$, ${\mathit{a}}_{\mathbf{11}}$, ${\mathit{a}}_{\mathbf{12}}$, ${\mathit{a}}_{\mathbf{13}}$, ${\mathit{a}}_{\mathbf{14}}$, ${\mathit{a}}_{\mathbf{15}}$, ${\mathit{a}}_{\mathbf{16}}$, ${\mathit{a}}_{\mathbf{17}}$, ${\mathit{a}}_{\mathbf{18}}$, ${\mathit{a}}_{\mathbf{19}}$, ${\mathit{a}}_{\mathbf{20}}$, ${\mathit{a}}_{\mathbf{21}}$, ${\mathit{a}}_{\mathbf{22}}$ are qualitative indicators, which cannot be calculated to obtain the evaluation value, and experts in the relevant fields need to be invited to score this evaluation value under the program. The scoring criteria are: the scoring criteria range from 1–10 points, the neutral value is 5 points, the higher the score, the better the effect, and the lower the score, the worse the effect.

3.1. Economic Efficiency

(1)

Income Growth Index ${\mathit{a}}_{\mathbf{1}}$

Reflect the performance of digital projects. The income growth index is a comprehensive assessment index, which examines the growth rate of the project’s own income, the growth rate of the project’s own profit, the average income growth rate of the same type of projects, the average profit growth rate of the same type of projects, and the relationship between them.

Income Growth Index = 25% × the growth rate of the project’s own income + 25% × the growth rate of the project’s own profit + 25% × the average income growth rate of the same type of projects + 25% × the average profit growth rate of the same type of projects

(2)

Degree of Cost Savings ${\mathit{a}}_{\mathbf{2}}$

Reflect the benefit status after the implementation of the digital project.

Degree of Cost Savings = (costs before the implementation of digital projects − costs after the implementation of digital projects)/costs before the implementation of digital projects × 100%.

(3)

Business Process Improvement ${\mathit{a}}_{\mathbf{3}}$

Reflect the performance of digital projects. This indicator mainly reflects the improvement of business processes and business functions after the application of the information system, including the deletion and streamlining of redundant processes. The evaluation value is obtained by scoring.

(4)

Data Multiplexing Effect ${\mathit{a}}_{\mathbf{4}}$

Reflect the efficiency of digital projects. This indicator mainly reflects the effect of digital projects on the use of existing data.

Data Multiplexing Effect = (number of fields using existing data in digitization project/total number of fields requiring data for digitization project) × 100%

(5)

System Efficient Construction Level ${\mathit{a}}_{\mathbf{5}}$

Reflect the efficiency of digital projects. This indicator mainly reflects the construction level of the digital system of the digital project, including the simplification of business processing procedures and the degree of functional perfection. The evaluation value is obtained by scoring.

3.2. Interconnection

(1)

Office Automation Operating System Application Degree ${\mathit{a}}_{\mathbf{6}}$

Reflect the status of office automation based on network applications in digital projects.

Office Automation Operating System Application Degree = (the number of functions realized by the office automation system of the digital project/the total number of office automation system functions) × 100%

(2)

Coverage of Digital Means of Information Collection ${\mathit{a}}_{\mathbf{7}}$

Reflect the ability of a digital project to effectively acquire external information.

Coverage of Digital Means of Information Collection = (number of fields in which digital means have been applied to information collection in digital projects/number of fields where digital means can be applied) × 100%

(3)

Data Multiplexing Field ${\mathit{a}}_{\mathbf{8}}$

Reflect the areas in which the data collected by the digitization project are utilized. This indicator mainly reflects the ability of digital projects to reduce the cost and expense of repeated development of digital technology.

Data Multiplexing Field = (number of fields that can be used for data collected by digital projects/total number of fields) × 100%

3.3. Intelligent Management

(1)

User-friendly Interface ${\mathit{a}}_{\mathbf{9}}$

Reflect how the digital project operating system operates. Friendly interface usually means that the software has a beautiful opening interface, easy to learn, easy to use, strong operability, promotes human-computer interaction, and gives people a beautiful, comfortable, and generous feeling. The evaluation value is obtained through the satisfaction survey of digital project operators.

(2)

Business Function Achievement ${\mathit{a}}_{\mathbf{10}}$

Reflect how the digital project operating system operates. This indicator mainly reflects the ability of the digital project to complete the business.

Business Function Achievement = (digital project actual business function/digital project expected business function) × 100%

(3)

Digitization Level of Core Business Processes ${\mathit{a}}_{\mathbf{11}}$

Reflect the nature of the digital project operating system itself. This indicator mainly reflects the depth and breadth of digitization of core business processes of digital projects. This indicator is examined through the scope of business processes covered by digital technology and whether the business process achieves optimal control.

(4)

System Reliability ${\mathit{a}}_{\mathbf{12}}$

Reflect the nature of the digital project operating system itself. This indicator mainly reflects the reliability of the digital project operation system. Check the fault tolerance of key functions, whether they can be recovered, and the mean time between failures.

(5)

Data Security ${\mathit{a}}_{\mathbf{13}}$

Reflect the nature of the digital project operating system itself. This indicator mainly reflects the ability of digital projects to protect data. Check the auditability of access, controllability of access, data confidentiality, and data security.

3.4. Value Release

(1)

Customer Service Improvement ${\mathit{a}}_{\mathbf{14}}$

Reflects the contribution of digital projects to the enterprise. This indicator mainly reflects the contribution of digital projects to serving customers and improving customer satisfaction. The evaluation value is obtained by conducting a satisfaction survey on the customer base targeted by the digitalization project.

(2)

Model Promotion Value ${\mathit{a}}_{\mathbf{15}}$

Reflects the contribution of digital projects to the industry. This indicator mainly reflects the contribution value of the digital project model to promoting the digital construction of the industry. The evaluation value is obtained by scoring.

(3)

Data Management Visualization ${\mathit{a}}_{\mathbf{16}}$

Reflects the degree to which data are utilized by the digitization project. This indicator mainly reflects the effect of digital projects in data management visualization. The evaluation value is obtained by scoring.

(4)

Information Mining Value ${\mathit{a}}_{\mathbf{17}}$

Reflects the degree to which data are utilized by the digitization project. This indicator mainly reflects the value of digital projects in exploring potential customers, discovering new business opportunities, and providing information services. The evaluation value is obtained by scoring.

3.5. Development and Innovation

(1)

Optimize the Power Supply Level ${\mathit{a}}_{\mathbf{18}}$

Reflect the development capabilities of digital projects. This indicator mainly reflects the contribution of digital projects to power supply optimization and reflects the ability of sustainable development. The evaluation value is obtained by scoring.

(2)

System Iteration Timeliness ${\mathit{a}}_{\mathbf{19}}$

Reflect the development capabilities of digital projects. This indicator mainly reflects the timeliness of system iteration during the implementation of digital projects. The interval period and the number of iterations of the system iteration during the inspection period are inspected.

(3)

Carbon Emissions Reduction ${\mathit{a}}_{\mathbf{20}}$

Reflects the development capabilities of digital projects. This indicator mainly reflects the functions of digital projects in terms of energy conservation and emission reduction. The evaluation value is obtained by scoring.

(4)

System Expansion Capability ${\mathit{a}}_{\mathbf{21}}$

Reflect the innovative ability of digital projects. This indicator mainly reflects the innovation vitality of the digital project in the system and reflects the development prospects of the project. The evaluation value is obtained by scoring.

(5)

Business Innovation Capability ${\mathit{a}}_{\mathbf{22}}$

Reflect the innovative ability of digital projects. Test the innovative vitality of the digital project in business, and reflect the development prospects of the project. The evaluation value is obtained by scoring.

4. Sensitivity Analysis of Weights of Comprehensive Evaluation Indicators for Digital Projects

The comprehensive evaluation of digital projects is inseparable from the determination of the weights of evaluation indicators. The weight distribution reflects the importance or contribution of various indicators. Determining the weight is the basis of comprehensive evaluation. Based on the experience of existing projects, the sensitivity analysis of the evaluation indicators is carried out, and the most universal evaluation indicator weights are obtained, which is conducive to the scientific and comprehensive comparison and selection of digital projects.

4.1. Variation Coefficient Screening Method Based on Index Discrimination

The coefficient of variation method was used to examine the degree of data difference between different programs under the same index. The greater the coefficient of variation, the greater the difference, and the greater the impact on the evaluation results [45]. Directly use the information of each index to reflect the gap between the evaluation indicators.

The formula for the coefficient of variation of each indicator is as follows:

$$V_i=\frac{{\sigma }_i}{\overline{x_i}} \left(i=1,2,\cdots \right)$$

(1)

$V_i$ is the coefficient of variation of the $i$-th indicator, ${\sigma }_i$ is the standard deviation of the $i$-th indicator, and $\overline{x_i}$ is the average of the $i$-th indicator. Criteria for screening indicators by the coefficient of variation method: delete the indicator with the smallest coefficient of variation under the first-level indicators.

4.2. Indicator Weight Generation Based on LSOM

The research on determining the weight has always been an important research topic in the social work profession. At present, there are many methods for determining the weight of attributes, which can be roughly divided into three categories: subjective weighting, objective weighting method, and subjective and objective weighting method. The subjective and objective weighting method is usually the derivation of the subjective weighting method and objective weighting method, so this paper only considers the subjective weighting method and the objective weighting method.

The weight of each evaluation index was obtained by using two subjective weighting methods, Delphi method and AHP, and two objective weighting methods, entropy method and mean square error method. According to the weights obtained by the weighting method, the maximum and minimum ranges of weight values are determined for the multiple groups of weights generated later. The maximum value of the obtained weight under each method is taken as the upper bound of the weight, and the minimum value is taken as the lower bound of the weight. The weight values are generated using the LSOM (linear stochastic optimized model). The model can be briefly described as follows:

$$\min f=\left|{{\displaystyle \sum }}_{i=1}^{n-1}{x}_i·\left(a_i-w_i\right)-Err\right|0\le x_i\le 1,\forall i\in N$$

(2)

$$Err=\{\begin{array}{l}{w}_n-a_n+m\left(a_n-b_n\right) w_n>a_n \\ b_n-w_n+m\left(a_n-b_n\right) w_n<b_n\end{array}$$

(3)

$f$ is the penalty function, $n$ is the number of evaluation indicators, $a_n$ is the upper bound of the evaluation index weight, $b_n$ is the lower bound of the evaluation index weight, $w_n$ is the weight value, and $m$ is a real number between 0 and 1.

Through this model, the expected weight of the evaluation index of the array can be obtained, which is convenient for the subsequent analysis of the weight sensitivity.

4.3. Weight Sensitivity Analysis Based on Cloud Model Theory

Cloud model theory is a theory for dealing with uncertain problems that completes the conversion between qualitative and quantitative concepts, and organically associates ambiguity and randomness. On the basis of the normal distribution function and the normal membership function, the cloud model reflects the integrity of the cloud concept and mainly uses the three digital features of expectation $E_x$, entropy $E_n$, and hyperentropy $H_e$ to represent the cloud concept as a whole. Expectation $E_x$ represents the central value in the universe of discourse; entropy $E_n$ represents a measure of the randomness of qualitative concepts and the degree of dispersion of cloud droplets; hyperentropy $H_e$ represents the cohesion of the uncertainty of all points of the language value in the universe of discourse and the association of ambiguity and randomness, which indirectly reflects the thickness of the cloud. The normal cloud model is a commonly used cloud model, which is based on the general applicability of normal distribution and normal membership functions, and is realized by a normal cloud generator [46]. This paper will use the normal cloud model for the analysis.

Only the change of one index value is considered at a time, and the other index values remain unchanged. The ranking changes of each scheme are counted, and the value range that keeps the optimal scheme unchanged is determined. Introduce cloud model theory for index sensitivity analysis: use the closeness $x_i$ obtained by each historical digital project under the change of index value as the digital feature of the cloud model, first use the reverse cloud generator to generate the respective cloud model, and then use the forward cloud generator to generate cloud map. Finally, the weight sensitivity of the evaluation index can be analyzed by comparing the size of $E_x$, $E_n$, $H_e$, and the number of overlapping parts of the cloud image. Specific steps are as follows:

(1)

Find the possible value range of the indicator $a_j$ as $\left(a_j^{min},a_j^{max}\right)$;

(2)

Assign an initial value of $a_j^{min}$ to $a_j$, and set the step size to $\Delta a=0.01$ until it reaches $a_j^{max}$;

(3)

The values of other indicators remain unchanged, the closeness of each scheme to the optimal scheme is calculated, and the ranking of each scheme is counted;

(4)

Let $a_j\to a_j+\Delta a$; repeat step (3) until $a_j=a_j^{max}$;

(5)

The above steps are repeated, and the degree of closeness $x_i$ of each alternative to the ideal point in the entire value interval of the change in other index values is counted in turn;

(6)

Calculate the sample mean $\overline{X}=\frac{1}{n}{\displaystyle \sum }_{i=1}^n{x}_i$, the first-order sample absolute center distance $D=\frac{1}{n}{\displaystyle \sum }_{i=1}^n\left|x_i-\overline{X}\right|$, and the sample variance $S^2=\frac{1}{n-1}{\displaystyle \sum }_{i=1}^n{\left(x_i-\overline{X}\right)}^2$ from the data $x_i$;

(7)

Find the mean as the expectation $E_x=\overline{X}$;

(8)

Calculate the entropy according to the first-order absolute center distance of the sample $E_n=\sqrt{\left(\frac{\pi }{2}\right)}D$;

(9)

According to the sample variance and the calculated entropy, the hyperentropy $H_e=\sqrt{S^2-E_n{}^2}$ is calculated, and the cloud model $\left(E_x,E_n,H_e\right)$ of the scheme can be obtained;

(10)

With $E_x$ as the expected value and $E_n$ as the standard deviation, a normal random number $x_i$ is generated;

(11)

With $E_n$ as the expected value and $H_e$ as the standard deviation, a normal random number $E_{n_i}^{\prime }$ is generated;

(12)

Calculate $\mu \left(x_i\right)=\exp \left[-\frac{{\left(x_i-E_x\right)}^2}{2{\left(E_{n_i}\right)}^2}\right]$; let $\left(x_i,\mu \left(x_i\right)\right)$ be the cloud drop;

(13)

Repeat steps (9)–(12) until $n$ cloud droplets are generated to form a cloud image.

After the above steps, the degree of overlap among the schemes can be judged from the cloud diagram with the same scheme ranking for each group of weights. The greater the degree of overlap, the more unstable the program ranking of the group of weights is in conducting program evaluation, and the less obvious the distinction of the obtained evaluation results.

In summary, the operation flow of the weight sensitivity analysis method of the comprehensive evaluation index of digital project proposed in this section is shown in Figure 5.

5. Problem Description and Example

Taking the selection of digital projects of a power grid enterprise in a certain place as an example, it is now necessary to make decisions on digital project plans $P_1$, $P_2$, $P_3$, $P_4$, and $P_5$, and there are historical decision-making plans $S_1$, $S_2$, $S_3$, and $S_4$ for digital projects that have been carried out.

The formulation of the calculation example in this paper and the measurement and recording of the verification data were completed with the support of the staff of State Grid Jilin Electric Power Company (Jilin, China) and State Grid Electric Power Academy Co., Ltd (Beijing, China). Besides, this article takes $P_1$, $P_2$, $P_3$, $P_4$, $P_5$, $S_1$, $S_2$, $S_3$, and $S_4$ as the analysis object, and hires 20 industry experts to statistically score the 15 qualitative indicators.

5.1. Indicator Screening

Based on the 22 indicators constructed in the third section, data statistics and standardization are carried out on the digital project plan $P_1$, $P_2$, $P_3$, $P_4$, and $P_5,$ and the coefficient of variation of each indicator is calculated and then screened. Coefficient of variation method screening index criteria: remove the index with the smallest coefficient of variation under the primary index. The indicators and screening results are shown in

Table 1. Screening of evaluation indicators.

Indicators	${\mathit{P}}_1$	${\mathit{P}}_2$	${\mathit{P}}_3$	${\mathit{P}}_4$	${\mathit{P}}_5$	Coefficient of Variation	Retain/Delete
$a_1$	0.00	0.42	0.14	0.66	1.00	0.91	Retain
$a_2$	1.00	0.95	0.80	0.81	0.00	0.57	Delete
$a_3$	0.42	1.00	0.50	0.00	0.73	0.70	Retain
$a_4$	0.13	0.31	0.91	0.00	1.00	0.97	Retain
$a_5$	1.00	0.75	0.72	0.53	0.00	0.62	Retain
$a_6$	0.00	0.52	0.84	0.07	1.00	0.92	Retain
$a_7$	0.74	0.00	0.91	0.23	1.00	0.76	Delete
$a_8$	0.11	0.24	0.00	0.57	1.00	1.05	Retain
$a_9$	1.00	0.58	0.30	0.00	0.59	0.75	Delete
$a_{10}$	0.67	0.16	0.00	0.57	1.00	0.83	Retain
$a_{11}$	0.65	1.00	0.94	0.00	0.16	0.83	Retain
$a_{12}$	0.80	1.00	0.00	0.13	0.34	0.95	Retain
$a_{13}$	0.00	0.54	0.77	1.00	0.04	0.94	Retain
$a_{14}$	0.71	0.38	0.12	0.00	1.00	0.94	Retain
$a_{15}$	0.76	0.15	0.83	1.00	0.00	0.81	Delete
$a_{16}$	0.79	0.00	0.86	0.02	1.00	0.91	Retain
$a_{17}$	0.19	0.49	0.82	1.00	0.00	0.84	Retain
$a_{18}$	1.00	0.15	0.84	0.49	0.00	0.87	Retain
$a_{19}$	0.28	0.66	0.00	1.00	0.04	1.08	Retain
$a_{20}$	0.00	0.83	0.73	1.00	0.29	0.73	Retain
$a_{21}$	0.00	0.90	1.00	0.28	0.05	1.06	Retain
$a_{22}$	1.00	0.41	0.85	0.69	0.00	0.67	Delete

.

Under the coefficient of variation method, the smaller the coefficient of variation, the smaller the difference between the indicators, and screening out the indicators with relatively small differences is beneficial to increase the difference between the evaluation results. After the index screening under the coefficient of variation method, the digital project’s “Degree of Cost Savings ${\mathit{a}}_{\mathbf{2}}$”, “Coverage of Digital Means of Information Collection ${\mathit{a}}_{\mathbf{7}}$”, “User-friendly Interface ${\mathit{a}}_{\mathbf{9}}$”, “Model Promotion Value ${\mathit{a}}_{\mathbf{15}}$”, and “Business Innovation Capability ${\mathit{a}}_{\mathbf{22}}$” were deleted.

5.2. Weight Processing and Generation

According to the selected indicators, the Delphi method, the AHP method, the entropy method, and the mean square error method are used to obtain the weights of the evaluation indicators in the historical decision-making schemes $S_1$, $S_2$, $S_3$, and $S_4$, and the upper and lower bounds of the weights of each indicator are formed. The obtained results are shown in

Table 2. Indicator weights calculated based on different weighting methods.

Indicators	Delphi Method	AHP	Entropy Method	Mean Square Error Method	Maximum Value	Minimum Value
$a_1$	0.0431	0.1126	0.0701	0.0577	0.1126	0.0431
$a_3$	0.0373	0.0644	0.0593	0.058	0.0644	0.0373
$a_4$	0.0428	0.0733	0.0579	0.0625	0.0733	0.0428
$a_5$	0.0387	0.0355	0.0426	0.0604	0.0604	0.0355
$a_6$	0.0684	0.1000	0.0591	0.0674	0.1000	0.0591
$a_8$	0.0765	0.0515	0.073	0.0593	0.0765	0.0515
$a_{10}$	0.0691	0.0318	0.0507	0.0548	0.0691	0.0318
$a_{11}$	0.0739	0.0095	0.0543	0.0552	0.0739	0.0095
$a_{12}$	0.0728	0.0205	0.0513	0.0588	0.0728	0.0205
$a_{13}$	0.0655	0.0445	0.0563	0.064	0.0655	0.0445
$a_{14}$	0.0523	0.0492	0.0499	0.0556	0.0556	0.0492
$a_{16}$	0.0285	0.0865	0.0589	0.0551	0.0865	0.0285
$a_{17}$	0.0654	0.0477	0.0604	0.0605	0.0654	0.0477
$a_{18}$	0.0369	0.0415	0.0477	0.0566	0.0566	0.0369
$a_{19}$	0.0679	0.0648	0.0873	0.0594	0.0873	0.0594
$a_{20}$	0.0946	0.0553	0.0760	0.0574	0.0946	0.0553
$a_{21}$	0.0663	0.1114	0.0452	0.0573	0.1114	0.0452

.

According to the obtained upper and lower bounds of the weight of the evaluation index, the maximum and minimum range of the weight of the index is formed, and five groups of weights are generated based on the LSOM model. The reason for using these five groups of weights as an example is that the differences that exist in the weight of each indicator in these five groups of weights are relatively obvious and facilitate the analysis in different situations. The details of the weights are shown in

Table 3. Indicator weights based on LSOM.

The Number of Index Weights	Weights
the first set of weights	(0.0435 0.0597 0.0733 0.0604 0.1000 0.0765 0.0691 0.0139 0.0249 0.0459 0.0556 0.0357 0.0654 0.0551 0.0873 0.0600 0.0737)
the second set of weights	(0.1039 0.0467 0.0441 0.0355 0.0596 0.0701 0.0488 0.0525 0.0571 0.0565 0.0492 0.0794 0.0594 0.0541 0.0725 0.0609 0.0497)
the third set of weights	(0.0927 0.0373 0.0633 0.0517 0.0808 0.0694 0.0506 0.0409 0.0466 0.0642 0.0495 0.0285 0.0520 0.0369 0.0828 0.0934 0.0594)
the fourth set of weights	(0.0959 0.0373 0.0428 0.0355 0.0794 0.0515 0.0595 0.0289 0.0252 0.0445 0.0517 0.0735 0.0509 0.0376 0.0847 0.0897 0.1114)
the fifth set of weights	(0.1121 0.0373 0.0428 0.0355 0.0894 0.0737 0.0639 0.0480 0.0547 0.0445 0.0520 0.0285 0.0477 0.0427 0.0594 0.0568 0.1110)

.

5.3. Cloud Model Generation and Analysis

Keeping the order of the digital project schemes unchanged, the evaluation value under each evaluation index of each scheme increases sequentially within a reasonable range by a step size of 0.01 and calculates the closeness of the index value to the optimal scheme after the change. Taking the closeness of each index as the sample data, through the cloud model of each program of reverse cloud and forward cloud computing, the results that are shown in (a)–(e) of Figure 6. a–e represents the sequential numbering of the cloud maps under the first to the fifth set of weights.

The cloud model consists of a number of cloud drops, each of which is a definite point that constitutes a cloud responsible for uncertainty. The concrete representation of the cloud model can be shown by a cloud map. From the distribution and shape of the cloud map, we can see the degree of difference between the four programs $S_1$, $S_2$, $S_3$, and $S_4$ under each group of weights. In Figure 6e, it is obvious that the four programs have a large degree of overlap, and the amplitudes are similar, indicating that, under the fifth set of weights, the values of expectation, entropy, and hyperentropy of the four programs are close. The use of this group of weights does not reflect the differences between programs well but will lead to unstable comprehensive evaluation values between programs, so this group of weights is excluded first. The remaining four groups of weights, that is, Figure 6a–d, are relatively similar in shape and have roughly the same distribution. Among them, under the second set of weights (Figure 6b), the distributions of $S_2$, $S_3$, and $S_4$ are more concentrated, the expectations of the three are relatively close, and the difference between entropy and hyperentropy is not obvious. Under the first group of weights (Figure 6a) and under the fourth group of weights (Figure 6d), $S_2$ and $S_3$ almost overlap, and the comprehensive evaluation results are prone to change. Relatively speaking, the cloud map under the third set of weights (Figure 6c) is clearer and has clear boundaries.

In summary, the third set of weights keeps the order of the four historical programs unchanged, and the evaluation results are the most stable when the index evaluation value changes, and it has better robustness when evaluating new projects. The results obtained by the other four sets of weights are less stable. Therefore, the third set of weights can be applied to the decision-making of new power grid enterprise digitization projects, and the most robust digitization projects can be decided based on historical experience.

After substituting the third group of weights into the specific index values of digital project plans $P_1$, $P_2$, $P_3$, $P_4$, and $P_5$ (their normalized processed data in Table 1), the obtained evaluation values of each plan are 0.309, 0.559, 0.514, 0.496, and 0.520, respectively. Therefore, based on this weight, the digital project $P_2$ with the highest score is finally selected as the decision result.

6. Discussion

Due to the real problems, such as the lack of experience in digitization projects and the complexity of evaluation perspectives of digitization project solutions in the STATE GRID Corporation of China, the comparison decision of current digitization project solutions is difficult. Therefore, this study develops a cloud model evaluation method for digitalization projects considering weight sensitivity to help the STATE GRID Corporation of China make decisions on digitalization project solutions. Based on this, we conducted an empirical analysis based on a real case study and came up with the following important conclusions.

Firstly, there are many aspects to be considered in the evaluation of digital project solutions. In the process of digital project program evaluation, the selection of indicators will largely affect the evaluation results and then the accuracy of the evaluation results. In order to avoid decision-making errors brought about by indicators, this study constructs a comprehensive evaluation index system for digital projects. The results of this study show that the index system meets the development requirements of the STATE GRID Corporation of China and covers the awareness level of the STATE GRID Corporation of China for digital transformation. In addition, it accurately measures the performance capability of digitalization projects in terms of economy, efficiency, development direction, and innovation. In fact, these factors are the key to achieving better value for the project. Through this indicator system, the STATE GRID Corporation of China can effectively assess the potential of digital project solutions and come up with an optimal selection strategy.

Secondly, there is variability in different digitization project programs under different evaluation indicators. After the comprehensive evaluation index system is established, not all groups of digitization project programs can have large differences under each index. Based on this, the coefficient of variation method is used to delete the smallest indicator with the smallest coefficient of variation under each group of first-level indicators. On the one hand, it widens the gap between the evaluation programs and avoids the situation of similar evaluation values. On the other hand, it ensures the completeness of the evaluation perspectives covered and the same number of indicator reductions under each first-level indicator. In the calculation example, after using the coefficient of variation method for indicator selection, the evaluation indicators are not only streamlined but the evaluation effect also becomes better.

Finally, an algorithmic framework is constructed to compare and select digitalization project solutions in grid enterprises. The results of the algorithm show that the evaluation results can fully absorb the experience of historical digitization projects and then compare and select new digitization project solutions. Moreover, the cloud model can clearly observe the overlap of program evaluation under each group of index weights and transform the quantitative evaluation values and weights into a qualitative presentation. At the same time, the cloud model can also better show the uncertainty of evaluation values in the actual evaluation process and comprehensively reflect the evaluation result situation under the change in evaluation values. Of course, there are still shortcomings in the study, such as not exploring whether the cloud model will have an impact on the results under a higher number of simulations, which will be the next research direction.

7. Conclusions

To address the problems in the comprehensive evaluation of digitalization projects in the STATE GRID Corporation of China (Beijing, China) under the wave of digitalization, such as the complicated evaluation perspectives of digitalization project solutions, the difficulty to learn from the experience of conducting smooth historical projects, and the inconspicuous differences in index values among solutions, it is necessary to establish an efficient and convenient comprehensive evaluation method for digitalization project programs in the STATE GRID Corporation of China. Therefore, in order to solve these problems, this paper first uses Citespace to visualize and analyze the historical literature of enterprise digitization and explore the development of digitization. Then, based on the objectives achieved by the digitalization project, a system of 22 evaluation indicators is established in five aspects: economic efficiency, interconnection, intelligent management, value release, and development innovation. In order to reduce the interference of the project decision-making process due to the small degree of variation among the indicators on the program decision, this paper adopts the coefficient of variation method for the selection of indicators. In addition, the four types of assignment methods, which are more frequently used in subjective and objective assignment, are taken as the boundaries to keep the weight values within a reasonable range, and the LSOM model is used to generate the weight values within the range. Then, the cloud model is used to reflect the randomness and fuzziness of the index values, and the sensitivity analysis of the comprehensive evaluation index weights of digital projects is conducted. The final obtained cloud distribution graph intuitively reflects the results, and the analyzed weight values have the best robustness and are applicable to multiple projects. Finally, an arithmetic example is analyzed based on the method proposed in this paper. Based on the empirical results, the theoretical contributions and practical implications of this study are sorted out, and the limitations of the study and the future development directions are pointed out.