Next Article in Journal
Energy Sufficiency in the Passenger Transport of Lithuania
Previous Article in Journal
Optimizing Traffic Flow in Smart Cities: Soft GRU-Based Recurrent Neural Networks for Enhanced Congestion Prediction Using Deep Learning
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Sustainable Iterative Product Design Method Based on Considering User Needs from Online Reviews

1
School of Industrial Design, Hubei University of Technology, Wuhan 430068, China
2
China-Korea Institute of New Media, Zhongnan University of Economics and Law, Wuhan 430068, China
*
Author to whom correspondence should be addressed.
Sustainability 2023, 15(7), 5950; https://doi.org/10.3390/su15075950
Submission received: 2 March 2023 / Revised: 27 March 2023 / Accepted: 27 March 2023 / Published: 29 March 2023

Abstract

:
Small and medium-sized manufacturing industries can use online reviews to add valuable user requirements, enabling them to iteratively and precisely upgrade their products based on user needs. However, a sustainable, iterative approach to product design requires the integration of a large amount of information about user requirements for accurate selection. Currently, product iterations are primarily focused on developing new solutions or upgrading a few components with little screening to see if the product iterations meet user needs. This leads to a large number of wasted resources and a shortened product lifecycle. To address these challenges, this paper proposes a sustainable iterative research method that mines user needs and provides comprehensive decision making for product design based on online reviews, using probabilistic semantic term sets (PLTS). The proposed method considers the hesitation and uncertainty among evaluating experts regarding indicators, and uses the decision-making trial and evaluation laboratory (DEMATEL) method to analyze the correlations between demand indicators. The DEMATEL correlation function is improved by reconstructing the PLTS acquisition score function and deviance into a DEMATEL correlation function, in the form of exact values using an improved binary semantic approach. This iterative design approach provides accurate feedback on how users feel about the use of product components and ensures that most product components are sustainably recycled. A drone case study is presented to demonstrate the feasibility of this approach. In-depth interviews with experts confirm that this approach is more sustainable and provides a new research methodology for sustainable iterative product design.

1. Introduction

The sustainability of Internet and Industry 4.0 development relies heavily on the manufacturing industry, particularly in terms of assessing its rapid development from an environmental perspective and rationing the use of limited global resources while safeguarding environmental sustainability [1]. Small and medium-sized manufacturing industries use most of the world’s resources, and studies have shown that natural resources are negatively correlated with a country’s ecological and economic growth, leading to the over-consumption of resources, water and air pollution, and the production of significant amounts of highly polluting waste [2]. The large-scale development of green products in small and medium-sized manufacturing industries has resulted in the rapid obsolescence of older, non-green products. This shortens the life cycle of existing products and generates a significant amount of waste [3].
As a result, there is widespread interest in sustainable products versus existing non-sustainable products, and small and medium-sized manufacturing industries need new ways to improve sustainability and slowly replace non-sustainable products [4]. Currently, the most cited sustainability theory for reducing the environmental impact of production and consumption is the Product Service System (PSS), proposed by Mont [5]. Feng et al. [6] proposed treating existing product modules as original systems and combined the Analytical Hierarchy Process (AHP), Lean Design-for-X (LDfX), Design Structure Matrix (DSM), and Pearson Correlation Coefficient (PCC). However, small and medium-sized manufacturing industries may require a new service system based on PSS theory to address sustainable product iteration solutions, which require significant resources. Furthermore, most small and medium-sized manufacturing industries are unable to solve this problem by themselves, not only due to their limited resources and service capabilities, but also because overly complex solutions increase their costs and risks [7].
The multi-attribute decision problem plays a crucial role in sustainable iterative product design, and previous work proposed an iterative conceptual design process (S-KFCF) that integrates the KANO model, FBS model, and FAHP method [8]. This paper proposes a new approach that leverages online reviews to identify user requirements and precisely corresponds to product structure modules. However, the real-time nature of online reviews and their continuous updating makes mining and filtering user needs a significant challenge. To address this issue, this paper proposes a sustainable iterative design methodology that considers the limited resources and service capabilities of small and medium-sized manufacturing industries. This approach extends product lifecycles, reduces waste, and reduces costs and risks for these industries.
This paper is organized as follows: Section 3 outlines the research methodology, mathematical algorithms, and models; Section 4 provides case experiments and validation of the results; and finally, Section 5 presents conclusions and future research directions.

2. Related Work

Small and medium-sized manufacturers can effectively use online reviews to make decisions and gain a competitive advantage [9]. In previous studies, it has been observed that decision making plays a crucial role in the various aspects of life. In product iteration, decision making is the fundamental process that leads to different directions of product development [10]. Yang et al. [11] found that traditional user research methods faced decision-making challenges in the process of product design and improvement. They proposed a new method to obtain useful online reviews from e-commerce platforms, constructed a product evaluation index system, and proposed product improvement strategies for online reviews. Yu et al. [12] believed that decision making occurs at all stages of product design, and proposed using multi-index decision making to improve the continuity and efficiency of product design and to solve problems in the product design process. Multi-attribute decision making (MADM) is a widely used method to select an optimal choice from a limited number of alternatives based on multiple attributes [13,14]. Thus, introducing MADM can effectively address the problem of decision making in the product iteration process. In MADM, it is essential to present information objectively under each attribute, and semantic terms are employed to represent significant information about the model, which can include evaluation information or qualitative assessment information [15,16].
In the face of rapid economic development, decision makers increasingly use a variety of semantic terms to provide evaluation information about product characteristics. Rodriguez et al. [17] proposed a hesitant fuzzy semantic term set (HFLTS) to represent the multiple semantic terms of a participant. Wang [18] further developed the extended hesitant fuzzy semantic term set (EHFLTS), which merges with HFLTS. The probability distribution of each semantic term is equal in either HFLTS or EHFLTS, but different participants may choose different semantic terms. To describe the level of preference for these semantic terms, Chen et al. [19] introduced a set of proportionally hesitant fuzzy semantic terms consisting of semantic terms and their corresponding probability distributions. Currently, participants may provide incomplete partial probability distributions or may ignore them. Pang et al. [20] proposed probabilistic linguistic term sets (PLTS) to define incomplete probability distributions, which can be considered a general form of PHFLTS, HFLTS, and EHFLTS. PLTS assigns different probabilities to the semantic terms to represent different degrees of preference, thereby precisely translating the hesitation and uncertainty present in the verbal evaluation of experts. PLTS is more flexible and comprehensive than other models and can appropriately describe complete and incomplete assessment information. Qualitative information is effectively addressed in the decision-making process and computed using expressions [21]. While PLTS can address hesitancy and uncertainty in expert scoring, it does not allow for the analysis of correlations between indicators. PLTS has been used to conduct several studies on probabilistic semantic environments, including the PLTS weight determination model [22,23] and probabilistic semantic decision-making methods. However, while PLTS can address the hesitancy and uncertainty of verbal evaluations in expert scoring, it does not allow for the analysis of correlations between indicators. For example, in a given product, the product components that users are concerned about are the same, but because of the limited range of user reviews there will be subtly different review expressions for the product, yet most real user review information is vague, uncertain, and inaccurate [24]. Therefore, it will be difficult for the model to identify a large amount of duplicate review information.
To address this problem, Baykasoğlu et al. [25] proposed the decision-making trial and evaluation laboratory (DEMATEL) to consider the interdependencies between problem attributes. Yi et al. [26] reconstructed PLTS elements and developed a multi-attribute decision approach based on the DEMATEL method, adding probabilistic information and extending the DEMATEL method to the re-customized PLTS framework. The DEMATEL method calculates deterministic values when computing the normalized and total correlation matrices between the indicators. Therefore, this paper proposes a new method to consider the hesitation and uncertainty of experts when evaluating the information extracted from user requirements. We first calculate the score functions associated with the indicators and then transform them into exact values using binary semantics [27,28]. This paper aims to use multi-attribute decision making to make optimal sustainable product iterative design solutions that meet user needs, reduce the cost and risk of iteration for small and medium-sized manufacturing industries, and improve the ability to evaluate and improve product iterations. The proposed method will allow for the analysis of correlations between indicators, addressing the issue of duplicate review information in real user review data, and enhance the ability to identify product components about which users are concerned.

3. A Sustainable, Iterative Approach to Product Design

This study employs online reviews to identify user needs and proposes a multi-attribute decision-making approach to address the sustainable product iteration problem. The approach presented in this paper explores how small and medium-sized manufacturing industries can use online review information to capture user needs and iterate products in a sustainable manner.
The sustainable iterative approach to product design proposed in this study involves the following steps: (1) Obtaining user needs information from a large number of online reviews. (2) Thematically organizing user requirements using the BTM model to establish a product iteration factor information base. (3) Dividing the corresponding products into structural modules to align user requirements with the structural modules as a second screening for user requirements. (4) Introducing the PLTS method to account for expert scoring hesitation and uncertainty. The DEMATEL method is used to consider the interrelatedness of indicators and is reconstructed into a DEMATEL correlation function in the form of exact values using an improved binary semantic approach. This step is a core element of the iterative design. (5) Analyzing the correlation rankings of the importance of the structural modules of interest to the users in order to prioritize the modules for the final iteration. (6) Discussing a sustainable product iteration plan in focus groups with professional product designers, structural engineers, brand owners, and product development technicians. The discussion focuses on the selected critical structural and technical modules; the functional characteristics of the components, user requirements, and the current state of the product are gradually analyzed to ensure the sustainable recycling of most modules and the iterative upgrading of the product.
The entire process of sustainable iterative product design involves an iterative filtered loop, where each horizontal plane clearly depicts the iteration stage, and each stage can be paused, as shown in Figure 1. This iterative process can be repeated with the original product to meet changing user needs whenever the latest online reviews become available.

3.1. Mining User Needs Based on the BTM Model

Yan et al. [29] proposed a biterm topic model (BTM) to comprehensively explore the topics in a corpus and tackle the issue of sparsity in short texts. BTM posits that two words in a biterm share a common topic, which is derived from a hybrid topic across the corpus. In this framework, a topic is represented as a distribution of words. Unlike most topic models that examine the thematic factors of a corpus by modeling document generation, BTM achieves this objective by modeling the co-occurrence of word pairs. Specifically, if two phrases appear together more frequently, they are more likely to be related to the same topic [30]. As shown in Figure 2.
Assuming that a biterm is composed of words wi,1 and wi,2, that is, b = { w i , 1 , w i , 2 } , the two words in each biterm are sampled from the same topic Ζ . N B Biterms form a set Β = { b i } i = 1 N B . In addition, the symmetrical θ and ϕ k in the Dirichlet prior are used, which have single-value hyperparameters α and β , respectively. The BTM is generated as follows:
  • The topic distribution is generated for α with parameters in the Dirichlet prior, θ D i r ( α ) ;
  • For each topic Κ [ 1 , k ] , the topic distribution is generated for β with parameters in the Dirichlet prior, ϕ k Dir ( β ) ;
  • For each biterm b i Β :
The topic distribution of the biterm b i in the topic distribution is θ : z i M u l t i n o m i a l ( θ ) . A word is generated according to the topic distribution z i : w i , 1 , w i , 2 M u l t i n o m i a l ( ϕ z i ) .
According to BTM, the probability of b i can be calculated by integrating θ and ϕ k . In addition, by multiplying the probability of each biterm, the entire corpus can be obtained. This can be stated as follows [30]:
P ( Β | α , β ) = i = 1 N B k = 1 Κ θ k ϕ k , w i , 1 ϕ k , w i , 2 d θ d Φ

3.2. Evaluating User Requirements and Technical Modules using Probabilistic Semantic Term Sets

Pang et al. [20] proposed the use of probabilistic linguistic term sets (PLTS) in their work. A PLTS is a semantic collection consisting of a set of semantic terms and their associated probabilistic information. In the case where the expert’s evaluation of the semantic set represented by PLTS is denoted as L(p), the corresponding definitions can be found.
In Definition 1 [31], suppose S = { s 0 , , s α } is a semantic term set, L ( p ) is called a PLTS on S:
L ( p ) = { L ( k ) p ( k ) | L ( k ) S , p ( k ) 0 , k = 1 , 2 , , # L ( p ) , k = 1 # L ( p ) p ( k ) 1 }
In Equation (2), # L ( p ) denotes the number of semantic sets in the PLTS, and L ( k ) ( p ( k ) ) represents the likelihood of the semantic set L ( k ) p ( k ) .
When k = 1 # L ( p ) p ( k ) = 1 , the semantic information in the experts’ evaluation is complete; when k = 1 # L ( p ) p ( k ) < 1 , the semantic information in the experts’ evaluation is an incomplete probability distribution and the PLTS needs to be standardized.
For example, assume that LTS is a semantic terminology set with five levels of granularity,   S 5 = ( S 0 = V L , S 1 = L , S 2 = M , S 3 = H , S 4 = V H ) . When the evaluation indicator is the evaluation of the product’s battery life, the user gives “The battery life of the product is good” as S4 with a probability of 0.6, and “I don’t know much about the battery life” as S3 with a probability of 0.3. At this time, L ( p ) = { ( S 1 , 0.3 ) , ( S 4 , 0.6 ) } . When the evaluation indicator evaluates the quality of the product, the first user gives a very good rating, the second user gives a very good rating, the third user gives a good rating, and the fourth an average rating, then the “quality of the product is very good” is S4 and the probability is 0.5. The probability of a product being of good quality is S3 and the probability is 0.25. The probability of the product being of average quality is S2 and the probability is 0.25, and L ( p ) = { ( S 2 , 0.25 ) , ( S 3 , 0.25 ) , ( S 4 , 0.5 ) } .
In Definition 2 [31], suppose L ( p ) is S the incomplete PLTS in the previous semantic information, it is standardized to:
L ¯ ( p ) = { L ( k ) p ¯ ( k ) | L ( k ) S , k = 1 , 2 , , # L ( p ) , p ¯ ( k ) = p ( k ) k = 1 # L ( p ) p ( k ) }
In Definition 3 [31], L 1 ( k ) and L 2 ( k ) are the K-th semantic sets in L ( p ) 1 and L ( p ) 2 , p 1 ( k ) and p 2 ( k ) are the probability information of semantic sets L 1 ( k ) and L 2 ( k ) , respectively, and the distance between L ( p ) 1 and L ( p ) 2 is:
d ( L ( p ) 1 , L ( p ) 2 ) = k = 1 # L ( p ) 1 [ p 1 ( k ) r 1 ( k ) p 2 ( k ) r 2 ( k ) ] 2 / # L ( p ) 1
where r 1 ( k ) and r 2 ( k ) denote the subscripts of L 1 ( k ) and L 2 ( k ) .
In Definition 4 [31], suppose L ( p ) is the S of the previous PLTS and r ( k ) is the subscript of the semantic term set L ( k ) , then the score function, degree of deviation, and degree of hesitation of L ( p ) are:
E ( L ( p ) ) = S α ¯
σ ( L ( p ) ) = [ k = 1 # L ( p ) ( p ( k ) ( r ( k ) α ¯ ) ) 2 ] 1 / 2 / k = 1 # L ( p ) p ( k )
H ( L ( p ) ) = 1 # L ( p ) k = 1 # L ( p ) [ p ( k ) ( r ( k ) α ¯ ) ] 2 α + 1
In Equation (8), α ¯ = k = 1 # L ( p ) r ( k ) p ( k ) / k = 1 # L ( p ) p ( k ) . If the two PLTSs are L ( p ) 1 and L ( p ) 2   , respectively, and if S F ( L ( p 1 ) ) > S F ( L ( p 2 ) ) , then L ( p ) 1 is superior to L ( p ) 2 .
In Definition 5 [31], suppose L ( p ) is the S of the previous PLTS, then L ( p ) is converted to an exact numerical function:
S F ( L ( p ) ) = α ¯ σ ( L ( p ) ) H ( L ( p ) )
The greater the α ¯ , the smaller the degree of deviation; the smaller the degree of hesitation, the better is the PLTS L ( p ) .

3.3. Calculating User Requirement Weights Using the Improved Probabilistic Semantic DEMATEL Method

The next step uses the enhanced probabilistic semantic DEMATEL method for calculating the weights of indicators in multi-attribute decision making. In multi-attribute decision making, indicator interactions are typically considered when applying factors to the exact numerical DEMATEL method and to probabilistic semantic improvements. This involves converting the semantic variables into probabilistic semantics and analyzing the indicators’ interactions using the DEMATEL method to determine their importance. Importance calculations are then performed to obtain the weights for each indicator. To calculate the index weight using the improved probabilistic semantic DEMATEL method, the following steps should be followed:
  • A direct correlation matrix Xk between the indexes was established. The LTS term collection for the correlation between the evaluation index is S r = { s g | g = 0 , 1 , 2 , , e } . The evaluation indexes are C j ( j = 1 , 2 , , n ) . Expert E k ( 1 k t ) evaluation of the correlation between indexes, according to the collection Sr, was used to establish a direct correlation matrix between indexes.
    X k = [ 0 s g ( 12 ) k s g ( 1 n ) k s g ( 21 ) k 0 s g ( 2 n ) k s g ( n 1 ) k s g ( n 2 ) k 0 ]
    where s g ( i j ) k represents the degree of influence of the use of the LTS evaluation index Ci by expert Ek on Cj. It takes a value of 0 if there is no influence.
  • All expert evaluations of the inter-influence of relationships between the indicators were assembled according to the example in Definition 1 to obtain a direct correlation matrix between the indicators in the form of a probabilistic semantic term set for all experts.
    X = [ 0 L ( p ) 12 L ( p ) 1 n L ( p ) 21 0 L ( p ) 2 n L ( p ) n 1 L ( p ) n 2 0 ]
    where L ( p ) i j represents the PLTS, and the exact numerical calculation is usually adopted during the calculation by the traditional DEMATEL method. The current data are still semantic information, which makes it impossible to calculate, so this study uses binary semantics to convert the semantic information into exact values. This is performed using Equations (5) and (6) to calculate the score function and the deviation degree σ ( L ( p ) ) of the probabilistic semantic L ( p ) i j . Subsequently, binary semantics were used to convert them into exact numerical values.
In Definition 6, β [ 0 , g ] is the result of the semantic term set L after integration, and let i = r o u n d ( β ) , i [ 0 , g ] . β can be expressed by the following function Δ as a binary semantic conformance [31]:
Δ [ 0 , g ] L × [ 0.5 , 0.5 ] Δ ( β ) = ( l i , α i ) { l i i = r o u n d ( β ) α i = β i α i [ 0.5 , 0.5 )
where r o u n d denotes the rounding operator, α denotes the transfer value. In contrast, let ( l i , α i ) be binary semantic information and the information is converted into a real number using a Δ 1 function.
Δ 1 : L × [ 0.5 , 0.5 ] [ 0 , g ] , Δ 1 ( l i , α i ) = i + α i = β
3
The score function and the degree of deviation were calculated and then converted into a direct correlation matrix after obtaining exact values using the Δ 1 function.
X = [ 0 Δ - 1 ( E ( L ( p ) 12 ) , σ ( L ( p ) 12 ) ) Δ - 1 ( E ( L ( p ) 1 n ) , σ ( L ( p ) ) 1 n ) Δ - 1 ( E ( L ( p ) 21 ) , σ ( L ( p ) 21 ) ) 0 Δ - 1 ( E ( L ( p ) 2 n ) , σ ( L ( p ) ) 2 n ) Δ - 1 ( E ( L ( p ) n 1 ) , σ ( L ( p ) n 1 ) ) Δ - 1 ( E ( L ( p ) n 2 ) , σ ( L ( p ) n 2 ) ) 0 ]
4
The directly normalized correlation matrices were then calculated. A common method for normalizing directly correlated matrices is based on the sum of the vector factors of every row of the matrix [32]. Let the normalization coefficient of X be λ , the normalization coefficient is calculated with the score function and the degree of deviation in the PLTS, the calculation form of λ is calculated as follows:
λ = 1 / max 1 i n ( Σ j = 1 n Δ 1 ( E ( L ( P ) i j ) , σ ( L ( P ) i j ) )
The normalized direct correlation matrix Z is:
Ζ = λ X
5
The total correlation matrix was calculated using T. According to references [33,34], T is calculated as follows:
T = [ 0 t 12 t 1 n t 21 0 t 2 n t n 1 t n 2 0 ]
T = Z ( 1 Z ) 1
6
Index importance ω j was calculated.
The sum of i in row I of matrix T is defined as Dj and the sum of j in column is defined as Fj.
D j = ( D j ) n × 1 = [ Σ j = 1 n t i j ] n × 1
F j = ( F j ) n × 1 = ( F j ) 1 × n = [ Σ i = 1 n t i j ] 1 × n
Let i = j , the formula for calculating the index importance ω j is as follows:
ω j = ( D j + F j ) 2 + ( D j F j ) 2
The index importance was normalized to obtain the index weight ω ¯ j ( 1 j n ) . The index weight set was ω ¯ j = W = [ ω ¯ 1 , , ω ¯ j , , ω ¯ n ] .

3.4. Sequencing Technology Modules with Consideration of User Requirements Interaction

Suppose that the number of experts is E k ( 1 k t ) , the expert weight is O k ( 1 k t ) , the user need is R j ( 1 j m ) , the technical module is C i ( 1 i n ) , and the weight of need is W j ( 1 j m ) .
The correlation between user needs R j and technical module C i is evaluated by the expert as:
G i j = k = 1 t [ S F ( L ( P ) k i j ) O k ]
The weight between user need R j is evaluated by the expert as:
H j = k = 1 t [ W j O k ]
The importance of technical modules considering the mutual influence between user needs is:
C i = j = 1 m H j G i j

4. Case Studies

4.1. Cases

In “The Use of UAVs as Research Products: A Comparative Analysis of User Reviews on Jindong Shopping Platform and Their Application in Experimental Data Analysis,” it was found that the versatility of UAVs (Unmanned Aerial Vehicles) was very attractive to customers and resulted in high sales volume [34]. In this study, we employed an UAV as the research product and conducted a comprehensive comparison of user reviews on the Jindong Shopping Platform (www.jd.com, accessed on 22 July 2022). The reviews were found to be relatively objective in content, abundant in information, and highly correlated with the products reviewed. A total of 15,303 reviews of a specific UAV product were collected and used as the data source for our experiment. We utilized Python web crawler technology to collect online reviews, including the review text, product type, review title, and review time. After eliminating invalid information, such as repeated reviews, we conducted data pre-processing, which involved segmenting the valid reviews into words and sentences using the Python Jieba word segmentation package. The stop word list by the Harbin Institute of Technology (HIT) was used to filter out stop words, resulting in 9085 pieces of valid comment information. Finally, we sorted and generated a corpus of online reviews for further analysis.
First, the parameters were set. The setting of relevant parameters, including prior hyperparameters α, β, and the number of topics k, was completed before running the UAV reviews in the BTM model. Second, the Niter was set to 1000 times by default, according to the sample data. During model training, the results were saved after every 100 iterations. Repeated tests on the reviews of the UAV product showed that when the number of topics, k, was set to 12, the extraction effects were the best; α was set to 10, and β was set to 0.5. Finally, the content of each topic was inferred according to the topic clustering results and ranked by probability from high to low. Each topic’s five high-frequency words and five low-frequency words were screened as keywords. The topic contents focused on product price, function, promotion, users’ needs for the product itself, experience, and services, etc. Figure 3 presents the results for these topics.
  • The user topics that needed to be extracted were divided by probability. The repeated user needs information was integrated and divided into six topics, from high to low: quality ( R 1 ), battery ( R 2 ), shooting ( R 3 ), convenient operation ( R 4 ), signal ( R 5 ), and cost performance ( R 6 ), as shown in Figure 4.
  • The technical structure modules were divided by analyzing the patents of this UAV brand and the experts’ advice. There were eight categories of technical modules: power module ( C 1 ), including the motor; photography module ( C 2 ), including photography component and picture transmission signal transmitter; gimbal module ( C 3 ), including gimbal motor; control module ( C 4 ), including remote control technology; interaction module ( C 5 ), including monitor and control panel; flight module ( C 6 ), including controller, propeller, and wing; efficiency module ( C 7 ), including battery power cable; and carrier module ( C 8 ), including the overall weight and volume of the UAV.
  • The evaluation and scoring were performed by an expert group. The expert group comprised six experts, including professional product structure designers, brand experts, and product development technicians. In this case, a collection of five-granularity semantic terms was defined: S 5 = { S 0 = Extremely low, S 1 = low, S 2 = medium, S 3 = high, S 4 = extremely high}. Owing to limited space, the evaluation of only one expert, E 1 , is presented herein. The evaluation indexes for user needs and technical structural modules by expert E 1 were converted into the form of a PLTS, as shown in Table 1. Expert E 1 converted the probabilistic semantic term evaluation of the user needs and technical modules into a score function, as shown in Table 2.
The degree of deviation between the expert E 1 evaluation and the correlation between user needs and technical modules can be calculated using Equation (6), as shown in Table 3.
The correlation between the expert and user needs and technical modules can be converted into exact numerical values using Equation (18). Subsequently, six experts ( E 1 E 6 ) were averaged as shown in Table 4.
Because there are a few repeated keywords in the BTM model topics, the mutual influence between user needs was considered in this study to obtain further user need information. The evaluation of the mutual influence between user needs was considered by expert E 1 and was converted into the form of a PLTS, as shown in Table 5. The evaluation of the mutual influence between user needs by expert E 1 was converted into a score function, as shown in Table 6.
The degree of deviation between expert E 1 ’s evaluation and user needs can be calculated using Formula (6), as shown in Table 7.
The normalization coefficient λ = 15.9246 can be calculated using Formula (12). Subsequently, the total correlation matrix T can be calculated using Equations (13), (14), and (19), as shown in Table 8.
The index weight based on the PLTS DEMATEL can be calculated according to Equations (15)–(17), as shown in Table 9.
According to Table 9, the normalized maximum reference weight is ω r = ω 1 = 0.195621 . The average normalized weights of all the experts are listed in Figure 5.
According to Figure 5, this can be obtained by normalizing the index importance as ω ¯ 1 r = 0.196792 , ω ¯ 2 r = 0.178659 , ω ¯ 3 r = 0.173552 , ω ¯ 4 r = 0.156606 , ω ¯ 5 r = 0.138382 , and ω ¯ 6 r = 0.156009 . Finally, the importance of the technology module, considering the influence between user requirements, is obtained by using Equation (20) combined with calculating the data in Table 4 and Figure 5, C = ( 2.277 ,   2.441 ,   1.939 ,   2.406 ,   2.396 ,   2.606 ,   2.674 ,   2.197 ) T .

4.2. Experimental Results

The importance of the technical modules that influenced user needs were calculated by combining the results of all experts with Formula (20) and resulted in: C 1 (2.277), C 2 (2.441), C 3 (1.939), C 4 (2.406), C 5 (2.396), C 6 (2.606), C 7 (2.674), and C 8 (2.197), respectively. The order of importance (from high to low) was C 7   > C 6   > C 2   > C 4   > C 5   > C 1   > C 8   > C 3 . The efficiency ( C 7 ), flight ( C 6 ), photography ( C 2 ), and control ( C 4 ) were the four most important modules. Therefore, priority was given to the iterative improvement of these four modules in the UAV component.
By conducting a focus group to assess user concerns related to UAV components and focusing on technical structures and functions, the upgrade of UAVs can be significantly advanced. Additionally, this approach can ensure the sustainable production and utilization of UAV components, thereby promoting the sustainable development of UAVs. Figure 6 displays the UAV module components that were analyzed in combination with their functional characteristics, based on meeting user needs.
This approach places the structure at the core, with a focus on the main component modules and user needs. Based on an analysis of the critical component modules, the primary structural components that met user needs were determined to be the battery module, body shell, propeller, shooting lens, fixed flight path data, and signal receiver. Following the identification of the iterative components, a focus group discussion was conducted to gather feedback on product improvement, as presented in Table 10.

5. Conclusions and Outlook

In this paper, we proposed a sustainable iterative research method for mining user needs and integrating decision-making products based on online reviews and utilizing probabilistic semantic term sets (PLTS). Our method leverages expert evaluations to address the hesitation and uncertainty that exists between user requirements and product modules, and uses the DEMATEL method to consider the correlation between requirement indicators. By improving the association function of DEMATEL using binary semantics, we reconstructed the PLTS acquisition score function with a deviation into an exact value DEMATEL association function, thereby providing accurate feedback on how users feel about the use of product components while ensuring the sustainable recycling of most product components.
Our research has three main innovations:
  • The introduction of multi-attribute decision making into sustainable product iterative design, which ensures longer product life cycles, reduces the costs and risks for small and medium-sized manufacturing industries, and minimizes the waste of available resources.
  • The improvement of the association function of DEMATEL, which accurately expresses the hesitation and uncertainty of expert evaluations, and solves the problem traditional multi-attribute decision making makes by not considering the interaction between indicators.
  • The improvement of the technical method of mining user demand information by combining it with the product structure module, which enables more accurate user demand screening and the identification of product components and modules for improvement using online reviews’ information about user requirements.
Our experiments have shown that our method of user demand mining allows companies to accurately identify iterative product component modules that can be updated to meet user demand while ensuring the sustainable recycling of most components, and can compete with similar products by predicting future product development through constantly updated online reviews.
This paper has certain limitations, as the case product analyzed was a single type and may have a limited area of application, affecting the universality of the method. Future research will cover a variety of product types to expand the scope of this method. At the same time, in the product iteration process there are many manual steps which affect the result. We will build on this research to develop applications to improve the automation of sustainable iterative product design processes.
In future work, we will explore the generalization of our method to different product types and investigate its applicability to diverse industries. We will also investigate the use of other techniques for mining user needs, and explore the impact of incorporating additional sources of data into our method.

Author Contributions

Research methodology, Q.W., S.W. and S.F.; software, S.W.; validation, S.F. and S.W.; formal analysis, Q.W.; survey, S.W.; data collation, S.W.; writing—original draft preparation, S.F.; writing—review and editing, S.W.; supervision, Q.W.; project administration, S.W.; funding acquisition, Q.W. and S.F. All authors have read and agreed to the published version of the manuscript.

Funding

This study was supported by the Ministry of Education of China, Humanities and Social Science Foundation Research Fund (No. 21YJC760081).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

All data included in this study are available upon request by contacting the corresponding author.

Conflicts of Interest

The authors declare there are no conflict of interest.

References

  1. Brozzi, R.; Forti, D.; Rauch, E.; Matt, D.T. The Advantages of Industry 4.0 Applications for Sustainability: Results from a Sample of Manufacturing Companies. Sustainability 2020, 12, 3647. [Google Scholar] [CrossRef]
  2. Khan, I.; Hou, F.; Le, H.P.; Ali, S.A. Do natural resources, urbanization, and value-adding manufacturing affect environmental quality? Evidence from the top ten manufacturing countries. Resour. Policy 2021, 72, 102109. [Google Scholar] [CrossRef]
  3. Sarkar, B.; Ullah, M.; Sarkar, M. Environmental and economic sustainability through innovative green products by remanufacturing. J. Clean. Prod. 2022, 332, 129813. [Google Scholar] [CrossRef]
  4. Chekima, B.; Wafa, S.; Igau, O.A.; Chekima, S.; Sondoh, S.L. Examining green consumerism motivational drivers: Does premium price and demographics matter to green purchasing? J. Clean. Prod. 2016, 112, 3436–3450. [Google Scholar] [CrossRef]
  5. Mont, O.K. Clarifying the concept of product-service system. J. Clean. Prod. 2002, 10, 237–245. [Google Scholar] [CrossRef]
  6. Feng, D.; Lu, C.F.; Jiang, S.F. An Iterative Design Method from Products to Product Service Systems-Combining Acceptability and Sustainability for Manufacturing SMEs. Sustainability 2022, 14, 722. [Google Scholar] [CrossRef]
  7. Gartzen, T.; Brambring, F.; Basse, F. Target-Oriented Prototyping in Highly Iterative Product Development. In Proceedings of the 3rd International Conference on Ramp-up Management (ICRM), Aachen, Germany, 22–24 June 2016; pp. 19–23. [Google Scholar]
  8. Zhou, J.Y.; Yu, M.; Zhao, W.; Zhang, K.; Chen, J.; Guo, X. An Iterative Conceptual Design Process for Modular Product Based on Sustainable Analysis and Creative Template Method. Processes 2022, 10, 1095. [Google Scholar] [CrossRef]
  9. Liu, Y.; Gan, W.X.; Zhang, Q. Decision-making mechanism of online retailer based on additional online comments of consumers. J. Retail. Consum. Serv. 2021, 59, 102389. [Google Scholar] [CrossRef]
  10. Ji, X.; Gao, Q.; Wang, H. A bilevel-optimization approach to determine product specifications during the early phases of product development: Increase customer value and reduce design risks. Expert Syst. Appl. 2022, 188, 116012. [Google Scholar] [CrossRef]
  11. Yang, C.; Wu, L.G.; Tan, K.; Yu, C.Y.; Zhou, Y.L.; Tao, Y.; Song, Y. Online User Review Analysis for Product Evaluation and Improvement. J. Theor. Appl. Electron. Commer. Res. 2021, 16, 1598–1611. [Google Scholar] [CrossRef]
  12. Yu, Z.Y.; Zhao, W.; Guo, X.; Hu, H.C.; Fu, C.; Liu, Y. Multi-Indicators Decision for Product Design Solutions: A TOPSIS-MOGA Integrated Model. Processes 2022, 10, 303. [Google Scholar] [CrossRef]
  13. Liu, P.D.; Chen, S.M.; Wang, Y.M. Multiattribute group decision making based on intuitionistic fuzzy partitioned Maclaurin symmetric mean operators. Inf. Sci. 2020, 512, 830–854. [Google Scholar] [CrossRef]
  14. Ramalingam, S. Fuzzy interval-valued multi criteria based decision making for ranking features in multi-modal 3D face recognition. Fuzzy Sets Syst. 2018, 337, 25–51. [Google Scholar] [CrossRef]
  15. Liu, H.C.; Quan, M.Y.; Shi, H.; Guo, C. An integrated MCDM method for robot selection under interval-valued Pythagorean uncertain linguistic environment. Int. J. Intell. Syst. 2019, 34, 188–214. [Google Scholar] [CrossRef]
  16. Zhang, Y.J.; Wei, G.W.; Guo, Y.F.; Wei, C. TODIM method based on cumulative prospect theory for multiple attribute group decision-making under 2-tuple linguistic Pythagorean fuzzy environment. Int. J. Intell. Syst. 2021, 36, 2548–2571. [Google Scholar] [CrossRef]
  17. Rodriguez, R.M.; Martinez, L.; Herrera, F. Hesitant Fuzzy Linguistic Term Sets for Decision Making. Ieee Trans. Fuzzy Syst. 2012, 20, 109–119. [Google Scholar] [CrossRef]
  18. Wang, H. Extended hesitant fuzzy linguistic term sets and their aggregation in group decision making. Int. J. Comput. Intell. Syst. 2015, 8, 14–33. [Google Scholar]
  19. Chen, Z.S.; Chin, K.S.; Li, Y.L.; Yang, Y. Proportional hesitant fuzzy linguistic term set for multiple criteria group decision making. Inf. Sci. 2016, 357, 61–87. [Google Scholar] [CrossRef]
  20. Pang, Q.; Wang, H.; Xu, Z.S. Probabilistic linguistic linguistic term sets in multi-attribute group decision making. Inf. Sci. 2016, 369, 128–143. [Google Scholar] [CrossRef]
  21. Liao, H.C.; Mi, X.M.; Xu, Z.S. A survey of decision-making methods with probabilistic linguistic information: Bibliometrics, preliminaries, methodologies, applications and future directions. Fuzzy Optim. Decis. Mak. 2020, 19, 81–134. [Google Scholar] [CrossRef]
  22. Lei, F.; Wei, G.W.; Gao, H.; Wu, J.; Wei, C. TOPSIS Method for Developing Supplier Selection with Probabilistic Linguistic Information. Int. J. Fuzzy Syst. 2020, 22, 749–759. [Google Scholar] [CrossRef]
  23. Liu, P.D.; Teng, F. Probabilistic linguistic TODIM method for selecting products through online product reviews. Inf. Sci. 2019, 485, 441–455. [Google Scholar] [CrossRef]
  24. Li, G.; Law, R.; Vu, H.Q.; Rong, J. Discovering the hotel selection preferences of Hong Kong inbound travelers using the Choquet Integral. Tour. Manag. 2013, 36, 321–330. [Google Scholar] [CrossRef]
  25. Baykasoglu, A.; Golcuk, I. Development of an interval type-2 fuzzy sets based hierarchical MADM model by combining DEMATEL and TOPSIS. Expert Syst. Appl. 2017, 70, 37–51. [Google Scholar] [CrossRef]
  26. Yi, Z.H. Decision-making based on probabilistic linguistic term sets without loss of information. Complex Intell. Syst. 2022, 8, 2435–2449. [Google Scholar] [CrossRef]
  27. Wan, S.P. 2-Tuple linguistic hybrid arithmetic aggregation operators and application to multi-attribute group decision making. Knowl.-Based Syst. 2013, 45, 31–40. [Google Scholar] [CrossRef]
  28. Nilashi, M.; Zakaria, R.; Ibrahim, O.; Abd Majid, M.Z.; Zin, R.M.; Farahmand, M. MCPCM: A DEMATEL-ANP-Based Multi-criteria Decision-Making Approach to Evaluate the Critical Success Factors in Construction Projects. Arab. J. Sci. Eng. 2015, 40, 343–361. [Google Scholar] [CrossRef]
  29. Yan, X.; Guo, J.; Lan, Y.; Cheng, X. A biterm topic model for short texts. In Proceedings of the 22nd International Conference on World Wide Web, Rio de Janeiro, Brazil, 13–17 May 2013; Association for Computing Machinery: New York, NY, USA, 2013; pp. 1445–1456. [Google Scholar]
  30. Cheng, X.Q.; Yan, X.H.; Lan, Y.Y.; Guo, J.F. BTM: Topic Modeling over Short Texts. IEEE Trans. Knowl. Data Eng. 2014, 26, 2928–2941. [Google Scholar] [CrossRef]
  31. Zhang, Y.Z.; Ye, C.M.; Geng, X.L. Probabilistic linguistic term set multi-criteria decision-making method considering psychological behavior of decision makers. J-Glob. 2020, 37, 3001–3024. [Google Scholar]
  32. Pan, Q.H.; Liu, X.S.; Bao, H.L.; Su, Y.; He, M.F. Evolution of cooperation through adaptive interaction in a spatial prisoner’s dilemma game. Phys. A-Stat. Mech. Its Appl. 2018, 492, 571–581. [Google Scholar] [CrossRef]
  33. Balachandran, A.; Voelker, G.M.; Bahl, P.; Venkat Rangan, P. Characterizing user behavior and network performance in a public wireless LAN. Perform. Eval. Rev. 2002, 30, 195–205. [Google Scholar] [CrossRef] [Green Version]
  34. Stankovic, M.; Mirza, M.M.; Karabiyik, U. UAV Forensics: DJI Mini 2 Case Study. Drones 2021, 5, 49. [Google Scholar] [CrossRef]
Figure 1. Flow chart of the sustainable product iterative design approach.
Figure 1. Flow chart of the sustainable product iterative design approach.
Sustainability 15 05950 g001
Figure 2. Schematic diagram of a BTM operation.
Figure 2. Schematic diagram of a BTM operation.
Sustainability 15 05950 g002
Figure 3. Subject extraction results.
Figure 3. Subject extraction results.
Sustainability 15 05950 g003
Figure 4. User requirements filtering diagram.
Figure 4. User requirements filtering diagram.
Sustainability 15 05950 g004
Figure 5. Normalized mean weights of the six experts.
Figure 5. Normalized mean weights of the six experts.
Sustainability 15 05950 g005
Figure 6. Analysis of the important component modules.
Figure 6. Analysis of the important component modules.
Sustainability 15 05950 g006
Table 1. Expert E1 evaluation of the indicators relating user needs to the technology modules.
Table 1. Expert E1 evaluation of the indicators relating user needs to the technology modules.
C1C2C3C4C5C6C7C8
R1{S3(0.4), S4(0.6)}{S3(0.3), S4(0.7)}{S3(0.5), S4(0.5)}{S3(0.4), S4(0.6)}{S2(0.25), S3(0.25), S4(0.5)}{S2(0.3), S3(0.7)}{S3(0.4), S4(0.6)}{S3(0.4), S4(0.6)}
R2{S1(1)}{S1(0.3), S2(0.7)}{S1(0.5), S2(0.5)}{S4(1)}{S3(0.4), S4(0.6)}{S2(0.4), S3(0.6)}{S3(0.5), S4(0.5)}{S3(0.3), S4(0.7)}
R3{S2(0.4), S3(0.6)}{S2(0.25), S3(0.5), S4(0.25)}{S2(0.3), S3(0.7)}{S2(0.2), S3(0.8)}{S2(0.3), S3(0.7)}{S4(1)}{S2(0.4), S3(0.6)}{S2(0.5), S2(0.5)}
R4{S1(0.4), S2(0.6)}{S1(0.5), S2(0.5)}{S1(0.4), S2(0.6)}{S0(0.4), S1(0.6)}{S1(0.2), S2(0.8)}{S3(0.5), S4(0.5)}{S1(0.2), S2(0.8)}{S0(0.25), S1(0.25), S2(0.5)}
R5{S3(0.3), S4(0.7)}{S3(1)}{S0(0.5), S1(0.5)}{S2(0.4), S3(0.6)}{S1(0.2), S2(0.8)}{S1(0.3), S2(0.7)}{S2(0.3), S3(0.7)}{S0(0.2), S1(0.8)}
R6{S3(1)}{S3(0.3), S4(0.7)}{S2(0.25), S3(0.25), S4(0.5)}{S3(0.5), S4(0.5)}{S3(0.4), S4(0.6)}{S3(0.3), S4(0.7)}{S3(0.4), S4(0.6)}{S3(0.5), S4(0.5)}
Table 2. Expert E1 score function for correlating user requirements with technical modules.
Table 2. Expert E1 score function for correlating user requirements with technical modules.
C1C2C3C4C5C6C7C8
R1S3.6S3.7S3.5S3.6S3.25S2.7S3.6S3.6
R2S1S1.7S1.5S4S3.6S2.6S3.5S3.7
R3S2.6S3S2.7S2.8S2.7S4S2.6S2
R4S1.6S1.5S1.6S0.6S1.8S3.5S1.8S1.25
R5S3.7S3S0.5S2.6S1.8S1.7S2.7S0.8
R6S3S3.7S3.25S3.5S3.6S3.7S3.6S3.5
Table 3. Degree of deviation of expert E1 from the correlation between user requirements and technical modules.
Table 3. Degree of deviation of expert E1 from the correlation between user requirements and technical modules.
C1C2C3C4C5C6C7C8
R10.33940.2970.35350.33940.18220.29690.33940.3394
R200.2970.353500.33940.33940.35350.2969
R30.33940.3540.29690.22620.296900.33940.3535
R40.33940.3540.33940.33940.22620.35350.22620.5901
R50.296900.35350.33940.22620.29690.29690.2262
R600.29690.18220.35350.33940.29690.33940.3535
Table 4. Precise function values of user requirements and technical modules by six experts.
Table 4. Precise function values of user requirements and technical modules by six experts.
C1C2C3C4C5C6C7C8
R13.32173.32173.13402.82173.06562.39433.24913.2491
R21.06701.39431.13403.31913.24911.69153.13403.3943
R32.24912.63822.39432.40892.39433.56702.24911.6340
R41.24911.13401.32170.24910.90893.13401.56870.6367
R52.51412.69710.13402.24911.06871.39432.39430.5687
R63.19713.39433.06563.13403.24913.39433.24913.1340
Table 5. Expert E1 evaluation of the probabilistic semantics terminology for the impact between user requirements.
Table 5. Expert E1 evaluation of the probabilistic semantics terminology for the impact between user requirements.
R1R2R3R4R5R6
R10{S2(0.75), S3(0.25)}{S3(0.2), S4(0.8)}{S0(0.1), S1(0.5), S2(0.4)}{S3(0.4), S4(0.6)}{S3(0.75), S4(0.25)}
R2{S2(0.25), S3(0.5), S4(0.25)}0{S3(0.5), S4(0.5)}{S3(0.75), S4(0.25)}{S0(0.1), S1(0.6), S2(0.3)}{S1(0.25), S2(0.5), S3(0.25)}
R3{S1(0.2), S2(0.8)}{S3(0.5), S4(0.5)}0{S2(0.3), S3(0.7)}{S0(0.3), S1(0.7)}{S3(0.25), S4(0.75)}
R4{S3(0.5), S4(0.5)}{S2(0.25), S3(0.75)}{S2(0.2), S3(0.8)}0{S0(0.2), S1(0.8)}{S1(0.25), S2(0.75)}
R5{S3(0.75), S4(0.25)}{S1(0.25), S2(0.75)}{S1(0.25), S2(0.5), S3(0.25)}{S1(0.25), S2(0.75)}0{S0(0.5), S1(0.5)}
R6{S3(0.5), S4(0.5)}{S1(0.3), S2(0.7)}{S0(0.1), S1(0.9)}{S0(0.2), S1(0.8)}{S2(0.1), S3(0.9)}0
Table 6. Expert E1 score function for the interconnection between user requirements.
Table 6. Expert E1 score function for the interconnection between user requirements.
R1R2R3R4R5R6
R10S2.25S3.8S1.4S3.6S3.25
R2S30S3.5S3.25S1.2S2
R3S1.8S3.50S2.7S0.7S3.75
R4S3.5S2.75S2.80S0.8S1.75
R5S3.25S1.75S2S1.750S0.5
R6S3.5S1.7S0.9S0.8S2.90
Table 7. The extent to which Expert E1 deviates from the interconnectedness of user needs.
Table 7. The extent to which Expert E1 deviates from the interconnectedness of user needs.
R1R2R3R4R5R6
R100.26510.22620.52880.33940.2651
R20.353500.35350.26510.26870.3125
R30.22620.353500.29690.29690.2651
R40.35350.26510.226200.22620.2651
R50.26510.26510.31250.265100.3535
R60.35350.29690.12720.22620.12720
Table 8. Total correlation matrix T.
Table 8. Total correlation matrix T.
R1R2R3R4R5R6
R10.99570.98241.09910.84480.90061.0155
R21.12160.81301.05960.88970.74520.9305
R31.01810.96310.80970.82580.69010.9623
R41.06560.90350.95420.64840.67480.8518
R50.91510.73940.79930.65980.51550.6756
R60.93190.72600.72900.60090.68750.6078
Table 9. DEMATEL indicator weights based on the probabilistic semantic term.
Table 9. DEMATEL indicator weights based on the probabilistic semantic term.
DjFjDj + FjDj − Fjωj ω j ¯
R15.83806.047911.88590.209911.88780.1956
R25.55965.127310.6868−0.432310.69560.1760
R35.26915.450910.72000.181810.72160.1764
R45.09834.46949.5677−0.62909.58840.1578
R54.30464.21378.5184−0.09098.51890.1402
R64.28305.04359.32650.76059.35750.1540
Table 10. Product iterative improvement design scheme.
Table 10. Product iterative improvement design scheme.
IndexUser RequirementsProduct ComponentsFocus Group Improvement ProgrammeSustainable Development Requirement
1Battery power, flight durationBattery modules Improving battery charging efficiency and battery capacity.It is recommended to use batteries that do not contain harmful substances, such as lithium iron phosphate, lithium polymer battery, lithium polymer battery, etc
2Wind resistance and stability of dronesFuselage shells, propellersLifting of the overall UAV housing counterweight to ensure flight; widening of the propeller to adjust propeller orientation in the event of wind.The overall materials of the drone body, such as polypropylene, polyamide and polycarbonate, should meet the specific environmental regulations of developing countries
3Picture clarityShooting footageProfessional photographers, with the possibility of configuring higher photographic components, offering individual choice of parts.Photographic components shall meet the technical requirements of environmental labeling products
4photography module Fixed flight path dataProfessional photographers, with the possibility of configuring higher photographic components, offering individual choice of parts.This demand is not sustainable development requirements
5Smart FollowSignal Receiver, Recording LensSet receiver module, drone sensor receiver module, intelligent following, more suitable for the elderly and children to go out to monitor and professional record life bloggers.This demand is not sustainable development requirements
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Wang, Q.; Wang, S.; Fu, S. A Sustainable Iterative Product Design Method Based on Considering User Needs from Online Reviews. Sustainability 2023, 15, 5950. https://doi.org/10.3390/su15075950

AMA Style

Wang Q, Wang S, Fu S. A Sustainable Iterative Product Design Method Based on Considering User Needs from Online Reviews. Sustainability. 2023; 15(7):5950. https://doi.org/10.3390/su15075950

Chicago/Turabian Style

Wang, Qi, Shuo Wang, and Si Fu. 2023. "A Sustainable Iterative Product Design Method Based on Considering User Needs from Online Reviews" Sustainability 15, no. 7: 5950. https://doi.org/10.3390/su15075950

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop