Next Article in Journal
Why Choose to Cycle in a Low-Income Country?
Next Article in Special Issue
Exploring the Possibility of Linking and Incorporating Future Design in Backcasting and Scenario Planning
Previous Article in Journal
Can Proper Funding Enhance Sustainable Tourism in Rural Settings? Evidence from a Developing Country
Previous Article in Special Issue
Future Design as a Metacognitive Intervention for Presentism
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Impact of Future Design on Workshop Participants’ Time Preferences

1
Department of Gastronomy Management, Ritsumeikan University, Biwako-Kusatsu Campus (BKC), Shiga 525-8577, Japan
2
Faculty of Economics and Law, Shinshu University, Nagano 390-8621, Japan
*
Author to whom correspondence should be addressed.
Sustainability 2020, 12(18), 7796; https://doi.org/10.3390/su12187796
Submission received: 26 June 2020 / Revised: 11 September 2020 / Accepted: 17 September 2020 / Published: 21 September 2020
(This article belongs to the Collection Future Design)

Abstract

:
In this paper, we examine the impact of Future Design (FD) on public workshops organized in Matsumoto city, Japan, for its city hall renovation plan. We ran an FD workshop and an ordinary workshop as a control, and the participants were randomly assigned to one of the two workshops. We identified the SVO (social value orientation) type (pro-social, pro-self, and other) and elicited time preference of each participant using simple questionnaires that were independent of the context of the workshops. We found that pro-self individuals tend to have shorter time perspectives than pro-social individuals before the workshops. While the pro-self individuals who went through the ordinary workshop became even more myopic, we did not detect such adverse effects in the FD workshop. This contrast between the ordinary and FD workshops is consistent with the qualitative differences in the policy outcomes between the two workshops. The discussions in the ordinary workshop tended to focus on the resolution of today’s needs, such as acquiring more rooms and more services, etc., while the discussions in the FD workshop focused on the more fundamental functions of the city hall that will be needed in the future, thereby leading to more constructive policy proposals. Such demand-based discussions in the ordinary workshop may have been a result of the growing myopia within the pro-self participants, who insisted on ensuring their current needs.

1. Introduction

‘Public participation’ is a commonly adopted method of policy making, particularly for local municipalities. The early stages of policy planning theory placed a large emphasis on ‘rationality’ along with optimism, where a policy planner was intended to be strictly neutral, and to offer alternatives backed by scientific evidence. Conversely, the public’s role was limited to advising based on values and preferences. Often, however, public workshops become fields of conflict among the relevant diverse stakeholders. Although in recent years, it has been legally required to employ public participation in government decision making in various forms of public review, citizens, agencies, and researchers are disappointed at these public participation procedures. Innes and Booher [1] noted that these procedures “do not work. They do not achieve genuine participation in planning or other decisions; … Most often these methods discourage busy and thoughtful individuals from wasting their time going through what appears to be nothing more than rituals designed to satisfy legal requirements” (p.419). Many studies—including those by Baum [2], and Hibbard and Lurie [3]—and many public administrators have found these procedures to be problematic.
Various practical efforts have been made to cope with this difficulty. Noteworthy attempts include ‘Responsible Research and Innovation’ (see Owen et al. [4] and Stilgoe et al. [5]), which introduced public participation to govern science and innovation as a form of social responsibility. Fitzgerald et al. [6] reported on the effectiveness of public participation in forming European Commission regulations. The authors also reported that, despite the help of well-prepared workshop management, “many citizens turned up on the day with a pet subject and tried to dominate the conversation” (p. 257). Communicative planning theory promotes ‘collaborative planning’ as an alternative, which incorporates all of the stakeholders in the community into a flat mutual dialogue network that helps to integrate individual and social interest. However, this approach is not enough to dissolve the ongoing frustrations, despite the scholarly efforts in making methodological improvements, such as those by Denhardt and Denhardt [7], and Kakabadse et al. [8].
This paper investigates the impact of ‘Future Design’ (FD), as proposed by Saijo [9,10,11], on the outcomes of public participation, as well as the valuation of the future in the minds of the participants. FD is fundamentally different from other approaches to public participation, in that it aims to solve not only conflicts between individuals and social interests but also conflict between the present and future generations, which are the source of the pressing issues of sustainability that many communities now face. FD is an attempt to redesign communities and societies to be more forward-looking by incorporating future generations as one of the vital stakeholders, which both democracy and markets fail to accomplish. Since these future generations are yet to be born, it is crucial to let public participants recognize and value the interest of future generations in practice. We followed the workshop method developed by Saijo et al. in the first FD attempt in Yahaba town, Japan (reported by Hara [12], and Hara et al. [13]), where participants were asked to play the role of an imaginary future generation. They were instructed to pretend not to be older versions of themselves, but rather to be their counterparts of the same age, with the same occupation, living 50 to 60 years in the future.
The field of future studies is closely related to FD, which increasingly involves public participation to induce cognitive changes in society for a sustainable future. The methods commonly used in the future studies to consider the desirability and feasibility of a future state include ‘backcasting’ (Quist and Vergragt [14], Robinson [15], Robinson et al. [16]) and ‘scenario planning’ (Peterson et al. [17], Patel et al. [18], Chakraborty [19], and Duckett et al. [20]). It should be noted that these studies share the problematic issue of communication under the collaborative planning theory. Referring to Habermas [21], Innes and Booher [22] argued that communication is not a mere neutral transmission of factual information, but is rather a form of acting on others. Habermas based his argument on Weber’s concept of ‘social action’ which is defined by the properties of ‘purposive-rationality’ (connecting values and means to ends) and ‘value-rationality’ (which involves screening consequences with a subjective intrinsic meaning). Weber added two more categories to his typology of social action—‘present affects’ and ‘long practice or tradition’—to generate coordination and consensus.
FD challenges the latter two types of rationality by freeing participants from ‘present affects’ and ‘present practice’ in order to obtain a wider and longer-term perspective to cope with social problems stemming from intergenerational dilemmas, especially sustainability. A possible shift in the participants’ perspectives can be found in the qualitative differences in the policy recommendations generated by the FD workshop and the non-FD workshop. Hara [12], Hara et al. [13], and Nakagawa et al. [23] identified such differences via transcription analyses of textual data from the first FD workshop performed in Yahaba. Based on the transcription data from the recent Yahaba FD attempt, Nakagawa [24] proposed a conceptual framework where he extracted four factors to free participants from their present constraints.
This paper takes a completely different approach from these previous FD studies. The goal of the paper is to identify changes in participants’ perspectives and determine whether the changes were caused by the FD workshop, based on the principles of the design of experiments. We elicited quantitatively individual time preferences from the workshop participants through questionnaires, whose contents were independent of the context of the workshops. We utilized a standardized question format in the field of economics that contains a list of simple intertemporal choices of monetary outcomes. The choice alternatives are purely monetary and have no contextual link to the topic of the main workshop. This facilitates the elicited time preferences to be a neutral measure in order to compare the ordinary and FD workshops. The questions were designed such that the answers can be mapped to a set of plain numbers, which are described in detail in the next section.
In Matsumoto city, Japan, we organized public participation workshops for policy-making purposes from November 2017 to February 2018. The target of the workshops was a 60-year-old city hall renovation plan. We ran two sets of workshops for the same target: one took the form of an ordinary workshop and the other took the form of an FD workshop. The discussion groups were formed randomly, and those in the ordinary workshop were labelled ‘C’ for the current generations and also for the control groups, while those in the FD workshop were labelled ‘F’ for the future generations, which are the treatment groups. Under this setting, any difference in participants’ time preference data between the C and F groups should reflect the influence of FD on the participants’ time perspectives.
We delivered the same set of questionnaires before and after the workshops to every participant in the C and F groups. As explained in detail in the next section, the questionnaires consisted of two sections of multiple-choice questions on simple monetary alternatives, all of which had no context-dependence with the workshops. In this way, we did not expect that conducting the questionnaires before the workshops would interfere with the main workshop discussions to follow. We did not reveal our research interests to the participants until the very end of all the workshops, in order to minimize the risk of data distortion.
The first part of the questionnaire was used to elicit the SVO (Social Value Orientation) index commonly used in the field of social psychology (see, for example, Van Lange and Kuhlman [25]) and thus identify individual social inclinations. We classified the participants into three SVO types: ‘pro-social’, those who value social welfare more than self-welfare; ‘pro-self’, those who ensure their self-welfare before social welfare; and ‘the other’. We did not provide these classifications to our participants. The second part contained multiple binary choices asking each participant to choose an amount of money today that would be equivalent in value to a given amount of money one year later. This equivalent amount of money today should reflect individual time preferences. If the amount is low, an individual is impatient and myopic. In addition to the above standard time preference questionnaire, Q1, we constructed Q2, which is a variant of Q1 created by replacing Q1’s intertemporal choices of individual monetary outcomes with intertemporal choices that incorporate not only own monetary outcomes but also the other person’s monetary outcomes. Q1 calls for purely individual decisions, while Q2 requires decisions for a pair. We thus obtained two kinds of time preference data per person.
In conjunction with the SVO classification, we compared the time preference data before and after the workshops and between the C and F groups in order to see if FD has any impact on time perspectives. Research on how time preference correlates with psychological personality traits such as the ‘Big Five’ has begun to develop in recent years (see Cohen et al. (2020) [26] for a survey). However, to our best knowledge, no other studies have examined time preference with SVO except for Hernuryadin et al. (2019) [27], who investigated the group-level time preferences in relation to the SVO type configuration within a group, and found mixed results.
Our main findings are two-fold. First, we found a strong correlation between individual SVO and individual time preferences, such that pro-self individuals were more myopic than pro-social individuals, in general. Second, the pro-self participants became even more myopic after the non-FD workshop, while the FD workshop did not produce such an adverse effect. For the pro-social group, a rough comparison of the group’s average time preferences before and after the workshop indicated that the participants in the F groups tended to have a longer time perspective after the FD workshop, but an individual-level comparison did not confirm this tendency.
These findings correlate with the clear qualitative difference observed from the textual data of the workshop discussions between the C and F groups. The city hall renovation design policies suggested by the C groups listed various needs—such as more parking lots, more counselling and registration windows, bigger rooms, bigger buildings—from a present-day perspective. In contrast, the counterparts from the F groups did not describe such need-base items. Instead, the F group participants began to question what the function of the local government should be in the future, and determined that the city hall should be equipped with the tools and capacities to cope with the expected social issues and problems. This type of qualitative difference exactly corresponds to that reported in the other FD attempts [12,13,23,24].
Furthermore, we found that the age of participants mattered. Those who were relatively younger (in their 20s and 30s) were more susceptible to the workshop’s form difference, with and without FD, than those who were older. Furthermore, we found that the city officers were more susceptible to the workshop’s form difference than the general public. All of these cases were specific to the pro-self participants.
Consequently, the contribution of this paper can be summarized as follows. Firstly, this paper highlights that measuring individual’s time preferences is a useful objective tool to evaluate the effectiveness of FD workshops, but it has not been included in extant research on public participation, both with FD and without FD. The importance of introducing measures of time preference that are independent of the context of the workshops should be thus emphasized. Without such measures, it would be difficult to clearly determine whether one individual’s recorded statement in the discussions in the F groups indicates a successful mind transformation towards future generations or a futuristic statement to merely conform with the statements of other group members who were successful. Examining individual time preferences brings us one step closer to understanding what occurs within participants’ minds, as reflected by the textual data generated by the workshop.
Secondly, since we followed the principle of the design of experiments, it is certainly the difference in the forms of the workshops, the ordinary or FD workshops, that caused the different responses of the participants’ time perspectives. This does not exclude the possibility of other factors, such as the SVO types of the participants. We will come back to this point in the last item in this paragraph. Thirdly, we combined our time preference analyses with the SVO typology. Thereby, we found a possible serious side-effect of ordinary workshops, which could drive the pro-self participants to behave even more myopically. This side-effect may have contributed to the aforementioned problematic phenomena of public participation that has been widely reported in the literature of various fields. The workshop itself possibly compelled some participants to become more eager to secure their own present-day interests and spoil the community discussions as the workshop proceeds, unless some additional care—such as the FD approach—was taken to design the workshops. Lastly, as has already been seen, this paper showed that heterogeneity among the participants in terms of the SVO typology matters for a successful workshop. We showed that the pro-self individuals tend to be more myopic than the pro-social individuals to begin with. Therefore, it is quite conceivable that the pro-self individuals could lead the workshop discussions to focus more on the current issues. We found that the FD workshop was effective particularly for the pro-self participants in preventing them from suffering the adverse effect of workshops otherwise. Naturally, this aspect of the FD workshop was more clearly spotted by the data based on Q1 (on individual decisions) than Q2 (on decisions for a pair).

2. Materials and Methods

We ran two sets of workshops, an ordinary workshop and an FD workshop, on the city hall renovation plan in Matsumoto city, Japan, from November 2017 to March 2018. Matsumoto is a town with about 240,000 residents, located in the dry and sunny highlands at an altitude of 600 m, surrounded by the 3000 m high mountains known as the Japanese Alps. Matsumoto is the commercial and cultural center of Nagano prefecture, which is a 2.5 h train ride from Tokyo or Nagoya. The 60-year-old city hall is situated in the city’s historic area next to Black Castle, which was built in 1504 and is a national treasure. Naturally, concern for the city hall renovation plan is substantial among Matsumoto’s citizens.
Both sets of workshops consisted of two-day sessions following the same procedure, and participants are strongly encouraged to attend both days. The interval between the first and the second day of each two-day workshop was about ten days. The workshop was run twice. The first two-day workshop was organized in November, 2017, when 63 city officers of various divisions volunteered, and the second was held in January and February, 2018, and was attended by 41 citizens who applied to an announcement via the city hall website, the local newspaper, or city flyers. The number of those who participated only on the second day of the first two-day workshop was seven. The number of those who participated only in the first day of the second two-day workshop was four. Those who participated in the second two-day workshop took a city hall tour before the first day workshop. On the first day before the workshop, each participant received a set of questionnaires to determine their SVO types and time preferences, and was asked to answer to all of the questions in the set. The same set of questionnaires was distributed at the very end of the second day after the completion of the workshop.
The city took the initiative to run the two-day workshops as a part of its formal policy making procedures. Each participant received a formal invitation letter in the name of the mayor, which explained that the workshop was organized for the purpose of constructing the city hall’s renovation plan. The letter also mentioned that a survey would be conducted by Shinshu University alongside the workshop discussions, strictly for academic purposes (see Appendix A.1 for details).
The questionnaire instructions distributed on the spot were self-explanatory. The instructions explained that the questionnaires were research material to analyze choice behavior, and that the answers would be treated as anonymous data identifiable only by an ID number for statistical analyses. We instructed the participants to answer the questionnaires without consulting any other participant or staff (Appendix A.2 provides the instructions for the questionnaires, translated into English). We provided no further explanations about the nature of the questionnaires until all of the stages of the second day workshop were completed. Just before the closing announcement, we briefly explained that the FD method was introduced to half of the groups, and a survey was then taken to evaluate the method.
As described in detail in Section 2.2, the content of the questionnaires was independent of the context and purpose of the two-day workshops. Moreover, a ten-day interval between the first and second day is presumably long enough for most of the participants to clear their precise memories of their exact choices on the first day. In this way, we minimized the risk of possible data distortion.
We paid each participant a fixed amount of show-up fee of 4000 yen (about 37 US dollars) at the end of the first day, as well as at the end of the second day. We could not pay the participants in the session for city officers due to regulations.

2.1. FD Workshop

At the beginning of the first day, the participants were randomly divided into 8 groups in the first workshop and 6 groups in the second workshop. Half of these groups were assigned the label ‘C’, and the other half were labeled ‘F’, randomly. However, the participants were not informed of these labels. After an ice-breaking session, a blank city map and chronological table (shown in Figure 1) were distributed to each group, and all of the groups discussed the state of living in Matsumoto as the current generation. The chronological table shows a graph of the annual city population and the age demographics from 1960 to 2060, revealing a projected decline of 34.9% from 2010 to 2045. The area under the graph presented a list of past symbolic events, such as the Tokyo Olympics (1964), the oil crisis (1973 and 1979), the first appearance of mobile phones, and the launch of Windows95, etc., as well as various scientifically predicted events for the future, such as climate change, the expected technological change of self-driving cars, AI robots, regenerative medicine, and other information that is available on the website of the Ministry of Internal Affairs. The C groups primarily used the blank city map to visualize the way of life in the city and used the chronological table as a reference, while the F groups mainly used the chronological table.
As the current generation, the F groups on the first day discussed the things that should have had been done in the past, or the things that were done in the past that they were thankful for. This past reflection work is called ‘past design’, and it helped participants act as an imaginary future generation during the study in Yahaba, in which no chronological table was used (as reported in Hara et al. [13] and Nakagawa et al. [23,28,29]). In our FD implementation in Matsumoto, we used a chronological table for the past design among the F groups. We found that the use of the chronological table amplified the positive effects of past design.
The main work on the second day was to determine a master plan for the city hall renovations, which each group presented in front of all of the participants at the end of the day. The F groups did so as the future generation living in Matsumoto city in 2060, while the C groups did so as the current generation. The details of the process and method that we employed to induce the participants to play the role of imaginary future generation are described by Nishimura et al. [30,31].

2.2. Questionnaires

We distributed the same set of multiple-choice questions twice: before and after the two-day workshops. The questionnaires consisted of two parts. The first part listed nine trinary choice problems to determine SVO types based on Messick and McClintock [32], where each participant was asked to choose one of three alternatives (A, B and C), as shown in Table 1. We asked the participants to imagine that each of them was paired with an unknown, anonymous, and unidentified person whom the participants would never encounter in the future. For example, in Q1 in Table 1, the participant receives 450 yen and the unknown other obtains 80 yen in alternative A; the participant obtains 540 yen and the other receives 280 yen in alternative B; and both receive 480 yen in alternative C. Let r = {r_1, r_2, r_3, r_4, r_5, r_6, r_7, r_8, r_9} denote the choices made in Q1 to Q9. There are three choice patterns of particular interest; rps = {C, B, A, C, B, A, A, C, B}, rind = {B, A, C, B, A, C, B, A, C}, and rcomp = {A, C, B, A, C, B, C, B, A}. According to the SVO typology, those who give six answers that are consistent with pattern rps are classified as being a ‘pro-social’ type, those who give six answers that are consistent with pattern rind are an ‘individualistic’ type, and those who give six answers that are consistent with pattern rcomp are a ‘competitive’ type. The latter two types are often merged as ‘pro-self’. The rest, who were not classified as either type, were labelled as ‘other’. We produced two SVO classifications: one was obtained from the first set of questionnaires and the other from the second set. Those who were classified as pro-social twice were considered to be ‘strong pro-social’.
The second part of each questionnaire was used to measure individual time preferences. Following the multiple-choice list (MCL) method described in Andreoni et al. [33] and Coller et al. [34]), which is often used to elicit individual time preferences in the field of economics, we presented the participants with a list of binary choices, Q1, as shown in Table 2. There were ten questions included (listed as ‘item’), each of which asked the individual participants to choose between option ‘W’ and option ‘M’. If he/she chose option W in item 1, he/she would receive 9800 yen today and 0 yen a year later, while if he/she chose option M, he/she would receive 0 yen today but 10,000 yen a year later. As the item number increased, the amount he/she could receive today in option W reduced by 400 yen, while option M remained the same. We expected most of the participants to choose option W in item 1 and option M in item 10. If the participants’ utility functions are non-decreasing in the prize amount, there should be some item number between 1 and 10 where the choices switch from option W to option M. Let us denote the minimum item number where the choice switch occurs in the Q1 list with q1. Here, a larger q1 value indicates a higher degree of impatience or myopia. We set q1 = 1 for the individual who chose option M from item 1, and q1 = 11 for the individuals who did not make a switch from option W to option M at any item number from 1 to 10.
The amount of today’s prize in option W in item q1—1 approximates the value of 10,000 yen a year later (When q1 = 1, the prize today is set to 10,000 yen). Let us denote that amount by x1. Then, we can find a number δ that satisfies u(x1) = δu(10,000), where the u function represents an individual’s utility from receiving a prize. Number δ is called the ‘discount factor’, which is one of the main building blocks in economic analyses of intertemporal decision making, such as savings and investments in general. The discount factor δ varies according to the length of time t before receiving 10,000 yen. There have been many theoretical and experimental studies on the functional form of δ(t), but the literature remains in disagreement (see Cohen et al. [26] for a survey). In this paper, we focus on identifying any change in an individual’s q1 before and after the FD workshops. This serves our purpose without trying to estimate the exact functional form of δ(t).
While Q1 focuses on the participants’ own rewards, we also propose another list, Q2 (shown in Table 3), where the binary choices relate to the rewards accrued simultaneously by two persons—a choice maker and his/her anonymous unknown counterpart, who has no role in the decision making. The social situation that Q2 depicts is an abstract of the workshops, where the participants discuss public policy, which may also affect the welfare of the non-participants in a community. Similarly to Q1, we focused on the minimum item number of q2 where the decision makers switched their choices for the first time from option W to option M.
In the next section, we report some statistical analyses on the change in q1 and q2 before and after the workshops, combined with conditional analyses on the SVO’s typology. Any change detected for the C groups reflects the impact of the ordinary workshops on the participants’ time perspectives, while any change in the F groups reflects the impact of the FD workshops.

3. Results

In this section, we investigate the possible impacts of FD workshops upon participants’ time preferences. Any impact should also be reflected as a difference in the workshop outcomes between the C and F groups. We briefly sketch our qualitative analyses of workshop outcomes first, and then describe the details of our statistical analyses on time preferences, along with the SVO classification of the participants.

3.1. Qualitative Difference in Workshop Output between the C and F Groups

One facilitator and one graphic facilitator were assigned to each of the 14 groups, as depicted in the two pictures of Figure 2, where the left picture features a discussion scene from one of the C groups, and the right picture is from one of the F groups. These facilitators are members of our research team. Most of them are city officers, who went through training provided by a non-profit organization, SCOP, in Matsumoto to learn the principles of facilitation. Note that all of the members in the right picture, including the two facilitators, are wearing the same green jackets. This is a technique commonly used in FD attempts (e.g., in Yahaba, Suita, etc.) that makes it easier and faster for participants to imagine their future selves. The participants undertook the two-day workshops to determine some of the desirable properties of the renewed city hall as a final output.
We digitally recorded all of the conversations in every group. The graphic facilitator in each group also documented the discussions via graphic recording. Based on the graphic outcomes with the help of the voice data, we were able to extract the core issues that emerged as focal points in the discussions. Figure 3 and Figure 4 show the issues extracted from the C groups and the F groups, respectively.
The expressions in the green square at the top center of each figure describe the desirable state of the city hall to be achieved by the renovation. The yellow dot items connected to the left red square are the focus issues for the benefit of the city officers, and the items connected to the right red square are the focus issues for the benefit of general citizens. There are substantial qualitative differences in the focus issues (marked by yellow dots) between the two groups (figures). Most of the focus issues raised by the C groups are ‘demands’, such as asking for more space, more facilities, and more services, which were labeled as ‘needs, wants, and complaints’, as shown in the upper left orange square in Figure 3. In contrast, there are no such ‘demands’ in the focus issues raised by the F groups (Figure 4). Instead, the items from the F groups are more forward-looking and investment-oriented, viewing the renovated city hall as a place for the training and schooling of the community, interactions and discussions among citizens about city planning and disaster prevention, and—most of all—for the coordination of various productive collaborations between citizens and professionals.
It is also interesting to note that the above qualitative differences are consonant with the observable differences between the C and F groups. We heard many laughs and witnessed youthfully-cheerful discussions in the F groups, which we did not find in the C groups. This observation, alongside the aforementioned outcome differences, suggests that there may have been some changes in the participants’ mind set between the C and F groups. The following analyses provide some supporting evidence.

3.2. Data Analyses of Questionnaires

There were 76 male and 28 female adult participants. In total, 46% were in their 40s or older, labelled ‘older’, with 36% among females and 50% among males. Figure 5a shows the corresponding ratios with the distribution of the SVO types based on the first set of questionnaires for the C groups and F groups. Figure 5b shows the distribution of gender with the SVO types within the younger subgroup and the older subgroup. Since the C and F groups were formed randomly, in Figure 5a, there is little difference between the two groups, while the distribution of the SVO typology shows gender and age dependence, as depicted in Figure 5a,b. The ratio of the pro-social type was higher among female than male participants, and was higher among the older participants of both genders.

3.2.1. Correlation between SVO and Time Preference

Our first finding is a curious correlation between SVO typology and time preference. Based on the first set of questionnaires conducted before the workshops, Figure 6 summarizes the average switch item number q in four cases of Q1 and Q2, conditional on the basic three SVO classifications from the first questionnaire (Figure 6a), and for Q1 and Q2, conditional on whether the SVO type was strongly pro-social or not (Figure 6b). Both the t-test and the Mann-Whitney U-test detected a significant difference in the average q between the pro-social and other types, but only in the right panel for Q2, with less than an 8% significance level. Since four cases out of the four produced an average q of the pro-social type that was nominally lower than that of the other type, the proportion test offers a less than 1% significance p-value to support the lower average q of the ‘pro-social’ type, suggesting that the pro-social type has a longer time perspective.

3.2.2. Quick Overview of the Group Data Comparison

To provide an overview, Figure 7 shows the average q from the first day questionnaire for all of the first day participants, indicated by the darker pillars labeled with ‘before’, as well as the average q from the second day questionnaire over the second day participants, indicated by the lighter pillars labeled with ‘after’. These results are displayed by SVO type, with the C groups labeled as ‘c_SVO type’, and the F groups labeled as ‘f_SVO type’.
For the F groups, the first day and second day differences were significant according to a t-test using heteroscedastic consistent standard errors at a 5% significance level only for the case of the Q2 questionnaire for the pro-social type. The lower q2 in the second day may reflect a tendency for the pro-social type to have a longer time perspective after the workshops, but this was not detected from Q1.
Notably, there was a large difference between before and after the workshops for the pro-self type in the C groups, marked by an orange dotted circle in both panels (Figure 7a,b). According to a t-test, this difference is significant at less than a 1% level, which may suggest a tendency of the pro-self type in the C groups to become even more myopic after the workshops. There was no corresponding tendency found for the pro-self type in the F groups (no indication by a dotted circle means no statistically significant difference in Figure 7.)
The above observation suggests that FD’s impact may be two-fold. One is an impact on the pro-social type, to induce them to have a longer perspective. The other is an impact on the pro-self type, to keep them from becoming more myopic; otherwise, their average q increased greatly in the ordinary workshops.

3.2.3. Analyses Based on Individual Data Comparison

To be more precise, we must examine the differences in the q values of the individuals who participated in both dates of the two-day workshops. The number of data available was reduced to 80 in total: 42 in the C groups, and 38 in the F groups. Table 4(a), below, shows the distribution of SVO types based on the first questionnaire, ‘b_SVO type’, as well as those based on the second questionnaire, ‘a_SVO type’. In order to account for the possible preference instability that may have occurred during the 10-day interval between the first and second date of two-day workshop, we classified the SVO types in the main analyses using ‘s_pro-social’ for those who are classified as pro-social twice, and ‘s_pro-self’ for those who were classified at least once as pro-self; the rest were classified by ‘s_other’. Table 4(b) lists the number of participants in each of the three ‘s SVO’ types in the C and F groups.
The kind of individual data to be considered is the difference in q before and after the workshops, denoted by dq, where dq = [q after] − [q before] for each individual. Because both q after and q before must be valid for the same individuals, our sample size was further reduced to 63 for Q1 (30 for the C groups, and 33 for the F groups) and 65 for Q2 (31 for the C groups, and 34 for the F groups). Each workshop’s schedule and size were strictly constrained by the actual policy-making schedule within the municipality, and it was practically impossible to replicate the same workshops to increase our sample size.
Figure 8 depicts the average dq of Q1 and Q2 in the C groups, as well as in the F groups. The dark blue bars show the average dq of all of the participants; the light green bars are for the s_pro-social type; the yellow bars for s_pro-self type; and the gray bars for s_other type. There are four columns; the left two columns are for the dq data from Q1 in the C groups (the first left column) and in the F groups (the second column). Similarly, the right two columns are for the dq data from Q2 in the C groups (the third) and in the F groups (the fourth). Figure 8 shows a contrasting pattern between the C and F groups, which is mainly reflected in the s_pro-self type (yellow bars), where the average dq of Q1 and Q2 in the C groups are positive, while in the F groups the average dq of Q1 is zero, and the average dq of Q2 is negative. This nominal pattern is consistent with the observation obtained in Figure 7.
We conducted difference-in-differences ordinary least squares (OLS) regressions on individual’s dq, as shown in Table 5. Models (1) and (2) are for Q1, while (3) and (4) are for Q2. The regressors are a dummy variable for the F groups (F), a dummy for the pro-self type individual (s_pro-self), a dummy for the other type individual (s_other), along with two cross terms (s_pro-self * F) and (s_other * F), where the pro-social type is the baseline. The OLS regressions in all of the models (1) to (4) were carried out using robust standard errors corrected by clustering with 14 groups and gender.
Models (1) and (3) show simple results where the regressors do not include the interactions with ‘F’. Both models generated no significant results. This implies that the change in choice of q by the s_pro-social individuals was not different from zero in both the C and F groups. However, by adding the interactions with ‘F’, both models (2) and (4) detect a significantly positive effect of ‘s_pro-self’, coupled with a negative effect of its cross term with ‘F’. The effect of ‘s_pro-self’ is significant in both models, but the significance level of the effect of its cross term is less than 10% in model (2), and 15% in model (4). This means that the behavior of dq for the s_pro-self type individuals is different between the C and F groups. The change in dq of the s_pro-self type relative to the s_pro-social type within the C groups is positive and larger, which means that the s_pro-self type in the C groups became more myopic after the workshop, while the s_pro-social type did not experience a change in time perspectives. The negatively significant effect of the cross term of ‘s_pro-self’ with ‘F’ in model (2) (weakly in model (4)) cancels out the positive effect of ‘s_pro-self’. This means that the FD workshop prevented the s_pro-self participants in the F groups from becoming more myopic after the workshop relative to the s_pro-social participants.
Table 6 reports the marginal effects of the probit regressions, where the dependent variable takes a value of one if dq > 0 for Q1 and for Q2, with the same regressors used in Table 5. It is meaningful to examine the likelihood of dq > 0, since an individual has to increase his/her switch point choice of q by at least one unit of the question items if dq > 0. As in Table 5, the regressions were carried out with the robust standard errors corrected by clustering according to groups and gender.
Table 6 shows somewhat stronger results than Table 5. This time, models (1) and (2) show a positive effect of ‘s_pro-self’, which implies that the s_pro-self individuals are more likely than the pro-social type to become more myopic after the workshop. By adding the interactions, model (2) detected a negatively significant effect of the cross term of ‘s_pro-self’ with ‘F’, which is not enough to offset the positive effect of ‘s_pro-self’, unlike model (2) in Table 5.The FD workshop had an impact on the s_pro-self type individuals in the F groups, making them less likely to become more myopic compared to the s_pro-self type in the C groups, but the s_pro-self type individuals were still more likely to become more myopic after the workshop relative to the s_pro-social type. Table 6 reveals the worse adverse effect of workshop on the s_proself type than Table 5. On the other hand, the effect of ‘s_pro-self’ crossed with ‘F’ is not significant in model (4). Thus, Table 5 and Table 6 provide observations that are consistent with the distinct behavior of the pro-self participants shown in Figure 7 and Figure 8 for Q1, while Table 5 weakly does so for Q2.
Next, we examined the dq of individuals in subgroups defined by the participants’ characteristics of age and profession. Table 7 and Table 8 report the OLS regressions on dq1, models (1) to (4) and dq2, models (5) to (8) where the main focus is on the younger and older (=not younger) subgroups, and on the subgroup of city officers and the subgroup of the general public (=not city officers), respectively. We selected the first set of subgroups because we witnessed first-hand during the workshops that the older participants had less difficulty in acting as an imaginary future generation. This impression is shared by many of those who have conducted FD workshops. The second set corresponds to two sets of two-day workshops organized on different days. All of the models in Table 7 and Table 8 corrected standard errors by clustering with groups and gender.
The regression results shown in Table 7 have a dummy variable for the younger participants who are in their 20s and 30s (young). Models (1) and (5) have ‘F’, ‘young’, and their cross term as the regressors. The other models gradually add the other regressors used in Table 5 and Table 6, and their cross terms with ‘young’. In all of the models, the constant term and ‘F’ are not significant. It follows that the participants in the older subgroup did not experience a change in their time perspectives before and after the workshop, regardless of the C or F groups.
Models (1) and (2) have no significant result. However, model (3) captures the positive effect of ‘s_pro-self’ and the negative effect of its cross term with ‘F’, which confirms the same result observed in Table 5 and Table 6; that is, the s_pro-self participants in the C groups tend to become more myopic after the ordinary workshop relative to the s_pro-social participants, but their counterparts in the F groups did not exhibit such a tendency after the FD workshop. It is model (4) that detects a significant effect of ‘young’ in its cross terms. The significantly positive effect of ‘young’ crossed with ‘s_pro-self’ means that it is the younger subgroup among the s_pro-self participants whose time perspectives became shorter. Since our sample size is small, let us mention the negative impact of ‘young’ crossed with ‘s_prosef’ and ‘F’, which offsets the positive effect of ‘young’ crossed with ‘s_pro-self’, though its significance level is 13%. This weakly implies that the younger s_pro-self participants in the F groups did not suffer the adverse impact of the workshops that their counterparts in the C groups experienced.
On the other hand, the Q2 questionnaire captured the tendency among the younger subgroup in the F groups to obtain the longer time perspectives after the FD workshop, compared to the younger subgroup in the C groups relative to their older counterpart, as shown by the negative significant effect of the cross term of ‘young’ with ‘F’ in model (5). This effect remains negative (only weakly) in model (6). The indication of a change in time perspectives among the younger subgroup in the C groups appears in model (8) in its positively-significant effect of ‘young’ crossed with ‘s_pro-self’. Similar to the result in model (4), it was the s_pro-self participants within the younger subgroup that obtained shorter time perspectives after the workshop.
To investigate an effect of the difference in professions among the participants, we ran the OLS regressions on dq1 and dq2 with the same regressors as in Table 7 but replacing ‘young’ with ‘officer’, a dummy variable for the city officers, as shown in Table 8. In all four models (1) to (4) of the left half of Table 8, the constant term is significantly negative, which implies that the base subgroup (=public) and particularly its pro-social type (the baseline in models (2) to (4)), obtained longer time perspectives after the workshop. Model (1) and (2) do not show any difference between the officer and public subgroups, but model (2) detects a positive significant effect of ‘s_pro-self’ indicating that the overall s_pro-self participants tend to become more myopic after the workshop relative to the s_pro-social participants. This effect remains positive in model (3) coupled with a significantly negative effect of its cross term with ‘F’, which again confirms our previous observations on the s_pro-self participants who became more myopic to the lesser extent in the F groups compared to the C groups in Table 5 and Table 6. At the same time, model (3) captures a positively significant effect of ‘officer’ at a less than 11% level, which indicates that the city officers tend to become more myopic relative to the general public after the workshop. Model (4) shows that the positive effect of ‘officer’ in model (4) comes from the positive effect of ‘officer’ crossed with ‘s_pro-self’, which means that such a change in the time perspectives of the officer subgroup is brought about by the s_pro-self type individuals within the officer subgroup. This effect is further divided into the effect of whether an individual is part of the F group or not, as depicted by the negative effect of ‘officer’ crossed with ‘s_pro-self’ and ‘F’. In this case, the total effect of the cross term of ‘officer’ with ‘s_pro-self’ and its cross term with ‘F’ is negative and significant at a less than the 1% level, which implies that the s_pro-self participants among the city officers in the F groups obtained longer time perspectives after the FD workshop relative to the pro-social baseline type than their counterpart in the C groups.
In contrast, the constant term in models (5) to (8) is not significant, such that Q2 questionnaire did not elicit a change in the time perspective of the base subgroup and its baseline pro-social type. Model (7) shows the same tendency for ‘s_pro-self’ and its cross term with ‘F’, as in model (3), but the effect of ‘officer’ is positive only nominally in model (7). Model (8) generates the same result as in model (4), only with a slightly weaker significance level.
As seen in models (4) and (8), the tendency of the s_pro-self individuals to become more myopic after the workshop is more pronounced among the city officers relative to the general public. The s_pro-self individuals among the city officers in the F groups could not only avoid becoming more myopic but also obtained longer time perspectives. Such a tendency is not observed for the general public in either the C or F groups. This difference between the city officers and the public subgroups does not seem to have stemmed from the young vs. old factor examined in Table 7, because the percentage of younger participants was about 50% in both subgroups.
Through the analyses up to this point, there seems to have been some difference in what Q1 and Q2 can detect. In Table 5 and Table 6, the absolute size of the coefficient associated with ‘s_pro-self’ and its cross term with ‘F’ on Q1 is nominally larger than those on Q2. So too are the absolute size of coefficient of ‘s_pro-self’ crossed with ‘young’ and its cross term with ‘F’ in Table 7. The cross-model Wald test detected that the coefficient of ‘s_pro-self’ crossed with ‘officer’ and its interaction with ‘F’ in Table 8 is larger in the Q1 regression (model (4)) than the Q2 regression (model (8)), with significance levels of less than 7% and 0.3%, respectively. This is indicative that the pro-self individuals were more sensitive to the individual choices in Q1 than the non-individual choices in Q2.
Furthermore, there are things that only Q2 revealed. A positive impact of FD on younger participants in model (5) of Table 7 was detected only by Q2. The behavior of the s_other type participants depicted in Figure 8 is captured only by model (4) on Q2 (as reflected in the negative coefficient of ‘s_other’ crossed with ‘F’ in Table 5 and Table 6), and not by Q1 of both tables. This means that those who are neither the s_ pro-social nor s_pro-self type obtained longer time perspective after the workshop compared to their counterparts in the C groups relative to the s_pro-social participants. Since the number of s_other type individuals is very small, we do not discuss the s_other type participants much further, but we will come back to the difference of Q1 and Q2 in the next section.

4. Discussion

In this study, we reported both qualitative and quantitative analyses on the impact of the FD workshop on the outcomes of workshop discussions and participants’ time preferences. We conducted two kinds of public participation workshops in Matsumoto city, Japan for its city hall renovation plan, following the method of the design of experiments: one took the form of an FD workshop, and the other was an ordinary workshop with no element of FD as a control. Before and after the workshops, we elicited participants’ time preferences via binary choice questionnaires, the contents of which were independent of the workshop’s context. In conjunction with the classification of the participants into three SVO types (pro-social, pro-self, and other) based on the standard nine trinary multiple choice questions, we found that: (i) the pro-social type was more likely to have a longer-term perspective than the pro-self type in general; (ii) the ordinary workshop without an FD element had an adverse effect, where the pro-self type became even more myopic; (iii) the FD workshop had a positive impact on the pro-self type and prevented them from becoming more myopic, or even helped them to obtain the longer time perspective when they are restricted to the city officer subgroup; and (iv) the tendency of (ii) and (iii) was more pronounced among younger participants than older participants, and among city officers than general public. Notably, the pro-self type city officers obtained longer time perspectives through the FD workshop; however, (v) no significant FD impact was found for the individual time preference of the pro-social type, although a rough group-wise comparison found their tendency to obtain longer time perspectives after the FD workshops.
These findings are consistent with our qualitative analyses of the discussion outcomes. The policy recommendations proposed by the ordinary workshop demanded the resolution of present-day needs, wants, and complaints. In contrast, the FD workshop induced the participants to focus more on the core functions of the city hall and how different the state of municipal services should be in the future. As a result, the FD discussion generated more constructive and efficient policy proposals by explicitly rejecting the idea of accommodating today’s needs. Matsumoto city adopted most of the recommendations proposed by the F groups into its basic policies for the renovation plan listed in the city’s web site [35].
These observations on the discussion outcomes are consonant with those of existing reports, including [12,13,23]. This study suggests that the change in the attitudes on the time perspective, particularly by those individuals of the pro-self type (e.g., (ii) to (iv)), may have been a driving force behind the distinctive nature of the FD workshop’s outcomes. Notably, we conjecture that the ordinary workshops induced the pro-self individuals, who were mostly concerned with their own stakes, to think more deeply about the current constraints that limit their welfare, such that they gave higher priority to resolving these constraints and ensuring their current welfare.
This paper is the first attempt to investigate if participating in a policy-making public workshop would cause a change in the participants’ mindsets through tracking their individual time preferences. Saito [36] calibrated the average discount rate based on choices observed in experiments using sequential dictator games with FD elements (reported in Kamijo et al. [37]), and showed that the average discount rate was substantially lower than the value estimated from macroeconomic data, such as savings and investments. The discount rate estimated in Saito [36] is based on group decisions in the games without a real timeline, and is game-context dependent. This paper provides more direct evidence of the impact of FD on an individual basis.
The paper demonstrates not only the positive impact of FD workshops but also the negative impact of ordinary workshops on the pro-self participants. The time preference was measured by means of materials that were independent of the workshop context or the physical and material conditions of the participants. This neutrality of the measurement method of the time preferences enabled us to explore the effect of the participants’ characteristics, such as age (i.e., younger versus older) and professions (i.e., city officers or otherwise) in addition to their SVO types. In order to elicit individual time preferences, we used a standard form of questions Q1, which involve only a decision maker’s own monetary payoffs. We newly devised a list of questions Q2, by modifying Q1 to make a decision incorporate not only the decision maker’s welfare but also that of another person. The observations (ii) to (iv) were captured by both regression analyses on Q1 and Q2, but more strongly by Q1. Therefore, we conjecture that the pro-self individuals were more sensitive to individual choices in Q1 than non-individual choices in Q2. On the other hand, we noticed there are things that only Q2 revealed. They include a positive impact of FD on younger participants in model (5) of Table 7, and a positive impact of FD on the s_other type participants in model (4) of Table 5 and Table 6. It is indicative that the response on the time preferences of individuals whose types are classified in-between the pro-social and the pro-self within the SVO spectrum is more sensitive to Q2 than Q1. Since the number of the s_other type individuals is very small in our samples, this issue of the heterogeneity of the participants remains a conjecture.
There are, however, at least three things to be improved: the first is our sample size, and the second is the fact that our questionnaires were hypothetical and the participants did not face real consequences based on their choices. These two points stem from the same root. We usefully collected data from workshops that were part of an actual policy making procedure, not a hypothetical one; however, this advantage also imposed two constraints. One is that we could not ask the city to reorganize the same workshops by calling for new sets of participants to increase our sample size. The other constraint is that it is unacceptable for the city to pay different amounts to different participants. Although experiments in the field of economics, in principle, should be incentivized (that is, the experimenters should pay according to each subject’s choices made during the experiments), there is a debate over the role of such incentives, especially in studies on individual time preference. Thus far, there is little evidence of systematic differences in choice behavior between incentivized and unincentivized experiments (for details, see [26]). The third issue is about possible factors other than the form of the workshops (i.e., unobserved confounders) that may have contributed to the generation of a difference between the C and F groups. Although we formed the C and F groups randomly, following the principle of the design of experiments, not every minute detail of the workshop procedures (other than the workshop forms) was perfectly controlled to be exactly the same between the two groups. The participants may have paid different amounts of attentions to the information from the various materials—such as the white maps and the chronological tables explained in Section 2—across the groups, for example.
To improve future research, we could utilize additional independent measurements, such as risk attitude or cognitive ability, as it is unreasonable to consider time preference as the only measure in the evaluation of the effects of FD or non-FD workshops. We could also improve the design of the workshops. It is plausible that the effectiveness of the FD workshops depends on the specific verbal guidance and reference materials used to induce participants to imagine themselves as future generations. The development of more effective methods and procedures for FD workshops is needed, which will be the topic of our future FD studies. Most importantly, we need to develop a method to design structurally a detailed procedure of workshop discussions, such as the creation of a format which tells how and what information should be conveyed, and how to induce participants to pay enough and balanced attentions to the various information provided, for example. Ideally, the method should be able to generate a controlled outcome irrespective of the facilitators’ skill levels.
As a last remark, let us mention that those who experienced the FD workshop recognized the transformation that occurred within their minds. This recognition stayed with the participants long after their FD participation, as reported by an article in the Asahi newspaper (Nishimura and Inoue [38]) that interviewed one of our participants. The same phenomenon was also reported for other FD workshops (Nakagawa et al. [23]).

Author Contributions

Conceptualization, N.N. and N.I.; methodology, N.N. and N.I. ; formal analysis, N.N., N.I., and H.M.; investigation, N.N., N.I., H.M., and T.M.; data curation, N.N.,N.I., and H.M.; writing—original draft preparation, N.N.; writing—review and editing, N.N., N.I., H.M., and T.M.; visualization, N.N., N.I., and H.M.; supervision, N.N.; project administration, N.N., N.I., and T.M.; funding acquisition, N.N., N.I., H.M., and T.M. All authors have read and agreed to the published version of the manuscript.

Funding

This study was supported by a Grant-in-Aid for Scientific Research (Research Project No. 18K18570) from the Japan Society for the Promotion of Science.

Acknowledgments

The authors thank K. Yamaoki and S. Sumi for their collaboration and inputs, as well as the anonymous referees for their helpful comments. We also deeply thank M. Yakubo, T. Yokouchi, Y. Miyao, Y. Yoshiike, T. Kashima, H. Fushimi, and the other members of the Matsumoto Policy Study Forum for their productive discussions and efforts in operating the FD workshops.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Appendix A.1. Invitation Letter

A call for participants in the city hall renovation workshop was announced on the city’s web site, the local newspaper, and the city flyer, which stated that Matsumoto city would organize a workshop to discuss the city hall renovation plan. Those who applied to the city received a formal invitation letter under the name of the mayor. The letter described the two-day schedule, the location, and what to do in the workshop: “All participants will be divided into groups of five to six people to have a discussion and determine some ideas to help establish the direction of the City Hall renovation plan. We ask you to participate both days as much as possible. In addition to discussions, we ask that you answer a questionnaire prepared by Shinshu University for academic purposes. For this workshop, you will be paid a gratuity of 4000 yen per day.” Along with the invitation letter, each applicant received a set of materials explaining the past reviews and discussions on the needs of the renovation, the current conditions of the city hall, and a tentative operational timetable for the renovation plan to be formed and executed.

Appendix A.2. Questionnaire Instruction

This questionnaire will be used as academic material to analyze choice behavior. You will be asked to make choices in an academically standardized format. This is not a test to measure your ability, since there is no “correct” answer to each question.
All collected answers will be treated as anonymous data identified only by an ID number for statistical analyses. No link between the ID number and your private information is traceable. Making a choice following your own feelings and thinking will be much appreciated.
Please do not hesitate to ask us if you have any questions. You can refuse to answer any question that you might feel is unacceptable. The anonymous data will be kept for a certain period of time and may possibly be open to academic researchers. The data will not be used for other purposes.
Please read all instructions carefully before you start answering the questions. Please answer all the questions in order. Do not come back to the earlier questions after you have answered the later questions. There are many questions. Please treat each question independently, not sequentially or jointly. Lastly and importantly, please do not consult with any other participant or staff.
Part 1:
Please consider the following hypothetical situation, and pick the alternative you prefer.
Suppose you participate in an experiment where you are paired with a person whom you have never met before and will never meet again. You will be asked to choose one of three alternatives of the following sort.
Example
Q1
ABC
Amount you get500 yen500 yen550 yen
Amount the other gets100 yen500 yen300 yen
In this example, you receive 500 yen, and the other person receives 100 yen if you choose alternative A. Likewise, for alternative B and C, your choice determines the amount of money you obtain, as well as the amount that the other person gets. There is no correct answer to this type of question. Please pick one out of three alternatives, A, B, and C as you wish.
Please circle one of the three alternatives for each of the following nine questions.
Question 1ABC
Amount you get450 yen540 yen480 yen
Amount the other gets80 yen280 yen480 yen
Question 2ABC
Amount you get560 yen500 yen500 yen
Amount the other gets300 yen500 yen100 yen
Question 3ABC
Amount you get520 yen520 yen580 yen
Amount the other gets520 yen120 yen320 yen
Question 4ABC
Amount you get500 yen560 yen490 yen
Amount the other gets100 yen300 yen490 yen
Question 5ABC
Amount you get500 yen560 yen490 yen
Amount the other gets100 yen300 yen490 yen
Question 6ABC
Amount you get560 yen500 yen490 yen
Amount the other gets300 yen500 yen90 yen
Question 7ABC
Amount you get500 yen500 yen570 yen
Amount the other gets500 yen100 yen300 yen
Question 8ABC
Amount you get510 yen560 yen510 yen
Amount the other gets510 yen300 yen110 yen
Question 9ABC
Amount you get550 yen500 yen500 yen
Amount the other gets300 yen100 yen500 yen
Part 2:
Please consider the following hypothetical situation involving various monetary rewards at different time intervals and pick the alternative you prefer for each question.
(1)
You are asked to choose between receiving 10,000 yen in one year (option M) or receiving a certain amount today (option W). The amount you can receive today in option W varies across the following ten questions. If you choose W, you will receive nothing in one year. If you choose M, you receive nothing today. Please choose W or M for each of all the ten questions from Q1 to Q10 and circle W or M of your choice in the rightmost column. Please consider each question independently, not sequentially or jointly.
QOption WOption MChoose W or M
1Receive 9800 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
2Receive 9400 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
3Receive 9000 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
4Receive 8600 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
5Receive 8200 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
6Receive 7800 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
7Receive 7400 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
8Receive 7000 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
9Receive 6600 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
10Receive 6200 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
(2)
You are paired with a person whom you have never met before and will never meet again. You are asked to choose between receiving 10,000 yen in one year (option M) and receiving a certain amount today (option W). If you choose option M, you and the other person will receive the same amount in one year. If you choose option W, you and the other person will receive the same amount today. The amount you and the other person will receive today in option W varies across the following ten questions. If you choose W, you and the other person will receive nothing in one year. If you choose M, you and the other person will receive nothing today. Please choose W or M for each of the ten questions from Q1 to Q10 and circle the corresponding mark of your choice in the rightmost column. Please consider each question independently, not sequentially or jointly.
Q Option WOption MChoose W or M
Amount You ReceiveAmount the Other ReceiveAmount You ReceiveAmount the Other Receive
1Today
1 yr. later
9800 yen
0 yen
9800 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
2Today
1 yr. later
9400 yen
0 yen
9400 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
3Today
1 yr. later
9000 yen
0 yen
9000 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
4Today
1 yr. later
8600 yen
0 yen
8600 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
5Today
1 yr. later
8200 yen
0 yen
8200 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
6Today
1 yr. later
7800 yen
0 yen
7800 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
7Today
1 yr. later
7400 yen
0 yen
7400 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
8Today
1 yr. later
7000 yen
0 yen
7000 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
9Today
1 yr. later
6600 yen
0 yen
6600 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
10Today
1 yr. later
6200 yen
0 yen
6200 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
Part 3:
Please circle one alternative for each of the next two questions.
(1)
You are (male or female).
(2)
You are in your (20s, 30s, 40s, 50s, and 60s, or older).

References

  1. Innes, J.; Booher, D. Reframing Public Participation: Strategies for the 21st Century. Plan. Theory Pract. 2004, 5, 419–436. [Google Scholar] [CrossRef]
  2. Baum, H.S. Ethical Behavior is Extraordinary Behavior, it’s the same as all other behavior: A case study in community planning. J. Am. Plann Assoc 1998, 64, 411–423. [Google Scholar] [CrossRef]
  3. Hibbard, M.; Lurie, S. Saving Land but Losing Ground: Challenges to Community Planning in an Era of Participation. J. Plan. Educ. Res. 2000, 20, 87–195. [Google Scholar] [CrossRef]
  4. Owen, R.; Macnaghten, P.; Stilgoe, J. Responsible Research and Innovation: From Science in Society to Science for Society, with Society. Sci. Public Policy 2012, 39, 751–760. [Google Scholar] [CrossRef] [Green Version]
  5. Stilgoe, J.; Owen, R.; Macnaghten, P. Developing a Framework for Responsible Innovation. Res. Policy 2013, 42, 1568–1580. [Google Scholar] [CrossRef] [Green Version]
  6. Fitzgerald, C.; MacCarthy, S.; Carton, F.L.; O’Connor, Y. Citizen Participation in Decision-Making: Can one make a difference? J. Decis. Syst. 2016, 25, 248–260. [Google Scholar] [CrossRef] [Green Version]
  7. Denhardt, R.B.; Denhardt, J.V. The New Public Service: Serving Rather than Steering. Public Adm. Rev. 2000, 60, 549–559. [Google Scholar] [CrossRef]
  8. Kakabadse, A.; Kakabadse, N.K.; Kouzmin, A. Reinventing the Democratic Governance Project through Information technology? A Growing Agenda for Debate. Public Adm. Rev. 2003, 63, 44–60. [Google Scholar] [CrossRef]
  9. Saijo, T. Future Design. In Future Design: Incorporating Preferences of Future Generations for Sustainability; Saijo, T., Ed.; Springer: Berlin/Heidelberg, Germany, 2020. [Google Scholar]
  10. Saijo, T. Future Design. In Future of Economic Design: The Continuing Development of a Field as Envisioned by Its Researchers; Laslier, M., Sanver, Z., Eds.; Springer: Berlin/Heidelberg, Germany, 2019. [Google Scholar]
  11. Saijo, T. Future Design: Bequeathing Sustainable Natural Environments and Sustainable Societies to Future Generations. Sustainability 2020, 12, 6467. [Google Scholar] [CrossRef]
  12. Hara, K. Participatory future design in pursuit of sustainability. Des. Eng. 2016, 51, 297–302. (In Japanese) [Google Scholar]
  13. Hara, K.; Yoshioka, R.; Kuroda, M.; Kurimoto, S.; Saijo, T. Reconciling intergenerational conflicts with imaginary future generations: Evidence from a participatory deliberation practice in a municipality in Japan. Sustain. Sci. 2019, 14, 1605–1619. [Google Scholar] [CrossRef] [Green Version]
  14. Quist, J.; Vergragt, P. Past and future of backcasting: The shift to stakeholder participation and a proposal for a methodological framework. Futures 2006, 38, 1027–1045. [Google Scholar] [CrossRef]
  15. Robinson, J. Future subjunctive: Backcasting as social learning. Futures 2003, 35, 839–856. [Google Scholar] [CrossRef]
  16. Robinson, J.; Burch, S.; Talwar, S.; O’shea, M.; Walsh, M. Envisioning sustainability: Recent progress in the use of participatory backcasting approaches for sustainability research. Technol. Forecast. Soc. Chang. 2011, 78, 756–768. [Google Scholar] [CrossRef]
  17. Peterson, G.D.; Cumming, G.S.; Carpenter, S.R. Scenario planning: A tool for conservation in an uncertain world. Conserv. Biol. 2003, 17, 358–366. [Google Scholar] [CrossRef] [Green Version]
  18. Patel, M.; Kok, K.; Rothman, D.S. Participatory scenario construction in land use 33 analysis: An insight into the experiences created by stakeholder involvement in the Northern Mediterranean. Land Use Policy 2007, 24, 546–561. [Google Scholar] [CrossRef]
  19. Chakraborty, A. Enhancing the role of participatory scenario planning processes: Lessons from reality check exercises. Futures 2011, 43, 387–399. [Google Scholar] [CrossRef]
  20. Duckett, D.G.; McKee, A.J.; Sutherland, L.A.; Kyle, C.; Boden, L.A.; Auty, H.; Bessel, P.R.; McKendrick, I.J. Scenario planning as communicative action: Lessons from participatory exercises for the Scottish livestock industry. Technol. Forecast. Soc. Chang. 2017, 114, 138–151. [Google Scholar] [CrossRef] [Green Version]
  21. Habermas, J. The Theory of Communicative Action: Lifeworld and System: A Critique of Functionalist Reason; Beacon Press: Boston, MA, USA, 1989. [Google Scholar]
  22. Innes, J.; Booher, D. A Turning Point for Planning Theory? Overcoming Dividing Discourses. Plan. Theory 2015, 14, 195–213. [Google Scholar] [CrossRef] [Green Version]
  23. Nakagawa, Y.; Hara, K.; Saijo, T. Becoming sympathetic to the needs of future generations: A phenomenological study of participation in future design workshops. Kochi University of Technology, Social Design Engineering Series, SDES-2017-4. Available online: http://www.souken.kochi-tech.ac.jp/seido/wp/SDES-2017-4.pdf (accessed on 21 June 2020).
  24. Nakagawa, Y. Taking a Future Generation’s Perspective as a Facilitator of Insight Problem-Solving: Sustainable Water Supply Management. Sustainability 2020, 12, 1000. [Google Scholar] [CrossRef] [Green Version]
  25. Van Lange, P.A.M.; Kuhlman, D.M. Social value orientations and impressions of partner’s honesty and intelligence: A test of the might versus morality effect. J. Personal. Soc. Psychol. 1994, 67, 126–141. [Google Scholar] [CrossRef]
  26. Cohen, J.; Ericson, K.M.; Laibson, D.; White, J.M. Measuring Time Preferences. J. Econ. Lit. 2020, 58, 299–347. [Google Scholar] [CrossRef]
  27. Hernuryadin, Y.; Kotani, K.; Kamijo, Y. Time preferences between individuals and groups in the transition from hunter-gatherer to industrial societies. Sustainability 2019, 11, 395. [Google Scholar] [CrossRef] [Green Version]
  28. Nakagawa, Y.; Kotani, K.; Matsumoto, M.; Saijo, T. Intergenerational retrospective viewpoints and individual policy preferences for future: A deliberative experiment for forest management. Futures 2019, 105, 40–53. [Google Scholar] [CrossRef]
  29. Nakagawa, Y.; Arai, R.; Kotani, K.; Nagano, M.; Saijo, T. Intergenerational Retrospective Viewpoint Promotes Financially Sustainable Attitude. Futures 2019, 114, 102454. [Google Scholar] [CrossRef]
  30. Nishimura, N.; Inoue, N.; Musha, T. Deliberation mechanism to let people speak for the future generations. Trends Sci. 2018, 23–26, 20–23. (In Japanese) [Google Scholar] [CrossRef] [Green Version]
  31. Nishimura, N.; Inoue, N.; Musha, T.; Masuhara, H.; Yamaoki, K. Development and Practice of Future Design in Matsumoto, Nagano; Staff Paper Series; Faculty of Economics and Law, Shinshu University: Nagano, Japan, 2020; Volume 1, pp. 1–36. (In Japanese) [Google Scholar]
  32. Messick, D.M.; McClintock, C. Motivational Bases of Choice in Experimental Games. J. Exp. Soc. Psychol. 1968, 4, 1–25. [Google Scholar] [CrossRef]
  33. Andreoni, J.; Kuhn, M.; Sprenger, C. Measuring time preferences: A comparison of experimental methods. J. Econ. Behav. Organ. 2015, 116, 451–464. [Google Scholar] [CrossRef] [Green Version]
  34. Coller, M.; Williams, M. Eliciting Individual Discount Rates. Exp Econ. 1999, 2, 107–127. [Google Scholar] [CrossRef]
  35. Matsumoto City’s Master Plan for City Hall Renovation, February 2020. Available online: https://www.city.matsumoto.nagano.jp/shisei/kikakuseisaku/shintyosya/shinchousha_kihonkeikaku.files/matsumoto-cityhall-kihonkeikaku.pdf (accessed on 10 August 2020). (In Japanese).
  36. Saito, M. Does a Present Generation Change Its Own Preferences as a Consequence of a Discourse with Imaginary Future Generations? Trends Sci. 2018, 23–26, 16–19. (In Japanese) [Google Scholar] [CrossRef]
  37. Kamijo, Y.; Komiya, A.; Mifune, N.; Saijo, T. Negotiating with the future: Incorporating imaginary future generations into negotiations. Sustain. Sci. 2017, 12, 409–420. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  38. Nishimura, N.; Inoue, N. A Thought on Sunday—Invitation to the future. Column Article Interviewed by Ohno, H in Japanese. Asahi Shimbun, 7 April 2019; the front page. [Google Scholar]
Figure 1. (a) Map work of Matsumoto city; (b) chronological table from 1960 to 2060.
Figure 1. (a) Map work of Matsumoto city; (b) chronological table from 1960 to 2060.
Sustainability 12 07796 g001
Figure 2. The second day of FD workshops, 2 February 2018: (a) C group; (b) F group.
Figure 2. The second day of FD workshops, 2 February 2018: (a) C group; (b) F group.
Sustainability 12 07796 g002
Figure 3. Issues extracted from the C groups.
Figure 3. Issues extracted from the C groups.
Sustainability 12 07796 g003
Figure 4. Issues extracted from the F groups.
Figure 4. Issues extracted from the F groups.
Sustainability 12 07796 g004
Figure 5. (a) Distribution of younger (20s and 30s) and older (40s and above) generations; (b) distribution of SVO Typology.
Figure 5. (a) Distribution of younger (20s and 30s) and older (40s and above) generations; (b) distribution of SVO Typology.
Sustainability 12 07796 g005
Figure 6. Average q and SVO Typology: (a) a case of three SVO types; (b) a case of two SVO types.
Figure 6. Average q and SVO Typology: (a) a case of three SVO types; (b) a case of two SVO types.
Sustainability 12 07796 g006
Figure 7. Average q for the C and F groups by SVO type: (a) questionnaire Q1; (b) questionnaire Q2.
Figure 7. Average q for the C and F groups by SVO type: (a) questionnaire Q1; (b) questionnaire Q2.
Sustainability 12 07796 g007
Figure 8. Average dq = [q after] − [q before] for the C and F groups, by SVO type. Note: as q increases, the individual becomes more myopic; thus, dq > 0 means that individuals become more myopic after the workshop, and dq < 0 means the opposite. Columns ‘Q1, C groups’ and ‘Q1, F groups’ show the average dq based on Q1, whereas columns ‘Q2, C groups’ and ‘Q2, F groups’ show the average dq based on Q2 in the C and F groups, respectively.
Figure 8. Average dq = [q after] − [q before] for the C and F groups, by SVO type. Note: as q increases, the individual becomes more myopic; thus, dq > 0 means that individuals become more myopic after the workshop, and dq < 0 means the opposite. Columns ‘Q1, C groups’ and ‘Q1, F groups’ show the average dq based on Q1, whereas columns ‘Q2, C groups’ and ‘Q2, F groups’ show the average dq based on Q2 in the C and F groups, respectively.
Sustainability 12 07796 g008
Table 1. Nine trinary choices for SVO classification.
Table 1. Nine trinary choices for SVO classification.
Q1Q2Q3
ABCABCABC
Amount you get450540480560500500520520580
Amount the other gets80280480300500100520120320
Q4Q5Q6
ABCABCABC
Amount you get500560490560500490500500570
Amount the other gets10030049030050090500100300
Q7Q8Q9
ABCABCABC
Amount you get510560510550500500480490540
Amount the other gets510300110300100500100490300
Units of the amount in each cell are in Japanese yen.
Table 2. Questionnaire Q1, to measure individual time preferences.
Table 2. Questionnaire Q1, to measure individual time preferences.
ItemOption WOption MChoose W or M
1Receive 9800 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
2Receive 9400 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
3Receive 9000 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
4Receive 8600 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
8Receive 7000 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
9Receive 6600 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
10Receive 6200 yen today and
Receive 0 yen one year later
Receive 0 yen today and
Receive 10,000 yen one year later
W/M
Table 3. Questionnaire Q2, to measure the time preferences for a pair.
Table 3. Questionnaire Q2, to measure the time preferences for a pair.
Item Option WOption MChoose W or M
Amount You ReceiveAmount the Other ReceiveAmount You ReceiveAmount the Other Receive
1Today
1 yr. later
9800 yen
0 yen
9800 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
2Today
1 yr. later
9400 yen
0 yen
9400 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
3Today
1 yr. later
9000 yen
0 yen
9000 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
4Today
1 yr. later
8600 yen
0 yen
8600 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
8Today
1 yr. later
7000 yen
0 yen
7000 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
9Today
1 yr. later
6600 yen
0 yen
6600 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
10Today
1 yr. later
6200 yen
0 yen
6200 yen
0 yen
0 yen
10,000 yen
0 yen
10,000 yen
W/M
Table 4. Distribution of the SVO types of individuals who attended both days.
Table 4. Distribution of the SVO types of individuals who attended both days.
(a)(b)
a_pro-sociala_pro-selfa_otherTotal CFTotal
b_pro-social641166s_pro-social333164
b_pro-self28010s_pro-self7512
b_other1124s_other224
total6710380total423880
Table 5. OLS Regressions on dq = [q after] − [q before] for Q1 and Q2.
Table 5. OLS Regressions on dq = [q after] − [q before] for Q1 and Q2.
Model(1)(2)(3)(4)
Q1Q1Q2Q2
F0.0760.3040.1220.411
(0.370)(0.444)(0.591)(0.709)
s_pro-self0.8121.504 *0.7491.425 **
(0.527)(0.753)(0.530)(0.551)
s_pro-self * F −1.504 * −1.461 #
(0.869) (0.984)
s_other0.1280.304−0.0920.625
(1.043)(0.332)(0.510)(0.399)
s_other * F −0.304 −1.411 *
(1.623) (0.830)
Constant−0.179−0.304−0.470−0.625
(0.302)(0.332)(0.365)(0.399)
observations63636563
R-squared0.0280.0520.0170.038
Robust standard errors corrected by clustering are in parenthesis. ** p < 0.05, * p < 0.1, # p < 0.15.
Table 6. Probit regressions for Q1 and Q2.
Table 6. Probit regressions for Q1 and Q2.
Model(1)(2)(3)(4)
Q1Q1Q2Q2
F0.1090.1340.0670.103
(0.091)(0.093)(0.059)(0.070)
s_pro-self0.202 *0.355 **0.150 #0.303 *
(0.121)(0.152)(0.114)(0.193)
s_pro-self * F −0.146 # −0.098
(0.060) (0.036)
s_other0.167−0.216 ***−0.169 ***−0.159 ***
(0.272)(0.058)(0.043)(0.046)
s_other * F 0.881 *** −0.071 ***
(0.040) (0.020)
observations63636565
The dependent variable takes on a value of one when dq = [q after] − [q before] > 0, and 0 otherwise. Robust standard errors corrected by clustering are in parenthesis. *** p < 0.01, ** p < 0.05, * p < 0.1, # p < 0.11.
Table 7. OLS regressions on dq = [q after] − [q before] for Q1 and Q2: young vs. old.
Table 7. OLS regressions on dq = [q after] − [q before] for Q1 and Q2: young vs. old.
Q1Q2
Model(1)(2)(3)(4)(5)(6)(7)(8)
F−0.133−0.129−0.0187−0.1380.8330.8420.9140.867
(0.612)(0.613)(0.636)(0.664)(0.740)(0.753)(0.792)(0.794)
young−0.210−0.400-0.587−0.8720.3000.1240.0162−0.160
(0.481)(0.459)(0.431)(0.648)(0.476)(0.470)(0.467)(0.547)
young * F0.3290.4320.7111.010−1.370 *−1.293 #−1.132−0.952
(0.789)(0.764)(0.782)(0.950)(0.790)(0.818)(0.833)(0.979)
s_pro-self 0.8881.716 **−0.003 0.860 *1.386 **0.107
(0.570)(0.794)(0.350) (0.495)(0.506)(0.548)
s_pro-self * F −1.744 * −1.114
(0.926) (0.908)
s_other 0.062−0.025−0.0716 0.1990.1950.540
(1.062)(1.052)(0.511) (0.543)(0.564)(0.512)
young * s_pro-self 2.303 * 1.593 **
(1.304) (0.708)
young * s_pro-self * F −2.371 # −1.248
(1.388) (1.271)
young * s_other 0.000 0.160
(1.693) (0.547)
young * s_other *F omitted −0.914
(1.158)
Constant0.0670.003−0.0460.072−0.500−0.566−0.599−0.540
(0.439)(0.468)(0.478)(0.511)(0.449)(0.464)(0.471)(0.512)
observations6363636365656565
R-squared0.0030.0340.0650.0870.0400.0600.0690.075
Standard errors corrected by clustering are in parenthesis; ** p < 0.05, * p < 0.1, # p < 0.13.
Table 8. OLS regressions on dq = [q after] − [q before] for Q1 and Q2: city officers vs. general public.
Table 8. OLS regressions on dq = [q after] − [q before] for Q1 and Q2: city officers vs. general public.
Q1Q2
Model(1)(2)(3)(4)(5)(6)(7)(8)
F−0.0220.0640.427−0.0120.2360.3230.6150.307
(0.275)(0.444)(0.519)(0.469)(0.769)(0.834)(0.853)(0.818)
officer0.4860.6850.892 ##0.391−0.0880.0480.174−0.277
(0.473)(0.522)(0.542)(0.584)(0.632)(0.643)(0.652)(0.801)
officer * F0.013−0.082−0.3290.246−0.217−0.338−0.510−0.165
(0.649)(0.785)(0.761)(0.813)(1.144)(1.205)(1.185)(1.289)
s_pro-self 1.000 *1.830 **0.639 0.7331.421 ***0.236
(0.558)(0.725)(0.559) (0.523)(0.483)(0.695)
s_pro-self * F −1.730 ** −1.462 #
(0.828) (0.977)
s_other 0.301−0.0840.480 −0.190−0.195−0.533
(1.195)(0.489)(1.830) (0.468)(0.514)(0.531)
officer * s_pro-self 3.438 *** 2.380 **
(0.767) (0.942)
officer * s_pro-self * F −3.735 *** −1.642 ##
(0.665) (0.997)
officer * s_other −0.403 1.149
(1.904) (0.828)
officer * s_other *F omitted omitted
Constant−0.286 **−0.571 **−0.809 **−0.468 *−0.313−0.484−0.655−0.338
(0.107)(0.242)(0.304)(0.257)(0.401)(0.443)(0.459)(0.488)
observations6363636365656565
R-squared0.0210.0620.0930.1200.0040.0200.0350.039
Standard errors corrected by clustering are in parenthesis, *** p < 0.01, ** p < 0.05, * p < 0.1, # p < 0.15, ## p < 0.11.

Share and Cite

MDPI and ACS Style

Nishimura, N.; Inoue, N.; Masuhara, H.; Musha, T. Impact of Future Design on Workshop Participants’ Time Preferences. Sustainability 2020, 12, 7796. https://doi.org/10.3390/su12187796

AMA Style

Nishimura N, Inoue N, Masuhara H, Musha T. Impact of Future Design on Workshop Participants’ Time Preferences. Sustainability. 2020; 12(18):7796. https://doi.org/10.3390/su12187796

Chicago/Turabian Style

Nishimura, Naoko, Nobuhiro Inoue, Hiroaki Masuhara, and Tadahiko Musha. 2020. "Impact of Future Design on Workshop Participants’ Time Preferences" Sustainability 12, no. 18: 7796. https://doi.org/10.3390/su12187796

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop