Smart Governance Toolbox: A Systematic Literature Review

Abstract

Smart city projects rely upon dynamic and complex multi-stakeholder collaboration. This collaboration can be challenging. In this study, we use an instrumental lens and argue that tools can help public professionals in dealing with smart governance challenges. Building upon smart governance and collaborative governance models, we conceptualize smart governance as a toolbox. Based on our “smart governance toolbox”, we assess the variety of tools available for professionals to initiate and support multi-stakeholder collaboration by reviewing academic and grey literature. This review results in the identification of a broad range of tools that research and practice have developed. However, we also demonstrate that certain parts of the ‘smart governance toolbox’ remain almost empty: there are few tools for assessing the smart collaborative governance context, facilitating the collaborative structure, tackling technology issues, and measuring outcomes of smart city practices. Future design research should focus on developing instruments needed to make the smart governance toolbox complete.

1. Introduction

Cities around the world face complex societal issues, such as securing a healthy and safe living environment, deploying sustainable means of transport, and the transition towards renewable energy [1]. Governments have set up smart city projects, through the use of digital technologies and urban data, to address these complex societal issues [2,3,4,5]. These societal issues exceed the scope of government organizations and require that different stakeholders, such as the government, citizens, research institutes, and businesses, collaborate [6,7,8].

Smart governance city literature emphasizes the importance of multi-stakeholder collaboration for the successful implementation of smart city strategies and practices [3,6,7,9]. Viale Pereira et al. [8] define smart governance as “the ability of governments to make better decisions through the combination of ICT-based tools and collaborative governance” (p. 144). However, different stakeholders might have different interests and expectations, leading to conflicts in collaboration [5,10]. These challenges in collaboration make it difficult for cities to implement digital technologies and capture the promising benefits of smart city practices [6,7,10].

In this study on smart governance, we use an instrumental lens. From this perspective, it is argued that public professionals can use tools to help them address challenges in collaboration [11]. A tools approach simplifies complexity and implies agency or control. There are many different instrumental approaches [12]. One strand of literature focuses on a “tools” or “instruments” as heuristic devices to make sense of the complexity of contemporary policymaking [12,13,14]. To illustrate, Howlett, Mukherjee and Woo [15] distinguish substantive policy tools designed to directly or indirectly affect the production, consumption, and distribution of goods and services in society (e.g., regulation, subsidies, and quotas), and procedural policy instruments that are designed to alter aspects of government’s workings (e.g., freedom of information legislation, reorganization of government agencies and public hearings). From this perspective, collaboration is one of the policy tools that can be chosen [14]. Another strand of literature [16,17] takes a managerial approach and focuses on tools for collaborative action, after the policy decision has been made to collaborate. Building upon this work, we focus in this study on management tools as the building blocks for collaborative action. We define tools as practical methods or instruments, such as checklists, guidelines, and templates, used by public professionals that shape and incentivize collaborative action [11]. A few scholars have focused on collaborative governance as a toolbox [11,16,17], and some have focused on specific governance components such as assessment tools [18,19,20] or participation tools in a smart city context [21]. However, to our knowledge, little is known about smart governance as a toolbox [22].

Our study aims to explore which tools and methods are available for smart city professionals that can help them facilitate smart governance. First, we conceptualize smart governance as a toolbox [11,14,23] by assessing, comparing, and synthesizing theoretical seminal models from smart governance [3,7,24,25] and collaborative governance literature [26,27,28,29]. Our conceptualization leads to a smart governance toolbox consisting of seven categories of tools for smart governance. Second, we conduct a systematic review of academic and grey literature. This results in an overview of 96 different smart governance tools available for smart city professionals. Third, we compare the found tools in the literature with our smart governance toolbox to identify gaps in research and practice. These gaps provide directions for future research and practice, and for multidisciplinary learning.

2. Theoretical Foundations of the Smart Governance Toolbox

2.1. Conceptualizing Smart Governance Tools

Our smart governance toolbox is based on two theoretical foundations: (1) conceptualizing the toolbox as a comprehensive set of methods and instruments for initiating and supporting multi-stakeholder collaboration; (2) understanding of smart governance as multi-actor collaboration in the smart city based on smart governance and collaborative governance literature. This section describes the first foundation.

In public policy and administration literature, there are many different conceptualizations, classifications and lists of instruments [13]. Scholars have developed different taxonomies to classify and categorize tools into smaller mutually exclusive categories to reduce the complexity of instrument choice [23,30,31,32]. The main dimension used to develop these taxonomies is their rationale or purpose [33], that is, the basic mechanism according to which the instruments are supposed to work. This purpose can be generic serving multiple purposes and activities, or specific serving a specific activity or purpose [14]. In this study, we focus on tools serving the purpose of multi-stakeholder collaboration as an activity in a smart city context.

In his seminal work, The Tools of Government, Salamon [23] focuses on public-private partnerships. He describes the increasingly collaborative nature of government action with third parties in terms of the shift from government to governance. At the heart of the governance approach is a “shift in the unit of analysis in policy analysis and public administration from the public agency or the individual public program to the distinctive tools or instrument through which public purposes are pursued” (p. 9). He argues that increasing collaboration requires the identification of policy tools or “tools for government action” that offer government and non-governmental actors options for addressing these problems. He defines tools as instruments or means to address public problems ([23], pp. 1–2). In Salamon’s approach, tools range from regulation, vouchers, and public information to grants and government loans, which can be implemented by a single public organization. However, as Prentice et al. [17] point out, Salamon [23] presents general approaches to policy problems rather than tools that public professionals can use in collaborative managerial practice in order ”to get things done”.

Building on Salamon’s work, Scott and Thomas [11] perceive collaborative governance as a strategic tool, or more specifically as a collection of strategic tools for achieving policy goals. They argue that collaborative governance itself is a toolbox representing a set of tools that public professionals wield for solving public problems. They define collaborative tools as methods or instruments for initiating and supporting collaboration [11]. Such tools include participation incentives, formal agreements, contracts, and other means to shape and incentivize collaborative actions. Scott and Thomas [11] do not provide a taxonomy for collaborative tools but focus on when and why professionals choose to use collaborative governance tools.

Prentice et al. [17] do classify collaborative tools. However, their conception of tools differs from Salamon’s [23] policy tools for governance and Scott and Thomas’s [11] collaborative governance tools. Instead of using a policy orientation, they focus on how managers work together. Based on a survey among public managers, they inductively classify collaborative governance tools among three dimensions: structural tools (e.g., contracts, resources), shared governance arrangement tools (e.g., forming partnerships), and commitment tools (e.g., sharing information and reaching agreement on goals). However, this taxonomy consists of a generic conceptualization of collaborative tools based on the concept of collaboration as an activity but not based on tools that professionals can use in collaborative practice. According to Prentice et al. [17], there is limited understanding in the literature on the assemblage of several types of practical tools that can help public professionals overcome challenges in collaboration. A stronger link between a conceptual and practice perspective on tools is needed [34].

Nilsson et al. [34] classify tools used in practice as a starting point of their research. They distinguish simple tools such as checklists, questionnaires, and process steps, formal tools which entail several analytical steps corresponding to predefined rules, methods, or procedures (such as scenario techniques and risk assessments), and advanced tools that attempt to capture more dynamic and complex aspects of developments through, for example, computer-based modeling or simulation. Nilsson et al. [34] show that simple and formal tools are used in day-to-day practice, but that the use of advanced tools is low, while these types of tools are especially needed to advance more complex relationships, such as collaboration.

Hence, a broad range of taxonomies of government tools exists. In this study, we focus on the smart governance toolbox. We conceptualize the smart governance toolbox, in line with a managerial approach to tools, as all tools, methods, or instruments, public professionals could need in practice for initiating and supporting smart governance. However, to assess what type of tools are available for smart governance, we need to further conceptualize and classify the tools based on their purpose. Whereas Prentice et al. [17] used an inductive approach, we use a deductive approach to further structure the toolbox based on the smart governance components identified in the literature.

2.2. Components of Smart Governance as the Structure for the Toolbox

The second theoretical foundation for the smart governance toolbox is an understanding of smart governance as multi-actor collaboration in the smart city based on two kinds of literature: the literature on smart governance and the literature on collaborative governance. This theoretical foundation is needed to position the various tools in a structured toolbox with clear components.

In smart city literature, governance frameworks are considered important for the realization of smart cities [7]. As Ruhlandt [7] points out, smart governance is enormously complex because it involves a broad range of stakeholder groups. In this light, several scholars have developed smart governance frameworks [3,7,24,25] in which they identify different components that influence multi-stakeholder collaboration.

Table 1. Components of smart governance based on smart city literature.

Components in Literature:	Bolivar & Meijer [3]	Lin [25]	Ruhlandt [7]	Tomor, Meijer, Michels and Geertman [24]	Overview Shared Smart Governance Components
Context		Institutions and economic, social, and environmental challenges ICT e.g., internet penetration, broadband, portals, social media	Local conditions Amount of autonomy	Features of the policy domain Trust within society Political & institutional environment Internet reach and use Socio-spatial characteristics	Features of the policy domain, Local (cultural, political, institutional, and societal) conditions, Socio-spatial characteristics
Strategies	Ideas e.g., vision Actions e.g., legislation, policy	Vision Policies and regulations Organizational transformation	-	-	Ideas/action (legislation, policy) Organizational transformation
Smart governance arrangements	Processes e.g., collaboration and participation, decision-making, E-administration Use of technology Innovation capacity	Collaboration and participation E-administration Decision-making E-administration Smart management Smart Services	Stakeholders, Structures & organizations (political, administrative and external) Processes Roles & responsibilities Technology & data Legislation & policies Exchange arrangements	Government organization Citizen participation Use of technology	Stakeholders, Structure & organizations, Processes Roles & responsibilities, Technology & data, Participation Exchange arrangements
Outcomes	First-order, second-order, third-order outcomes	First-order, second-order, third-order outcomes	Substantive outputs Procedural changes	Learning capacity Citizen participation	Procedural and substantive outcomes Learning capacity

provides an overview of these seminal frameworks and the different components. Based on an assessment of the different smart governance frameworks and their components, it can be observed that the frameworks emphasize the following components: (1) context components (e.g., policy domain, trust in government, the availability of technology, and the social/political/economic/institutional environment), (2) strategic components (e.g., an integrated vision, legislation, and policies), (3) smart governance arrangements (e.g., stakeholders, organization structure, processes, participation, roles, and responsibilities), and (4) outcomes (procedural and substantive outcomes).

To strengthen our understanding of the various components of smart governance, we also analyzed general collaborative governance literature. Attention to collaboration among a broad range of stakeholders is not new in public administration literature. Studies on collaborative governance emphasize processes and structures of public policy decision making and management that engage people constructively across the boundaries of organizations to carry out a public purpose [28]. There are many different collaborative governance frameworks [26,27,28,29].

Table 2. Components of smart governance based on collaborative governance literature.

Components in Literature:	Bryson, Crosby and Stone [27]	Thomson and Perry [29]	Ansell and Gash [26]	Emerson, Nabatchi and Balogh [28]	Overview Shared Collaborative Governance Components
Initial conditions	General Environment	Initial conditions Resource scarcity Resource dependence Complex issues	Incentives for participation Prehistory of cooperation or conflict Power-Resource-Knowledge asymmetries	Drivers such as leadership consequential incentives, interdependence, and uncertainty, policy and legal frameworks	Policy and legal frameworks Prehistory of collaboration Network connectedness Resource conditions Interdependence
Process	Formal and informal, Leadership, Legitimacy Trust, Conflict management, Planning	Governance Administration Organizational autonomy Mutuality Norms of trust and reciprocity	Trust building Commitment to process Face-to-face dialogue Shared understanding Intermediate outcomes Facilitative leadership	Principled engagement (e.g., Discovery, Definition, Deliberation, and Determination) Capacity for joint action (e.g., Leadership, institutional arrangements, resources) Shared motivation (e.g., mutual trust, mutual understanding, and shared commitment)	Leadership Mutual trust Shared understanding Communication Shared commitment Intermediate outcomes Facilitative, connective, leadership
Structure	Formal and informal structures Membership Structural configurations Governance structure		Participatory inclusiveness Clear ground rules Process transparency	Decision making Cross-boundary collaboration	Clear ground rules Process transparency Roles and responsibilities Decision-making Competing institutional logic Competing interests
Outcomes	First, second-order and third-order effects, resilience Inputs, processes, and outputs Results management system Relationship with political and professional constituencies	Achievement of goals Instrumental transactions Creation of new value partnerships Collective action to solve problems	Intermediate outcomes	Actions Impacts Adaptation	Intended goal Resilience Re-evaluation & adaption

provides an overview of the seminal collaborative governance frameworks and the components identified by the authors. Based on a comparison of these frameworks, we identified the following collaborative governance components that these frameworks have in common: (1) initial conditions (e.g., rules and regulation, prehistory of conflict and cooperation, existing relationships or networks, resource conditions, and interdependency), (2) process (e.g., trust, mutual understanding, commitment, leadership, and legitimacy), (3) structure (e.g., formal and informal rules, roles and responsibilities, and decision making) and (4) outcomes (e.g., intended objective, learning, and public value).

By comparing, contrasting, and synthesizing both smart governance and collaborative governance literature, we identified the following patterns:

• First, both strands of the literature stress the legal and policy frameworks and resources (such as budget) as part of the (system) context. According to Ruhlandt [7], unique policy and legal challenges emerge in smart governance such as data access and social justice.
• Second, the smart governance literature identifies stakeholders as a principal component. Stakeholders are those involved in collaborative smart governance in any way, such as government, residents, knowledge institutions, and private companies [6,7]. Collaborative governance literature [26,27,28,29] emphasizes the degree to which these stakeholders might have different or similar interests, aims, and expectations. Therefore, tactics should be used to manage potential conflict, and activities are used for trust-building and shared motivation [27,28].
• Third, the collaborative governance literature emphasizes structure, which refers to elements such as roles and responsibilities and the division of tasks [27]. Similarly, smart governance literature [7] stresses that organizational formation facilitates the interaction among the stakeholders, thereby making a distinction between management, political and external structures.
• Fourth, processes refer to the active involvement of, and participation of and collaboration with stakeholders [7]. In smart governance literature, there is attention to the local conditions of each stakeholder, including administrative cultures [3]. In contrast, collaborative governance literature also identifies so-called “soft” components of the process such as communication, and leadership.
• Fifth, all smart governance frameworks stress exchange arrangements that constitute the contractual or relational arrangements between different stakeholders in the form of, for example, contracts, business models, and tenders [7,24,25]. There is less emphasis on this aspect in collaborative governance literature. However, Emerson et al. [28] indicate the importance of procedural or institutional arrangements, both the informal norms of reciprocity and formal rules and protocols necessary to manage interactions over time of the collaborative network.
• Sixth, the smart governance literature [7,24,25] emphasizes technologies that include data, ICT (information and communication technologies), and technical skills as a central aspect to enhance the provision of services. These frameworks often perceive technology as part of the exchange arrangements. However, because the use of technology is a core component of smart city, we address technology as a separate component [c.f. 7].
• Seventh and last, outcomes are mentioned in both strands of literature in which a distinction is made between substantive outcomes in the form of, for example, economic growth and social inclusion and procedural outcomes that focus on changes in behaviors and procedural changes in implementation in terms of, for example, efficiency and transparency [7,24].

These identified patterns form the building blocks of a smart governance toolbox.

2.3. Smart Governance Toolbox

In this study, we conceptualize the smart governance toolbox as a comprehensive set of methods and instruments for initiating and supporting multi-stakeholder collaboration. We focus on managerial tools; tools that professionals can use in their daily practice in order “to get things done”. Building on the smart governance and collaborative governance literature, our smart governance toolbox consists of seven categories: context, stakeholders, structure, processes, exchange arrangements, technology, and outcomes. These seven categories and a description of these categories can be found in

Table 3. Seven categories of smart governance tools.

Tool Category	Purpose
1. Context tools	Tools aimed at identifying the rules and legislative local context and available resources in which the collaborative smart city practices take place
2. Stakeholder tools	Tools aimed at identifying stakeholders and their interest and commitment in collaborative smart city practices
3. Structure tools	Tools aimed at the organizational formation of collaborative smart city practices such as the identifying and supporting the division of roles and responsibilities in collaboration and gaining management and political support
4. Process tools	Tools aimed at facilitating communication and participation with stakeholders in smart city practices, tools aimed at building a collaborative culture and supporting leadership
5. Exchange arrangement tools	Tools aimed at the constitution of the relationships between different stakeholders in smart city practices, in the form of contracts, tenders, and business models
6. Technology tools	Tools that either facilitate decision making about the use of technology in smart city practices or the development of technical skills in collaborative smart city practices
7. Outcome tools	Tools aimed at measuring the substantive and procedural outcomes of collaborative smart city practices

.

This toolbox is partly in line with the toolbox developed by Prentice et al. [17] who distinguish collaborative structure, shared governance arrangements, and commitment to collaboration tools considering collaborative governance. However, based on our analysis of smart governance and collaborative governance strands of literature we have added several categories such as tools focused on stakeholders, process, technology, and outcome. Based on our toolbox, we will assess in the next section what tools are available for smart city professionals for the seven categories.

3. Methodology

In order to identify which tools are available in relation to our smart governance toolbox, we conducted a systematic review. A systematic review differs from other reviews in that it is replicable and transparent [35,36]. It involves several rigorous steps and thorough reporting on these review steps [35,37,38]. Systematic reviews are especially helpful when information and knowledge is spread out in different disciplines. Smart governance tools are pragmatic instruments that are sometimes presented in academic literature but also very often in research reports, magazine articles, policy documents, and white papers. We, therefore, used a systematic approach to analyze both academic and grey literature to identify which tools are available for smart governance.

3.1. Systematic Analysis of Academic Literature

An academic literature review was conducted based on the ‘Preferred Reporting Items for Systematic Reviews and Meta-Analyses’ (PRISMA) [36]. This methodology was chosen to ensure that articles were chosen systematically and transparently [36]. PRISMA distinguishes between study characteristics and report characteristics to define eligibility criteria.

Study Characteristics

• Type of studies: records should (1) develop/use a tool (2) support collaborative governance in smart city context.
• Topics: the records should either contain the words “smart city” and “collaborative governance”, “collaboration”, “partnership” in combination with words such as “instrument”, “tool”, “frame”, “method” or “lessons-learned”. The following search string was used:

TS = (“Smart City” AND (“Collaborative Governance” OR “Collaboration” Or “Partnership” OR “Cross-sector collaboration” OR “Public-private collaboration” OR “Public-private partnership” OR “triple helix collaboration” OR “quadruple helix collaboration”) AND (“Tool*” OR “Template*” OR “Instrument*” OR “Gadget*” OR “Device*” OR “Apparatus*” OR “Model*” OR “Format*” OR “Frame*” OR “Checklist*” OR” Method*” OR “Lessons learn*” Or “Lessons drawn” OR “Best practices” OR “principle*” OR “guideline*” OR “canvas”)

• Study design: Both empirical and theoretical studies were eligible. Only primary research was included in the scope of this research to prevent tools from being found more often.

Report Characteristics

• Language: Only English-written studies were selected.
• Publication status: academic articles, book chapters, and conference proceedings were included in the study.
• Year of publication: There was no set publication year for the start of the research. All articles published before December 2020 were included in this study.

In total, 446 articles were identified in the academic literature by searching the electronic databases of Scopus and Web of Science. These articles were screened based on their relevance according to the eligibility criteria described above. First, we checked for duplicates. Second, we screened the title, keywords, and abstract. Only (1) primary articles (2) that presented a tool to support collaborative governance were included. An Excel sheet was created to summarize the tools. The screening of all articles led to the inclusion of 54 studies. The process for selecting the records is presented in Figure 1.

3.2. Systematic Analysis of Grey Literature

In line with Hopewell, Clark and Mallet ([39], p. 49], we define grey literature as “that which is produced on all levels of government, academia, business and industry in print and electronic formats, but which is not controlled by commercial publishers.” Examples of grey literature include conference abstracts, research reports, book chapters, unpublished data, dissertations, magazine articles, policy documents, white papers, blogs, and newsletters [39,40]. We did not include grey data (such as tweets, Facebook status updates) or grey information (meeting notes, emails, personal memories) in our study [41].

One way to systematically search for grey literature is to search in online databases, databases that include grey literature, databases focused on specific grey literature types, web search engines, web repositories, and library catalogues [40]. An initial scan showed that this did not lead to fruitful outcomes for our research purpose. Another approach is hand searching and contacting specialists [40,41]. Our hand-searching approach started with selecting countries spread out geographically, based on the scientific literature, and on two international smart city indexes (IMD Smart City Index and IESE Cities in Motion Index) that are smart city frontrunners in governance. Within these countries, we selected frontrunner cities.

Table 4. Overview of cities included in our search.

Continent	City
Asia	Hong Kong
	Seoul
	Singapore
North America	Chicago
	Los Angeles
	New York
Europe	Amsterdam
	Berlin
	Cardiff
	Copenhagen
	Eindhoven
	Glasgow
	Helsinki
	London
	Rotterdam
	The Hague

provides an overview of these cities.

Within each city, we searched for flagship initiatives by looking for examples on the respective smart city websites and analyzed if and how these flagship initiatives related to smart governance and analyzed whether tools were used. Third, we used both Google Scholar and Google with the academic key terms, adding the name of the city and analyzing the first 10 pages found. The last search was conducted in November 2020. Finally, within each of these countries, we approached an expert in the smart city and asked whether they had information about tools. This led to four interviews with experts in the field. Additionally, three experts provided us with information via email.

3.3. Analysis of Documents in Academic and Grey Literature

Based on the results of academic and grey literature we created a list of 118 tools in total: 54 based on academic literature and 64 tools based on grey literature. Four researchers coded the tools found based on a coding scheme (see

Table 5. Coding scheme.

Coding Categories
General Characteristics	Year Academic or grey literature Smart City domain Quadruple helix partners
Type of Tool	Generic or Specific Simple, formal or advanced
Seven categories of smart governance tools	Context: laws and regulations, budget Stakeholders: interests Structure: political and management support, roles and responsibilities Process: participation and communication, leadership, culture Technology and data, technical skills Exchange arrangements: business model, tenders, contracts Outcomes: substantive and procedural

). The coding scheme provides an overview of the categories that were coded by the researchers. During weekly meetings, the researchers discussed the results and issues encountered during the coding. Once the researchers had coded all the tools, two researchers randomly selected the tools, coded the tools independently, and compared them with the initial coding results. The intercoder reliability was calculated at 90% on the subset of the articles.

Next, we scanned the found tools and eliminated some tools because they were not in line with our definition of a tool. For example, one of the tools turned out to be an empty table in a Word file. Of the 118 tools, 96 (38 tools from academic publications and 58 from the grey literature) were kept in the file.

Further, we coded the articles. First, we coded general characteristics of the tool such as the year published, in what domain the tools were used, and which stakeholders can use the tools in smart city collaboration: public sector, academia, industry, and citizens [42]. Second, we analyzed what type of tools we found, and whether these tools were general multipurpose tools or specific tools. In addition, we analyzed whether the tools found were simple, formal, or advanced tools [34]. Finally, we coded the tool in terms of the seven types of tools of the Smart Governance Toolbox as described in Section 2.3.

4. Findings

Following our code book, we first describe our findings in terms of the general characteristics found in the corpus of smart governance tools. Following, we describe the types of tools that we found and lastly which categories of tools we found based on our smart governance toolbox framework.

4.1. Description of the Corpus of Smart Governance tools

In our review, we identified a total of 96 smart governance tools (for an overview, see Supplementary Materials). As Figure 2, shows we can observe an increase in the number of tools over time, with a sharp increase observed as of 2014. This demonstrates a growing interest in the development of tools for smart city governance practices.

The tools were published in journals, conferences, and books of various disciplines: technology and engineering (33%), information management (29%), public policy and administration (22%), business administration (11%), and communication (4%).

Figure 3 shows that the included tools were used in a broad range of smart city domains. Tools were especially used in the domains of social inclusion and welfare, e-government services in general, and environmental resource management. Most tools were not used in a specific domain but had a general character.

Figure 4 demonstrates that the tools found can either be used by the government, or by the government in collaboration with other partners, such as research institutes or companies. Remarkably, we found few tools that can be used by citizens and/or NGOs.

4.2. Types of Tools

We assessed the different types of tools. Figure 5 demonstrates that we especially found what Nilsson et al. [34] call “formal tools”, which consisted of guides, templates, frameworks for a vision or guide, and living labs. Both the academic and the grey literature focus on formal tools. Notably, in academic literature, we especially found frameworks, and in grey literature, we especially found templates. In both academic and grey literature, we also found several advanced tools consisting of digital platforms and apps, dashboards, and other software tools such as simulations. For example, the city of Herrenberg uses an urban digital twin, a sophisticated data model, to aid collaboration with citizens [43]. We only found a few simple tools such as checklists, training, and workshop formats that can be used in the day-to-day activities of smart city professionals.

In addition, we assessed whether the tools are generic [14] or specific. A total of 40% of the tools found were specific tools, in that they served a single purpose in collaboration. In total, 39% of the tools served two purposes, 12% served three, and 2% four. The pattern is similar for both academic and grey literature. To illustrate, the Smart Nation Ambassador initiative in Singapore [44] involves 1,600 volunteers who try to involve a broader community in smart city activities but also help citizens in accessing digital government services. Thus, this initiative is both aimed at participation (as part of the smart governance process category) as well as technical skills (as part of the smart governance technology category). Some tools (7%) served all smart governance categories. These tools consisted of complete toolkits. To illustrate this, New York City uses Civic Services Design Tools and Tactics [45]. This toolkit is aimed at making public services more accessible and more effective for all New Yorkers. It contains a broad range of tools such as instruments for scanning the landscape, mapping out stakeholders and mapping a user journey. Hence, a few tools served all seven categories. Most tools in our review either served one or two purposes.

4.3. Tools Concerning the Seven Categories of Our Smart Governance Toolbox

Additionally, we assessed what tools are available considering the seven categories of our toolbox. As Howlett [14] indicates, it is important that professionals can choose instruments that cover all categories. As Figure 6 shows, the tools are unevenly distributed among the various categories. Most tools focus on the category process, some on the categories exchange arrangements and structure, but few focus on the categories context, stakeholders, technology, or outcomes.

We also analyzed whether there are differences between academic and grey literature tools. Figure 7 shows that in both academic and grey literature, we mostly found tools in relation to the category smart governance process. However, there are also some differences. Remarkably, in the grey literature we mainly found tools related to the categories smart governance structure, technology, and arrangements (business models), whereas in the academic literature, we mainly found tools in relation to the categories smart governance process and stakeholders.

Finally, we analyzed each smart government category more in-depth. For the first category of tools, the smart collaborative context, we found a few tools (see Table 6 for an overview of examples for each category). The found tools focused on tools aimed at facilitating resources such as budgets or are aimed at identifying rules and regulations that apply to smart city projects. To illustrate, the Privacy and Information Protection Principles developed by the city of Portland, are aimed at creating responsible data stewardship in the public sector. These principles include transparency, accountability, data openness, and equitable data management [46].

Remarkably, we found only a few tools for our second category, identifying stakeholders and their interests. Moreover, all tools found for this category consisted of multipurpose tools: next to the component stakeholders, the tools also served purposes related to structure (roles and responsibilities) or process (leadership). To illustrate, He et al. [47] developed a co-benefits approach implementation for public-private partnerships as a win–win strategy for climate change projects in China.

The tools found for the third category, the collaborative smart governance structure, focused on facilitating the roles and responsibilities of the different partners in the project. To illustrate this, in the academic literature, we found a model developed by Mayangsari and Novani [48] in Indonesia that identifies different stakeholder roles: enabler, provider, and utilizer. For each role, they also defined a value proposition. For example, “an enabler” such as a mayor or strategic smart city committee may want to avoid political bottlenecks, balance authority, and enhance the cooperation of stakeholders. Notably, few tools address internal organizational challenges such as gaining management and political support.

We found numerous tools in the fourth category, the smart governance process. These tools focus on facilitating participation and communication. Singapore, for example, uses SCOPE-Smart nation Co-creating with Our People Everywhere [44]. This is a co-creation platform that lets citizens try out new digital government initiatives and products. Citizens can provide instantaneous feedback via the platform. The insights gathered are used to enhance digital government services. Amsterdam uses the Toolkit Citizen Sensing, which provides concrete tools for citizen participation such as a mapping tool that describes how to map issues of concern for citizens in their neighborhood in collaborative workshops [49]. Few tools were found aimed at building a collaborative culture or supporting leadership. Seoul, for example, uses a Smart City living lab as a tool to build a cooperative ecosystem [50]. Regarding leadership, Paskaleva and Cooper [51] developed a Self-Assessment Toolkit that includes the skills and capacities needed by public managers to sustain effective co-production of smart city services. For example, the skill to enhance the social capital of stakeholders in terms of their capacity to contribute to co-design and in terms of their capacity to practice self-assessment.

Regarding the fifth category, the exchange arrangements, tools consist of business models. In academic literature, De Nicolo et al. [52] describe how software enables a gamified process aimed at business innovation. In the business innovation game, team members collaborate to elaborate innovation ideas eventually resulting in a shortlist of business model proposals. In the grey literature, we found the Business Model Canvas, used by Smart City London as part of the Development Impact You toolkit [53]. The canvas contains questions such as “Who will help you?” “What do you need?” and “What will it cost?”. We found a few tools aimed at contracts or tenders. The organization Smart Cities4All [54] developed a checklist for the government consisting of seven steps to adopt an ICT accessibility procurement policy and a checklist for implementation that is aimed at both contracts and tenders.

We found some tools in the sixth category, aimed at supporting decision making about the developments and use of technology. Together with the Utrecht Data School, the city of Utrecht developed the Data Ethics Decision Aid (DEDA) [55,56]. This tool helps data analysts, project managers, and policymakers to reflect on and recognize ethical issues in data projects, data management, and data policies. Hong Kong is stimulating training programs in schools and among professionals regarding technical skills [57]. Hudson et al. [58] examined a MOOC as a tool to facilitate attitudinal learning and participation in smart cities. They demonstrate that participants in the study reported high levels of attitudinal learning; however, only a small number of participants were actively seeking to engage in smart city activities within their city.

Remarkably, we found only a limited number of tools for the seventh category, which focused on measuring the outcomes of smart city practices. The tools that we did find focus on methods to measure procedural outcomes. For example, the software tool CITYkeys facilitates smart city performance measurement structured according to the categories of people, planet, prosperity, governance, and propagation [59]. The Impact Path [60] is used by the City of Rotterdam and provides a tool for demonstrating social impact. The tool allows organizations to assess whether they accomplish their mission and what value they create for society.

Table 6. Overview findings smart governance categories.

Smart Governance Categories	Number of Tools Found	Example
1. Context
Rules and regulations	3	Privacy and Information Protection Principles [46]
Budget	2	Business Model Canvas [53]
2. Stakeholders
Interests	5	Pathways to co-benefits approach among multi-scale stakeholders [47]
3. Structure
Political support	6	Problematizing data-driven urban practices: Insights from five Dutch ‘smart cities [55]
Management Support	3	Smart Cardiff: Cardiff Council’s Smart City Roadmap [61]
Roles and responsibilities	15	Stakeholders co-creation roles [48]
4. Process
Participation and communication	43	Toolkit Citizen Sensing [49]
Culture	17	Smart City Living Labs in Seoul [50]
Leadership	12	Innovations in Co-Created Smart City Services [51]
5.Exchange Arrangements
Business models	23	Business Model Canvas [53]
Contracts	6	Guide to Adopting an ICT Accessibility Procurement Policy [54]
Tenders	1	Guide to Implementing Priority ICT accessibility standards [62]
6. Technology
Technology and data	6	Data Ethics Decision Aid (DEDA) [56]
Technical skills	5	MOOC as a tool to facilitate attitudinal learning [58]
7. Outcomes
Substantive	1	The impact path [60]
Procedural	4	CITYkeys [59]

Table 6. Overview findings smart governance categories.

Smart Governance Categories	Number of Tools Found	Example
1. Context
Rules and regulations	3	Privacy and Information Protection Principles [46]
Budget	2	Business Model Canvas [53]
2. Stakeholders
Interests	5	Pathways to co-benefits approach among multi-scale stakeholders [47]
3. Structure
Political support	6	Problematizing data-driven urban practices: Insights from five Dutch ‘smart cities [55]
Management Support	3	Smart Cardiff: Cardiff Council’s Smart City Roadmap [61]
Roles and responsibilities	15	Stakeholders co-creation roles [48]
4. Process
Participation and communication	43	Toolkit Citizen Sensing [49]
Culture	17	Smart City Living Labs in Seoul [50]
Leadership	12	Innovations in Co-Created Smart City Services [51]
5.Exchange Arrangements
Business models	23	Business Model Canvas [53]
Contracts	6	Guide to Adopting an ICT Accessibility Procurement Policy [54]
Tenders	1	Guide to Implementing Priority ICT accessibility standards [62]
6. Technology
Technology and data	6	Data Ethics Decision Aid (DEDA) [56]
Technical skills	5	MOOC as a tool to facilitate attitudinal learning [58]
7. Outcomes
Substantive	1	The impact path [60]
Procedural	4	CITYkeys [59]

5. Discussion

Based on an assessment of the academic and grey literature review, we can make several observations. First, in line with Sharifi [19] we find a considerable increase in the number of tools for smart governance. Remarkably, we only found a few tools to be used by citizens and NGO’s. In the literature, there is increasing attention, for example, toward citizen science in a smart city context, where citizens use sensors and gather data [63]. From this perspective, citizens become partners in collaboration [63]. Our study demonstrates that more tools need to become available to stimulate these collaborative processes. Furthermore, the found tools mainly consist of formal and advanced tools and we found a few simple tools. Yet, according to Nilsson et al. [34] simple tools are especially used in practice due to capacity constraints and lack of time, whereas the use of advanced tools is low. Hence, there might be a gap between the tools available for smart governance practices and those needed by smart city professionals. Further research is needed to address this gap and to assess what tools are used in practice [22].

Second, conceptualizing smart governance as a toolbox implies that tools in combination can serve to shape and guide smart collaborative collaboration [11]. There need to be sufficient tools available for smart city professionals for each category in the smart governance toolbox to achieve a better, more efficient, or socially optimal collaboration. Our findings demonstrate that there are indeed tools available for each category. However, these tools are unevenly distributed and concentrate on three categories: tools related to the process (participation and communication), exchange arrangements (business models), and structure (roles and responsibilities). In line with earlier work [18,19,20], we find that there are few tools available related to stakeholders and outcomes. Furthermore, we also find that other parts of the toolbox (context, structure and technology) are almost empty. Gil-Garcia and Sayogo [4] demonstrate that roles and responsibilities, technology and budget are important for successful collaboration and information sharing. It is therefore problematic that except for roles and responsibilities, we found few tools for the other components that contribute to successful collaboration. Hence, more research is needed in these categories and tools need to be designed that can help public managers in addressing smart governance collaboration challenges.

Third, the lack of tools available for identifying the context, the interests of stakeholders, and supporting structure for facilitating management and political support might affect smart city practices over time. Gil-Garcia and Sayoga [4] demonstrate that political support is less important for collaboration. However, they do not focus on the long-term effect. In contrast, Van Lunenberg et al. [64] demonstrate the importance of the relationship between institutional factors (such as resources and network of stakeholders), structure (the way stakeholders organize their activities), pathway (in terms of mobilizing powerful “patrons” such as managers or politicians), and scaling outcomes. These components could partly explain why smart city projects often remain experimental and have difficulties scaling [5,64].

Fourth, assessing both the academic and the grey literature showed that there are also some differences in focus. Remarkably, the grey literature focuses on the smart governance structure, technology, and arrangements (business models), while the academic literature focuses on the smart governance process and stakeholders. Furthermore, there is a lack of attention in the academic literature for methods or instruments related to gaining management and political support. This implies a gap of focus between research and practice and requires more collaboration and knowledge exchange between research and practice.

Finally, this study also has some limitations. Our systematic review focused on the smart city domain. Due to this approach, we may have missed tools in other domains that can be applied to smart city practices. Furthermore, we found a broad range of tools in the grey literature in addition to the tools found in the academic literature. However, by focusing on frontrunners and flagship initiatives in our search, we did not cover all tools that are available to practitioners. Another limitation of our approach is that within our study, we did not measure the actual effectiveness or applicability of the tools. Further research should assess whether the existing tools are used and appreciated by public professionals in practice and in different socio-economic contexts [22,65].

6. Conclusions

This study aimed to assess what types of tools for smart governance are available for smart city professionals. Smart city literature emphasizes the importance of multi-stakeholder collaboration in smart city practices [3,7,66]. However, these collaborations can be challenging. We argued that tools could help public professionals in dealing with smart governance challenges. In this study, we focused on managerial tools, tools that can be used in the daily practice of city professionals in order to work efficiently [16,17]. We contributed to the smart city literature by conceptualizing smart governance as a collaborative toolbox and structured the toolbox based on seven types of tools of smart governance. We used our toolbox to assess what tools are available in research and practice. Our findings demonstrate that future research should focus on better understanding smart governance structure, in particular gaining management and political support, and (substantive) outcomes. Developing tools for these components can help practitioners in dealing with smart governance challenges. Based on these findings, we can draw several conclusions.

First, our conceptualization of smart governance as a toolbox provides a basis for international collaboration, knowledge exchange and multidisciplinary learning. Tools are knowledge products that, with caution and precision, provide a useful and important starting point for international learning about approaches to smart governance. After all, the strength of the toolbox approach is its parsimony and flexibility: it can be applied to different settings, and to simple and more complex forms of collaboration [17]. Second, the toolbox perspective provides an important basis for connecting different disciplinary perspectives on smart governance. The study highlights how knowledge from engineering, information management, business administration, and public administration is used to develop tools. The specific focus on tools can help to integrate insights from various fields.

Finally, our study highlights that much interesting work is not only presented in academic publications but in grey literature. This study highlights that grey literature provides a series of interesting tools. We, therefore, argue for a better connection between the practical development of tools and academic research. Academic researchers need to play a stronger role in rigorous testing of the variety of smart governance tools that are presented in the grey literature.