Next Article in Journal
Interplay of Workplace Sustainability, Sustainable Work Performance, Optimism, and Resilience: The Moderating Role of Green Creativity in Luxury Hotels
Next Article in Special Issue
Ways towards Transformation—Conceptual Approaches and Challenges
Previous Article in Journal
Chromium Poisoning in Buffaloes in the Vicinity of Contaminated Pastureland, Punjab, Pakistan
Previous Article in Special Issue
Assessing the Impact of Technology Assessment, Responsible Research and Innovation and Sustainability Research: Towards a Common Methodological Approach
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Reflexive Sustainable Technology Labs: Combining Real-World Labs, Technology Assessment, and Responsible Research and Innovation

1
Karlsruhe Institute of Technology (KIT), 76131 Karlsruhe, Germany
2
Vechta Institute of Sustainability Transformation in Rural Areas (VISTRA), University of Vechta, 49377 Vechta, Germany
3
KIT Center Humans and Technology, Karlsruhe Institute of Technology (KIT), 76133 Karlsruhe, Germany
4
Institute for Technology Assessment and Systems Analysis, Karlsruhe Institute of Technology (KIT), 76131 Karlsruhe, Germany
*
Author to whom correspondence should be addressed.
Sustainability 2022, 14(22), 15094; https://doi.org/10.3390/su142215094
Submission received: 2 March 2022 / Revised: 1 November 2022 / Accepted: 9 November 2022 / Published: 15 November 2022

Abstract

:
There is an increasing call for science to contribute more to real-world solutions for a sustainability transformation. At the same time, the scientific landscape encompasses different concepts, which are characterized as practice-oriented, addressing social challenges and wicked problems, which thus offer potential to contribute to a transformation, notably: technology assessment (TA), responsible research and innovation (RRI), and real-world labs (RwLs). This article deals with the question of how these different concepts can be better integrated to contribute to a transformation. With this goal in mind, we analyze the three concepts, show their core characteristics, and identify their common key similarities. TA and RRI have rarely been connected to transformative sustainability research in general or RwLs in particular. Based on the key similarities and respective strengths of the three concepts, we suggest four ways to integrate the approaches, including the novel endeavor of a Reflexive Sustainable Technology Lab as the highest level of the concepts’ synthesis.

1. Introduction

In accordance with the heightened political importance of sustainable development, sustainability science (also referred to as transformation research, see, e.g., [1]) has emerged as a novel discipline. It differs from traditional academic disciplines with its focus on solving real-world problems. Clark summarizes in this regard: “(…) sustainability science is a field defined by the problems it addresses rather than by the disciplines it employs” [2] (p. 1737). This is often combined with the aspiration to initiate practical change, typically referred to as ‘transformative science’ [3].
In this article, we focused on the real-world lab (RwL) as a key concept in transformative sustainability science [3] (see Section 2.1). Historically, both the term RwLs and its first implementations originate from transformative sustainability research [4]. From the very beginning, RwLs—with their direct links to urban and regional development—have been designed as a framework for societal research, transformation, and learning processes (cf. [5,6]). A RwL is a transdisciplinary research and innovation environment, located in specific geographic or thematic societal contexts, to facilitate experiments for a transition towards a culture of sustainability. It serves as a platform to initiate and perpetuate learning processes and transformative knowledge (cf. [7]).
Beside RwLs, there are other research strands which are practice-oriented, address social challenges and wicked problems, and aim to contribute to a better future. In this article, we relate to two such research strands: technology assessment (TA) and responsible research and innovation (RRI). TA emerged in the 1970s against the background of ever-increasing technological risks, and aimed to evaluate the future impact of novel technologies. Another more recent research field is RRI. It aims at responsible technology development, taking into account social and ecological needs and goals. Both TA and RRI have, so far, rarely been connected to transformative sustainability research, yet, as exceptions [8] show, they offer great potential for enriching transformative sustainability science in general and RwL approaches in particular.
In this article, we follow this promising line of research to show that while transformative research options (RwLs, TA, and RRI, respectively) have different origins, genesis, and immediate objectives, they all contribute to sustainability transformation and a sustainable future. By looking at their similarities—rather than emphasizing their differences—we connect the different research strands in order to integrate their knowledge for steering sustainability transformation.
For this endeavor, we build on the assumption that all three concepts—TA, RRI, and RwLs—are rooted in a common understanding of science engaged in and for society. Based on this common ground, we identified key similarities between all three concepts that provide the foundation for building bridges between them. Pre-empting the findings, six key similarities are mapped out in the following section to build joint concepts using the strengths of all three research strands: 1. Scientific approach; 2. Normativity and Responsibility; 3. Supporting practice; 4. Participation; 5. Reference to the future; 6. Learning.
This short list already illustrates that the heterogeneity of the field of sustainability transformation science and its neighboring research strands of TA and RRI can be seen as an asset rather than a problem. When combined, they offer a wide variety of analytic viewpoints and methods for sustainability transformation. We follow this perspective and aim at building bridges between these concepts.
To do so, we introduce RwLs, TA, and RRI as our key concepts in Section 2. Based on this analysis, we outline conceptual bridges between the three concepts in Section 3 and Section 4. These conceptual bridges provide the foundations for the novel research concept of a “Reflexive Sustainable Technology Lab” introduced in Section 5.

2. RwLs, TA, and RRI: Core Characteristics

In this section, we introduce RwLs, TA, and RRI and their respective key characteristics, starting with RwLs as our foundational concept to which the other two disciplines will be connected.

2.1. Real-World Labs

RwLs are part of a global “experimental turn” that is increasingly creating action-oriented and involve experimental research modes, especially in sustainability research (cf. [9]). The transformative approach of RwLs has expanded transdisciplinary research not only to include the production of knowledge for sustainable development, but also to develop practical impulses and contributions for sustainable development through research. Efforts to democratize science and direct it toward the common good are another ideational origin. Following a democratic understanding of society, science as a knowledge process is tied more closely to the actual legitimized subject of knowledge—to the sovereign, i.e., the population. The involvement of citizens, (local) government, and civil society groups in RwLs broadens the societal base of those people who participate in science and knowledge production, paving a direct way for the transfer of research results to society at large. Regarding our goal of connecting the three fields, it is furthermore notable that RwLs are open to all kinds of transition processes, e.g., including social innovation in contrast to other lab types focusing more on technological innovations [10,11,12]. Box 1 briefly presents a RwL in operation.
Based on the above understanding, the core characteristics of RwLs have evolved [7,13]; cf. [14,15,16]:
  • Research orientation: RwLs serve as scientific undertakings for sustainability and transformation research with the goal to gain and provide transformative knowledge.
  • Transformativity: RwLs are hybrid ventures, due to the goal to create scientific findings as well as significant changes in societal practice. They are supposed to make a difference, to walk the talk and be transformative.
  • Normativity: RwLs are oriented towards the guiding principle of sustainable development and explicate their normative assumptions, basis, and goals.
  • Transdisciplinarity: The core of RwLs are multiple co-operations of scientists and practice actors. The research starts from actual “real-world” problems, and not only theoretically derived ones.
  • Involvement of civil-society actors: There is a continuous focus on participation and mutual integration of civil society in the whole experimentation process (agenda-setting, co-design, co-operation, and co-evaluation).
  • Serving as models: RwLs are unique and bound to their specific context, but aim for transfer and upscaling of their insights, results, innovations, and methods to different and wider contexts.
  • Long-term perspective: RwLs are ideally established as long-term institutions with a time horizon of (several) decades. Therefore, they can design, conduct, and evaluate transformation processes which could not take place in the regular research project timeline of three years.
  • Laboratory character: The main methods are real-world experiments such as transformative approaches in specific geographic or thematic societal contexts that are tested for a defined period of time. RwLs constitute a transdisciplinary infrastructure to conduct these experiments.
  • Education: RwLs provide multiple learning processes on an individual level (insights, new ways of acting, and, especially for scientists, unusual but potentially fruitful ways of working) and on a systemic level (self-reflection and evaluation processes).
Box 1. Example of an operating Real-world Lab
“Quartier Zukunft—Labor Stadt” [English: “District Future—Urban Lab”] is a Real-world Lab of the Karlsruhe Institute of Technology (KIT) centered in Karlsruhe’s Oststadt, as a mature, typically European urban neighborhood. Quartier Zukunft functions as a creative platform, a space for experimentation, research and innovation and a stepping stone for a variety of civil society projects with a sustainability aspect.
Science and society are jointly seeking, testing, and researching how a culture of sustainability can emerge and be lived in an urban district. This aim is to increase people’s scope for action and capacity for transformation by testing and pioneering ideas and creative strategies for the future. This includes technical, organizational, and social innovations.
While many people think of sustainability primarily in terms of renunciation or efficiency, Quartier Zukunft is about creating new qualities of life for the people in the neighborhood. Sustainable development as a normative guiding principle is translated by the Integrative Concept of Sustainable Development (ICoS) [17], a scientific and ethical concept of the Helmholtz Association developed under the lead of ITAS.
https://www.quartierzukunft.de/en/ (accessed on 14 November 2022)

2.2. Technology Assessment

TA can best be understood as a critical reflection and scientific discourse which started in policy advice and still draws strongly on concepts of how to analyze complex socio-technical (and ethical, ecological, legal, etc.) systems or cases in a way that supports decision-making. Another key feature of TA is its focus on anticipation of future impacts. While TA has developed various strands and styles [18], including explicitly “participatory TA”, the focus on establishing a clear methodology has always been key to achieving scientific quality in a problem-oriented setting that includes the respective case context [19]. On a European level, TA was one of the discourses that lead to the broader concept of RRI. Core characteristics of the field include [18,20,21]:
  • Future orientation: TA evaluates the impact of technologies for future development.
  • Focus on technology: It is often inspired by new technologies and analyzes these developments from a socio-technical systems perspective.
  • Problem orientation: The choice of the analyzed technology is often problem-oriented (e.g., nuclear energy, AI).
  • Impact assessment: It uses a broad methodology for analyzing future impacts (e.g., Life-Cycle Assessment).
  • Scientific criteria: TA is an established (interdisciplinary) research field with high theoretical and methodological standards for impact assessment.
  • Policy advice: TA aims to support decision-making, especially in politics, but also more recently (fostered by RRI concepts) among further stakeholders such as companies.
Regarding our goal of connecting approaches, the role of technology in the research field of TA, which contrasts with the role of technology in real-world labs, needs critical reflection. Technologies improving, for example, ecological sustainability (e.g., biofuels or solar panels), might have a negative impact of other sustainability dimensions (economic, ecological, and social; see [22]). In their recent paper, Sovacool et al. [23] examined to what extent four different low-carbon technologies contribute or harm social equality. For example, they highlighted the demographic inequity of solar panels with adoption being mandated by gender roles and financial status (e.g., house ownership as precondition for household solar panels). Even if a technology is at a niche implication benefiting the social, economic, and ecological dimension of sustainability, this might change when being scaled up [24]. A good example are biofuels [25], e.g., first-generation biofuels, if scaled-up, increase monoculture, and thus negatively impact biodiversity as well as harming food security. Second-generation biofuels using waste could solve this issue but may counteract the efforts for waste reduction.
This remark on the sustainability of technology innovation is not made to state that technology is in general not a valuable asset for sustainability transitions but rather to point out that while acknowledging the asset of technology development we should critically monitor its overall sustainability impact, taking all three dimensions of sustainability (economic, social, ecological) into account rather than blindly following technology hypes (regarding technology hype cycles see e.g., [26,27,28]). TA research has a longstanding tradition in this regard and is therefore a crucial element for the new concept.

2.3. Responsible Research and Innovation

RRI can be understood as a policy concept, mainly supported by the European Commission (EC) over recent years [29]. In this EC context, certain “keys” of RRI (Gender, Ethics, Open Access, Engagement, and Science Education) have been identified, which each refer to discussions, measures, and activities already taking place in various areas [30,31], representing the main areas of focus for RRI, especially in European Union-funded projects. However, on a national level, RRI as a term is rarely explicitly used, even if many of the keys play an important role in science policy (for instance in Germany [32]). Other than at the EC level, the more academic discussion of RRI provides four “dimensions” that help frame the concept in a more procedural way [31,32,33].
  • Anticipation: RRI aims at an improvement of forecasting, resilience through systematic thinking for socially robust research and innovation, while recognizing the complexities and uncertainties of science and society’s co-evolution.
  • Reflexivity: RRI acknowledges the need for institutional reflexivity in governance, by being aware of activities, commitments, framings, and assumptions, as well as the limits of knowledge, values systems, and theories that shape science and innovation. It re-thinks concepts, assumptions, and demands of openness.
  • Inclusion: RRI implies the integration of stakeholders and the wider public regarding issues of science and innovation; this is still an experimental process.
  • Responsiveness: RRI involves the capacity to change the shape or direction of innovation processes in response to stakeholder and public values and circumstances by responding to new knowledge as it emerges, as well as emerging perspectives, views, and norms.
In line with TA, RRI studies often focus on technological innovation, e.g., in industry [30], rather than on business models or social innovation. However, they also offer a critical reflexivity in this regard, which could benefit RwLs aiming at involving technology innovations.

3. Key Similarities and Specifics of TA, RRI, and RwL Research

In chapter 2, we have identified the core characteristics of the concepts of RwLs, TA, and RRI based on central conceptual literature of each discourse. By comparing these characteristics (s. Figure 1), we identified six key similarities between all three concepts: science for society, normativity, supporting practice, participation, reference to the future, and learning. These key similarities can be used to build bridges, guiding the combination and synthesis of the three concepts. Until now, only TA and RRI have been linked systematically [34]; the links to RwL research have not yet been mapped out.
Therefore, we shall outline the six key similarities between the three concepts in more detail. We also highlighted the specific ways in how each of the concepts address the key similarities. This will serve as a basis for integrating their respective strengths.

3.1. Science for Society

All three approaches are also scientific endeavors facing the challenge of bringing science into broader societal processes. TA does research, using scientific methods and knowledge for decision support and policy advice [21], RRI is a research and development mode, combining technology development with societal research on its implications [34], and RwL research creates an environment for experimentation, transdisciplinary, and transformative research. In all three approaches, scientific criteria serve as a way to achieve knowledge that is more reliable, to avoid biases, to gain legitimacy, and to link learning processes from a specific project to a broader (scientific) discourse.
An especially relevant research practice is impact measurement via indicators. In RRI, effectiveness is usually measured with indicators relating to all four dimensions listed above, also to achieve responsiveness and adaptation. TA has a long-standing tradition of using indicator systems, but these are rarely used to measure the impact of TA itself (see Ladikas et al. in this volume). For RwL research, the challenge will be to develop an indicator system that covers both the level of a single experiment and the level of the lab as a whole.

3.2. Normativity and Responsibility

All three concepts either explicitly refer to a normative model (sustainability: RwLs; responsibility: RRI) or relate empirical and simulated data to normative agendas (TA). In TA, based on its tradition of scientific policy advice, normativity is handled carefully—e.g., by including different normative perspectives in scenarios, by referring transparently to broadly accepted concepts such as sustainability, or by guaranteeing transparency in the scientific process [35], and other contributions in that volume]. In RRI, responsibility is not necessarily linked to an explicit ethical framework. Instead, responsibility is integrated by the participation of various stakeholders and by processes designed for reflexivity and the iterative correction of plans. RwL research refers directly to concepts of sustainability [7], both as a scientific point of reference and as a development strategy. As science for society, all concepts (implicitly or explicitly) aim at basic normative ideas such as the precautionary principle [36] and the common good [37].

3.3. Supporting Practice

All three approaches are both problem- and solution-oriented. Depending on the field of practice, the way they offer support differs between decision processes (e.g., advice, assessment of alternatives, participatory decision-making) in TA [38], transformation processes starting in niches (e.g., experiments, transfer, and generalization of best practices) in RwLs [39], or innovation (multi-perspective feedback loops, prototyping, and testing) in RRI. In each concept, the support of practice is closely related to overarching societal goals as the normative framework. RRI, TA, and RwLs all favor a process-oriented work mode. In all three traditions, process criteria play a major and similar role, no matter whether the overall process is framed as one of decision, innovation, or transformation.

3.4. Participation

All three approaches have a methodological tradition of including different actors, either to include specific bodies of knowledge (TA, RRI, RwLs), to create a process that is acceptable in a democratic society (TA, RRI), to make sure that the results are ready for implementation (RRI, RwLs), or to contribute to sustainability transformation directly in scientific public-private partnerships (RwLs). While in TA, participation is only for producing knowledge, in RRI different actors are directly involved in innovation, and in RwLs they are involved in transformation processes. Their addressees—the participants—are numerous: key decision-makers, agents, and makers who realize innovative ideas, scientific communities, and a wide public audience. They bring in perspectives from politics, administration, economy, arts and culture, and civil society.
In TA, participation is often framed as a distinct type of project (“participatory TA”) [19]. In RwL research, both researcher and practice partners are heading for societal transformation and working in close cooperation.

3.5. Reference to the Future

One feature that all these traditions share, which not only causes practical difficulties but triggers fundamental epistemological questions, is their inherent reference to the future [40]. This is often harnessed in methodology (e.g., roadmaps, Delphi-surveys, or scenarios) to navigate between anticipation, planning, and visioning.
The way futures are addressed is not predictive but stays abstract in order to provide arguments for decision processes. In RRI, this is done mainly for technical or socio-technical innovation or its framework (e. g. changes in regulation) [41], and in TA as a resource for political decision-making. RwLs, on the other hand, may use narratives or conceptual ideas about the future, especially drawn from the discourse on sustainable development, but with an aim to contribute both knowledge and best practice examples to long-term sustainability transformations. A closer integration of futures thinking and (practical) transformation promises high potential for integrating TA, RRI, and RwL research.

3.6. Learning

RwLs, RRI, and TA are organizing societal and scientific learning, but have different notions of learning, which in many respects stay implicit. RwLs have been described as learning environments for various and interlinked learning processes [42]. RRI is conceptualized as an iterative innovation process with the need to include various stakeholders, amongst other reasons, to facilitate joint and mutual learning. In TA, learning concepts are either discussed in relation to higher education, or as the transfer and integration of knowledge of different types into a transdisciplinary process, e.g., to develop technology which is more reasonable. A perspective of “transformative learning” [43] can serve as a joint theoretical basis to link these discourses.

4. Combining RwLs, TA, and RRI—Synthesis and Potential

All three concepts are based on a common ground, have key similarities, and aim at shaping the future in a wiser, more reasonable way. Regarding their specifics, each of them can broaden the horizon of eco-socio-technological innovation in general, and sustainability transformation in particular. Looking at their core strengths, RRI focuses on anticipation, participation, and responsiveness; TA on technology, foresight, and impact assessment; and RwLs on sustainability, experimenting, collaboration, and building inter-institutions at the interface of science and society. Each of them can and does contribute to sustainability transformation. However, combining them might create an opportunity to establish synergies between their specifics, strengths, complementary perspectives, and leverage points. Mutual learning and the fruitful combination—the triangle of RwLs, TA, and RRI—offers the chance of a next step in a rational approach to sustainability transformation.
While TA and RRI are already linked in specific projects or methods [44], virtually no references have been made so far between TA and RwL research, or RRI and RwL research. That is why we derived options on how to combine TA and RRI with RwL research. The following points in this chapter are conceptual suggestions, showing how combinations between the three approaches can be implemented in a range of intensities, reaching from short-term one-directional links to an institutional synthesis.
  • The first step in bringing TA, RRI, and RwLs together is looking from the single perspective towards the other concepts, e.g., looking from TA or RRI at RwLs as research objects and case studies. At this stage, the concepts remain disparate, and insights can be gained into the neighboring concepts (see Section 4.1).
  • Inter-conceptual approaches, here, means that TA, RRI, or RwLs are integrating (single) aspects of the other concepts to widen their own work: TA studies are integrating real world experiments, for example. Therefore, TA learns from RwLs, or RWLs learn from RRI. The concepts ‘learn’ from each other and begin to merge (see Section 4.2 and Section 4.3).
  • The most far-reaching approach might be an endeavor that addresses all the core strengths of RwL, TA, and RRI together. To do so, we suggest a Reflexive Sustainable Technology Lab that combines and institutionalizes the triangle of all three concepts in one lab (see Section 5).
All of these modes of combination can and would foster sustainability transformation. In combination, what intensity is appropriate may differ, and depends on the case, context, and opportunities of implementation (See for the following also [8]).

4.1. Real-World Labs as the Research Objects in TA and RRI Case Studies

Numerous RwLs are suitable as case studies. Of course, RwLs with a focus on technology are particularly interesting for TA or RRI, e.g., new energy, climate protection/decarbonisation, information, or production technologies. The perspective on a RwL as a whole allows a TA study to examine new technologies as an eco-socio-technical system at an early stage. RRI could be interested in RwLs where tangible innovations are developed in an inclusive, reflexive cooperation of science and practice actors. The fact that RwLs can usually contribute from their own research system and orientation, and action knowledge can support TA research and RRI endeavors. In these cases, TA or RRI gain knowledge and learn from RwLs. However, the added value of TA or RRI case studies for the RwLs themselves is likely to be rather low.

4.2. TA- and RRI-Processes within Real-World Labs

RwLs can be used as a basis and framework for both TA and RRI. They can offer a social and epistemic framework and transdisciplinary infrastructure for TA studies or RRI processes. If an RwL is set up in a way that is suitable for a TA or RRI-topic (e.g., thematically, spatially, and in terms of its network of stakeholders), then TA or RRI projects can be carried out in it. As a rule, for TA, the more interactive approaches, especially participatory TA, seem appropriate. In general, long-term RwLs allow RRI and TA to follow the process of invention, innovation, or even the enculturation of new technologies longitudinally and at close range and with many research synergies. However, when using an RwL as a base for RRI or TA processes, special precautions must be taken in order to avoid negative repercussions for the RwL from project activities. The trust and social relations between the actors in the RwL are essential foundations of RwL work, and they can be damaged quite easily. In this respect, project participants must adhere to certain rules and ‘safety standards’ of the RwL. This includes transparency about the goals of the project and clear expectation management: What can the participants expect from the study? What are the perspectives beyond the project period? What about ownership (of the jointly developed) innovation?

4.3. Transformative Experiments as a New Methodology for TA and RRI

A third and—for TA and RRI—more fundamental option is to conduct the concepts in an experimental way. Looking at TA, RwLs can provide an appropriate and reliable epistemic and practical framework for TA experiments. Experimental work is not completely new in TA, e.g., a social bot for influencing opinions was programmed and used within the framework of a TA-study to investigate its effectiveness [45]. As a rule, however, such activities are too costly and lengthy within the framework of a TA study. RwLs allow the application of experimental working methods much more reliably, efficiently, and with better plannability. The decisive question that arises here is to what extent TA itself would like to proceed experimentally in the future.
A programmatic “Experimental TA” (eTA) would—like real-world experiments in general—be practice-oriented, application-oriented, transdisciplinary (participatory), and involve different actors under carefully monitored real conditions. The similarity of eTA to “Constructive TA” (cTA), where TA is directly implemented in technology development processes, is clear [46], but there is one small yet essential difference: In eTA, the outcome of the experiment is open. cTA accompanies a specific technology development process, which is already in large parts predefined, and in which it tries to minimize possible risks and side-effects prospectively, but cTA does not question the emergence of the technology per se. eTA would start much earlier, with far-reaching openness as to whether the development of a specific technology or eco-socio-technical innovation should be pursued further after the experimental phase or not. It is better suited to compare different technical solutions and their societal relations and ethical implications.
In contrast, RRI as a rational mode of innovation is, in a way, experimental. However, RRI projects normally aim at a specific technological or social innovation. Implementing transformative experiments into RRI means to widen the horizon of RRI in two ways: (1) it allows RRI processes to fail; (2) it means not only focusing on one specific possible innovation, but to a portfolio of different innovations contributing to a (specific) social challenge or need of transformation. Experimental RRI (ERRI) could become an endeavor where different innovations, or innovation-processes, heading for the same goal (contributing to a social challenge), could be guided simultaneously and compared. This would raise RRI to the next level of innovation, where the innovation itself is not of interest, but rather its societal purpose function. Thereby, RRI would become much more linkable to sustainability transformation. ERRI, in our opinion, should not replace RRI, but complement it. RRI is and remains useful and rational in all kinds of (specific) innovation processes. ERRI could complete this and guide more open search processes towards appropriate innovations for sustainable solutions. Contrastingly, transformative experiments in the future could be driven in a RRI-manner, driven by the four core criteria of RRI.

5. The Reflexive Sustainable Technology Lab—A Conceptual Synthesis of RwLs, TA, and RRI

The highest level of synthesis between the three approaches of RwL, TA, and RRI would be the establishment of what we would call a “Reflexive Sustainable Technology Lab” (RSTL). The RSTL would include TA and RRI as fundamental perspectives, approaches, and methodologies to a RwL. More precisely, an RSTL would be a full RwL in terms of its goals and core characteristics but would be enriched by a TA and RRI perspective (see Section 2). Looking from TA or RRI toward this RSTL expands the methodologies and practices of these concepts; namely, it would serve TA or RRI as infrastructure. As such, it would offer an environment for research, learning, and social transformation, with TA as an explicit objective and RRI as an innovation process guideline. In terms of methodology, the RSTL combines the common RwL approach of hands-on experimentation, transdisciplinary, and transformative research with the more distanced TA methods such as impact and sustainability assessment, scenario building, or multi-criteria decision analysis. Including TA and RRI into a technology-focused RwL increases the level of reflexivity on both sides: the reflexivity in the process of technology development as well as the reflexivity of a RwL as a structured learning environment (see “internal structure” below).
The overall objective of an RSTL is to contribute to a sustainability transformation via eco-socio-technological change, both in the form of innovation or even exnovation. Fostering technological innovations is one main strategy for a sustainable future worldwide (e.g., the decarbonization of national economies, see e.g., “The European Green Deal” [47]), while on the other hand “exnovation” describes the active phasing-out of unsustainable (socio-technical) systems which otherwise would create obstacles for alternative systems [48]. A starting point for an RSTL can be (a) a technological, eco-sociological, or socio-technological invention (or idea) that has the potential to contribute to sustainability transformation; (b) an existing (bundle of) technology transferred to a new context, or (c) different and competing technological solutions heading towards the same (sustainability) goal. The core questions are:
  • What potentials, risks, and side effects relate to the (emerging) innovation?
  • Which conflicts of goals and interests may arise?
  • How can the innovation process be accelerated and shaped in a socially and ecologically viable way?
  • What competencies do individuals, institutions, or societies need for this change process?
  • Which structures, governance, and institutions are needed?
  • What technology or common practice is outdated and has to be phased out, and how?
The focus on RSTL lies on sustainability transformation and not innovation itself. The key question here would always be whether emerging innovation really contributes to a sustainable future. This core question would steer the learning and innovation process. In contrast to today’s often pre-defined technology tests which aim to create acceptance of a given technology, it would be perfectly acceptable and a worthy result if the overall outcome of the RSTL was to stop the innovation and reject the tested eco-socio-technological invention. To fulfill this goal, its modus operandi is—like all RwLs—experimentation: Running experiments in, with, and for society in a framed, transparent, and reversible manner. The possibility that experiments can fail in their practical dimension—meaning that the innovation process stops by itself or is stopped intentionally—has to be taken into account and communicated transparently to all involved actors from the very beginning of the experiment. From a scientific perspective, a great deal of knowledge can be gained from a failing experiment (often more than from a ‘successful’ one), but in real-life failing is problematic. RwLs have lots of experience handling this difficulty with sensitivity.
Focusing on sustainability transformation and not on innovation opens the perspective to the possibility of RSTLs for exnovations. Sustainability transformation is not only a question of bringing new ideas to life, but also of addressing the phase-out of unsustainable practices and outmoded technological pathways, social practices, concepts, and mindsets [49]. For sustainability transformation, innovation and exnovation are equally important.
Thematically, RSTLs are open to all kind of sectors and subjects: energy, mobility, housing, materials, distribution and/or consumption processes, or any other topic related to a technological aspect and relevant for a sustainable future. For example, an RSTL on digitization, AI, and sustainability could be fruitful; an RSTL could cover new materials in the building sector; a city as an RSTL for a next generation of smart cities (including a TA-perspective from the very beginning) could be feasible; there is potential for autonomous mobility systems and alternative driving systems for sustainability, or, addressing a recent topic, transforming coal-mining regions to a regenerative future.
RSTL should accompany and support long-term transformation processes. Thus, they should last for several years, sometimes even for decades. Even though they are conceivable as small units dedicated to a specific, narrowly-tailored topic, their aspiration, complexity, and potential suggest that they should typically be bigger ventures.
For the internal structure or design of an RSTL, it is useful to organize it along the strength of the three concepts. From an epistemic point of view, the research and development activities within it are structured across three spheres: (a) the inner sphere of experimentation and development (RwL-research, RRI); (b) the outer sphere of reflection and assessment (TA); (c) the Lab (RwL) builds an epistemic and organizational framework and provides infrastructure.
The inner sphere (a) handles inner affairs from an inner perspective. There, the laboratory participants—from science and practice—look at and act in transformation processes in the RSTL from the inner perspective of a participant (or participatory observation). They are responsible for designing and driving the experiments, organizing the transdisciplinary (learning) process [50] as well as participation and stakeholder involvement, communication, and understanding. The inner sphere is the action and development unit of an RSTL—always driven in a RwL and RRI manner, or more specifically, by the respective core characteristics of these concepts. Research of and for sustainability transformation takes place from the position of an (active and/or observative) participant and can include implicit and tacit knowledge and produce experience-based ‘inner knowledge’.
The outer sphere reflects the inner sphere from an external perspective, and links the RSTL to outer (knowledge, political, and decision) contexts. It is realized in a classic TA manner. This department ensures a distanced perspective on the ongoing technological innovation and transformation processes in the RSTL. The laboratory researchers of the outer sphere are not directly involved in the innovation and transformation processes. They reflect the inner processes, organize assessment areas, open up the horizon, scale (up) and transfer the ‘inner knowledge’, and build upon it with scenarios and future options. Furthermore, they relate the RSTL work to bigger eco-socio-technological contexts, to general societal challenges and discourses, to social theory and (political) decision processes. The outer sphere is the reflection and consultation sphere. From this perspective, it can also provide critical and very substantial impulses to the inner work of the STL.
The third sphere is the lab itself. This ‘sphere’ provides both an epistemic framework for the research activities and the transdisciplinary infrastructure for the experiments and development processes [51]. First, serving the epistemic frame here means defining what is inside the lab (subject) and what is outside (context), what are the normative goals and presuppositions, which methodology and what approaches are appropriate, and what setting is useful for good research conditions. This sphere organizes the coupling and intertwining of the inner and outer spheres, enables reflection and learning processes, and a synthesis of different knowledge (types). Second, providing the RSTL with transdisciplinary infrastructure means organizing and taking care of rooms and spaces for the RSTL work, and allocating competencies, capacities, and methods for communication, moderation, participation, sometimes mediation (in cases of conflicts), knowledge transfer, and didactical support. The lab-sphere is the epistemic and practical enabling and maintenance unit.
To organize a RSTL along these three spheres, there must be a common construction plan and a strong commitment from all participants from the very beginning. The function of the spheres, the role of all participants, the modes of interaction, and the codes of conduct should be elaborated, explained, and written down in a kind of constitution (or ‘house regulations’). It should be explained that the lab is dedicated to sustainability transformation, the lab-sphere serves the inner and outer spheres, that the inner and outer knowledge is equivalently relevant, and how the reflection and learning circles in between the spheres are organized. All this (also) contributes to transparency and confidence building. Such a lab should be managed by a steering committee including representatives from all spheres, and a supervisory board with stakeholders, external experts and citizens.
To implement RSTLs successfully, they (in particular their subject) must fulfil a number of requirements. The most important are social relevance, relevance to the present, relevance for actors, and design potentials [8].
  • Societal relevance: Setting up an RSTL is only worthwhile if the topic is sufficiently relevant to society (also in order to acquire the required funding). Major societal challenges and transitions, such as those emerging in the course of the “Great Transformation”, particularly provide these preconditions. RSTLs could in this case support the discourse and decision-making process on technology pathways and transformation which are the subject of intense social controversy, such as energy system transformation.
  • Topical reference: The mainly experimental approach in RSTLs requires that at least some of the technologies (or aspects of the socio-technical system, models, processes, prototypes, etc.) can already be used experimentally or “experienced“ in some form at present. Thus, explicit RSTL should only be applied in the course of a technical invention, if necessary as a follow-up of a prospective TA study. Accompanying studies which are purely prospective could add to this.
  • Relevance for actors: In order for actors to be and—even more important—to remain involved in the long term, the issue being addressed must be of significant importance to them. Either they are directly or indirectly affected by the corresponding transformation process, or it must seem meaningful to them to initiate it.
  • Potential for shaping the future: An RSTL only makes sense if the technology field under investigation is sufficiently open for design, both with regard to the technologies themselves and with regard to their social, cultural, legal, economical, etc., embedding. The economic exploitability of innovations plays a role here—especially for necessary investments in the RSTL—but is ultimately only one aspect. It should accordingly not be overrated; in fact, it should always be subordinated to the goal of a public welfare-oriented, sustainable development.
An RSTL missing one of these requirements is most likely to fail, or rather, will never be realized because it takes a lot of effort and funds to build it up and operate it over many years. It needs (enough) social relevance, currency, relevance for actors, and design potentials to start this venture. For these reasons, RSTLs are adequate and appropriate in the context of great upheavals or transformation processes and should be realized largely by public authorities.
Concluding, in times of late modernism, RSTLs offer an adequate framework for transformative sustainability research and a mode of eco-socio-technological transformation under conditions of uncertainty, diversity and the increasingly urgent need for change. In RSTLs, innovation- and transformation processes are reversible, monitored, oriented toward the common good, future- and sustainability-oriented, and precautionary. By synthesizing the core characteristics of TA, RRI, and RwLs, their respective rationality can be combined to obtain a maximum of rationality in innovation processes.

Author Contributions

Conceptualization, O.P., P.B. and R.B.; Methodology, O.P., P.B., R.B., A.S., F.W. and J.H.; Project administration, A.S.; Writing—original draft, O.P., P.B., R.B., A.S., F.W. and J.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. WBGU. World in Transition—A Social Contract for Sustainability. Flagship Report; WBGU: Berlin, Germany, 2011. [Google Scholar]
  2. Clark, W.C. Sustainability science: A room of its own. Proc. Natl. Acad. Sci. USA 2007, 104, 1737–1738. [Google Scholar] [CrossRef] [Green Version]
  3. Grunwald, A. Transformative Wissenschaft—Eine neue Ordnung im Wissenschaftsbetrieb? GAIA Ecol. Perspect. Sci. Soc. 2015, 24, 17–20. [Google Scholar] [CrossRef]
  4. Schneidewind, U.; Scheck, H. Die Stadt als “Reallabor” für Systeminnovationen. In Soziale Innovation und Nachhaltigkeit; Rückert-John, J., Ed.; Springer: Wiesbaden, Germany, 2013; pp. 229–248. [Google Scholar] [CrossRef]
  5. De Flander, K.; Hahne, U.; Kegler, H.; Lang, D.; Lucas, R.; Schneidewind, U.; Simon, K.-H.; Singer-Brodowski, M.; Wanner, M.; Wiek, A. Resilience and Real-life Laboratories as Key Concepts for Urban Transition Research. 12 theses. GAIA Ecol. Perspect. Sci. Soc. 2014, 23, 284–286. [Google Scholar] [CrossRef]
  6. WBGU. Humanity on the Move—Unlocking the Transformative Power of Cities. Flagship Report; WBGU: Berlin, Germany, 2016. [Google Scholar]
  7. Parodi, O.; Beecroft, R.; Albiez, M.; Quint, A.; Seebacher, A.; Tamm, K.; Waitz, C. The ABC of Real-world Lab Methodology—From “Action Research” to “Participation” and Beyond. Trialog 2017, 126, 74–82. [Google Scholar]
  8. Parodi, O.; Beecroft, R. Reallabore als Möglichkeitsraum und Rahmen für Technikfolgenabschätzung. In Technikfolgenabschätzung—Handbuch für Wissenschaft und Praxis; Böschen, S., Grunwald, A., Krings, B.-J., Rösch, C., Eds.; Nomos: Baden-Baden, Germany, 2021; pp. 374–388. [Google Scholar] [CrossRef]
  9. Schäpke, N.; Stelzer, F.; Bergmann, M.; Singer-Brodowski, M.; Wanner, M.; Caniglia, G.; Lang, D.J. Reallabore im Kontext Transformativer Forschung. Ansatzpunkte zur Konzeption und Einbettung in den Internationalen Forschungsstand; Leuphana Universität Lüneburg, für Ethik und Transdisziplinäre Nachhaltigkeitsforschung: Lüneburg, Germany, 2017. [Google Scholar]
  10. McCory, G.; Holmén, J.; Schäpke, N.; Holmberg, J. Sustainability-oriented labs in transitions: An empirically grounded typology. Environ. Innov. Soc. Transit. 2022, 43, 99–117. [Google Scholar] [CrossRef]
  11. McCrory, G.; Schäpke, N.; Holmén, J.; Holmberg, J. Sustainability-oriented labs in real-world contexts: An explor-atory review. J. Clean. Prod. 2020, 277, 123202. [Google Scholar] [CrossRef]
  12. Schäpke, N.; Stelzer, F.; Bergmann, M.; Singer-Brodowski, M.; Wanner, M.; Caniglia, G.; Bernert, P.; Liedtke, C.; Loorbach, D.; Olsson, P.; et al. Jointly striving for transformation? A comparison of real world labs, urban transition labs, transformation labs, niche experiments and sustainability living labs. GAIA Ecol. Perspect. Sci. Soc. 2018, 27, 85–96. [Google Scholar] [CrossRef]
  13. Wagner, F.; Grunwald, A. Reallabore zwischen Beliebtheit und Beliebigkeit: Eine Bestandsaufnahme des transformativen Formats. GAIA Ecol. Perspect. Sci. Soc. 2019, 28, 260–264. [Google Scholar] [CrossRef]
  14. Parodi, O.; Steglich, A. Reallabor. In Handbuch Transdisziplinäre Didaktik; Schmohl, T., Philipp, T., Eds.; Transcript Verlag: Bielefeld, Germany, 2021; pp. 255–265. [Google Scholar]
  15. Bergmann, M.; Schäpke, N.; Marg, O.; Stelzer, F.; Lang, D.J.; Bossert, M.; Gantert, M.; Häußler, E.; Marquardt, E.; Piontek, F.M.; et al. Transdisciplinary sustainability re-search in real-world labs: Success factors and methods for change. Sustain. Sci. 2021, 16, 541–564. [Google Scholar] [CrossRef]
  16. Rose, M.; Maibaum, K. Meeting the challenge of (co-)designing real-world laboratories: Insights from the Well-Being Transformation Wuppertal project. GAIA Ecol. Perspect. Sci. Soc. 2020, 29, 154–160. [Google Scholar] [CrossRef]
  17. Schultz, J.; Brand, F.; Kopfmüller, J.; Ott, K. Building a ‘theory of sustainable development’: Two salient conceptions within the German discourse. Int. J. Environ. Sustain. Dev. 2008, 7, 465–482. [Google Scholar] [CrossRef]
  18. Grunwald, A. Technikfolgenabschätzung—Eine Einführung; Edition Sigma: Berlin, Germany, 2010. [Google Scholar]
  19. Böschen, S.; Grunwald, A.; Krings, B.-J.; Rösch, C. Technikfolgenabschätzung—Handbuch für Wissenschaft und Praxis; Nomos: Baden-Baden, Germany, 2021. [Google Scholar]
  20. Sotoudeh, M. TA in Unternehmen. In Technikfolgenabschätzung—Handbuch für Wissenschaft und Praxis; Böschen, S., Grunwald, A., Krings, B.-J., Rösch, C., Eds.; Nomos: Baden-Baden, Germany, 2021; pp. 165–178. [Google Scholar]
  21. Grunwald, A. Technology Assessment in Practice and Theory; Routledge: London, UK, 2019. [Google Scholar]
  22. Purvis, B.; Mao, Y.; Robinson, D. Three pillars of sustainability: In search of conceptual origins. Sustain. Sci. 2019, 14, 681–695. [Google Scholar] [CrossRef] [Green Version]
  23. Sovacool, B.K.; Newell, P.; Carley, S.; Fanzo, J. Equity, technological innovation and sustainable behaviour in a low-carbon future. Nat. Hum. Behav. 2022, 6, 326–337. [Google Scholar] [CrossRef] [PubMed]
  24. Lam, D.P.; Martín-López, B.; Wiek, A.; Bennett, E.M.; Frantzeskaki, N.; Horcea-Milcu, A.I.; Lang, D.J. Scaling the impact of sustainability initiatives: A typology of amplification processes. Urban Transform. 2020, 2, 3. [Google Scholar] [CrossRef]
  25. Tilman, D.; Socolow, R.; Foley, J.A.; Hill, J.; Larson, E.; Lynd, L.; Pacala, S.; Reilly, J.; Searchinger, T.; Somerville, C.; et al. Beneficial biofuels—The food, energy, and environment trilemma. Science 2009, 325, 270–271. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Jun, S.P. A comparative study of hype cycles among actors within the socio-technical system: With a focus on the case study of hybrid cars. Technol. Forecast. Soc. Chang. 2012, 79, 1413–1430. [Google Scholar] [CrossRef]
  27. Brand, U. Green Economy—The Next Oxymoron? No Lessons Learned from Failures of Implementing Sustainable De-velopment. GAIA Ecol. Perspect. Sci. Soc. 2012, 21, 28–32. [Google Scholar] [CrossRef] [Green Version]
  28. Paredis, E. Sustainability Transitions and the Nature of Technology. Found Sci. 2011, 16, 195–225. [Google Scholar] [CrossRef]
  29. Responsible Research and Innovation in Practice: Participants and Network. Available online: https://www.rri-practice.eu/participants-and-networks/affiliated-networks-and-related-projects/ (accessed on 12 August 2022).
  30. Martinuzzi, A.; Blok, V.; Brem, A.; Stahl, B.; Schönherr, N. Responsible Research and Innovation in Industry—Challenges, Insights and Perspectives. Sustainability 2018, 10, 702. [Google Scholar] [CrossRef]
  31. Van de Poel, I.; Asveld, L.; Flipse, S.; Klaassen, P.; Scholten, V.; Yaghmaei, E. Company Strategies for Responsible Research and Innovation (RRI): A Conceptual Model. Sustainability 2017, 9, 2045. [Google Scholar] [CrossRef] [Green Version]
  32. Ladikas, M.; Hahn, J.; Hennen, L.; Kulakov, P.; Scherz, C. Responsible research and innovation in Germany—Between sustainability and autonomy. J. Responsible Innov. 2019, 6, 346–352. [Google Scholar] [CrossRef]
  33. Stilgoe, J.; Owen, R.; Macnaghten, P. Developing a Framework for Responsible Innovation. Res. Policy 2013, 42, 1568–1580. [Google Scholar] [CrossRef] [Green Version]
  34. Von Schomberg, R. Prospects for technology assessment in a framework of responsible research and innovation. In Technikfolgen Abschätzen Lehren; Dusseldorp, M., Beecroft, R., Eds.; VS Verlag für Sozialwissenschaften—Springer Fachmedien: Wiesbaden, Germany, 2012; pp. 39–61. [Google Scholar] [CrossRef] [Green Version]
  35. Hahn, J. Towards a global Technology Assessment—Implications, challenges and limits. In Die Neutrale Normativität der Technikfolgenabschätzung; Nierling, L., Torgersen, H., Eds.; Nomos: Baden-Baden, Germany, 2020; Volume 23, pp. 175–194. [Google Scholar] [CrossRef]
  36. Reber, B. RRI as the inheritor of deliberative democracy and the precautionary principle. J. Responsible Innov. 2018, 5, 38–64. [Google Scholar] [CrossRef]
  37. Ostrum, E. Governing the Commons: The Evolution of Institutions for Collective Action; Cambridge University Press: Cambridge, UK, 1990. [Google Scholar]
  38. Bauer, A.; Kastenhofer, K. Policy advice in technology assessment: Shifting roles, principles and boundaries. Technol. Forecast. Soc. Chang. 2019, 139, 32–41. [Google Scholar] [CrossRef]
  39. Augenstein, K.; Bachmann, B.; Egermann, M.; Hermelingmeier, V.; Hilger, A.; Jaeger-Erben, M.; Kessler, A.; Lam, D.; Palzkill, A.; Suski, P.; et al. From niche to mainstream: The dilemmas of scaling up sustainable alternatives. GAIA Ecol. Perspect. Sci. Soc. 2020, 29, 143–147. [Google Scholar] [CrossRef]
  40. Fisher, E. Entangled futures and responsibilities in technology assessment. J. Responsible Innov. 2017, 4, 83–84. [Google Scholar] [CrossRef] [Green Version]
  41. Nieminen, M.; Ikonen, V. A future-oriented evaluation and development model for Responsible Research and Innovation. In Assessment of Responsible Innovation; Routledge: London, UK, 2020; pp. 248–271. [Google Scholar] [CrossRef]
  42. Beecroft, R. Transdisciplinarity in Real-world Labs. In Handbook for Transdisciplinarity; Unpublished Work; Lawrence, R.J., Ed.; Edward Elgar Publishing: Cheltenham, UK, 2021. [Google Scholar]
  43. Singer-Brodowski, M.; Holst, J.; Goller, A. Transformative Wissenschaft. In Handbuch Transdisziplinäre Didaktik; Schmohl, T., Philipp, T., Eds.; Transcript Verlag: Bielefeld, Germany, 2021; pp. 347–357. [Google Scholar] [CrossRef]
  44. Nentwich, M. A short response to van Lente, Swierstra and Joly’s essay ‘Responsible innovation as a critique of technology assessment’. J. Responsible Innov. 2017, 4, 262–267. [Google Scholar] [CrossRef]
  45. Hennen, L. TA und Öffentlichkeit—TA in öffentlichen Technikdebatten und öffentlicher Politikberatung. In Technikfolgenabschätzung—Handbuch für Wissenschaft und Praxis; Böschen, S., Grunwald, A., Krings, B.-J., Rösch, C., Eds.; Nomos: Baden-Baden, Germany, 2021; pp. 144–156. [Google Scholar] [CrossRef]
  46. Konrad, K. Constructive Technology Assessment—TA als konstruktives Element im Innovationsprozess. In Technikfolgenabschätzung—Handbuch für Wissenschaft und Praxis; Böschen, S., Grunwald, A., Krings, B.-J., Rösch, C., Eds.; Nomos: Baden-Baden, Germany, 2021; pp. 209–220. [Google Scholar]
  47. European Commission. The European Green Deal. Brussels, Belgium, 2019. Available online: https://ec.europa.eu/info/sites/info/files/european-green-deal-communication_en.pdf (accessed on 12 August 2022).
  48. David, M.; Gross, M. Futurizing politics and the sustainability of real-world experiments: What role for innovation and exnovation in the German energy transition? Sustain. Sci. 2019, 14, 991–1000. [Google Scholar] [CrossRef]
  49. Loorbach, D. To Transition! Governance Panarchy in the New Transformation; Communications Office Faculty of Social Sciences & DRIFT: Rotterdam, The Netherlands, 2014. [Google Scholar]
  50. Bergmann, M.; Jahn, T.; Knobloch, T.; Krohn, W.; Pohl, C.; Schramm, E. Methoden Transdisziplinärer Forschung. Ein Überblick mit Anwendungsbeispielen; Campus Verlag: Frankfurt, Germany, 2010. [Google Scholar]
  51. Parodi, O.; Waitz, C.; Bachinger, M.; Kuhn, R.; Meyer-Soylu, S.; Alcántara, S.; Rhodius, R. Insights into and recommendations from three real-world laboratories: An experience-based comparison. GAIA Ecol. Perspect. Sci. Soc. 2018, 27, 52–59. [Google Scholar] [CrossRef]
Figure 1. Key similarities as strong conceptual bridges (green) between RwL research, RRI, and TA, based on [7,14,19,21,33] (authors’ visualization).
Figure 1. Key similarities as strong conceptual bridges (green) between RwL research, RRI, and TA, based on [7,14,19,21,33] (authors’ visualization).
Sustainability 14 15094 g001
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Parodi, O.; Bögel, P.; Beecroft, R.; Seebacher, A.; Wagner, F.; Hahn, J. Reflexive Sustainable Technology Labs: Combining Real-World Labs, Technology Assessment, and Responsible Research and Innovation. Sustainability 2022, 14, 15094. https://doi.org/10.3390/su142215094

AMA Style

Parodi O, Bögel P, Beecroft R, Seebacher A, Wagner F, Hahn J. Reflexive Sustainable Technology Labs: Combining Real-World Labs, Technology Assessment, and Responsible Research and Innovation. Sustainability. 2022; 14(22):15094. https://doi.org/10.3390/su142215094

Chicago/Turabian Style

Parodi, Oliver, Paula Bögel, Richard Beecroft, Andreas Seebacher, Felix Wagner, and Julia Hahn. 2022. "Reflexive Sustainable Technology Labs: Combining Real-World Labs, Technology Assessment, and Responsible Research and Innovation" Sustainability 14, no. 22: 15094. https://doi.org/10.3390/su142215094

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop