When Wrong Is Right: Leaving Room for Error in Innovation Measurement

Abstract

To date, measuring innovation has not been an exact science. As in many areas of organizational life, errors in measuring innovation are a recurring fact. Innovation researchers and practitioners alike have become increasingly interested in understanding the occurrence of organizational errors and how these errors affect innovation and its measurement. This empirical study aims to address this under-explored area by utilizing a qualitative in-depth case study at the innovation department of an organization with production sites and sales organizations worldwide. A total of 28 semi-structured interviews at several organizational levels were conducted, with innovation managers, project managers, senior managers, and staff. Based on the findings in this case study, three explanations are presented on how organizational errors occur when using innovation KPIs (key performance indicators). The first explanation can be connected to the increasing complexity of innovation and its intangible nature. Another explanation can be traced to the difference between innovation strategy and innovation KPIs. Lastly, room for organizational errors can be related to the multitude of individuals and organizational levels involved in innovation and its measurement. The implications for practitioners are that innovation KPIs are not precise metrics but should be seen as estimates with organizational errors. Whether or not these innovation KPIs can be used as tools to turn innovation into competitive advantages largely depends on whether wrong is right. Future research should focus on the metrics that are implemented and actually in use, as this future path would highlight the function and dysfunction that organizational errors in innovation KPIs can have.

1. Introduction

In response to a complex and dynamic business environment, organizations try to improve their performance and become more competitive (Alfaro García et al. 2015; Okwir et al. 2018). In this context, innovation is seen as an important source of competitive advantage for companies (Porter 1990) and significant for the survival of the organization (Dziallas and Blind 2019; Ortt and van der Duin 2008).

An interesting issue in innovation management is its measurement (Alfaro García et al. 2015; Gimbert et al. 2010). In ever-changing competitive dynamics, metric systems should function as a link between strategy, implementation and value creation (Davila et al. 2012; Melnyk et al. 2014). To track progress in innovation, organizations usually implement metrics such as key performance indicators (KPIs) (Richtnér et al. 2017). Thus, KPIs play a central role in enabling managers to fulfil the primary purpose of innovation, to create new opportunities, or to exploit existing ones (Brattström et al. 2018; Damanpour and Wischnevsky 2006; Drucker 1998).

Existing literature indicates problems with these KPIs, stemming from the complexity of quantifying, evaluating, and managing innovation practices and competence (Davila et al. 2012; Tidd and Bessant 2018). To further explain these unintended deviations between what should be measured in innovation and what actually is measured, this paper takes a closer look at phenomena called organizational errors in the use of innovation KPIs.

In an ideal world, KPIs in innovation should fit the underlying context of innovation (Brattström et al. 2016; Brattström et al. 2018; Richtnér et al. 2017). Recent literature on innovation and its measurement shows that the underlying context of innovation has become more complex (Okwir et al. 2018; Ollila and Yström 2020). In response to this increased complexity in innovation, KPIs seem to have evolved towards more “intelligent” and contemporary performance measurement systems (Franco-Santos et al. 2012; Henri and Wouters 2017; Schrage and Kiron 2018). This might explain why KPIs in innovation may be used with room for error, as the complexity of the context of innovation calls for continual updates (Chenhall and Moers 2015; Edmondson and Verdin 2018; Harris and Tayler 2019). Although links have been made between KPIs, innovation, and errors, the ‘error-proneness’ of innovation calls for special attention to the use of KPIs in innovation and the occurrence of organizational errors (Edmondson and Verdin 2018; Harris and Tayler 2019).

To address the missing links between organizational errors, innovation, and its measurement, this study aims to examine how KPIs in innovation (innovation KPIs) create room for error. The study suggests that the gap between what KPIs in innovation are supposed to measure and what they actually measure can partly be explained by the occurrence of organizational errors. The interplay and consequences of this occurrence of organizational errors in innovation and its measurement are unknown territory in innovation measurement literature. Thereby, the following research question is addressed: how and where does room for organizational errors occur in measuring innovation, and what are the implications of this room for error occurring?

The study contributes to the literature in several ways. Firstly, it adds to the body of knowledge on the KPIs in innovation and organizational errors. Following Frese and Keith (2015), innovation and error management are related, and knowledge on how measurement, innovation, and organizational errors interact will be highlighted in this case study. Secondly, this study shows that measurement, metrics and organizational errors in innovation are linked. Although links have been made between measurement, metrics and organizational errors, the ‘error-proneness’ of innovation calls for specific attention (Edmondson and Verdin 2018; Harris and Tayler 2019; Richtnér et al. 2017). Thirdly, the case study shows the connection between performance measurement, metrics, organizational errors and learning. The discussion on error detection, error prevention and error management shows that organizational errors do not need to have a negative organizational consequence per se (Edmondson and Verdin 2018; Frese and Keith 2015). This study highlights how innovation KPIs are differently interpreted on strategic and operational levels. The room for organizational errors between strategy and metrics on multiple organizational levels can thus create learning opportunities that are essential to positive organizational outcomes in innovation (Edmondson and Verdin 2018; Frese and Keith 2015; Goodman et al. 2011; Weick 2012). Lastly, it shows that the actual use of KPIs in innovation can differ from the strategy-as-planning (Edmondson and Verdin 2018). This more interpretative and relational perspective on KPIs in innovation shows the variation that occurs when participants at multiple levels are involved, and highlights the room for error (Goodman et al. 2011). This multilevel analysis gives a unique perspective on KPIs, organizational errors and innovation. The actual use of KPIs in innovation is explained using an interpretative and relational perspective on these KPIs. Comparing their actual use to the strategy-as-planning shows that variation occurs when participants at multiple levels are involved (Goodman et al. 2011).

The remainder of this paper is structured as follows: Section 2 discusses the existing research related to this study and Section 3 presents the qualitative research methodology undertaken. The results of the case study are described and discussed in Section 4 and Section 5, while the conclusions and future work are covered in Section 5.

2. Theory Background

2.1. Innovation, KPIs and Organizational Errors

Before discussing innovation measurement, it is essential to understand innovation (Dewangan and Godse 2014). Numerous definitions of innovation have been proposed in the literature, and to date there is no clear and authoritative definition of innovation in existing scientific literature; various definitions and understandings of it can, therefore, vary greatly from one another (Baregheh et al. 2009; Dziallas and Blind 2019). Thus, innovation warrants clarification. In this study innovation is defined as the process of moving ideas into value (Davila et al. 2009; Davila and Oyon 2009; Davila 2012). Our definition, further, builds on the Eurostat (2018) definition that innovation includes the implementation of a new or significantly improved product, process, or service (Dewangan and Godse 2014; Dziallas and Blind 2019).

A key performance indicator (KPI) is defined as an instrument used to quantify the efficiency and/or effectiveness of action in innovation (Melnyk et al. 2014; Neely et al. 2005). Related terms that can be found in performance measurement literature are “measurement”, “performance” and “metrics”(Goshu and Kitaw 2017; Micheli and Mari 2014). Keong Choong (2014) finds that many of the articles in their review treat measurement and performance measurement as synonymous. Keong Choong (2014) further suggested that “measurement is executed prior to any performance measurement”. The review quotes Neely et al. (1995), who state that measurement is “the process of quantification and action leads to performance” and performance measurement the “process of quantifying the efficiency and effectiveness of action.” As a result, this study operationalizes KPIs, performance measurement and measurement according to these reviews (Goshu and Kitaw 2017; Keong Choong 2014; Taticchi et al. 2012).

Organizational errors represent unintended deviations from organizational expectations about how to execute the work, and involve multiple individuals acting within formal organizational roles; these errors carry a risk of harm and are primarily caused by organizational conditions, including values and rewards (Edmondson and Verdin 2018; Harris and Tayler 2019; Lei et al. 2016). “Errors” are essentially unintended deviations from rules or procedures and should be distinguished from failures (Goodman et al. 2011). Errors are said to be a recurring fact of organizational life and thus merit study in their own right as an important phenomenon of growing theoretical and managerial significance (Goodman et al. 2011; Hofmann and Frese 2011; Lei et al. 2016). Not every error leads to failure: errors can be detected and corrected immediately, or they may occur in a safe environment and thus not lead to failure (which is a consequence of an error) (Frese and Keith 2015). In the next sections it will be explained how the room for organizational errors can arise in innovation and its measurement and how, as a result, KPIs in innovation have no consensus and mean the same for different things (Keong Choong 2014).

2.2. Innovation Measurement

Innovation measurement provides different tools and mechanics that are necessary in the process of moving ideas into value (Davila et al. 2009; Davila and Oyon 2009; Davila 2012). These metrics seem to give strategy and innovation form (Harris and Tayler 2019). Innovation, in today’s dynamic and turbulent environment, changes either the business environment or the business strategy, which in turn can lead to the need for new or revised measures and metrics (Melnyk et al. 2014). Many research studies indicate that a contingency approach—an alignment—between the metric chosen and underlying context of innovation, is recommended (Chenhall 2003, 2009; Chenhall and Moers 2015).

A few decades ago, innovation KPIs mainly were concerned with the input side of innovation, for example, spending on research and development (R&D), the number of acquired patents and the amount of highly qualified human resources (Alfaro García et al. 2015). Other studies, such as Davila et al. (2012), suggest to advance these KPIs to include the process, outcome and output sides of innovation. These holistic frameworks of innovation KPIs are also presented in Adams et al. (2006), Crossan and Apaydin (2010) and more recently by Edison et al. (2013) and Alfaro García et al. (2015).

Previous studies have presented many alternative conceptualizations and models for innovation measurement and metrics (Liao and Wu 2010; Melendez et al. 2019). However, disagreement exists on how to quantify and measure innovation: for example, in a recent study, 82 unique indicators to assess innovation were identified, indicating that consensus in what metrics to use does not exist (Dziallas and Blind 2019). Metrics in innovation are crucial as they can provide clearer directions, and coordinate behaviors and actions where strategy may otherwise seem too amorphous to have an impact (Harris and Tayler 2019). Still, a number of industry surveys confirm the discontent of firms with their current metrics for innovation (Dewangan and Godse 2014). It could be concluded that most metrics, not only for innovation, are imperfect at some level (Davila et al. 2012; Harris and Tayler 2019).

Innovation measurement is a multifaceted phenomenon that does not prescribe one way of measuring innovation (Brattström et al. 2018; Edison et al. 2013; Richtnér et al. 2017). The choice of a metric in innovation is dependent on the definition of innovation used and the conceptualization between strategy, innovation and metrics. The room for organizational errors in innovation measurement is further explored and explained in the next sections.

2.3. Organizational Errors in Innovation Measurement

Failures and errors are an integral part of innovation, learning and growth (Frese and Keith 2015; Khanna et al. 2016). Innovation is inherently error-prone (Bledow et al. 2009; Frese and Keith 2015). Frese and Keith (2015) argue that organizational errors and innovation are related because of the serendipitous, creative, experimental, ambidextrous, and explorative nature of innovation. These organizational errors are said to lay the foundation for outcomes such as innovation and learning (Hofmann and Frese 2011). By nature, innovations are not possible without making errors, because any innovation implies actions in a new and therefore unknown environment (Frese and Keith 2015).

In previous sections, it could be seen that innovation and its measurement leave room for organizational errors. Innovation, being error-prone and not having one authoritative definition, leaves room for organizational errors in its definition (Baregheh et al. 2009; Dziallas and Blind 2019; Frese and Keith 2015). Innovation requires the value creation and value capture of several stakeholders inside and outside of the organization (Garcia et al. 2019). Organizational errors here can be related to the definition of innovation, as each individual member has their own definition of innovation and an organization does not have an agreed-upon definition of innovation. As a result, room for organizational errors is caused due to the different definition of innovation of each individual member in the organization. Once acted upon, these different definitions of innovation may lead to organizational errors. Organizational errors are thus an explanation of the challenge that emerges because of the incongruence in the definition of innovation that exists at multiple levels of the multi-stakeholder network (Garcia et al. 2019). Secondly, the deviations between innovation, strategy, and metrics may also cause room for organizational errors. As stated earlier, KPIs in innovation are imperfect at some level (Choong 2014; Davila et al. 2012; Harris and Tayler 2019). Most literature on KPIs in innovation recommends choosing KPIs that align with external and internal conditions, such as strategies, and are adapted to firm-specific objectives of measurement (Bititci et al. 2018; Goshu and Kitaw 2017). Insights in KPIs in innovation show that research is moving away from traditional, mostly financial metrics—measuring the rate of return on investment, cash flow, and the profit margin—towards integrated and contemporary metrics (Bititci et al. 2012; Bititci et al. 2018; Bourne et al. 2018; Franco-Santos et al. 2012; Goshu and Kitaw 2017). These necessary adjustments in strategy and/or metrics are not automatic and can create room for organizational errors, as they create deviations from procedures at several levels of the organization (Nixon and Burns 2012). A third area where organizational errors can occur in measuring innovation is when the pre-determined, or designed, innovation KPIs are compared to the actual utilization of information obtained from the implemented innovation KPIs (Janssen et al. 2011). In literature, the difference between the designed innovation KPIs and the innovation KPIs that are implemented and used is discussed widely (Bititci et al. 2018; Bourne et al. 2018; Ferreira and Otley 2009; Neely and Bourne 2000). Currently, research has focused on the design of innovation KPIs and not on observing their actual uses, nor on the utilization of information obtained from them (Janssen et al. 2011; Nudurupati et al. 2011).

In sum, there seem to be three main reasons why room for errors can be created while using KPIs in innovation (see Figure 1). The first one is connected to the context of innovation. Innovation, is complex and evolutionary as a chain from idea to value, making quantification in KPIs vulnerable to room for error. Secondly, translating strategy and innovation into metrics leaves room for errors, as most metrics are imperfect on some level (Harris and Tayler 2019). Thirdly, there is room for error in the usage of KPIs in innovation, as there may be a space between when the KPIs are set and when they are used, due to time or the division of labor.

3. Methods

3.1. Research Design

This research study is conducted using a case-study approach (Yin 2014) and applies a multi-disciplinary perspective to innovation and the phenomenon studied (Brattström et al. 2018; Fried 2017). Because existing literature regarding KPIs in innovation and organizational errors is still scarce, an exploratory study using case studies is chosen as the preferred methodology to build knowledge about the phenomenon (Yin 2014). The motivation behind studying organizational errors in innovation KPIs is twofold. First, organizational errors in KPI usage in innovation are understudied empirical contexts (Bansal et al. 2018). Secondly, the innovation department was a distributed unit consisting of several departments operating in different geographical markets. Hence, the innovation KPIs and organization in this study can be considered a unique case, with several subcases (Yin 2014). The case was selected based on recommendations from industry experts and by using snow-ball selection by stakeholders at the case company.

In many instances, case studies have been found to be an effective methodology to investigate and understand complex issues in real world settings (Bansal et al. 2018; Harrison et al. 2017). In this study, a case study has been used to deduce new knowledge that can introduce new theory or guide existing theory in new directions (Bansal et al. 2018). Furthermore, in this study, in-depth and face-to-face interviews were the suitable choice for obtaining knowledge of informants’ experiences and how they make sense of concepts such as innovation, its measurement, and organizational errors (Alvesson 2003; Brinkmann and Kvale 2015; Kvale and Brinkmann 2009).

3.2. Data Collection

To capture the occurrence of organizational errors in KPIs in innovation, a qualitative case study was conducted at an innovation department in a single organization. The innovation department, instated at least a decade ago, is part of a company in the process industry that operates production sites and sales offices worldwide. The parent company generates annual revenues of around EUR 1.5 billion, and the company’s headquarters are in a Nordic country. It has sales and production in the Americas, Asia, and Europe, but most of its business is within European countries. The organization has end customers in many sectors: Automotive, Construction, Electronics, Medical, Feed & Food, to name a few.

Like those at many other large organizations, the innovation department and their teams are cross-functional. In addition to R&D and engineering, employees from marketing and manufacturing are also involved in the projects to obtain a nuanced view. Because the innovation department is diffused worldwide, project members usually work internationally, located in different subsidiaries.

In this study, potential validity threats were addressed by spending considerable time in the field, using multiple methods of observation and being aware of one’s behavior as a field researcher (Ahrens and Chapman 2006; McKinnon 1988). Both informal and formal meetings were attended, and days on location were deliberately scheduled. In the study, the researchers moved back and forth between data, theory and related literature to make sense of the made observations (Ahrens and Chapman 2006; Jørgensen and Messner 2010).

The fieldwork of this study started in September 2017. Over a 5-month period, 28 interviews were conducted within this organization regarding KPIs in innovation. To prepare the interview rounds and select the case, two preliminary meetings in the organization’s headquarters were initiated. First, exploratory meetings were held with the responsible innovation controller and the senior innovation advisor. Notes were taken during the meeting, and internal documentation regarding the organization of innovation over the years were provided. Second, the researcher conducted 28 interviews with innovation personnel, senior management, and project management and business/sales employees mostly lasting between 20 min and an hour (See

Table 1. Details of Case Study.

Date of Interviews	Number of Interviews	Average Time Interview	Job Title	Data Collected
Autumn 2017 Spring 2018	28	1/2–½ h	Innovation personnel (5), controller (3), sales manager (3), R&D manager, sales and technical support manager, senior consultant innovation, project manager (4), program manager, business managers, production managers (2), patent manager, CEO, director of innovation.	Interviews, Notes, Reports Powerpoints Presentation Emails

). The respondents were selected to give their views on the existing KPIs in innovation. All interviews were semi-structured, with an initial set of questions given to respondents relating to their role, individual responsibilities and tasks, and their views on innovation. This was followed by open-ended discussions regarding the KPIs in innovation in use, and their perceptions regarding the KPIs’ levels of functionality, development possibilities and challenges. Different informants were thus asked similar questions to acquire different perspectives on the same issues and/or to confirm individual accounts. Each of the interviews was recorded and later transcribed in full. Finally, the internal documents were reviewed, such as the presentation slides from meetings or those distributed after meetings, as well as public documents, such as the parent organization’s recent annual reports. Among these internal documents, a written documentation clearly stating the KPIs of innovation was obtained. This written documentation was used as to see whether “official” KPIs indeed were found in the respondents ‘answers.

3.3. Data Analysis

The data analysis of the interview consisted of three steps of coding, condensation and interpretation, following Brinkmann and Kvale (2015) and Bloomberg and Volpe (2018). In the first phase, the codes (in vivo codes) generated from the collected empirical materials are compared with the themes (a priori categories) identified in the literature. In the second stage, the data were condensed by structured readings, re-readings and writing of the thematically structured data. Finally, in the third phase, the condensed data were interpreted in the light of the literature on innovation, its measurement, and organizational errors, to be able to fully explain the findings.

4. Results

These sections describe how KPIs in innovation are used and how room for organizational errors is created. As a basis for understanding how KPIs in innovation are used, illustrations are given regarding strategy, innovation, and KPIs (Melnyk et al. 2005). As innovation involves multiple stakeholders at multiple organizational levels, individuals with different organizational tasks and positions were interviewed (see Table 1). Here, it was observed how KPIs in innovation are used as compared with the “officially” agreed upon KPIs. These “officially” agreed upon KPIs were found in formal, written documentation and were used to compare the answers of the respondents.

4.1. Room for Organizational Error 1: Innovation and Its Defintion

Innovation has no authoritative definition (Adams et al. 2006; Baregheh et al. 2009; Edison et al. 2013), which explains why different definitions of innovation can exist simultaneously on different levels of organizations. These incongruencies and deviations in the definitions of innovation may explain the room for organizational errors within organizations. The following answers illustrate that each of the respondents has a different view on what innovation is. The business controller introduces the issue:

“We need to understand what innovation implies in our organization. We need to focus on our customer promise and deliver on it and understand how innovation delivers customer and business value. During my years as a controller [not in the process industry] I have never encountered such a complicated relationship between innovation and customer & business value.”

Business controller

Other views on innovation exist at other levels in the organization. This is illustrated by the answers given by innovation staff and management:

“In my previous organization I was responsible for delivering value to business through innovation [not in the process industry]. In this organization everything seems to be more unclear and the complexity of what is really innovation seem to make it more difficult.”

Director innovation

“I am used to approximately 10 innovation projects per month passing a tollgate [not in the process industry] but those projects were usually homogeneous. Here at this organization all the projects seem to be unique and it takes longer time to understand them. This might explain why less projects pass a tollgate.”

Innovation staff

During the interview, it became clear that innovation has been defined differently at different points in time. In the beginning, innovation only involved the staff at the research and development (R&D) department, but over time, more and more stakeholders became involved in innovation. The following quotes illustrate this shift:

“Before [the] innovation [department] was just supplying us with new products, now we want [the] innovation [department] to be our speaking partner.”

Business account manager

“I have been to conferences on innovation management, and I have seen how other companies track their innovation. After this I have made a lot of suggestion to management how to change their KPIs.”

Project manager

As can be seen, not only one level, but several levels of the organization are involved with innovation in some way. The respondents indicate that innovation is not just one department’s business (the R&D department), but instead is becoming everyone’s business. Innovation does not only include producing new products in the research and development department; instead, it is a concern to all members of the organization. Innovation is not seen as limited to the developers of new products; it is rather seen as a co-creator of new solutions to secure the company’s competitive edge. There is a wish for the innovation to work as a team with the rest of the organization, as is stated in the official statement: “innovation should act as one on a common agenda”.

The above respondents illustrate that innovation is defined differently by each of the respondents. A common definition of innovation seems to not have been defined and formally communicated in this company, although there is a clear agenda that all respondents should act as one. Innovation seems to have evolved from being the responsibility of few, to becoming the responsibility of many. The risk that the traditional departmental “silo” view on innovation applies to innovation KPIs for organizational wide endeavors is also illustrated by these citations. In both of these cases, different definitions of innovation and innovation as an organizational wide endeavor, may cause deviation from the intended goal to “act as one on a common agenda” (Frese and Keith 2015; Goodman et al. 2011).

4.2. Room for Organizational Error 2: Innovation Strategy vs. Innovation KPIs

In another part of the interview, respondents were asked about the linkages between innovation strategy, innovation KPIs and corporate value. This question addresses how outcomes or consequences from errors at one level of analysis affect outcomes at other levels of analysis (Goodman et al. 2011). In this part, the researcher is asking how changes in innovation and innovation strategy at one level may lead to changes in the innovation KPIs at a different level. The senior innovation expert describes the innovation strategy as follows:

In industry, there is a generic triangle. At the center of the triangle are safety, health and the environment. […]. Each company defines a number of trends; they are connected [environmental] trends and [subsequently] described in our common strategy and our strategy for innovation. We currently have multiple production platforms. Then we have selected several market segments, and you should preferably fit this [innovation strategy] into a [scorecard with KPIs] which [portrays] areas we are adequately skilled at managing.

During his career at the company, which started in 1995, he has seen how the linkages between innovation strategy and innovation KPIs may create room for error:

[…] Now one must instead pedagogically explain that the connection is [between innovation strategy and innovation KPIs]. I saw before when we linked the innovation strategy to the innovation KPI by means of the amount spent on innovation. The personal bonus was conditional on [this KPI] and we were not allowed to spend more than 70 percent in December. Suddenly, all staff disappeared. It was in this situation that [the linkages between innovation strategy and innovation KPIs] become clear. Most people during that time tried to do their best to change the margin and spent less. What happened was that, that year, the price of gasoline was just very high. Then you start to question the linkages, and how can I [on an individual level] affect innovation strategy or innovation KPIs? If you do not know these cycles, it can be difficult to motivate yourself towards the KPIs in innovation that you should reach.

This example shows that room for organizational errors is created between innovation strategy and innovation KPIs. Although this room for organizational errors is unintended, the risk of the occurrence of organizational errors between innovation strategy and innovation KPIs in this company should be identified. This occurrence might be related to the inherent imperfection of KPIs, which is addressed by Harris and Tayler (2019).

4.3. Room for Organizational Error 3: Official KPIs vs. Used KPIs in Innovation

In this part of the case study, the focus is on which KPIs in innovation are in place and how they are used: that is, how the information obtained from these KPIs in innovation is utilized. Organizational errors can occur when the KPIs that are designed for innovation in practice provide little or no information that is utilized by the organization. In the written documentation, the company outlines its innovation strategy and the associated “official” KPIs (a spreadsheet with all the information was provided by senior management). The provided documentation shows on each organizational level the innovation strategy and 5–10 KPIs. Asking the respondents about which KPIs are in use, and which information they derive from them, led to unexpected responses:

Actually, I do not really know what the KPIs in innovation are. So, I cannot really answer the question, is that a problem?

Project manager

The information from KPIs that are relevant for my job is limited. I have been doing this job [managing the ideation part of innovation] for quite a while, so I know I have to keep an eye on tollgate 1 and tollgate 4. I know exactly how many ideas we need to deliver upon our own targets.

Senior researcher

Can you please give me a definition of what a KPI is? I do not understand the difference between this and the goal and scorecard we use in the organization. And because I am new to this organization, I feel that I do not have access to all of the information and KPIs that I should know. In my previous organization everything was much more clear and accessible.

Application manager

A review of the responses shows that most of the respondents use the official KPIs in innovation to measure and follow-up their work, in part or not. This is an unintended deviation between “official” and actual use and can create room for organizational errors. An explanation can possibly be found in the time lapse between 2014, when the official KPIs were determined, and when they were used (in 2017). Another explanation can be found in the difference between the organizational level at which the “official” KPIs were set—that is, the top management level—compared to the levels the innovation KPIs were used—the lower operational levels. This could indicate that the “official” KPIs in innovation were part of the strategy-as-planning (Edmondson and Verdin 2018) and created room for organizational errors when used in different levels in the organization.

5. Discussion

This paper has explored how the use of KPIs in innovation displays room for organizational errors. The case study shows that there are three areas where room for organizational errors is created in KPIs in innovation. The first room for organizational errors in using KPIs in innovation is connected to the increasing complexity of innovation and the intangible nature of innovation. As can be seen in the findings, the complexity of innovation and its measurement required more people to become involved (Okwir et al. 2018; Ollila and Yström 2020). The interview series provides evidence that innovation required several organizational levels and individuals to “act as one on a common agenda” (official written documentation). During the interview, the business controller mentioned that it is difficult to connect the customer and business value created (innovation output/outcome) to the endeavors made in innovation (innovation input). An explanation why it is difficult to connect innovation input and output could be related to complexity. It could furthermore be observed that the view on innovation has shifted over the years. At first, innovation involved one department that developed new products and supplied research and development. Slowly, over time, innovation moved towards innovation including many processes and participants at several organizational levels, from idea generation to execution and value capture (Adams et al. 2006; Davila et al. 2012). In this case, it was observed that innovation shifted from being somebody’s business to becoming everybody’s business. Following the definition of organizational errors (Frese and Keith 2015; Goodman et al. 2011), room for organizational errors was created because the KPIs in innovation were designed to measure somebody’s business but moved to being used to measure everybody’s business.

Secondly, room for organizational errors was created between innovation strategy and innovation KPIs. Many researchers have investigated the alignment between innovation strategy and KPIs, and advocate that KPIs need to “fit” strategy (Chandler 1990; Harris and Tayler 2019). In the literature, these innovation KPIs are acknowledged to be flawed proxies of innovation strategy (Harris and Tayler 2019). The room for error between strategy and KPIs is a well-documented problem in contingency theory and can be related to KPIs being flawed proxies. Their findings showed how higher prices in gasoline created a room for error in the KPIs compared to the innovation strategy. No individual action could change the collective result of keeping the limits set by the KPIs and the innovation strategy. Eventually, this led to individuals resigning from their jobs, as they were unable to influence and reverse the organizational errors. The level of origin of the innovation strategy, the strategic level, and the level of operation of the KPIs, the operational level, can in part explain the discrepancy between innovation strategy and innovation metrics. In this case, considering the use of these flawed proxies created room for error indicates that strategic and operational consequences of KPIs and their uses require attention (Dahlin et al. 2018; Edmondson and Verdin 2018; Frese and Keith 2015; Harris and Tayler 2019; Lei et al. 2016). The room for organizational errors between strategy and metrics on multiple organizational levels can create a learning opportunity essential to positive organizational outcomes in innovation (Edmondson and Verdin 2018; Frese and Keith 2015; Goodman et al. 2011; Weick 2012).

Thirdly, room for organizational errors is found between the ‘official KPIs in innovation’ and the KPIs in innovation in use. The mismatch between what is supposed to be measured in innovation and what actually is measured can be derived either from the underlying context of innovation or from the design of KPIs, or both. This room for error originates not only from different organizational levels (strategic vs. operational) but also from the time lapse between when official KPIs were established and when they were used. Concerning the room for error that can appear between strategic and operational levels, the findings show that the innovation strategy at the strategic level cascaded down to those in the organization using innovation KPIs. The findings indicate that innovation KPIs still are not dynamic and resilient to changes in the internal and external environment of the firm (Melnyk et al. 2014; Nudurupati et al. 2011; Okwir et al. 2018). The consequences of innovation and KPIs not portraying this ideal fit can explain why room for error is observed in practice (Fried 2017; Harris and Tayler 2019).

In our view, it seems inconclusive whether the use of KPIs in innovation supports or misdirects organizational actions. As said by Frese and Keith (2015), not every error leads to failure: errors can be detected and corrected immediately, or they may occur in a safe environment and thus not lead to failure. This room for error in the use of innovation KPIs increases the risk of harm and negative consequence (Edmondson and Verdin 2018; Harris and Tayler 2019). To manage this risk, the literature suggests that organizations need to look at the associated organizational conditions that caused the room for error. This room for error may lay the foundation for outcomes such as innovation and learning but might also end up creating dysfunctional and unwanted organizational and individual behavior (Harris and Tayler 2019; Hofmann and Frese 2011). The findings here illustrate how measurement, innovation, and organizational errors interact, and thus provide a preliminary indication that innovation and error management are related (Frese and Keith 2015).

6. Conclusions

The findings of this study try to explain and understand the occurrence of organizational errors in innovation, specifically when innovation KPIs are used. In the evidence gathered it is shown that, in this case, room for organizational errors was created while using innovation KPIs. Three explanations for this are provided. The first explanation is related to changes in the definition of innovation. The second explanation is related to differences in the formulated innovation strategy and innovation KPIs used. Thirdly, room for organizational errors occurs between the ‘official KPIs in innovation’ compared to the KPIs in innovation in use.

This paper contributes to research in several ways. Firstly, it addressed the missing links between organizational errors and innovation and its measurement and organizational errors. It tries to explain how KPIs, designed for measuring innovation, are used in practice with room for organizational errors. The room between what KPIs in innovation are supposed to measure and what they actually measure can be explained in part by the occurrence of organizational errors. Secondly, it highlights that the perceived view and definition of innovation on different organizational levels and different points of time affect the use of KPIs in innovation. The findings show that innovation is no longer a departmental duty, but concerns all organizational levels responsible for the chain from idea to value: whereas innovation used to be somebody’s business, it now is everybody’s business: (Davila et al. 2012). Thirdly, this paper adds to the discussion on whether the formulation of strategy and metrics in innovation are too focused on finding the ‘right’ metrics in the strategy-as-planning and not focused enough on the occurrence of organizational errors in usage and how these can be managed in strategy-as-learning (Edmondson and Verdin 2018; Harris and Tayler 2019; Richtnér et al. 2017).

Besides its contributions, this paper provides both limitations and possibilities for future research. This study only involves a single case study at one organization and can be seen as explanatory with limited generalizability. The challenges found between KPIs, innovation and organizational errors provide researchers with possibilities to further explore and explain (1) the occurrence of organizational errors in innovation KPIs, (2) the variation in definition and interpretation of innovation not only among different representatives of the same company, but also among different organizations and companies from various industries, (3) how the occurrence of organizational errors in innovation KPIs can influence present and future innovation strategy and KPI reviews, updates and design, (4) and how multiple organizational level analysis can shed light on the occurrence of organizational errors in innovation KPIs, and how this in turn influences future innovation strategy and KPI reviews, updates and design. The implication for practitioners is that innovation KPIs are not precise metrics but should be seen as estimates with organizational errors. Whether or not these innovation KPIs can be used as tools to turn innovation into competitive advantage largely depends on whether wrong is right. Future research should focus on the metrics that are implemented and actually in use, as this future path would show the function and dysfunction that organizational errors in innovation KPIs can have.