Governance of Web-Based Idea Management System Rewards: From the Perspective of Open Innovation

Abstract

Digitisation of business processes has attracted practitioner attention across a wide range of industries as it enables enterprises to better manage their processes and improve results. The rate of digitisation has grown over the last decade and has become essential for enterprises to handle unforeseen disruptions and promote better management of resources. The COVID-19 pandemic is one example of a recent major disruption that caught enterprises off-guard and had a major impact on their value chains. The adoption of digitisation of business processes has significantly sped up to improve enterprise and their value chain resilience and enable enterprises to survive and potentially thrive in today’s uncertain environment. Through this movement, innovative companies have embraced idea management to develop new business models and achieve competitive advantage. One of the major digital tools that support enterprises in their idea management efforts is web-based idea management systems. This study aims to show how enterprises can manage the rewards of platforms, such as web-based idea management systems, to achieve a first-in-class quality and quantity of ideas sourced from the stakeholders involved in these platform-based ecosystems. To reach this aim, a global survey study was conducted involving over 400 responses from enterprises that use web-based idea management systems and consequent results were analysed. The impact of intrinsic and extrinsic rewards on the results of enterprises is a well-researched area. To further verify the results, the authors have conducted ten expert interviews and a qualitative analysis of a data set sourced from a database that contained 129 company entries that utilise web-based idea management systems and was available for this research that was conducted over a two-year period. Results strongly indicate that mixed and financial reward type tasks result in a higher quantity of ideas created, however, higher idea quality—i.e., ideas selected—is higher for financial reward tasks. The research highlights that there could be differences between different web-based idea management systems application groups.

1. Introduction

Open innovation helps companies to manage knowledge across organisations’ boundaries [1] due to the continuous inflows and outflows of knowledge [2,3], including creativity [4]. This makes open innovation an essential part of modern-day innovation ecosystems [5,6]. Furthermore, the growing popularity of open innovation, digitalisation and increasing remote work (accelerated by COVID-19) has led to the application of different virtual tools that help organisations to manage their innovation ecosystems and facilitate open innovation. Knowledge sharing is one of the key elements in rewards-based idea management (IM), which could be defined as outside-in open innovation. According to Markovic et al. [7], continuous improvement of employee and dedicated team knowledge and skills are likely to bring behavioural changes, increasing the level of outside-in open innovation.

Digitalisation changes the nature of the information flow, therefore digital convergence becomes imperative [8], including open innovations, technology-related readiness [9] and web-based communication aspects [10]. There are different information management systems, such as electronic messaging systems [11], collaborative systems [12], group decision support systems, etc. By leveraging the capabilities of information management systems, innovative companies have embraced IM to develop new growing businesses and web-based idea management systems (IMS). Web-based IMS is one of the digital tools that can facilitate enterprises in their IM efforts [13]. IMS provides organisations with a systematic and manageable process of IM, whereas IM [14] is one of the first steps in the innovation process [15,16], which could be defined as a two-step process—idea generation and evaluation.

There are several business models that can boost innovations. Figure 1 [17] presents a typology of ten business model families for innovations. While each business model family is unique, it does not prevent businesses from combining and mixing these models. In this paper, the focus is on the IM business model that provides potential benefits and opportunities when combined with any of the other models, as shown in Figure 1.

There are different types of IMS, for example, an internal type (mainly, the involvement of employees and internal stakeholders), an external type (mainly, the involvement of external parties—clients, crowds, academics, etc.) and a mixed type, which combines both the internal and external type characteristics. In general, IMS is a multi-stakeholder (platform-based) ecosystem for innovations. It is advantageous, as nowadays there is a growing trend toward multi-stakeholder—platform-based—collaborative ecosystems [18].

Web-based IMS are applied by many well-known and globally recognised organisations, such as Panasonic, Virgin, Sony, Fujitsu, Volvo, NASA, European Commission, etc. There is an abundance of cases that prove the positive effects a web-based IMS can have on an organisation’s performance [19]. In practice, during the IM process, enterprises often face several challenges when it comes to managing idea generation, evaluation and other management aspects of IM [20,21]. Although there is a great variety of web-based IMS and has shown they are successfully applied by well-known organisations, the situation that was described in a legendary publication Capturing the Complexity in Advanced Technology Use: Adaptive Structuration Theory by DeSanctis and Poole (1994) is still topical, as both the developers and the users of these information management systems see the potential of these systems. However, positive results rarely occur consistently because of the way organisations adopt these systems [22]. In this research, the authors will contribute to closing this gap by answering the following question: What types of rewards have to be applied to achieve better results, and in this case—what rewards lead to a higher idea quality and quantity? See Figure 2.

This research will have both academic and practical contributions by filling the following gaps:

Knowledge gap: No research looks at intrinsic and extrinsic rewards leading to better results in a web-based IMS application type context. There is existing research on rewards in IM within innovation and descriptive articles, however, in this research, authors specifically look at web-based (digital) IMS and IMS application types and therefore this research has a novelty factor.

Practical knowledge gap: Web-based IMS application types in professional practices are not covered in existing research and there are only a few articles where researchers look at how different application types influence IM results. For companies, it is important to understand how they can apply web-based IMS more effectively and the key aspects they need to pay attention to in the IMS application. Within this research, the authors focus on the rewarding mechanisms to facilitate better results. Thus, the authors will provide practitioners with an answer to the following question: What rewards should be used based on the web-based IMS application type? By answering this question, practitioners will be able to make better-informed decisions based on the IMS application type and desired IM outcome.

Empirical Gap: There are literature reviews that include descriptions of the importance of rewards in web-based IMS; however, there is no further focus or elaboration through empirical research.

Theory Gap: There are theories about rewards and the adaptation of different information management systems, but in this paper, the authors will try to fill a gap identified in the literature review—how the adaptation of different reward mechanisms influences results based on the IMS application type (see in Figure 3. An adaptive structuration theory (AST) is considered an appropriate theoretical framework for this study, as it could reveal how structures and systems interact and present the results. AST states that the technology application by an organisation is determined by several forces: the technology, the organisational systems, the task and the way structures emerge and update in the appropriation process [23]. From the start, AST has been used to study decision systems, and it is a strong theoretical framework that can be used to study other more advanced information technologies [24], such as web-based IMS. Although AST provides the framework to understand the interaction between systems and structures, there is little evidence on how specific ICT tool applications may affect results associated with structures [25]. Here, the structures will be web-based IMS application types, focusing on outputs related to idea quality and idea quantity, while the rewards will be the systems that will interact with the structure (See Figure 3).

Despite the practical experience and authors’ prior research base for this topic, the existing literature on IM lacks the evaluation of different reward system results. In the literature, there is no common view of what is the best kind of motivation for encouraging people to provide idea suggestions [26]. Some researchers have based their assumptions on that intrinsic motivation is very important, such as the possibility to express ideas, apply and improve creativity, learn something new, see the ideas being applied, or improve conditions through creating new ideas [27,28]. In contrast, other researchers state the importance of an extrinsic motivation that is based on monetary rewards [29,30]. This study aims to show how enterprises can manage the rewards of such platforms as web-based IMS to achieve a first-in-class quality and quantity of ideas sourced from the stakeholders involved in these platform-based ecosystems. To reach this aim, a global survey study was conducted of over 400 responses from enterprises that use web-based IMS and the subsequent results were analysed. To verify the results, the authors have conducted ten expert interviews and a qualitative analysis of a data set sourced from a database that contained 129 company entries that utilise web-based IMS and was available for this research that was conducted over a two-year period. The focus of this study is on an organisational level, researching web-based IMS applications within the AST framework. In the next sub-section, the authors outline the main IMS and reward literature trends and verify the research novelty.

Figure 4 shows the general steps of the research design. The research is based on a systematic literature review that supports the research of existing theories and identifies research gaps. Based on the systematic literature review, the authors have defined the research question and selected the appropriate research methods. From the survey results, the authors defined sub-questions to be further researched. The triangulation approach was used by authors to verify results by using three different data collection methods.

2. Theoretical Framework

Many researchers not only concentrate on exploring the different information technologies but also their applications and impact. According to the basic scanning of seven scientific databases, there are over 100 papers, proceedings and notes that aim to explain it. There are two main trends in the literature. The first one is to explore the overall IT management and the different use aspects, for example, malleable IT uses [31], a technology adaption in groups [32], IT management responsibilities [33], an information system project control [34], social inclusion and IT [35], ICT and the wellbeing of nations [36], IT impact on performing companies [37], etc. The aim of these studies is to explore specific aspects of ICT. The second trend is to explore the specific type of IT, for example, group support systems [38], crowdsourcing [39], gaming platforms [40], knowledge management systems [41], etc. The aim of these studies is to research the specific ICT types and related aspects. In this paper, the authors rely on the second construct—exploring the specific aspect related to the specific ICT because it is a more focused view on the issue. The growing number of publications on IM has shown that there is an interest in the academic community about IM, however, still, there is a lack of academic research papers on many crucial aspects of IM. IM and IMS have been researched by various researchers (e.g., Brem, Bothos, Bjork, Barczak, Westerski, Bakker) and the gained results are quite similar, emphasising the potential of IM and IMS. Most researchers examine IM and its features, while the focus on IMS, and especially on web-based IMS, is rare in the scientific literature. The scarcely available literature forms a theoretical basis for future research. However, it must be noted that there is little theoretical guidance on how the application of different rewards influences the results of different IMS application types.

2.1. IMS Application Types and Outputs

The paper is built on the assumption that IMS is a tool, a tool kit or a complex system which facilitates IM, while IM is a systematic and manageable process of idea generation, evaluation and the repeated idea generation and evaluation if it is needed [42]. There are different web-based IMSs available on the market. The authors apply two classifications of IMS as IMS application types: based on a task focus and on an involvement focus.

According to the task focus, all IMSs could be divided into active or passive IMS. The passive IMS are without a task focus (participants are welcome to submit all kinds of ideas without a specific task); however, in an active IMS, the application process provides the opportunity to create separate tasks (participants generate ideas for the specific task or tasks). Based on the involvement focus, all IMS could be divided into internal, external or mixed focus. Internal focus provides only the involvement of idea creators from within the organisation’s bounds (mostly employees, separate departments, etc.); external IMS implies the possibility to involve external idea creators (e.g., crowds, clients, experts, etc.) and the mixed focus combines both approaches by involving both internal and external stakeholders in the IM process.

IM results can be measured in two ways: based on idea quality (ideas selected for further development/implementation), and idea quantity (ideas created). The quality of ideas is the average amount of selected ideas for further development, while idea quantity is the number of ideas created [43,44,45,46,47]. Rewards are an important part of IMS as IM is a suggestion process, thus by involving rewards [48] the enterprises are more capable of motivating idea submissions and better results [49,50].

In previous research, authors have conducted a systematic literature review on the available IM literature. The theoretical framework development is based on a theoretical research method, namely, by use of a critical literature review. Data collection was conducted in 4 stages: (1) Scientific database research was conducted to explore the literature using the following terms (conducted in Q4, 2022): “idea management”, “idea management systems”; each of these terms was looked up in four different scientific databases; (2) only the literature directly about IM, IMS was selected; (3) we excluded all articles that were duplicates and/or articles that did not have full-text availability; (4) the selected literature was applied in a further systematic literature analysis to find the research gaps. Detailed literature source counts at the different stages are reflected in

Table 1. Idea management literature review.

Stages:	Stage 1—in Article Title and/or Keywords in the Article:		Stage 2—Directly about (Full Text Available):		Stage 3—Unique Sources:
	IM	IMS	IM	IMS	IM	IMS
Scopus	66,459	28,860	126	42	234	86
Google Scholar	5,200,000	5,080,000	82	29
Ebsco	94	4	12	4
Web of Science	406	38	62	13
Sum:	5,266,959	5,108,902	282	88

Every term was searched for in the different databases, based on the scientific database availability.

.

By analysing the content and future research directions of the articles, it was concluded that there are only a few papers where rewards are described; however, no paper focuses on web-based IMS types and the impact of rewards on idea quality and/or quantity.

2.2. Rewards

The role of reward systems in innovation is a well-researched field according to the number of papers in the scientific database Scopus—with over 1427 articles at the time of this research. Authors have started the analysis of publications with an initial data collection through the search of pre-defined terms: TITLE-ABS-KEY (Innovation rewards) AND (LIMIT-TO (SUBJAREA, “BUSI”) OR LIMIT-TO (SUBJAREA, “SOCI”)). It means the author limited the literature research to the social sciences and business management journals. The search results were filtered and then the refined results were used for further network analysis. The network analysis provides qualitative scientific explorations with a focus on publication correlations and their impact on the publisher network. In the network analysis, keywords in the titles were appraised. The initial search was conducted in the Scopus database to collect the publications for network analysis. For the network analysis, the visualisation tool VOSViewer was applied. It is effective at clustering and visualising data, and therefore, it is used for the network presentation of keyword correlations [51]. The whole procedure is visible in Figure 5.

To identify new trends in the field network, an analysis of the keywords was executed, thus, keywords in the title, abstract and keywords by using the following function: “(TITLE-ABS-KEY (“innovation rewards”))”. This was limited to the social sciences, business, management and accounting. This led to identifying 1427 documents. For further investigation, “.RIS” and “.CVS” files were exported from the Scopus database for further analysis.

There is no doubt that the question about rewards in innovation processes is gaining traction year by year, based on the growing number of papers. See the Scopus database publication number by the years in Figure 6.

According to the Scopus database, the most influential authors are Griffin, A., Chandy, R.K., Ford, E.W., Mazzucato, M., Prabhu, J.C., Sergeeva, N., Vossen, A. This differs from the results of the VosViewer network analysis where the most cited articles and linkages are created by Hauser J., Lin, H.J., West, J. A total of 238 works were clustered in 27 clusters and 316 links (see Figure 7).

The identified keywords were investigated using the tool VOSviewer to detect their interconnections. It is possible to observe the development of the topic and deduce new perspectives. The results can be seen in Figure 8.

The system of linked keywords is represented as a network. The colours point to the clusters in which most of the keywords of a group occur together. The size of the node shows the connection degree of the keywords. A total of 509 keywords included three clusters, 55,492 links found, and a total link strength of 142,980. As seen, the most affiliating keyword is “employee”. This is also the most connected keyword in its cluster with the terms “team”, and “dimension”. The second-largest node is the keyword “incentive”. The third is “platform” and “community”. For the Scopus database, the main keywords show other words (see Figure 9) for example: “innovation”, “human”, “reward”, “motivation”, and “knowledge management” can be explained by the meaning of the network analysis—not the frequency but the weighting of the connections is emphasised.

The first cluster shows that the research focuses on internal reward systems where employees are involved in R&D and new product development.

The second cluster reveals incentives aspects of this topic, thus, there are different things related to motivation, for example, finances, IPR, positions and achievements.

The third cluster demonstrates platforms, community, ICT and their differences. As strong keywords, there were differences that highlighted the importance of researching these differences.

Network analysis revealed that there are no studies that link ICT or platforms with incentives and elements from the first cluster. It provides additional grounds to prove that a gap exists: there is no research about web-based IMS (platform, ICT) types (differences) and results (achievements). The same conclusion came from the qualitative research reading of all 1427 article abstracts—there is no research looking at these specific elements.

Rewards in IM are more important for idea creator motivation than anything else [52,53,54]. There are different rewards, for example, extrinsic rewards (compliance-based), such as monetary benefits, a salary, a bonus or commission. In this research, extrinsic rewards are defined as financial rewards. In contrast, there are intrinsic rewards (purpose-based), such as joy, meaningfulness, recognition, reputation, etc. In this paper, intrinsic rewards are defined as non-financial rewards. These types of rewards could be also defined as tangible and non-tangible ones [55]. See Figure 10.

A common factor in all these theories is that they note the interlinks between intrinsic and extrinsic types of rewards. This paper will research which of these rewards bring the higher idea of quality and quantity or what happens when these rewards are mixed. Rewards are a very important part of IMS because they help to boost motivation and the feeling of appreciation [56]. There is still a discussion about which motivation is the most important—an extrinsic or an intrinsic motivation? For example, some researchers declare that extrinsic motivation is the most important since it is based on the possible extrinsic rewards grounded on the IM outcome [30]. Others have a contrasting opinion that extrinsic motivation has a negative influence on creativity, leading to the idea that it could lead to a decrease in the quantity and quality of ideas [57]. In this paper, the authors will evaluate this assumption, since there is research that has come to opposite results indicating that extrinsic motivation can facilitate creativity [58].

A proper reward is critical, as it could negatively affect the self-determination of IM participants [59]. The most common distinct reward is a financial one [29], however, there could be some bureaucratic limitations to delivering financial cash rewards [30]. Therefore, the alternatives are vacations or tickets to interesting events, vouchers to purchase goods or services, profit sharing or some other consumer goods, such as cameras, bags [60,61], etc. Intrinsic rewards also motivate participants to become involved in a web-based IMS application, for example, the best idea creator receives recognition (a name and a photo in the frame or on the company’s website, in a journal, etc.) or recognition through a public award ceremony [29,62].

Rewards are an important tool to encourage crowdsourcing intrapreneurial contributions and according to the previous studies, only “promotion within the organisation” (mixed- extrinsic and intrinsic) and “monetary bonus rewards” (extrinsic/ financial) had a positive and significant influence on performance [63]. This paper showed the internal, active and mixed IM types and the different reward connections. This paper also expands upon existing research by using the IMS application type in the process to measure the impact of different rewards on direct IMS results (idea quality and quantity). However, external crowdsourcing case research has proved that intrinsic rewards impact weakens when the level of extrinsic (financial) incentives is high [64]. However, these and other papers do not reveal whether these differences are based on the IMS application type by comparing the impact of rewards on IM results.

Based on previously mentioned concerns, the authors deliver the main composite hypothesis.

Hypothesis H1.

Different reward type applications in different web-based IMS application types have different impacts on idea quality (ideas selected) and idea quantity (ideas created).

The main elements of the research are described in Figure 11.

In this paper, reward receivers are most commonly described as idea creators. However, there are other possibilities to further provide rewards to idea evaluators. Idea creators could be one group of reward receivers, however, there are two opportunities to receive a reward as a group or as an individual. How can a reward be split properly in a group? Some researchers state rewards should be equally divided among all group members or based on the individual team member’s contribution degree [62].

The rewards could be distributed at the different stages of the process: participants could be rewarded directly after the selection or after the successful implementation of the idea [65], but also during the process of IM itself.

Figure 12 demonstrates both the elements that were included and left out of the research. The elements highlighted in grey are the included elements, while the elements in white are the limitations of the research and serve as an indication of future research directions.

There are several different theories reflecting extrinsic and intrinsic rewards and motivation [66,67,68,69,70,71].

The main composite hypothesis for this study is as follows: Different rewards type applications in different web-based IMS application types have a different impact on idea quality (ideas selected) and idea quantity (ideas created). Based on the previously described theoretical grounds and the aim of the research, five sub-hypotheses were created:

• The use of just non-financial rewards provides a larger number of ideas created per task than the use of only financial ones.
• The use of just non-financial rewards provides a larger number of ideas selected per task than the use of only financial ones.
• The use of mixed (financial and non-financial) rewards provides a larger number of ideas created per task than the use of only financial or non-financial ones.
• The use of mixed (financial and non-financial) rewards provides a larger number of ideas selected per task than the use of only financial or non-financial ones.
• The impact of mixed (financial and non-financial) rewards on the number of ideas generated and selected is different for different IMS.

To answer the main research question “How to reward better to gain higher idea quality and quantity?” the authors have defined the sub-questions:

• Do just non-financial rewards provide a larger number of ideas created/selected per task than only financial ones (hypothesis 1–2)?
• Do mixed rewards provide a larger number of ideas created/selected per task than only financial/non-financial ones (hypothesis 3–4)?
• Does the impact of mixed (financial and non-financial) rewards on the number of ideas generated and selected differ depending on the different IMS (hypothesis 5)?

To answer these three questions and to test the five hypotheses, two main results of IMS are included (idea quality, idea quantity); two main classifications of web-based IMS have been applied (classification based on the focus: an active and passive IMS; the classification based on involved sources: internal, external and mixed IMS). See the research framework in Figure 13.

Interestingly, prior research proves that managers understand the importance of knowledge sharing among their employees, but the expected rewards are not significantly related to the attitude towards their knowledge sharing (see Figure 14). Scientists prove that a positive attitude to knowledge sharing leads to more actual knowledge sharing behaviours [72]. Further research could focus on the attitudes and competencies of IMS. In this paper, the authors limit their research to non-financial and financial rewards.

In the next section, the methodology is described on how to answer the question: How to reward better by financial, non-financial or mixed rewards to reach higher idea quantity and quality depending on IMS application type?

3. Methods

3.1. Quantitative Research Instrument and Data Collection

How to reward better? To answer this question, the authors have conducted a global survey among enterprises that have been applying commercially available web-based IMS. The research is grounded in the adaptive structuration theory (AST) framework. In this paper, the authors have analysed in detail the elements of the rewards. This survey has allowed the pooling of data on IMS corresponding to the type of AST in 8 blocks: (1) IMS; (2) tasks; (3) an organisation system; (4) the adaptation and the type of use; (5) IMS results; (6) benefits; (7) new structures; (8) problems with IMS. In previous research, authors have analysed the part of the tasks [74] and benefited the part of creativity [75]. In this paper, the authors apply the organisation system questions about rewards, the use and IMS results.

Table 2. Survey questions about quantity and quality.

What Is the Average Number of Ideas Created Per Task?
Using internal idea management	None Up to 10 11–100 101–1000 1001–5000 5001–10,000 More than 10,001
Using external idea management
Using mixed idea management
Using active idea management (focused task)
Using passive idea management (unfocused task)
What Is the Average Number of Ideas Created Per Task?
Using internal idea management	None 1–5 6–10 11–20 21–50 51–99 More than 100
Using external idea management
Using mixed idea management
Using active idea management (focused task)
Using passive idea management (unfocused task)

Source: created by the authors.

shows the block questions about the results of idea quality and ideal quantity. An additional question was created related to the reward types for these cases by evaluating them from strongly disagree to strongly agree.

The survey was conducted on the survey platform “The QuestBack” (https://www.unipark.com/ (accessed on 2 April 2022)) created by UNIPARK. This platform was chosen because (1) it focused on academic surveys; (2) it is widely recommended by world-class researchers; (3) it provides data security required by IMS representatives—it is a BSI-certified data centre in accordance with ISO 27001; (4) it complies with the requirements of the EU General Data Protection Regulation. In

Table 3. Data collection and analysis steps.

Data Gathering Method	Data Analysis	Period	Steps
Survey of the enterprises that apply web-based IMS (Sample size n > 400)	Statistical analysis	3rd and 4th quarter of 2020	Survey development is based on literature analysis and developed classifications. 3 Round survey test (data not included in the analysis). Survey distribution to 107 web-based IMS developers that they could distribute to their clients. 400 valid surveys. Standard deviations to evaluate the data consistency for the analysis. Data analysis through selected methods.

Source: created by the authors.

see the main data collection steps.

There was no data on how many organisations globally apply web-based IMS. This is the reason why the authors have created separate research to summarise the information about the existing commercially available web-based IMS and collected information about the approximate number of their clients. According to that research, these 107 IMS companies have approximately 120,000 clients (companies that apply web-based IMS). In the research analysis paper, 447 responses were included from enterprises applying web-based IMS globally (n > 400 answers were collected, but not all of them were appropriate for analysis because of missing data or filling errors).

3.2. Research Framework and Data Analysis

Survey data were analysed by use of analytical statistical methods. For the study, the answers to the survey respondents were sorted into three groups depending on the answer regarding the type of the reward applied in a company:

• Confirms (agree or strongly agree) the mixed-use of financial and non-financial rewards (F&N);
• Confirms (agree or strongly agree) the use of only financial rewards (FR);
• Confirms (agree or strongly agree) the use of only non-financial rewards (NR).

The Granger causality test was performed using the R function “grangertest” to assess the statistical significance of the rewards causality on ideas created and selected.

As one can see from

Table 4. Granger causality test statistics by IMS and reward types.

IMS Type	Reward	Ideas	F-Stat	p-Value	Significance
Active	Financial	Created	12.963	0.0004	***
Passive	Financial	Created	4.0121	0.0452	*
Internal	Financial	Created	3.4522	0.0638	.
External	Financial	Created	51.776	0.0000	***
Mixed	Financial	Created	6.0445	0.0143	*
Active	Financial	Selected	7.8796	0.0052	**
Passive	Financial	Selected	3.6451	0.0553	.
Internal	Financial	Selected	8.7913	0.0032	**
External	Financial	Selected	10.210	0.0015	**
Mixed	Financial	Selected	5.5665	0.0187	*
Active	Non-financial	Created	9.3584	0.0023	**
Passive	Non-financial	Created	13.103	0.0003	***
Internal	Non-financial	Created	6.8891	0.0090	**
External	Non-financial	Created	4.6263	0.0323	*
Mixed	Non-financial	Created	14.291	0.0002	***
Active	Non-financial	Selected	10.882	0.0010	**
Passive	Non-financial	Selected	6.8975	0.0089	**
Internal	Non-financial	Selected	6.4158	0.0116	*
External	Non-financial	Selected	3.7024	0.0549	.
Mixed	Non-financial	Selected	4.3474	0.0376	*
Active	Mixed	Created	14.219	0.0002	***
Passive	Mixed	Created	4.1253	0.0425	*
Internal	Mixed	Created	3.7841	0.0512	.
External	Mixed	Created	25.217	0.0000	***
Mixed	Mixed	Created	17.287	0.0000	***
Active	Mixed	Selected	7.0893	0.0081	**
Passive	Mixed	Selected	3.8753	0.0487	*
Internal	Mixed	Selected	3.0621	0.0760	.
External	Mixed	Selected	3.4441	0.0641	.
Mixed	Mixed	Selected	7.4953	0.0064	**

Significance codes: *** = 0.001; ** = 0.01; * = 0.05; . = 0.1. Source: created by the authors.

, in the absolute majority of cases, the impact of the rewards type on the number of ideas generated and selected is confirmed at the confidence level of 0.95, while in other cases the confidence level is slightly lower but not below 0.92. It can be considered that sufficient justification has been obtained to carry out a comparative analysis of the impact of the reward types on the number of ideas generated and selected.

Based on the survey results, main descriptive statistics were calculated. The mean of grouped data was calculated as follows:

$$\overline{x}=\frac{{{\displaystyle \sum }}_1^K{f}_{i }\ast m_i}{n}$$

(1)

where $m_i$—ith class (group) midpoint, $f_i$—frequency of the ith class (interval), $n$—sample size, $n={{\displaystyle \sum }}_1^K{f}_i$.

The standard deviation of grouped data was calculated as follows:

$$s=\sqrt{\frac{s^2}{n}\ast \left(1-\frac{n}{N}\right)}$$

(2)

where $s^2=\frac{1}{n-1}{{\displaystyle \sum }}_1^K{f}_{i }\ast {\left(m_i-\overline{x}\right)}^2$, $m_i$—ith class (group) midpoint.

As sample sizes and standard deviations differ, the test statistics were calculated as follows [76]:

$$t=\frac{{\overline{x}}_{i }-{\overline{x}}_j}{\sqrt{\frac{s_i^2}{n_i}+\frac{s_j^2}{n_j}}}$$

(3)

where ${\overline{x}}_{i }$ and ${\overline{x}}_j$ are the means of comparable group variables, $s_i^2$ and $s_j^2$ are the variance (standard deviation squared) of comparable group variables, $n_i$ and $n_j$ are the sample sizes of comparable groups and are compared with t-statistics critical values $t_{\nu ,\alpha }$, where the degrees of freedom $\left(df\right)$ for the t-test statistics were calculated as follows:

$$df=\frac{{\left[\left(\frac{s_i^2}{n_i}\right)+\left(\frac{s_j^2}{n_j}\right)\right]}^2}{\frac{{\left(\frac{s_i^2}{n_i}\right)}^2}{\left(n_i-1\right)}+\frac{{\left(\frac{s_j^2}{n_j}\right)}^2}{\left(n_j-1\right)}}$$

(4)

Additionally, p-values were calculated for the given test statistics and the degrees of freedom. The p-value is the probability of obtaining a value of the test statistic as extreme or as more extreme than the actual value obtained when the null hypothesis is true. Thus, the p-value is the smallest significance level at which a null hypothesis can be rejected, given the observed sample statistic.

The decision rule is to reject H₀ in favour of H_A, if the t-statistic (t_stat) is larger than the t-critical $\left(t_{\alpha }\right)$ values for ($\alpha =0.05$):

$$t_{\mathrm{stat}} > t_{\alpha }$$

(5)

In the next chapter, the authors of the article apply these methods to reveal the secrets about the rewards of web-based IMS.

3.3. Triangulation

For triangulation to verify the results, the authors have conducted 10 expert interviews with IM and design thinking experts. Five questions were included related to the main five IMS application types, and each question had two sub-questions about what rewards create greater idea quality and quantity from their experience. The last question was looking at whether they agree with the statistical data analysis results from this research, or they do not think that it matches with reality. All interview data were anonymised.

Expert criteria: (1) A person with over 3-years of practical experience in IM as a professional that provides these services to enterprises or is the responsible person for it in the enterprise; (2) highest education in related fields. In

Table 5. Summary of informants.

Identifier	Sector	Position	Experience (Years)	Education
INT1	Academical, private	Innovation consultant	10	Business administration
INT2	Private	Innovation Lead	4	Economics
INT3	Academical, private	Innovation consultant	5	Business administration
INT4	Public, academical	Innovation consultant	6	Economics
INT5	Private	Innovation Lead	4	Business administration
INT6	Academical, private	Innovation consultant	7	Business administration
INT7	Private, public	Innovation Lead	8	Economics
INT8	Private	Innovation consultant	9	Business administration
INT9	Private	Innovation Lead	15	Business administration
INT10	Academical, private	Innovation consultant	11	Business administration/Economics

Source: created by the authors.

, see the information about informants.

The main question fields are represented in

Table 6. Expert interview question fields.

Main Question Field by IMS Type	Sub-Questions about Rewards in Relation to the Results
Active IMS application	Best rewards to increase idea quantity Best rewards to increase idea quality
Passive IMS application
Internal IMS application
External IMS application
Mixed IMS application

Source: created by the authors.

. According to these questions, data analysis was made—it was used as the code map.

Interviews were conducted in a one-on-one and Zoom settings. The interviews ranged from 45 min to 65 min long and the data were transcribed to prepare for analysis. The authors used a semi-structured interview guide. The semi-structured design was selected as it provides consistency between interviews while allowing a natural flow conducive to an open discussion and the emergence of unanticipated topics.

The authors gained access to one web-based IMS entry of 129 company tasks in the period of 2018–2020, with the main limitation that data could be shown only as summary results about rewards, and that the results in the five IMS application types, name of IMS and their clients could not be shared publicly. The main analysis fields are represented in

Table 7. Databases analysis.

Main Analysis Fields	Sub-Questions about Rewards in Relation to Results
Active IMS application	Type of rewards Idea quality Idea quantity
Passive IMS application
Internal IMS application
External IMS application
Mixed IMS application

Source: created by the authors.

.

4. Results

How to reward better? The answer is provided in the results. Based on the survey results, the main descriptive statistics were calculated (See

Table 8. Descriptive statistics on ideas created and selected by the rewards’ types.

Reward’s Type	Observations	Ideas Created		Ideas Selected
		Mean	Standard Deviation	Mean	Standard Deviation
Non-financial	207	1616.3	2569.5	9.4	14.5
Financial	51	3171.2	4048.0	19.2	31.3
Mixed	171	3202.8	3270.2	12.6	14.9

Source: created by the authors.

) to characterise the average number of ideas created and ideas selected in the tasks by the different reward types.

The data represents that the higher the number of ideas is created in the mixed and financial reward type tasks, the idea of quality (ideas selected) is higher in the financial reward tasks. It also gives the statistical grounds for testing the pre-defined hypothesis.

The next question is whether there are differences according to the IMS application type. Figure 15 demonstrates that there are differences in the average number of ideas created per task depending on the reward and IMS types.

Figure 16 demonstrates that there are differences in the average number of ideas selected per task depending on the reward and IMS types.

4.1. Do Just Non-Financial Rewards Provide a Larger Number of Ideas Created/Selected per Task Than Only Financial Ones?

The answer to this question is provided by testing the first two hypotheses:

1. The use of just non-financial rewards provides a larger number of ideas created per task than the use of only financial ones.

2. The use of just non-financial rewards provides a larger number of ideas selected per task than the use of only financial ones.

Calculated t-statistics, degrees of freedom (df), critical values (t_c) and p-values are aggregated in the following

Table 9. Hypothesis test statistics for non-financial and financial rewards comparison.

Impact Type	Non-Financial Reward		Financial Reward		t-Statistic	df	t-Critical	p-Value
	Meani	ni	Meanj	nj
Ideas created	1616.3	207	3171.2	51	−2.6162	60.28	2.0003	0.9944
Ideas selected	9.4	207	19.2	51	−2.1720	55.37	2.0040	0.9829

Source: created by the authors.

.

As the calculated t-statistics are not larger than the critical values for the comparable reward types pairs, the authors cannot reject the corresponding null hypothesis and conclude that the sample data does not provide enough evidence to support the alternative hypothesis—there is little chance that the use of non-financial rewards alone will provide a larger number of ideas created and a larger number of ideas selected than the use of financial rewards alone (ca. 0.6% and 1.7%, respectively).

4.2. Do Mixed Rewards Provide a Larger Number of Ideas Created/Selected per Task Than Only Financial/Non-Financial Ones?

The answer to this question is provided by testing the second two hypotheses:

3. The use of mixed (financial and non-financial) rewards provides a larger number of ideas created per task than the use of only financial or non-financial rewards.

4. The use of mixed (financial and non-financial) rewards provides a larger number of ideas selected per task than the use of only financial or non-financial rewards.

The following

Table 10. Hypothesis test statistics for mixed rewards comparison with non-financial and financial rewards.

Impact Type	Comparable Rewards Pairs	Meani	ni	Meanj	nj	t-Statistic	df	t-Critical	p-Value
Ideas created	F&N vs. FR	3202.8	171	3171.2	51	0.0511	70.57	1.9944	0.5203
	F&N vs. NR	3202.8	171	1616.3	207	5.1626	319.1	1.9674	0.0000
Ideas selected	F&N vs. FR	12.6	171	19.2	51	−1.4509	56.94	2.0032	0.9238
	F&N vs. NR	12.6	171	9.4	207	2.1001	358.3	1.9666	0.0182

Source: created by the authors.

summarises the statistics to test the third and fourth hypotheses. As can be seen, the use of mixed rewards increases the number of ideas generated and selected. However, it is not clear whether these increases are statistically significant, or if they can be explained by a sampling error.

As the calculated t-statistics are not larger than the critical values for the mixed and financial rewards, the authors cannot reject the corresponding null hypothesis and conclude that the sample data does not provide enough evidence to support the alternative hypothesis—the chance that the use of mixed rewards will provide a higher number of ideas created is only slightly higher than with a financial reward alone, and there is a little chance that the use of mixed rewards will provide a larger number of ideas selected than the use of financial rewards alone (ca. 7.6%).

As the calculated t-statistics are larger than the critical values for the mixed and non-financial rewards, the authors would reject the corresponding null hypothesis and conclude that the sample data provide strong evidence to support the alternative hypothesis. The conclusions imply that the application of mixed rewards has resulted in more ideas created than the application of only non-financial rewards and that the application of mixed rewards has resulted in more ideas selected than the application of only non-financial rewards is supported also by low p-values (<0.02).

4.3. Does the Impact of the Mixed (Financial and Non-Financial) Rewards on the Number of Ideas Generated and Selected Differ Depending on the Different Types of IMS?

The following

Table 11. Mean values of ideas created and selected by the rewards’ types.

IMS Type	Means of Ideas Created				Means of Ideas Selected
	F&N	FR	NR	Total	F&N	FR	NR	Total
Active	5240.6	4948.7	2628.5	3694.9	21.4	24.5	15.6	17.7
Passive	386.1	2119.4	385.2	633.3	3.4	14.4	4.4	5.3
Internal	821.2	1710.5	1083.9	1016.9	11.5	13.9	10.8	10.9
External	4841.9	5993.8	1572.5	3268.5	7.9	21.2	4.6	7.7
Mixed	4724.3	1083.3	2411.5	2930.8	18.8	22.0	11.6	14.6

Source: created by the authors. Dominating reward style is highlighted.

summarises the statistics on the impact of different rewards types on the number of ideas created and selected for different types of IMS. As can be seen, the most significant effect of the mixed remuneration is on the average number of ideas created in cases of the application of mixed, external and active IMS. The mixed remuneration does not affect the number of ideas created in cases of the application of passive and internal IMS. In terms of the number of ideas selected, it is less than with ideas created, however, it still has a significant impact in the application’s case of mixed and active IMS.

The answer to this question is provided by testing the fifth hypothesis:

5. The impact of mixed (financial and non-financial) rewards on the number of ideas generated and selected is different depending on the different types of IMS.

The following

Table 12. Hypothesis test statistics for mixed rewards impact by IMS types.

Impact Type	Comparable IMS Type Pairs	Meani	Meanj	t-Statistic	df	t-Critical	p-Value
Ideas created	Active vs. External	5240.6	4841.9	1.1994	338.7	1.9670	0.1156
	Active vs. Mixed	5240.6	4724.3	1.4978	340.0	1.9670	0.0676
Ideas selected	Active vs. Mixed	21.4	18.8	1.4140	324.3	1.9673	0.0792

Source: created by the authors.

summarises the statistics to test the fifth hypothesis. Though it is not clear whether these differences in mean values of ideas are created and selected and are statistically significant, or if they can be explained by a sampling error.

As the calculated t-statistics are not larger than the critical value for the comparable IMS types pairs, the authors cannot reject the corresponding null hypothesis and conclude that the sample data does not provide enough evidence to support the alternative hypothesis—the difference in the mean values of ideas created when mixed rewards are applied in the cases of active, external or mixed IMS types usage, and the difference in the mean values of ideas selected when active or mixed IMS types are applied, are not statistically significant at a 95% confidence level. However, the possibility that in the case of a mixed reward usage with active IMS applications will be generated and selected, a larger number of ideas is high enough (>92%).

4.4. Expert Interview Results

Expert interviews verified statistical data analysis results that different IMS should use different rewards to obtain better results and also that financial and mixed rewards more often lead to better results than just non-financials alone. Detailed interview results are in

Table 13. Summary of interview results.

Identifier	Active IMS Application		Passive IMS Application		Internal IMS Application		External IMS Application		Mixed IMS Application
	Created	Selected	Created	Selected	Created	Selected	Created	Selected	Created	Selected
INT1	F&N	F&N	FR	FR	F&N OR FR	FR	F&N	FR	F&N	F&N or FR
INT2	F&N	F&N	F&N	F&N	F&N	F&N	F&N	F&N	F&N	F&N
INT3	F&N	FR	FR or F&N	FR	F&N ON FR	FR	F&N	FR	F&N	F&N or FR
INT4	FR OR F&N	FR	FR	FR	FR	FR OR F&N	FR OR F&N	FR	F&N	F&N
INT5	FR OR F&N	FR	F&N	FR	NR OR FR	FR OR F&N	FR OR F&N	FR OR F&N	F&N or FR	F&N or FR
INT6	F&N	F&N	FR	FR	F&N	FR	FR	FR	F&N	F&N or FR
INT7	FR OR F&N	FR OR F&N	F&N or FN	FR	NR	FR OR F&N	FR	FR OR F&N	F&N or FR	F&N or FR
INT8	FR OR F&N	FR OR F&N	F&N or FN	FR	FR	FR OR F&N	FR	FR	F&N	F&N
INT9	F&N	FR	F&N	FR	NR or FR	FR OR F&N	F FR OR F&N	FR OR F&N	F&N or FR	F&N or FR
INT10	F&N	F&N OR FR	FR	FR	F&N	F&N	F&N	FR	F&N	F&R

Source: created by the authors.

.

According to the interview results, it was proved that financial or mixed rewards will lead to higher idea generation and selection results, and when looking at ideas created and selected, idea selection is stimulated by financial rewards more than mixed or non-financial alone.

Just one of the interview subjects has highlighted that overall, in all cases, the best results are with mixed rewards; however, they explain that there will be differences between what will be the proportion of financial and non-financial rewards in different web-based IMS applications. For example, they explain that in passive and external IMS types, financial reward portions should be bigger and the non-financial part is used just as a marketing element to attract the attention to the IM process. Thus, this additional explanation resonates with other expert perspectives because all other experts expressed that there are differences in what kind of IMS type organisation applies and rewards impact on results.

According to the interviews, the dominating opinion is that passive and external IMS applications should be followed by financial rewards more than other IMS application types, both to create greater idea quality and quantity. However, in mixed IMS application types more mixed rewards are suggested to create more ideas.

Experts have also mentioned some examples of more specific, interesting rewards for mixed, financial, and non-financial reward types:

• Financial rewards: for each generated idea, EUR 1 and a bonus to the next salary is given (EUR 5000 for the best idea); a prize of a EUR 500 gift card in-game shop; a 10% salary increase for the best idea author; The top ten authors receive each EUR 100, but the top three receive a EUR 1000 gift card in different shops.
• Non-financial rewards: one day as the boss; special badges for the number of generated ideas; one free day for the top ten contributors; a special concern for the person’s department, the potential to choose the food for the next office party.
• Mixed rewards: a surprise box from the company goods (value: EUR 75) and the title “Idea guru”; EUR 50 cents for each generated idea and badges in the IMS system; EUR 500 for the best idea creators and hockey tickets with the opportunity to meet the hockey team after the game.

According to the interviews, rewards should be appropriate and resonate with the tasks inactive IMS application cases (should be a good story about the reward), however, in passive IMS applications, these rewards should be something that will lead to interest in the long term and something that will not lose motivation.

4.5. Databases Qualitative Analysis Results

The web-based IMS experience of 129 companies was analysed by counting the dominating IMS application types. In these 129 cases, the successful idea generation results were defined based on application. Thus, the authors counted the dominating rewards based on the system application. The main results are represented in

Table 14. Databases analyses of the ideas created.

	N	F&N	FR	NR
Active IMS application	100	60%	29%	11%
Passive IMS application	29	31%	69%	0%
Internal IMS application	68	10%	87%	3%
External IMS application	51	24%	75%	2%
Mixed IMS application	10	70%	20%	10%

Source: created by the authors. Dominating reward style is highlighted.

, and these results are consistent with the survey results.

To analyse the best idea selection results, the authors have filtered and selected cases where at least 10% of the ideas were selected for development. See

Table 15. Databases analyses of the ideas selected.

	N	F&N	FR	NR
Active IMS application	90	57%	32% ↑	11%
Passive IMS application	23	17%	83% ↑	0%
Internal IMS application	67	10%	87%	3%
External IMS application	51	24%	75%	2%
Mixed IMS application	7	57%	29% ↑	14%

Source: created by the authors. “↑” means in comparison with ideas created.

.

According to this analysis, the dominating types for good cases do not change, however, it is important to see if the proportion is increasing or decreasing. Based on these analysis results, selected ideas are stimulated by financial rewards in active, passive and mixed IMS application types. Statistically, these differences are not tested because in some groups, the sample size is of insufficient size.

5. Discussion: Web-Based Idea Management System Rewards and Open Innovation

5.1. How to Reward Better?

The answer based on the descriptive statistics makes us believe that there is a higher number of ideas created for the mixed and financial reward type tasks, however, idea quality (ideas selected) is higher for the financial reward tasks. It is also highlighted that there could be differences between different web-based IMS application groups. The impact of intrinsic and extrinsic rewards on the results related to enterprises is a well-researched area. Some research proves that intrinsic rewards have a higher impact than extrinsic ones [77]; this research has proved that in a web-based IMS context, financial and mixed approaches provide better idea quality and idea quantity results. These results provide the following further research directions: (1) What are the differences based on the enterprise’s characteristics (e.g., size, industry etc.); (2) what is the impact of rewards on other results of IMS (e.g., different innovations, involvement, etc.)

Do just non-financial rewards provide a larger number of ideas created/selected per task than only financial ones? Using non-financial rewards alone will provide a larger number of ideas created and a larger number of ideas selected than the use of only financial rewards.

Do mixed rewards provide a larger number of ideas created/selected per task than only financial/non-financial? The chance that the use of mixed rewards will provide a higher number of ideas created is only slightly higher than with financial rewards alone, and the use of mixed rewards will provide a larger number of ideas selected than the use of only financial rewards.

However, the application of mixed rewards will cause more ideas created than the application of non-financial rewards alone, and the application of mixed rewards has resulted in more ideas being selected than the application of non-financial rewards alone.

These results consist of the self-determination theory that extrinsic rewards can enhance an intrinsic motivation [68] and the general interest theory, which extracts rewards, where that can enhance an intrinsic motivation [70].

Does the impact of mixed (financial and non-financial) rewards on the number of ideas generated and selected differ depending on the different types of IMS?

The most significant effect of mixed remuneration is on the average number of ideas created in cases of the application of mixed, external, and active IMS. The mixed remuneration does not affect the number of ideas created in cases of the application of passive and internal IMS. In terms of the number of ideas selected, it is less than with ideas created, but it still has a significant impact in the application cases of mixed and active IMS.

5.2. Web-Based Idea Rewards System, and Its Way to Motivate Open Innovation

Mixed and external IMS relate to open innovation, which is an important approach to collecting globally distributed knowledge and for enterprises increasingly searching for external ideas. The obvious issue is how to ensure the high-quality contribution of all participants [78,79,80]. The response to this problem, from a web-based IMS perspective, is to select the most appropriate rewards to increase the involvement and end results. Web-based IMSs are tools that could help to boost open innovations with external and mixed IMS application types. Open innovation highlights that competitive advantage often comes from inbound connections [81] where external knowledge flows inside the firm [82]. From the research results, it is clear that the highest idea quality and quantity could be stimulated by financial rewards or financial rewards in combination with non-financial rewards.

Open innovation via crowdsourcing [83] is a frequently applied method in organisations and crowdsourcing is one of the IMS sub-types. Community crowdsourcing, where involved actors are preselected on the basis of specific resources or required knowledge [84], could be defined as an active IMS application. Thus, to boost this type of crowdsourcing enterprise, the use of rewards that suit best for active IMS should be adopted. Open crowdsourcing, on the other hand, is a passive IMS where there is no pre-selection of contributors. Crowdsourcing via innovation intermediaries, where the external broker connects the company with potential contributors [85]—in this case, it could be either active or passive IMS—depends on the pre-selection process that the broker selects.

External and mixed IMS are great for boosting open innovation across enterprises’ boundaries [2,3], and creating the innovation ecosystem [5,6]. In future research, authors should explore in detail how different crowdsourcing groups could be influenced by different types of rewards, because there could be different crowdsourcing types, such as scientific crowdsourcing [86]. The combination of internal and external IMS as mixed IMSs become more popular with the trend to move away from delivering innovative activities primarily internally, to outsourcing innovative activities externally. IMS could help organisations to support “multidexterity” from a perspective that it helps the organisation to simultaneously carry out a multiple ideas search and selection activities based on diverse strategic logic and levels of knowledge to generate innovative outcomes [87], thus bringing to life new business models [88].

6. Conclusions

6.1. Theoretical and Practical Implications

This paper offers several theoretical implications for scholars and researchers. First, rewards have been researched in a variety of research contexts, however, they have not been extensively researched in a web-based IMS context.

Second, the results provide some insights that may help in designing future studies. They highlight the importance of empirical and theoretical research to select detailed financial and non-financial rewards. The results also show that there are a lot of possible elements to research in the future. In future studies, researchers should evaluate and select the most appropriate reward methods.

The main theoretical implication is the answer to the composite research question: What types of rewards have to be applied that could lead to better results in different IMS application types, and in this case—what rewards lead to a higher idea quality and quantity?

The paper filled several gaps: (1) knowledge gap—research focuses on intrinsic and extrinsic rewards leading to better results in a web-based IMS application type context. There was existing research on rewards in IM within the innovation and descriptive articles, however, this research looks at web-based (digital) IMS and IMS application types that provide this research with novel elements; (2) theory gap—there are theories about rewards and the adaptation of different information management systems; however, in this paper, authors filled a gap that was identified in the literature review—how the adaptation of different reward mechanisms influences results based on the IMS application type; (3) practical knowledge gap—web-based IMS application types in professional practices are not covered in existing research and there are only a few articles where it was discovered how different application types influence IM results. Thus, the authors provide practitioners with an answer to the following question: What rewards to use based on the web-based IMS application type?

This research contribution is part of a series of publications about the types of IMS and the use of IM methodology in different industries and business ecosystems. For example, in previous and outgoing research studies, the authors have discovered a relationship between web-based IMS and creativity based on the managerial survey results [74]; the IMS outcome’s influence on IMS creates benefits [75] and the results of the IMS application in goal setting and achieving, and decision making [89] and tasks of IMS [74]. In these papers, the authors analyse the outcomes and benefits of web-based IMS; however, in this paper, the authors analyse elements of the rewards that have led to such elements.

Web-based IMS fall in line with the growing importance of information communication technologies, and the spread of open innovation and co-innovation; therefore, indirectly, this paper creates a contribution to open innovation and co-innovation approaches.

For practical implications, the results proved that mixed reward usage with active IMS applications will provide higher idea quality and quantity; therefore, managers should focus on this combination of approaches.

The practical contribution of the research results helps to understand what kind of results enterprises could be expected from different IMS application types and rewards.

The research results highlight the benefits/implications of adopting different IMS for organisations.

This paper provides managers with a richer set of theoretical tools, letting them make better decisions regarding the selection of IMS that is the best for achieving the results in a context. Web-based IMS types and their impact on the IMS results could help to understand the potential application of these systems in different application scenarios.

6.2. Limitations

These results must be interpreted with caution and several limitations should be taken into account. To deal with many of these limitations, the authors applied the triangulation method to check the same areas of questions with other methods—expert interviews and databases analyses.

The main limitations based on the basic research approach are discussed below.

Issues with samples and selection, whereby only commercially available web-based IMS clients were invited into the surveys because of the approach to reach them through IMS developers. IMS developers distributed the survey to their selected clients, who approved the survey reception. In future studies, also non-commercial IMS application cases should be included.

There was an insufficient sample size for statistical measurements to research the data by industry. Industry perspectives could bring some interesting exploration based on the industry approach differences. Additionally, database analysis provided only descriptive results because of the small amount of data.

Regarding the lack of previous research studies on rewards in different web-based IMS, depending on the specific research topic, prior research studies that apply to the paper are limited. Here, it can be considered an important opportunity; presenting a gap with the need for further development in the study.

The methods/instruments used to collect the data included the survey instrument which provided limited data about rewards. In future research, authors could use detailed database analysis and case studies to verify and test results. Authors have to realise that they could also create more detailed survey questions from another viable perspective of rewards. For example, the value of rewards could be considered because of the already extensive survey with over 186 variables; however, the authors could not include an important question about the values and sizes of the rewards in the survey.

Regarding the limited access to data, the limited access to these respondents was only through IMS providers. Because of this limited access, authors needed to redesign and/or restructure their research depending on the circumstance. Findings are still reliable and valid despite this limitation.

Time constraints negatively affected the study because it does not provide the possibility of dynamics. Therefore, a future study—a longitudinal study—is needed to address this limitation.

6.3. Future Research Directions

How large should the reward be? This is an important question for future research because research has proved that very large rewards can be less effective. After all, idea creators would be scared to share their ideas, with the assumption that their idea is not good enough [90]. However, some researchers even describe that there should be no rewards at all because IM is part of their job, for example, for employees [28]. No rewards will lead to lower idea quality and quantity. This is proved in this paper (see Figure 7 and Figure 8). Some research even explains that there should be a reward for each idea—even for those that are not implemented [30].

When to reward? In this paper, rewards are described in the context of IM—at the end of the idea generation or during the process. Several authors considered rewarding after implementation [56,61], but others considered improving the idea generators’ satisfaction, suggesting an option for participants to choose their rewards in software [30]. Furthermore, the time of the reward could be examined in the research.

Idea management systems as the intrinsic motivation element? There are studies that explore people’s motivations to become involved in IM idea contests [91] and innovation processes [92], however, as web-based IMS is one tool that sometimes has integrated gamification and other intrinsic motivation elements, it could be interesting in the future to research IMS as possible intrinsic motivation tools.

Reward-based idea management from a multi-theoretical approach could be appropriate for future study fields. There is a theoretical basis for discussions from researchers about crowdfunding that are reward-based [93], and also how visualisations could help to improve results of idea management [94].