Next Article in Journal
Bureaucratic and Societal Determinants of Female-Led Microenterprises in India
Next Article in Special Issue
The Police and Citizens as Co-Producers of Crime Prevention in Johannesburg
Previous Article in Journal
Women’s Risk-Taking Behaviour during COVID-19 Pandemic: Will Work–Family Enrichment and Work Satisfaction Prevent Turnover Intention?
Previous Article in Special Issue
Healthy and Happy Citizens: The Opportunities and Challenges of Co-Producing Citizens’ Health and Well-Being in Vulnerable Neighborhoods
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Assessing the Quality of Digital Coproduction: An Interdisciplinary Model

Department of Public Administration, Institute for Management Research, Nijmegen School of Management, Radboud Universiteit, 6525 AJ Nijmegen, The Netherlands
*
Author to whom correspondence should be addressed.
Adm. Sci. 2023, 13(3), 69; https://doi.org/10.3390/admsci13030069
Submission received: 22 December 2022 / Revised: 15 February 2023 / Accepted: 21 February 2023 / Published: 27 February 2023

Abstract

:
The digital transformation of society raises high hopes for the improvement of the design and implementation of coproduction processes, which may lead to higher levels of acceptance, satisfaction, and trust with respect to public services. Yet and despite a growing number of studies on the role of digital technology on coproduction, our knowledge on how to assess the quality of digital coproduction is still very limited. The difficulty of defining quality indicators relates to three issues: the complexity of defining quality in public services, the lack of a clear understanding of quality in a coproduction process and the tendency to evaluate coproduction using the same approach as with other digitalised services. In this article, we adopt an interdisciplinary perspective to develop a conceptual model that builds on and bridges theoretical approaches commonly used to assess the quality of digital services, while adding elements that are specific to coproduction.

1. Introduction

This article discusses how to conceptualise and measure the quality of digital coproduction. The digital transformation of society raises high hopes for the improvement of the design and implementation of coproduction processes, which may lead to higher levels of acceptance, satisfaction, and trust with respect to public services (Fledderus 2015, 2016). Yet and despite a growing number of studies on the role of digital technology on coproduction, our knowledge on how to assess the quality of digital coproduction is still very limited (Lember et al. 2019; Clifton et al. 2020; Rodriguez Müller 2021; Cagigas et al. 2021). Policy documents show optimistic notions of digital coproduction processes and their benefits, such as customisation, efficiency, better public services, and higher engagement. However, what does quality mean in this instance?
As in any concept of quality, the underlying values may compete (for instance, effectiveness vs. efficiency). Thus, quality of coproduction encompasses a range of potential combinations of focus points and even trade-offs rather than a fixed standard. This is not surprising since coproduction often takes place over a longer period of time, in complex multi-actor settings and with varying stakeholders. These stakeholders have different roles, responsibilities, and perspectives. Expectations regarding the quality of coproduction are therefore highly context-dependent and should not be conceptualised rigidly in terms of one standard. Instead, a comprehensive framework is needed that offers a blueprint to normatively and empirically investigate stakeholders’ demands regarding quality and resulting tensions.
What makes coproduction in a digital context even more complex is that perspectives on quality are in practice at least partly driven by a technological perspective, which emphasizes values that would not normally play a significant role in assessing coproduction. From a social science perspective, some of these considerations may seem counterintuitive. However, in practice they shape not only the design and implementation but also the evaluation of coproduction processes. Adopting an interdisciplinary perspective, we will thus analyse and bring together different theoretical perspectives. Our aim is to arrive at a multidimensional conceptual and analytical framework to assess the quality of digital coproduction.
This article was developed as an interdisciplinary conceptual and empirical effort, as part of the EU-funded INTERLINK project (https://interlink-project.eu, accessed on 15 January 2023). Due to the interdisciplinary nature of our research consortium, we were able to incorporate concepts derived from various strands of the literature and disciplines that are not always easily compatible. This chance to present a more encompassing and richer perspective on quality, however, also raises new questions about the overall assessment of quality and the negotiation of tensions and trade-offs.
The remainder of this article is structured as follows. Section 2 discusses the background of digital coproduction and the way digital coproduction can be conceptualised. In Section 3, we discuss the challenges of defining and assessing quality in the context of digital coproduction, while Section 4 presents our inter-disciplinary and multi-dimensional model to analyse the quality of digital coproduction. Section 5 shares our concluding remarks and reflection on the development and further use of this framework.

2. Conceptualizing the Digital Coproduction of Public Services

To assess the quality of digital coproduction, we first need to define coproduction and consider what makes it digital. The concept of coproduction has been widely discussed in the literature (see: Brandsen and Honingh 2016; Nabatchi et al. 2017; Bovaird and Loeffler 2012). In this paper, we rely on the relatively narrow definition of coproduction developed by Brandsen and Honingh (2016) as ‘a relationship between the employee of an organization and (groups of) individual citizens that requires a direct and active input from these citizens to the work of the organization.’
In this article, the focus is on digital coproduction. What is it exactly that makes a coproduction process digital? It is certainly not difficult to come up with examples showing how the relationship between citizens and public administrations can change significantly under the influence of digital technologies. Often it is stated that opportunities to coproduce have increased due to the implementation of ICT. Lember (2018) states that it is often assumed that new technologies will foster coproduction and cocreation by making these processes more effective and more efficient, for example, through lower barriers for citizens to engage in such processes. Citizens can fill out forms from home or report dirt, loose paving stones, and broken lampposts. More substantive contact can also take place online, for example, through online citizen fora and referenda.
Digital coproduction could therefore be defined as coproduction in which ICT plays a significant role. That definition is of course rather imprecise. Should ‘digital’ be used to include only those processes which take place online? This would narrow down the object of our inquiry considerably, but it would also be misleading. Suppose we only have e-mail contact with someone; is that a ‘digital’ relationship? And does that change once we happen to meet that person at a physical conference, or in an online meeting where we see the person’s face? What if the digital interaction takes place against the background of largely analogue processes—say between two organisations and most of the actual work of the collaboration occurs in person? The distinction between analogue and digital coproduction is in practice blurry, so a strict definition is unlikely to be useful. Theoretically, it is sufficient to acknowledge that digital technologies play an important role in how the coproduction process is shaped, even if that intertwines with analogue processes. This is a gradual rather than an absolute demarcation.
This fuzzy definition is in fact less problematic than the fact that the different disciplinary perspectives that can be applied to the phenomenon each bring with them different understandings of quality, some of which are hard to reconcile. In the next section, we will point out the main difficulties which also need to be taken into consideration in the formulation of a framework for the quality of coproduction.

3. Quality: What Is Specific to Digital Coproduction in Public Services?

Quality can be considered what Pollitt and Hupe (2011) have termed a ‘magic concept.’ The very high degree of abstraction and the notion’s solely positive normative connotations overcome any opposition and have meant that aiming for qualitative public services has become an important objective across domains. However, its blurriness also means that defining and implementing a framework in the attempt to ‘assess’ quality is difficult. We will here break the difficulty down into three issues of how quality must be understood in the context of (1) public services, (2) coproduction, and (3) digital processes.

3.1. The Quality of Public Services

To understand the difficulties in defining—and ultimately measuring—the quality of digital coproduced public services, it is important to understand their origins: The concept of quality management spilled over from the industrial sector to the service sector, and finally to public services (Cole 1994; Reichwein and Broekmate 2010). This means there are three main difficulties in defining and measuring quality as a performance indicator in the public sector (Donabedian 1988).
First, many public services are delivered in complex settings, as is the case in coproduction processes. This means that stakeholders with different perspectives are involved, spanning public administration at the national, regional, and local levels, as well as public, not-for profit, and private service providers. They all see the colours of the quality prism filtered through their own lenses, since quality assessments are in the end normative judgements based on the values of the respective stakeholders (Oechler 2009; Beckmann et al. 2004). Defining quality thus usually includes the coordination of different perspectives rooted in divergent values and interests. This of course also matters in a business context, but in the public sector there are more (kinds of) actors of which to take account. Complexity is further increased in the context of digital public services as new players, like IT service providers, enter the stage.
Second, standardisation of quality, though common in the industrial sector, is more complicated in the case of public service provision as unpredictable human interactions are often an integral part of these kinds of services (Oppen 1995, pp. 20–31). This leads to a number of issues when trying to identify quantifiable (and even objective) indicators of quality, e.g., the immateriality and intangibility of the ‘product’ or the nonexistence of uniformity in service provision. Other difficulties are inherent to the public sector, such as the centrality of legal requirements and the client’s relative lack of independence compared to services in the private sector.
Finally, it is often unclear who is responsible for the level of quality that is ultimately achieved. This is because various interdependent actors are involved in creating public services and the effects of their contributions are hard to separate. This is even more complex in the context of services in which the consumer of public services is a coproducer (Beckmann et al. 2007; Honingh et al. 2020). Thus, their collaboration is necessary for the production of public services and, moreover, the acts of producing and consuming coincide. Who then is responsible for which parts of quality in public services?
For these reasons, an objective approach to quality, based on a standardised understanding of quality, is difficult in the context of public services. When designing quality indicators, it is important to include different perspectives on quality. We will do this by developing a multidimensional perspective on quality.

3.2. The Quality of Coproduction

In coproduction research, there has been relatively little attention given to the issue of quality and the control thereof. This is perhaps due to the use of coproduction in practice still being so fragmented. Perhaps fostering coproduction and giving shape to such an interactive process may have already been seen as a success and therefore as an aim or mark of quality in itself. Or perhaps the difficulty of defining good indicators has discouraged scholars from engaging with the issue. Or perhaps it has been regarded as a type of process that should not be subjected to quality control at all, because it may encourage efforts towards (undesired) standardisation. We recognize the latter risk, but also the equally real risk that coproduction will in practice be evaluated anyway, regardless of whether good indicators have been developed or not. As we will argue below, it is important to influence this process.
So where to start? There is a handhold in the academic literature as it has previously described the potential benefits of coproduction. These usually come down to the following: (1) a greater role for citizens in service delivery is an impulse for democracy (i.e., of intrinsic value), (2) it leads to greater feelings of empowerment among users, (3) it leads to a better relationship between user and provider, and (4) it leads to a functionally better service (Brandsen and Pestoff 2006). Preferably, all these benefits should be considered in developing indicators for coproduction processes.

3.3. The Quality of Digital Processes

If coproduction is conducted through digital means, such as a platform, there is a tendency to evaluate it using the same approach as with other digitalised services. This has led to reliance on two types of quality approaches. One is the technology acceptance literature, which uses an information systems perspective to assess how users accept and apply a technology. The other is consumer-based quality systems such as SERVQUAL that focus on perceptions and expectations in relation to the service that is ultimately delivered.
In one way, this fits well with existing practices, since the quality discourse in the public sector is highly embedded in a trend towards the rationalisation, marketisation, and managerialisation of public services (Hoggett 1996; Speck 1999; Power 1997). However, it does tend to emphasize certain values over others. The coalition of market-inspired and engineering perspectives can lead to an overwhelming emphasis on efficiency. Moreover, quality systems based on a consumer perspective tend to measure satisfaction rather than the actual quality of the service. In, for example, a music streaming service, that distinction is less important than in education or health care. Finally, there is the risk that less tangible, less measurable effects of coproduction (e.g., effects on trust) are ignored.
Our point is not that there is one superior way of measuring the quality of digital coproduction. The realistic approach is that all these perspectives must be taken into account. In the following section, we will suggest a conceptual model that builds on existing approaches, while simultaneously building in elements specific to coproduction.

4. A Model to Measure the Quality of Digital Coproduction

Definitions of quality can differ wildly, depending on their origins. The analysis above has shown that this will be especially the case for a fuzzy concept such as digital coproduction, which can be claimed as the stomping ground of various disciplines.
To start a process of reconciling the different perspectives, we will use and merge previously developed categorisations of quality that groups them into three types (Walsh 1991; Loeffler 2002; Alzaydi et al. 2018):
  • a product-based definition understands quality as the degree to which a particular service or product conforms to its specifications;
  • a user-based definition understands quality as the extent to which attributes of a product meet the customer’s requirements;
  • a value-based definition understands quality as the extent to which services are in line with normative expectations regarding public services.
Most definitions of quality tend to be multidimensional in one respect or another combining different factors. However, to comprehensively assess the quality of digital coproduction and to allow a multistakeholder perspective, it is necessary to combine these three categories. By doing so, it is possible to tackle many of the issues mentioned earlier, because the three approaches jointly cover each other’s ‘blind spots’.
Product-based quality is the strictest of the three interpretations, as it ties quality to concrete standards to which a service must adhere, such as compatibility and reliability. This allows precise measurement and a clear ranking of assessments, as in a school report. A justified criticism of New Public Management was that it applied such a product-based logic to public services, although these are inherently different. However, while we agree with this regarding the service element of digital coproduction, the digital element of the process does rely on a product. Thus, the product-based understanding of quality is relevant to digital public services. For instance, when several organisations are involved in delivering a service, but the incompatibility of software precludes smooth collaboration, that incompatibility constitutes a lack of quality. Within the three categories, there are of course many potential criteria, leading to more detailed subcategories. One of the most important sources of variation is whether to focus on processes and interactions or on outcomes (Parasuraman et al. 1985; Brady and Cronin 2001). In our design of quality indicators, we have included both of these perspectives, to achieve a comprehensive measurement of quality.
User-based quality incorporates those indicators that reflect perceptions, such as whether the service is easy to access and whether it meets expectations. The understanding of users in the context of digital public services is twofold. It encompasses both the recipient of a service (for example, elderly or unemployed citizens) and public officials using previously developed digital tools (for example, street-level-bureaucrats). As noted earlier, a criticism of this type of measure is that it tends to equate with satisfaction, although satisfaction itself can be the result of user-based quality assessments (for example, how flexible a service is to individual circumstances). Not purely focusing on satisfaction may also mean a more objective approach, especially in the case of highly complex public services for which the outcome and thus the eventual degree of satisfaction can be hard to observe in the short term, if at all. For instance, satisfaction with hospital care tends to be determined as much by the friendliness of staff and by the perceptions of interactions as by the quality of medical treatment. Thus, the criticism against one indicator loses its sting when applying a multidimensional framework. The weaknesses of any single criterion are balanced out by the multitude of incorporated angles. Furthermore, we cannot neglect that (the subjective) user experiences are essential elements.
Value-based quality refers to those indicators that assess quality based on higher norms. In public services, they play a crucial role, as the renewed debate about ‘public values’ shows. In contrast, a business context mainly refers to value in relation to price—the ‘bang for your buck’—even though there is a debate on higher norms, for example, in the context of corporate social responsibility. Overall, this category of quality shows a noticeably high level of tensions and trade-offs between factors—especially in the context of the digital coproduction of public services. For example, there is the issue of how to efficiently handle highly personal data while maintaining privacy standards. There is also the question of how to empower citizens, one of the fundamental values and assumptions of coproduction, while also paying attention to efficiency, privacy, flexibility, and ease of use. Furthermore, democratic values (e.g., empowerment) are intrinsic aims of coproduction. However, they must be balanced against other values such as efficiency (coproduction can be costly and time-consuming) and accountability (coproduction can muddy the division of responsibilities). It is only in considering the trade-offs that the relevance and weight of values can be truly grasped.
Figure 1 shows our overall multidimensional conceptualization of quality. Above all, this figure reveals that bringing together different perspectives and criteria does not mean to get rid of tensions when assessing quality. Instead, the framework helps to make them more visible and to investigate empirically how the stakeholder perspectives might differ regarding the assessment of quality. We are convinced that an open discussion will help to identify and overcome barriers of respective collaborative processes.
In the conceptualisation of specific indicators, we have deliberately sought not to create something entirely new and detached from what was previously developed, but have, insofar as is possible, aimed to integrate existing measurements, indices, and indicators on quality, most notably the PSQ and SERVQUAL models (Guenoun et al. 2016; Sabadie 2003). These are important reference points for observing product-based and user-based quality in public services, with constituent dimensions that take into account the specific context of the public sphere (e.g., tangibles, reliability, responsiveness, comprehension). We have also included insights from the literature on service design and satisfaction in the context of digitalised services as well as the well-known ISO frameworks on Product Quality and Quality in Use. As there is a lack of specification of the value-based understanding of quality in the literature, we could not refer to existing measurements for this category, but ourselves designed items inspired by the literature of public values and by the specific requirements of service delivery in the public sector (Weber 1922; Li and Shang 2020; Kim et al. 2021).
These indicators have three important limitations. First, public services differ regarding contextual settings. Hence, the proposition of general indicators might not always capture the specific understanding of quality in a public service under investigation. Second, as quality is a result of negotiation processes and the power to define quality is based on existing accountability relationships, its assessment is also time-sensitive. Changes in the assessment of quality thus do not stem only from changes in the quality but also from a change in the dominant perspectives in judging quality. A third and final limitation relates to the fact that some of the indicators might refer to outcomes as well as to the process of coproduction (for example, efficiency, empowerment, and democratic values). As it is hard, if not impossible, to provide a clear conceptual demarcation, the most realistic strategy to deal with potential confusion is to clearly state whether the focus of the assessment is on the process or on the outcomes.

5. Conclusions

In this article, we have aimed to develop an interdisciplinary, comprehensive framework for measuring the quality of digital coproduction. Clearly, this is only the skeleton of what should eventually become a more refined, coherent framework, but it is nevertheless a useful starting point for addressing the main issues such a framework must face. Such issues include: (1) interdisciplinary differences in the emphasis on particular values; (2) differences in the notions of users in the disciplines; (3) the need to take account of contextual differences when applying the framework; and (4) difficulties in distinguishing the process from output/outcome measures.
One of the most important questions concerns what values the term quality represents, an issue that even within disciplines is a complex one. The literature identifies various such values: effectiveness, efficiency, satisfaction, inclusion, and more. Disciplines tend to stress some values over others, which complicates interdisciplinary discussions. For example, efficiency and ease of use are dominant values in the development of technical systems, a frame which is then transplanted into public service contexts. In contrast, values of privacy/security or Weberian principles might be more important from a public administration perspective. In working towards a reconciliation of different values within an interdisciplinary team, there is a logical tendency to compromise by simply adding up different categories of values. Yet, that can be problematic: first, because it gives no clear guidelines on how to reach an overall assessment, and second, because it tends to favour those values which can be more easily measured.
A related Issue is how the role of users is perceived. When they are seen as customers, quality is interpreted as a measure of how satisfied they are and how likely they are to remain loyal. Of course, in a public service context, satisfaction matters as well. However, such an approach can be problematic, because the use of public services is often not voluntary and there is no effective competition. This has indeed been one of the major criticisms of market-oriented public sector reforms. A more fundamental issue is whether the use of a public service is the only type of involvement that should be measured. Although in a business context there are many examples of users codesigning products and services, this is usually seen as a way of raising the perceived quality of those products and services, but is kept separate from the quality measurement itself. In a public service context, the inclusion of users in the design and delivery of public services can be a democratic aim in itself, apart from whether or not that leads to better services in a more technical sense.
Public services are also part of a more political context, in which the way that services are delivered, or the fact that they are delivered at all, is seen as an expression of democratic choices. That means the assessment of a public service partly derives from a wider context, beyond the specific characteristics of the service. To some extent this is also the case in a commercial context: one’s satisfaction with a company’s services are often influenced by one’s feelings about the brand and the reputation of the company. In the public sector, such a connection is especially strong: assessments of public services have sometimes been found to be mostly unrelated to technical quality. There is known to be a strong connection between general trust in and expectations of the public sector, and the evaluation of services. If this is so, choosing quality measurements that only incorporate aspects of the service itself is a mistake.
Moreover, it is important to acknowledge that the quality of coproduced public services might refer to the process of coproduction itself as well as to the final output. Some of the variables we have presented in Figure 1 refer both to the process as well as to the output. Think, for example, about the values of empowerment or efficiency. Both of these indicators play a role over the course of the coproduction process, as they are directly related to the way coproduction gets shaped. Ultimately, the output might be considered inefficient or the citizen might even feel disempowered. This requires the assessors to be very explicit about the focus and starting point of their assessment.
All of this influences how the measurement of quality is set up, with respect to indicators, but also with respect to the process: how and when users are approached. Achieving a unified approach in an interdisciplinary context is hard work, for various reasons. The concerns of one discipline can seem marginal in the eyes of another. Specifically, the emphasis in both IT and business on efficiency can lead to a dominant place of this value in an evaluation. Furthermore, the concepts used by various approaches are overlapping and fuzzy, which means long and difficult discussions are required between scholars who might have difficulties learning each other’s scientific languages and who may regard the concerns of the other disciplines as either shallow or overly finicky. Yet, this collaboration is necessary in order to reach a comprehensive framework.
This article has attempted to bring together the most important concepts encountered when measuring the quality of digital coproduction, in the hope that this will encourage further exploration of how the strengths of different perspectives can be combined.

Author Contributions

Conceptualization, I.R., N.H., T.B. and M.H.; methodology, I.R., N.H., T.B. and M.H.; resources, I.R., N.H., T.B. and M.H.; writing—original draft preparation, I.R., N.H., T.B. and M.H.; writing—review and editing, I.R., N.H., T.B. and M.H.; visualization, I.R., N.H., T.B. and M.H.; project administration, T.B.; funding acquisition, T.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by European Commission, grant number 959201.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

No data is available as this is a conceptual article.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Alzaydi, Zyad M., Ali Al-Hajla, Bang Nguyen, and Chanaka Jayawardhena. 2018. A review of service quality and service delivery: Towards a customer coproduction and customer-integration approach. Business Process Management Journal 24: 295–328. [Google Scholar] [CrossRef]
  2. Beckmann, Christof, Andreas Schaarschuch, Hans-Uwe Otto, and Mark Schrödter. 2007. Quality Management and Formalization in Social Service Organizations—A Survey on Home-Based Family Intervention Services. Social Work & Society 5: 78–93. [Google Scholar]
  3. Beckmann, Christof, Hans-Uwe Otto, Martina Richter, and Mark Schrödter. 2004. Negotiating Qualities—Ist Qualität eine Verhandlungssache? In Qualität in der Sozialen Arbeit. Edited by Christof Beckmann, Hans-Uwe Otto, Martina Richter and Mark Schrödter. Wiesbaden: VS Verlag für Sozialwissenschaften, pp. 9–31. [Google Scholar]
  4. Bovaird, Tony, and Elke Loeffler. 2012. From Engagement to Coproduction: The Contribution of Users and Communities to Outcomes and Public Value. Voluntas: International Journal of Voluntary and Nonprofit Organizations 23: 1119–38. [Google Scholar] [CrossRef]
  5. Brady, Michael K., and J. Joseph Cronin Jr. 2001. Some new thoughts on conceptualizing perceived service quality: A hierarchical approach. Journal of Marketing 65: 34–49. [Google Scholar] [CrossRef] [Green Version]
  6. Brandsen, Taco, and Marlies Honingh. 2016. Distinguishing different types of coproduction: A conceptual analysis based on the classical definitions. Public Administration Review 76: 427–35. [Google Scholar] [CrossRef]
  7. Brandsen, Taco, and Victor Pestoff. 2006. Coproduction, the third sector and the delivery of public services: An introduction. Public Management Review 8: 493–501. [Google Scholar] [CrossRef]
  8. Cagigas, Diego, Judith Clifton, Daniel Diaz-Fuentes, and Marcos Fernández-Gutiérrez. 2021. Blockchain for public services: A systematic literature review. IEEE Access 9: 13904–21. [Google Scholar] [CrossRef]
  9. Clifton, Judith, Daniel Díaz Fuentes, and Gonzalo Llamosas García. 2020. ICT-enabled coproduction of public services: Barriers and enablers. A systematic review. Information Polity 25: 25–48. [Google Scholar] [CrossRef] [Green Version]
  10. Cole, Robert E. 1994. Different Quality Paradigms and their Implications for Organizational Learning. In The Japanese Firm: The Sources of Competitive Strength. Edited by Masahiko Aoki. Oxford: Oxford University Press, pp. 66–83. [Google Scholar]
  11. Donabedian, Avedis. 1988. The quality of care. How can it be assessed? JAMA: The Journal of the American Medical Association 260: 1743–48. [Google Scholar] [CrossRef] [PubMed]
  12. Fledderus, Joost. 2015. Does User Coproduction of Public Service Delivery Increase Satisfaction and Trust? Evidence From a Vignette Experiment. International Journal of Public Administration 38: 642–53. [Google Scholar] [CrossRef]
  13. Fledderus, Joost. 2016. User Coproduction of Public Service Delivery: Effects on Trust. Ph.D. Thesis, Radboud University, Nijmegen, The Netherlands, January 15. [Google Scholar] [CrossRef]
  14. Guenoun, Marcel, Kiane Goudarzi, and Jean-Louis Chandon. 2016. Construction and validation of a hybrid model to measure perceived public service quality (PSQ). International Review of Administrative Sciences 82: 208–30. [Google Scholar] [CrossRef] [Green Version]
  15. Hoggett, Paul. 1996. New Modes of Control in the Public Service. Public Administration 74: 9–32. [Google Scholar] [CrossRef]
  16. Honingh, Marlies, Elena Bondarouk, and Taco Brandsen. 2020. Coproduction in primary schools: A systematic literature review. International Review of Administrative Sciences 86: 222–39. [Google Scholar] [CrossRef] [Green Version]
  17. Kim, Yaeri, Qiran Wang, and Taewoo Roh. 2021. Do Information and Service Quality Affect Perceived Privacy Protection, Satisfaction, and Loyalty? Evidence from a Chinese O2O-Based Mobile Shoppingapplication, Telematics and Informatics. Telematics and Informatics 56: 101483. [Google Scholar] [CrossRef]
  18. Lember, Veiko. 2018. The Increasing Role of Digital Technologies in Coproduction and Co-Creation. In Coproduction and Co-Creation. Engaging Citizens in Public Services. Edited by Taco Brandsen, Trui Steen and Bram Verschuere. New York: Routledge, pp. 115–27. [Google Scholar]
  19. Lember, Veiko, Taco Brandsen, and Piret Tõnurist. 2019. The potential impacts of digital technologies on coproduction and co-creation. Public Management Review 21: 1665–86. [Google Scholar] [CrossRef] [Green Version]
  20. Li, Yan, and Huping Shang. 2020. Service quality, perceived value, and citizens’ continuous-use intention regarding e-government: Empirical evidence from China. Information & Management 57: 103197. [Google Scholar] [CrossRef]
  21. Loeffler, Elke. 2002. Defining and Measuring Quality in Public Administration. Available online: https://www.nispa.org/files/publications/ebooks/nispacee-betterquality2002.pdf#page=15 (accessed on 15 January 2020).
  22. Nabatchi, Tina, Alessandro Sancino, and Mariafrancesca Sicilia. 2017. Varieties of Participation in Public Services: The Who, When, and What of Coproduction. Public Administration Review 77: 766–76. [Google Scholar] [CrossRef]
  23. Oechler, Melanie. 2009. Von der Dienstleistung zur Dienstleistungsqualität. In Dienstleistungsqualität in der Sozialen Arbeit. Wiesbaden: VS Verlag für Sozialwissenschaften, pp. 77–110. Available online: http://link.springer.com/chapter/10.1007/978-3-531-91603-3_4 (accessed on 23 August 2013).
  24. Oppen, Maria. 1995. Qualitätsmanagement. Grundverständnisse, Umsetzungsstrategien und ein Erfolgsbericht: Die Krankenkassen. Berlin: Edition Sigma. [Google Scholar]
  25. Parasuraman, Anantharanthan, Valarie A. Zeithaml, and Leonard L. Berry. 1985. A conceptual model of service quality and its implications for future research. Journal of Marketing 49: 41–50. [Google Scholar] [CrossRef]
  26. Pollitt, Christopher, and Peter Hupe. 2011. Talking About Government. Public Management Review 13: 641–58. [Google Scholar] [CrossRef]
  27. Power, Michael. 1997. The Audit Society: Rituals of Verification. Oxford and New York: Oxford University Press. [Google Scholar]
  28. Reichwein, Alfred, and Loes Broekmate. 2010. Qualitätsmanagement in Kommunen. In Wege zum Qualitätsmanagement. Verwaltungsressourcen und Verwaltungsstrukturen. Edited by Hermann Hill. Baden-Baden: Nomos. [Google Scholar]
  29. Rodriguez Müller, A. Paula. 2021. Making smart cities “smarter” through ICT-enabled citizen coproduction. In Handbook of Smart Cities. Cham: Springer, pp. 1–21. [Google Scholar]
  30. Sabadie, William. 2003. Conceptualisation et mesure de la qualite’ perçue d’un service public. Recherche et Applications en Marketing 18: 1–24. [Google Scholar] [CrossRef]
  31. Speck, Otto. 1999. Die Ökonomisierung sozialer Arbeit, München. Basel: Ernst Reinhardt Verlag. [Google Scholar]
  32. Walsh, Kieron. 1991. Quality and public services. Public Administration 69: 503–14. [Google Scholar] [CrossRef]
  33. Weber, Max. 1922. Wirtschaft und Gesellschaft. Grundriss der verstehenden Soziologie. Tübingen. Available online: https://www.textlog.de/7355.html (accessed on 15 January 2023).
Figure 1. A multi-dimensional approach to the quality of digitally coproduced public services.
Figure 1. A multi-dimensional approach to the quality of digitally coproduced public services.
Admsci 13 00069 g001
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Radtke, I.; Hoevens, N.; Brandsen, T.; Honingh, M. Assessing the Quality of Digital Coproduction: An Interdisciplinary Model. Adm. Sci. 2023, 13, 69. https://doi.org/10.3390/admsci13030069

AMA Style

Radtke I, Hoevens N, Brandsen T, Honingh M. Assessing the Quality of Digital Coproduction: An Interdisciplinary Model. Administrative Sciences. 2023; 13(3):69. https://doi.org/10.3390/admsci13030069

Chicago/Turabian Style

Radtke, Ina, Noortje Hoevens, Taco Brandsen, and Marlies Honingh. 2023. "Assessing the Quality of Digital Coproduction: An Interdisciplinary Model" Administrative Sciences 13, no. 3: 69. https://doi.org/10.3390/admsci13030069

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop