Next Article in Journal
Choices We Make in Times of Crisis
Previous Article in Journal
Twitter’s Messages about Hydrometeorological Events. A Study on the Social Impact of Climate Change
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

The EU Health Technology Assessment and the Open Method of Coordination: A Relation with Potential in the Context of Network Governance

Ani Matei
Adrian-Stelian Dumitru
Corina-Georgiana Antonovici
Faculty of Public Administration, National University of Political Studies and Public Administration, 010643 Bucharest, Romania
Author to whom correspondence should be addressed.
Sustainability 2021, 13(6), 3582;
Submission received: 18 January 2021 / Revised: 19 March 2021 / Accepted: 19 March 2021 / Published: 23 March 2021
(This article belongs to the Section Health, Well-Being and Sustainability)


The open method of coordination (OMC)—a tool which was formalized in the early 2000s—has generated the interest of both the researchers and practitioners in the context of the new EU governance. This article is examining the literature of both network governance and OMC, with the focus particularly on one main question: is OMC a useful instrument in health policies in order to achieve concrete results by outlining norms and legislation where EU exercise limited power? Analyzing a field in which the EU competence is limited—given the budgetary implications of medicines reimbursement—from the results of the existing collaboration within EUnetHTA, we will observe the added value in this particular case of the OMC application, and the possible consequences in shaping the supranational competences. Given that the EU, with some exceptions provided by the Treaties, may only exercise actions to support, coordinate or complement the action of the Member States in the health policy, the OMC proves to be a useful tool, both from the perspective of the Member States but especially of the supranational level.

1. Introduction

The feedback pulse of the European Union against the economic crisis of 2008 which laid out the weaknesses of the EU economic governance from that moment emerged in a debate in order to find out a viable solution from the intergovernmental and supranational actions through balancing the scales. That debate brought into discussion the need that intergovernmental ‘soft’ methods such as the open method of coordination (OMC) to be extended and used in a broader sense.
This article aims to examine the use of OMC in a policy area that has been carefully kept as a national competence by the Member States that is health care. As scholars mention, ‘’health policy in the EU has a fundamental contradiction at its core’’ [1]. The reason for this statement is that on the one hand, the Treaties, as the primary EU law, expound explicitly that health care resides in the Member State competence. But, given that health systems involve interactions with patients, goods and services, all of which are granted freedom of movement across borders by the same Treaties, various national health tasks are in fact subject to EU law and policy.
Given those fundamental contradictions EU health policy is facing, associated with the hesitation of Member States to transfer power in this area to the EU, other policy approaches have grown over time, including in the field of health care [1] . As such, we first focus on network governance concept and its meaning. The new focus on ”governance”—a new concept which might be difficult to convert across European languages—was generated by the problematization of the traditional processes of governing. We consider the various attempts to define network governance and try to identify if may be possible a generic understanding commonly accepted.
We introduce a concise overview of the EU network governance meaning, and later focus on OMC sense and evolution over years as well as its introduction into the EU health policy field. Regardless of the ”lack of formalized EU welfare policies, a patchwork of law, governance and policy, particularly in the areas of public health, employee protection and cross-border health care provision, is noticeable’’ [2].
The OMC processes have been nominated by the European Commission for monitoring and aiding existing legislation in fields such as health and safety, environmental protection, immigration, disability and fundamental areas where the EU has very limited powers. The introduction of the OMC has prompted much debate over the role of such soft laws in EU governance. These various OMCs have been classified from “weak” to “strong” by reference of three criteria which are related to the choice of the common guidelines; the sanctions that could be imposed and finally the roles provided to the variety of actors [3] (. As one of the best-known example of soft law, the OMC involves the European Commission as ‘’something of a broker or facilitator between Member States, with the burden of work falling to transnational networks of policy experts” [1].
Further, we concentrate on health technology assessment domain in the EU and after a brief introduction into this specific scientific field we examine the EUnetHTA collaboration network which was established as a response to an expressed need of EU Member States and the European Commission to establish a sustainable network for health technology assessment (HTA) in Europe. The various EU-funded projects generated a specific understanding of HTA as a scientific and policy-making field.
We discuss how this new governance instrument in EU health policies—the OMC—“promise to induce law-like behavior by creating norms and networks, whether they will have that effect, or are intended to have that effect, varies”, as Greer and Vanhercke mention [4].
We find that the OMC can be popular because it strengthens networks among officials and advocates, and it potentially will interact with, channel and shape ‘harder’ law in the health policies and moreover might be seen as a mechanism to develop further supranational competences, as we will see in our study case.


Given the complexity of the politically and policy-making system in the EU, we are interested in the significance of network governance concept and its implications for the development of EU competences. What does network governance mean? Is there a general understanding of this concept? In the EU institutional framework what is the network governance role?
After a general analysis of the network governance concept, our attention is directed to another EU specific proliferated approach, which is the OMC. Our analysis focuses around one main question: may OMC be considered a new way of governance? We are interested how this new mechanism first appeared, its intended primary policy domain and further development. Furthermore, we try to identify the importance of the OMC to achieve the goal of the networks.
Later, our analysis approaches a very interesting area of health policy, namely HTA. We call it interesting because according to EU regulations, the Member States are responsible for the definition and organization of their health policies, including funding and reimbursement decisions, so that the EU has limited power in this area. We address the origins of HTA process, its evolution over time in the EU and its technical peculiarities. Further, we concentrate our attention on the EUnetHTA collaboration network between EU’s Member States. We are interested to analyze and identify how started this collaboration, what were its intended objectives, Member States and European Commission involvement as well as its impact on developing homogeneous practice all over the EU.
Based on the results of the collaboration mentioned above, we argue that the success of this network stands as evidence that OMC as an instrument of the network governance in this case served to develop and strengthen homogeneous practices across EU’s HTA domain.
In order to support our argument, three main quantitative indicators were identified:
The number of Member States and actors participating in the network; a high number of Member States from the total 27 being a prove of their interest and a recognition of the network added value;
The number of projects developed from the initial collaboration; the continuation of network collaboration showing the relevance given to it by the European Commission and Member States;
Benefits of network from the participants’ perspective: the prevalent benefits classified as “very useful”; the number of respondents that classified their benefits from the network to “very useful” can be seen as a success criteria.
In the concluding remarks, having in mind our findings, we try to understand if this network collaboration (EUnetHTA) has provided new leverage to EU supranational level in order to strengthen its competences using OMC/OMC-like instruments. To all these questions we intend to provide an overview and a potential interpretation.

2. Theoretical Considerations

2.1. Network Governance

The concept of network, like the governance one, is widely used in countless disciplines, being a common term in the case of social sciences disciplines.
Network concepts have a long and rich history as being used to study organizational form and the dissemination of information within social structures. Berry et al. [5] identify the origin of social network analysis from Hawthorn’s early experiments from 1924 to 1932, marking the first use of “network configurations to analyze social behavior.” These social experiments are often seen as an important “milestone” in the evolution of management and organizational development, leading Koliba, Meek, and Zia [6] to conclude that network analysis is part of the field of research for some time and is incorporated into studies of hierarchies.
The importance of network governance in the context of governance is thus given by a precise delimitation of the sphere of interest—“the attempt to facilitate that coordination in and through negotiated interactions between a plurality of political actors” [7].
Since the early 2000s, Klijn and Koppenjan have tried to define the concept of network governance, in the sense of which it is insoluble by the term “governance”, now reaching the following definition: “models of social relations, more or less unstable, between mutually dependent actors that gather around a problem that needs to be solved, a draft policy, and/or a set of resources that arise, are supported and adapted/changed through a series of interactions” [8].
For Hajer and Versteeg [9] network governance usually exists where the constitutional regulatory framework is not very well defined, which is why the actors involved need to focus on strengthening network trust as well as bringing to a common denominator the meanings assigned to the purpose of the network (policy formulation).
According to a prominent definition, a network is “a set of relatively stable relationships that are non-hierarchical and interdependent in nature that link a variety of actors who share a common interest in (public) policy and who exchange resources in to achieve common interests, recognizing that cooperation is the best way to achieve common goals” [10].
A network includes all public and private actors involved in the development and implementation of policies in a given policy area. Network governance refers to a process of governance in the absence of a central authority in which the political arena is populated by public and private actors who are linked together by a variety of interdependent resources [11].
The structure of the network is often understood and described in the form of connected “nodes” through coordinated actions and resource exchange [6,12,13]. Governance networks differ from other forms of social networks by the characteristics of network actors and by the types of collective functions and actions they take. These functions are aligned with the pursuit of one or more policy flows.
Although there are different types of networks in the literature, network theorists tend to argue that “the relative stability of belonging, their openness to individuals and groups and the level of interdependence of resources between actors determine the relative influence of different actors and the substantial content of EU policies” [14].
As such, what do we understand by “network governance”? From the analysis of the literature so far, we have noticed as many interpretations and manifestations of governance as disciplines that approach the concept. However, according to some authors, the conjugation of terms—network governance—which takes place mainly in the field of political and administrative sciences, is “a specific manifestation of governance, a term however insufficiently clear which “vaguely refers to non-hierarchical attempts to coordinate public and private interests, actions and resources” [7].
Torfing [7] considers that the importance of network governance in the context of governance is thus given by a precise delimitation of the sphere of interest—“the attempt to facilitate that coordination in and through negotiated interactions between a plurality of political actors”.
For other authors, network governance usually exists where the constitutional regulatory framework is not very well defined, which is why the actors involved need to focus on strengthening network trust, as well as bringing to a common denominator the meanings assigned to the purpose of network (policy-making). Based on the common observation that the traditional state actor is currently facing difficulties in resolving various political, social, economic issues, etc., network governance emphasizes the importance of collaboration between national authorities and various institutions, either similar but from different states, either particularized, such as sub-, trans-, inter- or supranational organisms [15].
Hajer and Versteeg [9] perceive the well-established traditional hierarchical order (local-regional-national-international) which has long described the relationships between different levels of authority as a “matryoshka” system, very different from the proposed governance networks which can be described as “a polycentric collaboration, often transnational and almost by intercultural necessity of several actors” [9].
Sørensen and Torfing [16] consider that it is necessary to advance some warnings in order to avoid certain common misunderstandings. First, the proliferation of interactive forms of network governance is not, as some authors suggest [13,17,18], resulting in the “hollowed-out state”.
The growth of interactive forms of governance in the form of networks and partnerships does not replace the role and the impact of the state. The state may have lost its privileged position in the public policy-making process, but many of its former state powers have remained at its disposal and new capabilities are being developed as long as local and central state agencies take over the ability to manage networks to different levels. Thus, the two authors emphasize that the power of the state is not reduced, but transformed and exercised in new ways.
Second, governance networks, despite the recent proliferation of the term and the focus of researchers’ attention, are by no means a new phenomenon [16]. In many states and arenas of public policy making, there is a long tradition of corporate involvement from the social partners (business associations, trade unions) and other relevant actors in the formulation and implementation of public policies. In fact, the interaction between public and private actors is a key component of modern government and a constitutive feature of liberal democracy [16].
What is really new is that central decision-makers and political theorists increasingly perceive governance networks as effective and legitimate mechanisms of public governance. This is highlighted by the increased trust in governance networks, present at all levels of government [19].
Sørensen and Torfing [16] point out that the only way to solve complex and improperly defined problems in the face of conflicting demands and objectives is by pooling the relevant actors involved and facilitating a collaborative problem-solving process, in order to encourage mutual learning and to strengthen the common ownership of new bold solutions.

2.2. Network Governance and the EU Policy-Making Process

From what perspectives can the applicability of the “network” type as a way of governance can be understood within the EU? First, some authors believe that, on the one hand, the European institutional system—the multilevel structure, the combination of supranational and intergovernmental elements—has an important role, to which is added the role of the judiciary, and, on the other hand, the decoupling the competencies associated with the various stages of a policy, the design being most often associated with the supranational level (with a tendency towards consensus, formal or informal), and the implementation of the national one, is also an important feature [11].
Ion [15], referring to the above characteristics, notes that there are certain keywords that can describe this system, namely: “fragmentation” (if we consider the extreme specialization existing within the European institutions), “homogeneity” and “fluidity”, mainly due to the involvement of several actors only in the policy formulation phase, not the implementation that takes place at national level, often with the same people. The competencies of those involved therefore differ not only from one policy to another, but also in the different stages of a policy-making cycle.
To understand the phenomenon of policy-making within the EU, we need to understand what kind of policies we can talk about within the EU. Thus, in this question, we find a first perspective in Helen Wallance [20] who presents five types of European policies differentiated according to the changes that have occurred over time at both supranational and national level:
  • The traditional Community method;
  • the way of EU regulation (for “micro” decisions on the implementation of the acquis communautaire);
  • The distributive modality of the EU (distribution of resources by involving several types of actors);
  • Policy coordination (for example using the open method of coordination);
  • Intensive transgovernmentalism (what in other authors appears as intergovernmental cooperation—the cases of EMU, CFSP, the former JHA field [20].
Wallace’s argument is that policy-making patterns in the EU are different, not only because of the ongoing controversy over policy responsibilities shifting from national to European processes, but also because of functional differences between policy areas and changing perspectives on development of contemporary government and governance [20].
Wallace also points out that “most sectorial policy areas do not clearly fall into a single policy method and there is a strong variation over time both within policy sectors and in response to events and contexts” [20]. Following the case studies analyzed in his research, Wallace concludes that variations are maintained, and hybridization between these ideal types is widespread.
Next, Simon Hix [21] considers that existing policies within the EU can be classified as follows:
  • Regulatory policies
  • Expenditure policies
  • Macroeconomic policies
  • Citizens’ policies and foreign policies
Hix notes that one of the main differences between these types of policies, apart from the scope, is the way decisions are made. While the first two types of policies, as well as segments of c. and d. are adopted by the community method, the other types of policies are dependent on an intergovernmental decision-making process [21].
Warleigh-Lack and Drachenberg [22] divide the decision-making process into two basic categories: (a) that of “historical proportions” and (b) the category of “day-to-day”. With regard to the first category—issues of major importance, such as establishing an EU-wide strategy for a period of years, or agreeing to treaty changes, Member State governments have full decision-making power.
At the meetings of the Head of State or Government at the European Council, important and complex negotiations take place to ensure that the package of measures resulting from these summits is acceptable to all, using the rule unanimity. In this way, any state can exercise its veto if a proposal is considered unacceptable [22].
With regard to the second category, that is ”day-to-day” decisions, the standard operating model is pillar I, which in recent years has become a “formal legislative exchange of power between Member States—through the EU Council—and the European Parliament” [22].
This process, known formerly as co-decision, has had a fundamental impact on the history and relevance of the European Parliament, in particular by increasing its legislative role—from a marginal one to one equal to that of the EU Council.
However, the emerging standardization of legislative decision-making is not the only story, complemented by the increasing use of so-called “new” or “soft” governance tools such as benchmarking or the open method of coordination which in a debatable way takes place outside the pillar-type structure [22].
Warleigh-Lack and Drachenberg [22] point out that these relatively new governance instruments adopted relatively recently do not give any essential role to either the EU Court of Justice or the European Parliament, although they may involve a variety of actors in the civil society and can produce decisions at European level but at a different level: not legislation, but recommendations, advice and examples of good practice and guidance.
Moreover, when Member States note the benefits of co-operation at EU level in a public policy area, but want to ensure that this co-operation remains under their full control, or as close as possible, Member States create new “containers” for this cooperation.
Next, another question arises, namely what do most of these different types of policies have in common? Ion [15] wishing to provide an answer to the above question, to which he adds the adjective “extreme” when referring to existing differences, considers that the process of their realization most often takes place within a network that appeared in the context of the development of specific EU policies. From the Commission’s point of view, the emergence of specialized networks may be the result of the process of European integration, of increasing the level of expertise in various fields, but also of the intensification of global interdependence [15].
Beyers and Kerremans [23] note that the emergence of a network is conditioned by the demand and supply of government, which is why, at EU level, it would be interesting to analyze “what resources mobilized by societal actors attract the attention of public officials working in the EU institutions and stimulates the supply of access (to institutions) and whether the networks thus created succeed in reproducing at the supranational level the political cleavages within the member states or remain suspended in the simplistic pro-integration/anti-integration dichotomy” [23]. Other authors do not consider that networks would appear and develop only in those sectors known as belonging to the low politics area, arguing that the analysis of policy networks could be extended to the systemic level [15].
Jönsson and Strömvik [24] highlight that the shortest definition of a network is “a set of interconnected nodes”, that change the territorial perception of the political space and has implications on the analysis of actors’ positions within networks. These positions are determined by the relationship with different network nodes in a context in which the perceived node falls not as an organization, but as a person. In the case of the EU, networking “remains a combination of know-how with know-how” [24].
Thus, in the EU negotiations, the role of these informal networks would be: (i) to facilitate cooperation between states and other types of actors, with most decisions being heavily negotiated in so-called “package deals”; (ii) a new perception of statehood is thus reached, “sovereignty being reduced to a certain legal authority which states use as a lever in negotiations”. The authors point out that we are thus talking about a change in the role of the state, a situation that transcends the reified cleavage between the defenders and contestants of the state actor. We must not forget that these networks “create multiple and diffuse loyalties and identities”, foster trust between actors and increase the rate of policy implementation, although the decision-making process is often criticized for opacity [24].
We may observe that from the perspective of the two authors, in the process of carrying out a policy several categories of actors are really involved, differentiated according to the administrative levels at which they operate, as well as according to their constitutive character: public-private-non-profit that configures their own agendas of various interests.
Hofmann and Türk [25] argue that any discussion of “transforming forms of government and governance in Europe” should start from the analysis of the stages of the public policy-making process involving public actors at four levels: subnational, national, supranational and international, and the institutional configuration that determines the degree of involvement of the supranational level in this process (direct involvement, action through states or mechanisms to influence only the national regulatory framework).

2.3. The Open Method of Coordination—A Novel Approach of Governance?

The question as to whether or not the OMC is a new governance method is hotly debated in the OMC literature. Büchs [26] argues that “answers depend on the reference point of comparison. If one acknowledges that the EU has used ‘soft’ governance before the OMC was introduced, the OMC does not appear to be so new. If one however emphasizes the particular design of the OMC in EU employment and social policy as established in the late 1990s one has to conclude that the OMC presents a new governance method”.
The introduction of the OMC originally praised as a fresh and bold step towards integrating new policy areas into the spectrum of EU activities, has been very much at the core of the debate [27].
The OMC understood as a governance tool based on flexible proposals and acts without binding legal value aims at:
  • Setting general guidelines for the Union, in conjunction with precise programs to achieve the objectives they have set in the short, medium and long term;
  • Establishing, where appropriate, qualitative and quantitative indicators and targets to be achieved, related to the highest performance worldwide and customized to the needs of different Member States and sectors of activity, as a means of comparing best practices;
  • Transformation of these indicators into national and regional policies with specific objectives, taking into account national and regional particularities;
  • Regular monitoring, evaluation and careful review, organized as shared learning processes [28].
João Rodrigues [29] whom in some authors opinion is credited with being the architect of the OMC [30] , defines the open method of coordination as:
  • A concrete way of developing modern governance using the principle of subsidiarity;
  • An instrument to organize a learning process at European level in order to stimulate exchange and the emulation of best practices, and in order to help Member States improve their own national policies;
  • A way of encouraging management by objectives by adapting European guidelines to national diversity;
  • An inclusive method for deepening European construction;
  • An instrument to be added to a more general set of instruments;
  • An important tool to improve transparency and democratic participation”.
The purpose of OMC is not do define a general ranking of Member State in each policy but rather to help Member States to “learn” from one another and obviously to improve their domestic policies [31]. Among the characteristics of the OMC, we can highlight the following: a totally decentralized approach at EU, national, regional and local level, involving the social partners and civil society; secondly, the abandonment of the traditional concept of “top-down” governance and, thirdly, the method abandons the traditional concept of proposing comprehensive policies and relying on binding normative acts [32]. While within some of the Treaty-based coordination processes, and particularly in the case of economic policy coordination, binding rules and legal enforcement do play a role, nonetheless in most of the OMCs and OMC-like processes legally enforceable or binding norms are scarcely present, or they have a much less prominent role [30].
Looking at its design, Craig and de Burca [30] believes that the OMC is clearly intended “to promote flexibility and openness, and to facilitate interaction between levels of governance in framing and developing policies”.
Although it appeared and was formalized in the early 2000s, the method dates back to 1992—the Maastricht Treaty—where a similar form of governance was used for economic coordination. We also identify such a form of governance in the case of the Treaty of Amsterdam (1997) in which the first regulations that encourage coordination of national social policies in the area of combating social exclusion were introduced. Here we identify specific aims like “improving knowledge, developing exchanges of information and best practices” and “evaluating experiences”. Barcevičius, Weishaupt and Zeitlin [33] emphasize that the Nice Treaty (2001) maintained this provision and extended its reach to a wider number of social policies. “The various mutual learning formats were expected to provide evidence and forums for deliberation which could help policy actors to adjust their preferences so that they could work together towards shared objectives”, authors suggest.
In addition, Kjaer [34] states that similar practices already existed in “transnational governance”. Even if the OMC has not been established as a broad method of coordination in the EU Treaties, several provisions of the Treaty on the functioning of the European Union refer to it in substance, without directly naming it: art. 168: health policy, 149: employment, 181: research and technology, 153 and 156: social policy, 173: industry.
The reason for the formal establishment of the OMC in 2000 was to be able to identify and promote appropriate and effective social policies. This form of European governance that takes place in the areas mentioned has been considered much less invasive by the Member States.
The Lisbon Summit acknowledged that a decentralized approach would be taken in accordance with the principle of subsidiarity in which the European Union, the Member States, the regional, local levels, civil society and social partners would take an active role.
The Lisbon Strategy represented a double breakthrough for the development of social policy at EU level. First, it forcefully stated that the European integration process should no longer be dominated by ”negative integration”. Second, the Lisbon summit formally introduced a new governance instrument to achieve these ambitious goals: the OMC. In relation to the OMC the Commission stated that it is used on a case-to-case basis.
Thus, according to the ”White Paper on European Governance”, the OMC “is a way to encourage cooperation, exchange of best practices and acceptance of common goals and guidelines for Member States, sometimes taking into account national action plans (…). It is based on regular monitoring of progress towards these objectives, allowing Member States to compare their efforts and to learn from the experience of others” [35] . It was argued that the Commission was already playing an active coordination role and was “ready to do so in the future, but using the method should not disturb the institutional balance or alter the achievement of the common objectives of the Treaty. (…) The open method of coordination should be a complement rather than a substitute for community action” [35].
Subsequently, with the evolution of the priorities of the European Union over the years, the OMC consolidated its own evolution, and in 2005 it will be transformed into a single OMC on the social field [31]. It was revised and reintroduced in the following years, especially with the revised Social Agenda, which provided for the social impact of all European policies to be taken into account.
The Lisbon Strategy was set to expire in 2010 and thus the discussion on a future EU-flagship strategy started a few years earlier. At the beginning, the majority of participants in this discussion acknowledged that the Social OMC was useful (although not without its flaws), but argued that it also needed further consolidation and reinforcement [33].
In the 2010s, the role of the EU has been extended, given that the Member States were required to submit a progress report on the Europe 2020 goals, as well as the powers of the Council and the European Commission to make individual recommendations to the governments of the Member States within the European Semester [31]. The Europe 2020 strategy clearly makes an effort to address the key concern of the social actors by setting ‘inclusive growth’ as one of its three ‘mutually reinforcing’ priorities (the other two being ‘smart growth’ and ‘sustainable growth’).
Coordination processes, in particular the OMC, have undergone a period of expansion, differentiation and formalization in recent years. Since their origins in employment policy, ever more fields have been brought under this kind of ‘coordination’, while the specific form of the procedure has been adjusted to each particular case [27]. Thus, at the moment the OMC covers a wide variety of fields such as social inclusion, insurance and health systems, research, innovation, education, development, environment, information society, business environment, etc.
Even if there are differences between the OMC variants depending on the domain concerned, certain principles and procedures remain constant in the EU. The Council of the EU is usually the first to propose the policy objectives that are then applied by the Member States according to their needs and availability [31]. The implementation is subsequently evaluated according to the qualitative and quantitative indicators, established in the process by the participants. Finally, the results of these evaluations are in turn evaluated and compared with the best practices of the Member States. The results of the evaluations are not explicitly mandatory for the Member States, but there is an element of informal pressure exerted within the OMC that can cause states to act on those issues when they would not normally do so. As for the role of the European Parliament, it is limited to providing recommendations, while that of the Commission seems to be just as limited, being responsible for monitoring and surveillance [31].
Although started as an intergovernmental process of informal coordination, it has been gradually assimilated by the European Commission, which has understood faster than national actors that soft law informal mechanisms are sometimes extremely effective in facilitating—more or less assumed—“the transfer of competences to the supranational level”. From this perspective, the community method will remain the classic policy-making solution, but constrained by a series of “informal practices that have grown up around it” [36].
An interesting approach regarding the development of EU policy-making in the context of OMC we find at Héritier and Lehmkuhl [37] for which some typical process of development appears to be the so-called process of sedimentation or layering. The metaphor of «layering» notes that “new modes and new instruments are introduced while the old modes of governmental intervention are left in place. The simple addition of new modes and instruments to already existing ones offers the advantage that no political costs arise in overcoming the resistance of those opposing the abolition of the old ones” [37].
Moreover, the OMC is receiving increasing support from national actors, especially those who participate in the European public policy-making process, as evidenced by the study of Susana Borrás and Anders Ejrnæs [38]. In fact, the authors refer to stakeholders at national level, by this phrase understanding—on a neofunctionalist line—a broader category than that of national elites: all bodies that have declared interest in a specific policy area, in the national politics context [38].

2.4. The Proliferation of the OMC in the Health Care and Its Importance for Achieving Network Goals

As we already mentioned, the determination of the Member States to maintain a minimal EU role in terms of public health, and almost non-existent in the national health services, can be seen in the form of Treaties.
The foundation for EU legal action in the area of public health is and has been limited. Art. 152, for example, from the old EC Treaty being an evidence for modest competences that were granted by the Member States. Words and expressions like “complementary” and “encouraging cooperation” were intended to highlight that the EU can only complement the action of the Member States which represent the main actors of health policy [39]. This involves public health, an area in which treaties state a competence that the EU can “use” to supplement by normative acts in certain matters, such as substances of human origin, blood derivatives or pharmaceuticals.
This provision of the Treaty outlines an attractive image of the EU, which is seen as an official within the reach of the Member States, having power and policies only where the Member States have wished to give it powers [37]. Subsequently, the Lisbon Treaty created a more coherent and categorical health competence that shows the sensitivities of the Member States and again emphasizes the supranational complementary role towards the Member States.
Currently, Title XIV of the TFEU, respectively art. 168, regarding “public health” states that ”a high level of human health protection shall be ensured in the definition and implementation of all Union policies and activities”. Thus, the Union action ”shall be directed towards improving public health, preventing physical and mental illness and diseases, and obviating sources of danger to physical and mental health”.
At the same time, paragraph 1 provides that this action includes ”the fight against the major health scourges, by promoting research into their causes, their transmission and their prevention, as well as health information and education, and monitoring, early warning of and combating serious cross-border threats to health”. Additionally, paragraph 2 states that the EU “encourages cooperation between the Member States” in the fields covered by art. 168, and “if necessary, lend support to their action”. The importance of the EU in improving the complementarity of the health services of the Member States in the cross-border regions is also underlined here.
Last but not least, the Treaty provides that the Member States shall coordinate, in cooperation with the Commission, the policies and programs in the areas mentioned above. “The Commission may adopt, in close contact with the Member States, any useful initiative to promote this coordination, in particular initiatives aimed at establishing guidelines and indicators, organizing the exchange of best practices and preparing the necessary elements for periodic monitoring and evaluation”, all with informing the European Parliament.
However, by way of derogation from art. 2, paragraph 5 and from art. 6 (a) and in accordance with art. 4, paragraph 2 (k) from the TFEU, the European Parliament and the Council are empowered that in accordance with the ordinary legislative procedure to contribute to the achievement of the objectives mentioned by art. 168 by adopting measures that establish high standards of quality and safety of medicines and medical devices, inter alia.
Apart from the actions for which the competence of the Union and the Member States is shared, art. 168 regulates, first of all, the horizontal policy feature of the action in the field of health carried out by the OMC. As regards the attributions of the institutions, it is foreseen the competence of the Commission to adopt “initiatives” (“not measures”), which include guidelines, indicators, exchange of practices [32]. The instruments adopted include, as in the case of the other competences of support of the Union, measures of encouragement, which are adopted by the ordinary legislative procedure, with the consultation of the Economic and Social Committee and the Committee of the Regions, as well as recommendations.
The OMC in the health and long-term care field was formally established in 2004 under the umbrella of the Social Protection Committee. The operative phase started in 2006, when the Council merged the three social OMC processes (for pensions, social inclusion, and health and long-term care). This action was rather surprising in view of the fact that the European Commission had tried (but failed) to secure a mandate in this field, from the March 2004 Spring European Council. As a matter of course, in its annual Spring Report, the Commission asked the European Council to ‘(e)xtend the open method of coordination in the social protection field to the modernization of healthcare schemes’ [4]. Significantly, the 2004 Spring European Council declined to adopt the proposal.
Some authors suggest that adopting as a general rule the implementation of the OMC in the field of health care was Member States willingness. Greer and Vanhercke [4] argue that a first sign for this statement is that the OMC was launched with a provisional institutional architecture (provisional common objectives, no common set of indicators, preliminary reports instead of action plans, etc.). Furthermore there, the Social Affairs Council not only underlined that this OMC should be initiated in a progressive and flexible manner, while placing a firm emphasis on added value’, but also decided it should ‘not impose an excessive administrative burden; health ministries should be directly involved in the OMC process. Additionally, the ministers for health opted to keep control of the European health care agenda in the Council. Thereby, in 2005, “health ministers agreed to draw up a statement on the core values and shared principles that unite the health systems of the Member States” [4].
Lamping and Steffen argue that the OMC “has gradually widened the Commission’s room for maneuver in policy areas to which the EU has no legal access”. Authors suggest that this instrument “is an attempt to politically coordinate the shift in specific areas of public policy among Member States and to channel them as similarly as possible via a postregulatory approach” [40].
Applied into the health area, the OMC “has above all enabled the Commission to switch from “aggressive leadership” unlikely to work in the health sector, to a softer and somewhat ideational political leadership” [40]. This leadership revolves around this kind of «issue networks», as we will see in the EUnetHTA case, characterized by a deliberate restricted admission into the network, limited fluctuation of membership, common frames of reference, stable and complex interactions between members as well as sustainable outcomes.
The OMC helps the European Commission to avoid many of the obstacles to supranational regulation of the health care sector such us: “weak official legitimacy, a vague policy mandate, and lack of political consensus (Member State interest in own health policy), weaknesses in policy coherence and policy formulation resulting from the intergovernmental system of negotiation as well as the internal structure of the health policy sector” [40].
EU networks join together Member State agencies and homogenize and sometimes create those agencies by putting together experts who work together and legitimate a European model in each of those sectors [4]. Some authors suggest that “this European model served a reference point when the experts proposed new organizations or more resources in their home governments” [4].

3. A Brief Introduction of the Health Technology Assessment in the EU

3.1. Origins

Most authors agree that Health technology assessment (HTA) in Europe started on small scale with the late 1970s with both formal and informal initiatives in different countries of the continent [41,42,43,44,45]. The first assessment of technologies decree was to be issued in the USA, in 1972, with which, the Office of Technology Assessment (OTA) was established ”to develop and disseminate HTA and demonstrate its usefulness to the political representatives” [46]. Although, some authors suggest that the term “health technology assessment” was most likely first used around 1967 [47,48].
In the 1980s HTA gained a lot of interest among academics and practitioners both in the USA and Europe. As concerns the development and diffusion of HTA in Europe, it maybe stated that the importance of assessment in Europe started to be understood when the World Health Organization (WHO), within the program “Health for Hall”, indicated “that European states identify a formal mechanism for a efficient assessment of the use of medical technologies to determine their effectiveness, efficiency, safety and acceptability” [46].
A significant addition to the development of HTA in the EU Member States was that many agencies in the field were settled up between the 1980s and 1990s. The first national agency was established in 1987, that is, The Swedish Council on Technology Assessment in Health Care [39,44,45,46,49]. The Swedish agency was soon followed by the establishment of other agencies and HTA funding programs in Spain, France and the Netherlands. Since then, the number of organizations or programs mandated to support decision-making in healthcare has developed continuously, especially in western Europe. As such, in the 1990s new agencies were established in Austria, Denmark, Finland, Norway, Germany, Hungary and the United Kingdom [43,50]. Later, these countries were followed by Ireland, Belgium, Poland, Italy, and now most of the EU Member States.
The Swedish Council on Technology Assessment in Health Care’s mandate ”was to provide evidence-based information on matters of health technology to guide health policy and practice”. As Garrido et al. [48] emphasize, ”it was made explicit that the agency should synthesize research findings and present this information in a manner understandable to both experts and the lay public”. Additionally, an important aspect was that the agency ”should focus not only on clinical aspects, but also on the economic, ethical and social implications of different technologies, procedures and programs for preventing, diagnosing and treating disease”, as authors argue [48].
Another important step for the development of HTA in Europe was the creation of the HTA Program in the United Kingdom. It all started by understanding the ”need for the NHS to identify its research needs and ensure that knowledge from research is transferred to services, the NHS National Research and Development Program—in which HTA has been most prominent—was established with solid funding in the mid 1990s” [51].
Similarly, another boost in HTA domain was the establishment of the National Institute for Health and Clinical Excellence (NICE) in 1999. The main areas of activity of this institution are ”the implementation of guidelines and evaluation of new health care technologies already in use: drugs, medical devices, diagnostic tests, clinical procedures and aspects concerning health prevention” [46].
Jonsson [41] mentions that these developments were observed with great interest in the rest of Europe. As a result, other countries also started HTA programs during this time, partly in cooperation with university departments, leading to the creation of national agencies in the 2000s [48]. Banta, Kristensen and Jonsson [43] argue that ”all agencies experienced the need to account for country-specific circumstances, such as the actual healthcare system, financing of health care, demography, disease panorama, available resources, and wealth”.
In the 21st century, HTA has continued to develop more and more. Existing agencies have grown and HTA has become a talked-about field in universities and think tanks, in industry, and in some consumer-based organizations [45].
Banta, Kristensen and Jonsson [43] also note that ”the involvement of the European Commission arguably became a very important factor in promoting HTA at the European level, along with other organizations” such us International Society on Technology Assessment in Health Care, International Network of Agencies for Health Technology Assessment, the World Bank or WHO—Regional Office for Europe.

3.2. HTA Basic Considerations

Although our approach does not aim to debate the specific HTA scientific field, we believe that in order to understand the phenomena it is useful and at the same time mandatory to shortly introduce the unfamiliar reader into some of the main and general considerations about HTA.
It should be noted that every country has a structure of health policies that influence health technology. Banta [45] reflected on the relation of health policies and HTA and highlighted that ”from its beginnings, HTA has focused on these policies, especially policies related to regulation, quality and payment for care, as a target for its work. This is in accord with the definition of HTA as a form of policy analysis”.
For Scaletti [46], a very broad definition that appears to be more responsive to the scientific context, particularly in business studies, is the one proposed by OTA according to which “all the tools, equipment, medicine and procedures used in dispensing health services, as well as the organizational and support systems through which health care is provided” fall into same category.
Kristensen et al. [42] argue that HTA offers an approach for improving the knowledge base for healthcare policy and decision making across a broad range of technologies. They indicate that HTA is “a multidisciplinary process that summarizes information about the medical, social, economic, and ethical issues related to the use of health technology in a systematic, transparent, unbiased, robust manner. Its aim is to inform the formulation of safe, effective, health policies that are patient focused and seek to achieve best value” [42]. The same meaning can be found in the Directorate-General for Health and Food Safety of the European Commission’s study entitled ‘Mapping of HTA national organizations, programs and processes in EU and Norway’ [52].
A quite similar principle related to HTA can be found at Drummond [53] for which ”HTA is a dynamic, rapidly evolving process, embracing different types of assessments that inform real-world decisions about the value (i.e., benefits, risks, and costs) of new technologies, interventions and practices”.
More often than not, its focus tends to be on a health economic summary of “value for money” of new pharmaceuticals, which is used in Europe today to give guidance to public health-care payers if a new drug should be funded or not [54]. The cost issue was also addressed by Sorenson, Drummond and Kanavos [50] in which belief an HTA’s principal aim “is to provide a range of stakeholders (typically those involved in the funding, planning, purchasing and investment of health care) with accessible, usable and evidence-based information to guide decisions about the use and diffusion of technology and the efficient allocation of resources”. As such, can be argued that this instrument also contributes to the sustainability of national health systems.
In Scaletti’s view, ”HTA is based on the evaluation of the most relevant knowledge available on a given issue. It is a process that takes advantage of and adopts both the techniques of research, that are strictly scientific, and the managerial administrative ones, focused on decision-making analysis, creating a bridge between the scientific model (science paradigm) oriented on performance analysis technology and decision-making activities (policy paradigm) aimed at evaluating the effective and efficient use of resources” [46].
Generally, ”the ability to conduct an HTA as a complex multidisciplinary process requires the expertise of several areas of research: clinical opinion, patient preferences, information specialists, biostatistical analysis, economical evaluation expertise, ethics and policy impact analysis, among many others” [55].
With regard to the objectives of an HTA, Hopkins and Goeree [55] refer to International Network of Agencies for Health Technology Assessment’s perspective. These objectives are as follows:
”Identify evidence, or lack of evidence, on the benefits and costs of health interventions
Synthesize health research findings about the effectiveness of different health interventions:
  • Evaluate the economic implications and analyse cost and cost-effectiveness
  • Appraise social and ethical implications of the diffusion and use of health technologies as well as their organizational implications
  • Identify best practices in health care”.
While looking at the scope of the HTA, in 2017, at the EU level 23 Member States indicated having an HTA system that includes assessment of pharmaceuticals, 20 Member States indicated having an HTA system that includes assessment of medical devices, and 17 Member States indicated having an HTA system that includes assessment of other technologies [52].
Of course, except the scope already mentioned above, the scope of the HTA may be viewed by other potential users as follows: (i) Clinicians and patients with the scope of appropriate use of health care interventions for a particular patient’s clinical needs and circumstances; (ii) investors and companies for venture capital funding, acquisitions and divestitures and other transactions concerning health care product and service companies; (iii) Standards-setting organizations having in mind manufacture, use, quality of care and other aspects of health care technologies [55].
Regarding the role of HTA in decision-making, the European Commission’s study show that HTA ”is used to inform primarily pricing and reimbursement decisions, with a majority of the EU countries (24) indicating using HTA to inform reimbursement decisions on pharmaceuticals” [52]. Concerning the decision-making on medical devices, fewer EU countries (19) apply HTA to inform reimbursement decisions and clear minority (9 EU countries) apply HTA to inform pricing decisions.

4. European Collaboration Regarding HTA and EUnetHTA—A Very Fertile Field for Using OMC

4.1. EU’s Health Policy Particularization and HTA Interaction: Developing a Network Approach

First, we must take into account that in all of the 27 countries of 2020 (as the UK left the EU), which make up the European Union, the State has the responsibility for providing healthcare and universal coverage, whether tax- or insurance-based. All Member States have to deal in general with almost similar challenges for their healthcare systems: ageing populations, and complaints about waiting times, accessibility, and quality of care. In this context, a major common problem appeared, that is “the escalating costs of health services at the same time as there is widespread public and professional demand for new healthcare technology” [43].
As Chowdhury emphasize, some scholars have characterized EU’s health regulatory processes as “new architecture of experimental governance” highlighting a set of specific attributes of European governance such us framework goals that are agreed jointly by member states and EU institutions, self-rule to local bodies within member states to device strategies and mechanisms to implement those rules as well as to engage in a peer review process that regularly assess their performance [1].
The shift from law-based to network-based governance within the EU by focusing on such processes as the OMC has been highlighted over the years by various scholars [1].
As a general rule in the EU, pricing and reimbursement decisions belong to member states competence and are taken at national level, although they have to conform to overall EU regulation [56]. Likewise, EU member states “show overall similarities in pricing, as well as reimbursement”, nevertheless there are differences related to the peculiar policies [56].
Against that background, the European Commission has supported and funded several projects to promote collaboration of Member States on HTA. These were mostly done under the program on health services research, and HTA was not seen strategically at that point in time, but was viewed as yet another form of health services research. The various EU-funded projects created a specific understanding of HTA as a scientific and policy-making field [54]. However, by the mid 1980s, the Health Services Research Committee of the European Commission began to favor HTA with contracts on economic appraisal, variations in use of particular technologies, and mechanisms for regulating expensive health technologies in different countries [43].
Between 1994 and 2002 the European Commission has funded three major projects that sought to support collaboration on HTA methods and working: EUR-ASSESS, HTA-Europe and ECHTA/ECAHIP. The later projects “stressed the need for a permanent structure to support HTA coordination in Europe to avoid duplication, maximize scarce resources, strengthen HTA in Member States and ultimately contribute to the better health of all European citizens” [57].
In May 2002, the High Level Group on Innovation and Provision of Medicines in the EU (G10) recommended that the European Commission organize a European process to reflect on how Member States could improve ways of sharing information and data requirements. The purpose would be to achieve greater certainty and reliability for all involved, even if their policy decisions might differ. One objective was to foster the development of HTA, including clinical and cost-effectiveness, in the Member States [42].
This was soon followed by a report on healthcare developments which was presented in 2003. According to Kristensen et al. [42] this document stated that “HTA could assist policy makers in making informed decisions by providing evidence on medical, social, economic, and ethical issues concerning healthcare policy and practice”. The report recommended inviting the European Commission to consider how a sustainable network and coordination function for health technology assessment could be organized and funded and to make an appropriate proposal.
Moreover, on the eve of the Eastern Enlargement, the Commission juxtaposed HTA and inequalities of access in its 2004 Communication by mentioning its agenda to “ensure universal access to high-quality services” and by using EU-level HTA initiatives to “help to ensure that patients throughout Europe benefit from care reflecting the latest advances in medical technology” [58].
The High Level Group on Health Services and Medical Care established by the European Commission in 2004, identified an urgent need to establish a sustainable network for HTA and proposed several steps starting with a 3-year project supported by the EU Public Health Program, endorsed also by the Council of Ministers [42]. Thus, the Council of Ministers was calling HTA a political priority: ”(…) the European Council concluded that the exchange of expertise and information through HTA may be enhanced through systematic EU-wide cooperation, in order to assist the Member States to plan, deliver and monitor health services effectively, based on the best available scientific evidence on the medical, social and economic implications of health technology” [59]. The EUnetHTA project was cofunded by the European Commission and participating partner organizations during the period of 2005–2008.

4.2. EUnetHTA Objectives

The European Network for HTA (EUnetHTA) was formed to connect public national/regional HTA agencies, research institutions and health ministries, in order to enable effective exchange of information and support Member States’ policy decisions [50,59]. The project intended to create an effective and sustainable network for HTA across Europe that could develop and implement practical tools to provide reliable, timely, transparent, and transferable information to contribute to HTAs in Members States [42].
The EUnetHTA project involved 64 organizations: one main partner (DACEHTA in Denmark), 34 associated partners, and 29 so-called collaborating partners. In total, 33 countries (Europe: 25 EU and 2 European Economic Area countries (Norway, Iceland), Switzerland and Serbia; outside Europe: Australia, Canada, Israel, United States) participated in the project. More than 300 individuals were directly involved in the project, including several from the new members of the EU [43].
Manifestly, no member states wish to be excluded from cooperation on HTA. Even countries with almost no HTA capacity nominate at least one formal contact person, often from the Ministry of Health. “The empowerment, or effective creation, of networks of experts with the legitimacy to formulate desirable course of action is a key characteristic of the policy stream in the case of European HTA” [54].
The main aim of those involved in the project was the development of practical tools for transnational collaboration and that a permanent coordination and communication for HTA would be set up at the conclusion of the project, funded by the European Commission.
The strategic objectives of the EUnetHTA Project were “to reduce duplication of effort and promote effective use of resources for HTA, increase HTA input to decision making in Member States and the European Union to increase the impact of HTA, strengthen the link between HTA and healthcare policy making in the European Union and its member states, and support countries with limited experience with HTA” [42,60]. Table 1 presents an overview of the of the EUnetHTA strategic objectives as well as the specific objectives. The objectives were developed in 2005 and were adjusted reflecting the experience, needs and outcomes from the work performed in the project and changing healthcare systems policy environment.
The three-year work program was developed and supported by a well-organized management function. On this firm basis, the EUnetHTA Project “quickly established an open network supported by state-of-the-art communication tools to promote exchange of information and development of tools to assist the coordinated provision of HTA information” [57].
Regarding specific objectives, 8 specific objectives were defined to facilitate rapid, productive collaboration that would lead to the development of a range of practical tools to deliver the strategic objectives. Those were followed by corresponding key deliverables.
Concerning the membership, The EUnetHTA project emphasized involvement of stakeholders in its processes. EUnetHTA set out to identify relevant groups, develop contact and consultation, collect feedback and advice, and discuss the future of EUnetHTA. The first involvement of stakeholders was in a half-day Forum organized by EUnetHTA in the European Health Forum Gastein 2006, where representatives from hospital management, industry, and healthcare management discussed European HTA with representatives from public health, government, the EU Commission, international funders, and HTA agencies [42].
To ensure the responsiveness of the EUnetHTA Project to the needs of the Member States and the EU, regular updates on the progress of the Project were given to DG SANCO and the High Level Group on Health Services and Medical Care. Additionally, the Secretariat regularly monitored and informed the Executive Committee and all EUnetHTA Partners about healthcare policy developments at the EU level. Partners “were also encouraged by the Executive Committee to make contact with their Ministry of Health to discuss the work of the EUnetHTA Project and gain support for ongoing work nationally” [57].

4.3. Results

During the 3 years in which The EUnetHTA Project spanned it comprised eight Work Packages (WPs). It developed a structure for coordination, management, and governance to support the work facilitated by three WPs. As Kristensen et al. [41] mention, the WPs developed “annual work plans that were shared with other WPs in the overall coordination of the project management structure. A wide spectrum of methods were applied in the WPs, for example, literature searches, survey questionnaires, Delphi surveys, pilot and applicability testing of tools, structured reviews of drafts, and meetings among experts and other forms of collaboration to build consensus”.
The EUnetHTA Project progressed this work by placing emphasis on developing practical tools, systems and structures that would allow application of the good methodological guidance on HTA in a transnational HTA collaboration. The purpose of its work was “to avoid duplication and ensure better use of resources available for HTA work, and enhance effective uptake of evidence-based input to health policy and planning” [57]. Therefore, the EUnetHTA project aimed to create tools and systems (concrete outputs) to facilitate sharing of information and coordination of HTA activities [57].
The governance, management structures, and organizational tools for effective collaboration on HTA in Europe proved able to effectively support the scientific work to meet the defined objectives. In an overview of results of the practical methodologies and tools for HTA developed in EUnetHTA, Kristensen et al. [41] argues that EUnetHTA achieved its objectives and delivered tangible results complying with its project description, while involving a large group of organizations and individuals.
In the Report following the end of the project, it is stated that “the EUnetHTA Project achieved its specific objectives, with some additional achievements. However, it was not just production of the deliverable, but the quality and usability of the output that was paramount in this Project which intended to deliver practical tools as well as ‘real-time’ transnational collaboration made possible by the processes and facilities developed through the project” [57].
To conclude, the work of the EUnetHTA Project has involved two clear strands:
Delivering tools and information to support HTA in Europe; and
Developing a well-functioning network of national HTA organizations that can share information and undertake joint work.
The practical nature of the EUnetHTA Project, and a transparent governance and management structure, helped to achieve tangible results that should create added value for HTA in Europe. It enabled close collaboration among many organizations and individuals across national borders, cultures, and the systems facilitating change.
This has positioned EUnetHTA at the center of committed collaboration on HTA between EU Member States and the Commission, to ensure the continuation and development of HTA in the EU [42].
The EUnetHTA Project succeeded in building practical tools for several of the key areas of HTA:
“Setting up a new agency;
Informing about new technologies;
Facilitating new evidence generation;
Performing and reporting actual cross-border assessments to support timely, relevant, high-quality Core HTA information that can be used for national/regional reporting;
Adapting information from one setting to another; and understanding better the relation between HTA and health policy”
In order to assess this collaboration from the participant’s perspective, annual self-completion online questionnaires were sent to identify their views about the project processes. As such, when it comes about the perceived benefits of the collaboration, information sharing, developing of methods and tools for doing HTA as well as formation of a sustainable network were reported by the beneficiaries [61]. The two most common benefits classified as “very useful” were: sharing information and networking. Other benefits were cited as capacity building and training [61].
EUnetHTA has prepared the necessary organizational framework, the collaborative working process, and the main tools to facilitate daily work. This structure has built a solid foundation for concrete European collaboration in HTA [41].

4.4. Further Collaboration and Steps towards a Mandatory National Uptake of the Joint Clinical Assessments

As a result of a decision in early 2009 between the EU and Member-State-appointed HTA bodies and representatives, a 3-year Joint Action (2010–12) under the EU Health Program (2008–13) represented the basis for continuation of European networking in HTA and further work on relative effectiveness assessment of pharmaceuticals.
EUnetHTA was asked to bring the Joint Action forward, thus implementing the aim of sustainability of a European network for HTA at the request of governments and the European Commission [42].
The Cross Border Directive (2011) provided the political and regulatory framework for Joint Action 1 (JA1) and the succeeding JA2 and JA3, stating in article 15 that “The Union shall support and facilitate cooperation and the exchange of scientific information among Member States within a voluntary network connecting national authorities or bodies responsible for health technology assessment designated by the Member States” [62].
The third JA (JA3) period was 2016–2020 but it was extended until May 2021 [49]. The objectives evolved through the project itself and the subsequent JAs was “to support efficient production and use of HTA in countries across Europe through the reduction of redundancies and duplication of effort, and by strengthening the link between HTA and healthcare policy making” [60]. In JA1 the main objective was “to enable an effective exchange of information and support of policy decisions. In JA2 the focus was laid on establishing an effective and sustainable HTA collaboration in Europe and strengthening the practical application of tools and approaches to crossborder HTA collaboration”. In EUnetHTA JA2, core HTA information—both full and rapid assessments—is produced in collaboration with methods developed in EUnetHTA JA1 [63].
Finally, the systematic development and establishment of quality management for EUnetHTA processes and products were initiated in Joint Action 3 (2016–2021) in order to improve the efficiency and quality of joint work and also to define and implement a sustainable model for European cooperation post 2020 [49].
In January 2018, the European Commission published the “Proposal for a Regulation of the European Parliament and of the Council on health technology assessment and amending Directive 2011/24/EU”. The proposal has the specific objectives “to promote convergence in HTA tools, procedures and methodologies; to ensure efficient use of resources and strengthen the quality of HTA across the EU and to improve business predictability.”
Erdös et al. [60] argue that this proposal is based on “intensive mapping of HTA structures and methodologies and impact analyses of all member states and can be interpreted as a sign of commitment and a strong wish from all parties to strengthen the cooperation and not lose the achieved results”.
Vella Bonanno et al. [64] indicate that Member States acknowledge the challenges posed by the different methodologies, tools, and models used for the utilization of HTA across countries, and generally consider the standardization of approaches positively. Authors suggest that benefits of HTA collaboration include unification and implementation of common criteria for Member States and companies, streamlining of activities, avoidance of duplication, more effective use of resources through the development of joint methodologies, synergies between experts and the authorities, availability of best expertise to cover future challenges, the possibility of having a common framework to support the process and the possibility of one joint submission [64].
However, a number of concerns were expressed with the Proposal. There is a belief that “transferring the authority to conduct comparative benefit assessments of medicinal products to a single body with a binding effect on Member States is a violation of primary law, and this could have severe consequences among national pricing authorities in ensuring the cost-effective provision for their medicinal products” [64].
Thus, if such a proposal will be adopted this will mean a step forward for the EU overall interest as an harmonized EU process will be used by all national bodies and furthermore this will be a mandatory one (i.e., mandatory clinical assessments). We should emphasize that at the moment the dossier is still under discussion in the Council. The Romanian Presidency (January–July 2019) continued the technical negotiations initiated by the Austrian Presidency in 2018 in the Council and held not less than height meetings in which they addressed the tasks and the composition of the coordination group, procedural issues of joint clinical assessments, and scientific consultations.
During the Finnish Presidency (July–December 2019), views were primarily exchanged about the fact that the European assessments should gradually begin, which and how many health technologies should be subject to mandatory Joint clinical assessments as well as which obligations should be imposed on pharmaceutical companies and the member states [65]. The current pandemic context produced little progress in the Council working group under the Croatian Presidency but it was revitalized again between July–December 2020 when Germany held the Presidency of the Council of the EU.

5. Final Remarks

This paper has analyzed network governance with an emphasis on concept, actors and structure. A careful attention was directed to the EU case. We found that there is no general acceptance regarding a definition unanimously accepted, but we have noticed that looking from some narrow contexts, the concept might be directed to certain characteristics accepted by a majority of scholars.
As such, for Börzel [10], Rhodes [13], Eising and Kohler-Koch [11], Torfing [7], Klijn and Koppenjan [8] and Bevir and Phillips [14] some certain common elements in their approaches can be identified as peculiar to network governance: a variety of actors who share a common interest in policy-making process and who exchange resources in order to achieve common interests, as well as recognizing that cooperation is the best way to achieve common goals.
In our opinion, with regard to the literature review presented in this paper, network governance stands as a way by which certain policy issues can be addressed within an environment governed by a plurality of actors with various specific objectives but wishing to achieve the same general objective. Being aware of the network abilities, take the opportunity and agree to share resources and values which they manage together.
We observed, looking at the EU case, that the fragmentation resulted from several actors involvement in the policy-making cycle creates a fertile frame to use network governance. In this context, the OMC appears to be a very useful tool given that it was defined from the begging as a concrete way of developing modern governance; an instrument to organize a learning process in order to stimulate exchange and the emulation of best practices; and a way of encouraging management by objectives. These are just some of the OMC characteristics assigned to it by João Rodrigues [29]. We have seen that there are differences between the various OMC types depending on the domain concerned. Nevertheless, certain principles and procedures remain constant in the EU case.
In the section dedicated to our case study, which covers the HTA system in the EU, we analyzed and identified the origins of this domain, its significance for health systems and the main features. Although it is a very technical field, we introduced the inexperienced reader succinctly in the field of HTA and later we focused on the existing collaboration at EU level, respectively EUnetHTA.
After a brief analysis of the emergence of this network, we focused on its objectives and later on the results achieved. The success of the EUnetHTA collaboration, based on the results achieved within the network, encourage the idea that ”the EU’s capacity to impact on Member State health systems and policies is reaching far beyond its rather modest formal competency in this field” [40].
The number of Member States participating in the collaboration indicates the acknowledgement of the added value that network provides to their cause. We noted that 25 out of the 27 Member States are active participants of the network, while 25 Member States indicated that they use EUnetHTA tools in their national HTA processes [56]. That means that even though there is no legal binding legislation, a uniform approach has been proliferated among Member States.
At the same time, an interesting element we found is that on the basis of this OMC-type collaboration, European Commission has initiated a Proposal which aims, inter alia, to transfer the authority on clinical assessments of medicines to a single EU body. Their results will have a binding effect for Member States. Counter-intuitive as it can be, whether the Proposal will not be adopted in the initial form proposed by the Commission, the future legislation might be seen as a contribution of the works of EUnetHTA, designed to strengthen the supranational role in this field of policy.
Although the study of the effect of OMC as a governance tool is quite recent and still provisional, as reflected by the “recent emergence of EU health policy issues and the reluctance of Member States to permit even this relatively unthreatening expansion of EU competence, there are conditions, identified in studies of the most-researched mechanisms (the OMCs for other policy areas), that allow us to judge the likelihood of an effect” [4].
Therefore, it can be stated that in this particular case, from using a ”soft law” instrument (voluntary and non-binding) has been developed a framework that eroded what is known as ”hard law”. As a consequence, if the discussed proposal will be adopted in this form, the EU competences in the health policy are going to be the real winner.
Such instruments of EU network governance in this particular field of health care are here to stay. It may serve the divergent objectives of numerous actors and “is often a simple recognition of networks that already exist” [4]. The challenge is to figure out when, how and why it matters.

Author Contributions

This article had been elaborated under the direct coordination and supervision of A.M.; conceptualization, formal analysis and investigation, A.-S.D. and C.-G.A. All authors have read and agreed to the published version of the manuscript.


This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Mossialos, E.; Permanand, G.; Baeten, R.; Herve, T. Health systems governance in Europe: The role of European Union law and policy. In Health Systems Governance in Europe: The Role of European Union Law and Policy; Cambridge University Press: Cambridge, UK, 2010. [Google Scholar]
  2. Radaelli, C. Europeanization: Solution or problem. In European Union Studies; Cini, M., Bourne, A., Eds.; Palgrave Macmillan: Bsingstoke, UK, 2006. [Google Scholar]
  3. Hatzopoulos, V. Why the OMC is bad for you: A letter to the EU. Eur. Law J. 2007, 13, 309–342. [Google Scholar] [CrossRef]
  4. Greer, S.; Vanhercke, B. The hard politics of soft law: The case of health. In Health Systems Governance in Europe; Mossialos, E., Permanand, G., Baeten, R., Hervey, T., Eds.; Cambridge University Press: Cambridge, UK, 2010; pp. 186–230. [Google Scholar]
  5. Berry, F.S.; Choi, S.O.; Goa, W.X.; Jang, H.; Kwan, M.; Word, J. Three traditions of network research: What the public management research agenda can learn from other research communities. Public Adm. Rev. 2004, 64, 539–552. [Google Scholar] [CrossRef]
  6. Koliba, C.; Meek, J.W.; Zia, A. Governance Networks in Public Administration and Public Policy; CRC Press: Boca Raton, FL, USA, 2010. [Google Scholar]
  7. Torfing, J. Governance network theory: Towards a second generation. Eur. Political Sci. 2005, 4, 305–315. [Google Scholar] [CrossRef]
  8. Klijn, E.-K.; Koppenjan, J. Governance Networks in the Public Sector; Routledge: New York, NY, USA, 2016. [Google Scholar]
  9. Hajer, M.; Versteeg, W. A decade of discourse analysis of environmental politics: Achievements, challenges, perspectives. J. Environ. Policy Plan. 2005, 7, 175–184. [Google Scholar] [CrossRef]
  10. Börzel, T.A. Organizing Babylon: On the Different Conceptions of Policy Networks. Public Adm. 1998, 76, 253–273. [Google Scholar] [CrossRef]
  11. Kohler-Koch, B.; Eising, R. The Transformation of Governance in the European Union; Routledge: London, UK, 1999. [Google Scholar]
  12. Wasserman, S.; Faust, K. Social Network Analysis. Methods and Applications; Cambridge University Press: Cambridge, UK, 1994. [Google Scholar]
  13. Rhodes, R.A.W. Understanding Governance; Open University Press: Buckingham, UK, 1997. [Google Scholar]
  14. Bevir, M.; Phillips, R. Genealogies of European governance. Comp. Eur. Politics 2016, 15, 685–704. [Google Scholar] [CrossRef] [Green Version]
  15. Ion, O.-A. Governance of the European Union: Current Approaches; Polirom: Iaşi, Romania, 2013. [Google Scholar]
  16. Sørensen, E.; Torfing, J. Theories of Democratic Network Governance; Palgrave Macmillan: London, UK, 2007. [Google Scholar]
  17. Milward, H.B.; Provan, K.G. Governing the hollow state. J. Public Adm. Res. Theory 2000, 10, 359–379. [Google Scholar] [CrossRef]
  18. Rhodes, R.A.W. The Hollowing-Out of the State. Political Q. 1994, 65, 138–151. [Google Scholar] [CrossRef]
  19. Marcussen, M.; Torfing, J. Democratic Network Governance in Europe; Palgrave Macmillan: Basingstoke, UK, 2007. [Google Scholar]
  20. Wallace, H. An Institutional Anatomy and Five Policy Modes. In Policy-Making in the European Union, 6th ed.; Wallace, H., Pollack, M.A., Young, A.R., Eds.; Oxford University Press: New York, NY, USA, 2010; pp. 70–100. [Google Scholar]
  21. Hix, S. The Political System of The European Union, 2nd ed.; Palgrave Macmillan: Basingstoke, UK, 2005. [Google Scholar]
  22. Warleigh-Lack, A.; Drachenberg, R. Policy Making in the European Union. In European Union Politics, 3rd ed.; Cini, M., Perez-Solorzano Borragan, N., Eds.; Oxford University Press: Oxford, UK, 2011; pp. 209–224. [Google Scholar]
  23. Beyers, J.; Kerremans, B. Bureaucrats, Politicians, and Societal Interests: How is European Policy Making Politicized? Comp. Political Stud. 2004, 37, 1119–1150. [Google Scholar] [CrossRef]
  24. Jönsson, C.; Strömvik, M. Negotiations in networks. In European Union Negociations. Processes, Networks and Institutions; Elgström, O., Jönsson, C., Eds.; Routledge: Abingdon, UK; New York, NY, USA, 2005; pp. 13–28. [Google Scholar]
  25. Hofmann, H.C.H.; Türk, A.H. An introduction to EU administrative governance. In EU Administrative Governance; Hofmann, H.C.H., Turk, A.H., Eds.; Edward Elgar Publishing Limited: Cheltenham, UK; Northampton, UK, 2006; pp. 1–18. [Google Scholar]
  26. Büchs, M. New Governance in European Social Policy. In The Open Method of Coordination; Palgrave Macmillan: New York, NY, USA, 2007. [Google Scholar]
  27. Diedrichs, U.; Reiners, W.; Wessels, W. The Dynamics of Change in EU Governance; Edward Elgar Publishing Limited: Cheltenham, UK; Northampton, UK, 2011. [Google Scholar]
  28. Craig, P.; de Búrca, G. EU Law: Text, Cases, and Materials, 4th ed.; Oxford University Press: Oxford, UK, 2009. [Google Scholar]
  29. João Rodrigues, M. The Open Method of Coordination as a New Governance Tool. In L’evoluzione della Governance Europea’, Special Issue of ‘Europa/Europe’; Telò, M., Ed.; Fondazione Istituto Gramsci: Rome, Italy, 2001; pp. 2–3. [Google Scholar]
  30. Craig, P.; de Búrca, G. EU Law: Text, Cases, and Materials, 5th ed.; Oxford University Press: Oxford, UK, 2011. [Google Scholar]
  31. Prpic, M. The Open Method of Coordination; (EP 542.142). 2014. Available online: (accessed on 15 October 2020).
  32. Gâlea, I. Treaties of the European Union: Comments and Explanations; C.H. Beck: Bucharest, Romania, 2012. [Google Scholar]
  33. Barcevičius, E.; Timo Weishaupt, J.; Zeitlin, J. Assessing the Open Method of Coordination. In Institutional Design and National Influence of EU Social Policy Coordination; Palgrave Macmillan: New York, NY, USA, 2014. [Google Scholar]
  34. Kjær, P.F. Between Governing and Governance: On the Emergence, Function and Form of Europe’s Post-National Constellation; Hart Publishing: Oxford, UK, 2010. [Google Scholar]
  35. European Commission. European Governance. A White Paper; Bruxelles. 25 July 2001. Available online: (accessed on 24 April 2019).
  36. Wincott, D. Backing into the future? Informality and the proliferation of governance modes (and policy participants) in the EU. In Informal Governance in the European Union; Thomas, C., Piattoni, S., Eds.; Edward Elgar Publishing Limited: Cheltenham, UK; Northampton, UK, 2003; pp. 226–236. [Google Scholar]
  37. Héritier, A.; Lehmkuhl, D. Governing in the Shadow of Hierarchy. New Modes of Governance in Regulation. In New Modes of Governance in Europe. Governing in the Shadow of Hierarchy; Héritier, A., Rhodes, M., Eds.; Palgrave Macmillan: New York, NY, USA, 2011; pp. 48–74. [Google Scholar]
  38. Borrás, S.; Ejrnaes, A. The legitimacy of new modes of governance in the EU: Studying national stakeholder’s support. Eur. Union Politics 2011, 12, 107–126. [Google Scholar] [CrossRef]
  39. Greer, S.L. The Politics of European Union Health Policies; Open University Press: Berkshire, UK, 2009. [Google Scholar]
  40. Lamping, W.; Steffen, M. European Union and Health Policy: The “Chaordic” Dynamics of Integration. Soc. Sci. Q. 2009, 90, 1361–1379. [Google Scholar] [CrossRef]
  41. Jonsson, E. Development of health technology assessment in Europe. Int. J. Technol. Assess. Health Care 2002, 18, 171–183. [Google Scholar] [CrossRef] [PubMed]
  42. Kristensen, F.B.; Lampe, K.; Chase, D.L.; Lee-Robin, S.H.; Wild, C.; Moharra, M.; Garrido, M.V.; Palmhøj Nielsen, C.; Røttingen, J.-A.; Allgurin Neikter, S.; et al. Practical tools and methods for health technology assessment in Europe: Structures, methodologies, and tools developed by the European network for Health Technology Assessment, EUnetHTA. Int. J. Technol. Assess. Health Care 2009, 25, 1–8. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Banta, D.; Børlum Kristensen, F.; Jonsson, E. A history of health technology assessment at the European level. Int. J. Technol. Assess. Health Care 2009, 25, 68–73. [Google Scholar] [CrossRef] [Green Version]
  44. Luce, B.R.; Drummond, M.; Jonsson, B.; Neumann, P.J.; Schwartz, J.S.; Siebert, U.; Sullivan, S.D. EBM, HTA, and CER: Clearing the Confusion. Milbank Q. 2010, 88, 256–276. [Google Scholar] [CrossRef] [Green Version]
  45. Banta, D. Dissemination of Health technology assessment. In Health Technology Assessment and Health Policy Today: A Multifaceted View of their Unstable Crossroads; del Llano-Señarís, J.E., Campillo-Artero, C., Eds.; Springer International Publishing: New York, NY, USA, 2015; pp. 147–154. [Google Scholar]
  46. Scaletti, A. Evaluating Investments in Health Care Systems. In Health Technology Assessment; Springer: London, UK, 2014. [Google Scholar]
  47. Banta, D. The development of health technology assessment. Health Policy 2003, 63, 121–132. [Google Scholar] [CrossRef]
  48. Garrido, M.V.; Børlum Kristensen, F.; Palmhøj Nielsen, C.; Busse, R. Health Technology Assessment and Health Policy-Making in Europe Current Status, Challenges and Potential; Observatory Studies Series No 14; World Health Organization on Behalf of the European Observatory on Health Systems and Policies: Copenhagen, Denmark, 2008. [Google Scholar]
  49. Waffenscfhmidt, S.; van Amsterdam-Lunze, M.; Gomez, R.I.; Rehrmann, M.; Harboe, I.; Hausner, E. Information specialist collaboration in Europe: Collaborative methods, processes, and infrastructure through EUnetHTA. Int. J. Technol. Assess. Health Care 2020, 37, 1–6. [Google Scholar] [CrossRef]
  50. Sorenson, C.; Drummond, M.; Kanavos, P. Ensuring Value for Money in Health Care, The Role of Health Technology Assessment in the European Union; World Health Organization on behalf of the European Observatory on Health Systems and Policies: Copenhagen, Denmark, 2008. [Google Scholar]
  51. Stevens, A.; Milne, R.; Burls, A. Health technology assessment: History and demand. J. Public Health Med. 2003, 25, 98–101. [Google Scholar] [CrossRef] [Green Version]
  52. European Commission. Mapping of HTA National Organisations, Programmes and Processes in EU and Norway; Publications Office of the European Union: Luxembourg, 2017. [Google Scholar]
  53. Drummond, M. Health technology assessment and its Interface with regulation, Policy and Management. In Health Technology Assessment and Health Policy Today: A Multifaceted View of Their Unstable Crossroads; del Llano-Señarís, J.E., Campillo-Artero, C., Eds.; Springer International Publishing: New York, NY, USA, 2015; pp. 3–12. [Google Scholar]
  54. Greer, S.; Löblová, O. European integration in the era of permissive dissensus: Neofunctionalism and agenda-setting in European health technology assessment and communicable disease control. Comp. Eur. Politics 2016, 15, 394–413. [Google Scholar] [CrossRef]
  55. Hopkins, R.B.; Goeree, R. Health Technology Assessment: Using Biostatistics to Break the Barriers of Adopting New Medicines; CRC Press: Boca Raton, FL, USA, 2015. [Google Scholar]
  56. Vogler, S.; Martikainen, J.E. Pharmaceutical Pricing in Europe. In Pharmaceutical Prices in the 21st Century; Din Babar, Z.-U., Ed.; Springer International Publishing: New York, NY, USA, 2015; pp. 343–370. [Google Scholar]
  57. EUnetHTA Secretariat. EUnetHTA Project. Overview of Results 2006–2008. March 2009. Available online: (accessed on 13 January 2021).
  58. Commission of the European Communities. Communication from the Commission to the Council, the European Parliament, the European Economic and Social Committee and the Committee of the Regions e-Health—Making Healthcare Better for European Citizens: An Action Plan for a European e-Health Area. Bruxelles. 30 April 2004. Available online: (accessed on 13 January 2021).
  59. Del Llano-Señarís, J.E. What Health technology assessment System Do we really Need? A Critical review of the Current Situation of Health technology assessment in Five European Countries. In Health Technology Assessment and Health Policy Today: A Multifaceted View of Their Unstable Crossroads; del Llano-Señarís, J.E., Campillo-Artero, C., Eds.; Springer International Publishing: New York, NY, USA, 2015; pp. 109–125. [Google Scholar]
  60. Erdös, J.; Ettinger, S.; Mayer-Ferbas, J.; de Villiers, C.; Wild, C. European Collaboration in Health Technology Assessment (HTA): Goals, methods and outcomes with specific focus on medical devices. Wien. Med. Wochenschr. 2019, 169, 284–292. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  61. Woodford Guegan, E.; Cook, A. European network for Health Technology Assessment Joint Action (EUnetHTA JA): A process evaluation performed by questionnaires and documentary analysis. Health Technol. Assessement 2014, 18, 1–285. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  62. Directive 2011/24/EU of the European Parliament and of the Council of 9 March 2011 on the Application of Patients’ Rights in Cross-Border Healthcare. OJ L 88. 4 April 2011. Available online: (accessed on 13 January 2021).
  63. Woodford Guegan, E.; Huić, M.; Teljeur, C. Eunethta: Further steps towards european cooperation on health technology assessment. Int. J. Technol. Assess. Health Care 2014, 30, 475–477. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  64. Vella Bonanno, P.; Bucsics, A.; Simoens, S.; Martin, A.P.; Oortwijn, W.; Gulbinovič, J.; Rothe, C.; Timoney, A.; Ferrario, A.; Gad, M.; et al. Proposal for a regulation on health technology assessment in Europe—opinions of policy makers, payers and academics from the field of HTA. Expert Rev. Pharm. Outcomes Res. 2019, 19, 251–261. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  65. Behring, A. Benefit assessment in the European context—The G-BA’s perspective. Interdiscip. Platf. Benefit Assessement 2020, 11, 52–61. [Google Scholar]
Table 1. Overview of the EUnetHTA strategic and specific objectives.
Table 1. Overview of the EUnetHTA strategic and specific objectives.
Strategic ObjectivesSpecific Objectives
(1) Reduce overlap and duplication of effort and hence promote more effective use of resources;
(2) Increase HTA input to decision-making in Member States and the EU and hence to increase the impact of HTA;
(3) Strengthen the link between HTA and health care policy making in the EU and its Member States;
(4) Support countries with limited experience with HTA.
(1) To establish the organizational and structural framework for the network with a supporting secretariat
(2) To effectively disseminate and handle HTA results, information sharing and coordination of HTA activities through the development and implementation of elaborate communication strategies and description of clearinghouse functionality
(3) To produce generic core models for HTAs on two essential categories of health technology questions: interventions and treatment, as well as core HTAs on selected topics for each category
(4) To develop and implement generic tools for adapting assessments made for one country to new contexts
(5) To develop and implement effective tools to transfer HTA results into applicable health policy advice in the Member States and EU—including systems for identification and prioritization of topics for HTAs and assessment of impact of HTA advice
(6) To structure prioritization for HTA and provide health care decision makers with policy relevant information on new and emerging technologies
(7) To provide tools to monitor the development of health technologies and to share data and results of this monitoring
(8) To establish a support system for countries without institutionalized HTA activity
Source: Own creation based on available information from EUnetHTA Secretariat, [57].
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Matei, A.; Dumitru, A.-S.; Antonovici, C.-G. The EU Health Technology Assessment and the Open Method of Coordination: A Relation with Potential in the Context of Network Governance. Sustainability 2021, 13, 3582.

AMA Style

Matei A, Dumitru A-S, Antonovici C-G. The EU Health Technology Assessment and the Open Method of Coordination: A Relation with Potential in the Context of Network Governance. Sustainability. 2021; 13(6):3582.

Chicago/Turabian Style

Matei, Ani, Adrian-Stelian Dumitru, and Corina-Georgiana Antonovici. 2021. "The EU Health Technology Assessment and the Open Method of Coordination: A Relation with Potential in the Context of Network Governance" Sustainability 13, no. 6: 3582.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop