Internet in the Middle of Nowhere: Performance of Geoportals in Rural Areas According to Core Web Vitals

Abstract

The spatial planning system in Poland is undergoing a fundamental reform. It emphasises the digital representation of spatial data. Low performance of geoportals, no Internet access, or poor connectivity can contribute to the exclusion from the spatial planning process, and consequently to the exclusion from a specific part of public life. Considering these developments, the present study seems relevant by pointing out the issue with geoportal performance and availability of quality Internet in rural areas. The primary contribution of the article is (1) results of performance measurements for selected geoportals; (2) presentation of measuring tools and performance indices combined with methods for ad-hoc performance measuring; and (3) presentation of potential actions to improve geoportal performance on the device with which it is used. The article offers case studies where the performance of selected geoportals was tested in rural mountainous areas with limited Internet access. Five geoportals were tested with PageSpeed Insights (PSI), WebPageTest, GTmetrix, Pingdom, and GiftOfSpeed. Core Web Vitals indices were analysed: Largest Contentful Paint (LCP), First Input Delay (FID), Cumulative Layout Shift (CLS), and First Contentful Paint (FCP). The author verified values of the Speed Index and Fully Loaded Time along with other performance indices, like GTmetrix Structure. The study failed to provide unambiguous evidence that radio link users in rural areas could experience problems with geoportal performance, although the results seem to suggest it indirectly. PSI Lab Data and Field Data tests revealed a relatively low performance of the geoportals. The Performance index remained below 50 in most cases, which is ‘Poor’ according to the PSI scale. The fully loaded time exceeded 10 s for all the geoportals and 20 s in some cases (Lab Data). It means that the perceived performance of the tested geoportals on a radio link in rural areas is most probably even lower. The case studies demonstrated further that the user has limited possibilities to speed up map applications. It is possible to slightly improve the geoportal experience through the optimisation of the device locally, but the responsibility to ensure geoportal performance is mainly the publisher’s.

1. Introduction

Mobile devices, especially smartphones, significantly improved Internet access. Mobile Internet access, and more recently, satellite access, can serve as the only alternative in rural and mountainous areas with sparse development and non-existent telecommunications infrastructure. Research shows that modern hardwired access infrastructure is installed first in cities and then in rural areas. This leads to a poorer availability of hardwire infrastructure in rural areas and a continued access divide between urban and rural areas [1]. Furthermore, studies revealed significant differences in the availability of modern telecommunications infrastructure between eastern and western Poland and northern and southern parts of the country [2]. The discrepancies are mostly due to geographical and socioeconomic factors. Eastern and north-eastern Poland is mostly rural with low or very low population density and scattered buildings. In such conditions, investment in telecommunications infrastructure entails such large costs, often exceeding the expected revenue, that it is infeasible in the long term [1]. Note the disparity in Internet access in rural and urban areas. In 2022, 93.3% of households in Poland had access to the Internet, which was 0.9 pp. more than in 2021. The type of access and connection depended on the kind of household, place, and degree of urbanisation. In terms of place of residence, more households could access the Internet in cities than in towns and villages. Similarly, the most urbanised areas had the largest share of households with Internet access. In addition, households with children had Internet access more often than those with no children [2].

Małopolskie Voivodeship in southern Poland, especially its southern rim, is mountainous. Developments there are often scattered, and the scattering issue is deteriorating [3]. It makes the installation of network infrastructure, also telecommunications infrastructure, such as optic fibre, difficult or even impossible. Moreover, the substantial scattering of buildings in rural areas makes them a daunting challenge for potential ICT investors due to high risk and long payback time. Despite this obstacle, the number of rural households with Internet access grows.

About 60% of the Polish rural population uses the Internet. While it is still less than in cities (85%), the percentage of rural households with Internet access grew from 15.5% in 2006 to 72.5% in 2018 [4]. The better availability of the Internet in cities is caused mostly by user dispersion (scattered developments and lower population density) in rural areas, which entails higher infrastructure costs. Urban areas are free of such problems because the high density of individuals and organisations fuels a greater intensity of Internet infrastructure use. Moreover, a smaller number of ISPs in rural areas can drive up prices, also for lower-quality services. Even if prices are comparable, the price-to-quality ratio is worse in rural areas [5]. In short, rural households usually can access the Internet but with a lower connection quality than in urbanised areas where buildings and infrastructure are denser.

Janc and Jurkowski [5] noted that Internet access is no longer a problem in most highly-developed European countries. More often than not households can access the Internet. Still, advances in digitalisation and developments in electronic services increasingly more often require high-quality connection [6]. Both download and upload parameters matter. Hence, a faster connection facilitates increased consumption of content and greater web user comfort. It also helps with more efficient work with sophisticated software used online. These circumstances open greater possibilities to learn, have fun, and work remotely or online [7]. In addition, official business can be conducted faster with online forms.

One type of web application provided by local authorities is the geoportal. Geoportals are integrated web-based systems providing tools for open spatial data sharing and geo-information management online. They usually give access to distributed data systems with maps, data search functions, and data downloads. Some offer online analysis and processing services, enhanced semantic search engines, and dynamic visualisation tools [8]. Geoportal users can view spatial data and search spatial databases and services, such as those related to spatial planning [9]. Therefore, the primary purpose of geoportals is to provide the public, business, and administration access to spatial information of appropriate quality from official registers. These applications are usually complex. This leads to the question of user comfort regarding map applications in rural environments in the context of poor-quality Internet access.

1.1. Motivation and Aim

Studies on Internet availability tend to adopt a global outlook and present aggregate data for a region, country, or continent [10,11]. Statistical data are often analysed and interpreted [12,13]. Rural internet access research focuses on its impact on rural household income and spending and feasibility, possibilities, and determinants of broadband growth in rural areas. Fewer researchers present in-depth investigations into the comfort of use of various applications, including GIS portals under limited Internet access. Furthermore, there are not so many studies on the geoportal usability as perceived by a single user during regular, everyday use, especially in rural areas. This poses a certain research gap worth investigating. Therefore, the present article offers case studies where the performance of selected geoportals was tested in rural mountainous areas with limited Internet access.

The objective is to measure the performance of selected geoportals through poor-quality rural Internet connection. Performance is defined as the comfort of website use considering user experience and equated to portal quality. Web portal performance, combined with its other quality attributes constitutes a network of interrelations. Hence, recognition of this problem has numerous positive outcomes and is critical for page experience evaluation. The performance was assessed with such indices as Google’s Core Web Vitals [14].

The spatial planning system in Poland is undergoing a fundamental reform [15]. Its foundations include digital services related to spatial planning and increasing digitalisation of spatial planning in line with global trends [9] aimed to improve public participation in the spatial planning process. The purpose of public consultation is to pursue tasks of local governments, voice the expectations of the public, and collect requests and comments. Geoportals and geo-questionnaires play an important role in this system. They help the public have more impact on local authorities’ decisions. Were it not for the opportunity to view geoportals and fill in geo-questionnaires efficiently, the possibility of the public participating in spatial planning remotely would be limited to non-existent. This means that the low performance of geoportals, no Internet access, or poor connectivity can contribute to the exclusion from the spatial planning process, leading to the exclusion from public life to a certain degree. Considering these developments, research pointing out the issues with portal performance and access to quality Internet in rural areas seems even more relevant.

1.2. Novelty and Contribution

The most common results of research on website performance are ranking lists and/or recommendations to optimise critical areas in need of improvement. Therefore, research most often suggests how to improve website loading times and reduce the loading of specific components or identifies those elements that are detrimental to content loading times. Audit recommendations provide guidelines for improving technical aspects of the website quality. Such suggestions are intended for publishers, administrators, or editors of web portals. They usually guide the improvement of the portal to enhance its quality for users and crawlers. The present study embarks on a different approach. According to the best of the author’s knowledge, it is one of a few projects attempting to advise users (the audience) on how they could optimise their devices to improve browsing performance. The expected contribution of the study is not to produce guidelines for administrators on how to optimise their portals, which is the most common idea in the literature but rather to guide users on what to do when their Internet access is poor. The paper investigated the following research questions:

•

How can a user with poor Internet access speed up a geoportal?

•

Can a website’s performance be measured for a specific location, at a specific connection speed, and with a specific device (here and now)?

The article emphasises the need for better design solutions to improve geoportal performance and develop new functions [16]. It is crucial because there are still many locations worldwide where Internet access is of poor or very poor quality. The primary contribution of the article is:

•

Results of performance measurements for selected geoportals conducted in rural and urban areas and then juxtaposed;

•

Presentation of measuring tools and performance indices combined with methods for ad hoc performance measuring;

•

Presentation of potential actions to improve geoportal performance on the device with which it is used, followed by a verification of the effort.

1.3. Organisation of the Article

This article is structured into six sections, beginning with an introduction and a literature review, where relevant key aspects of Internet availability and quality in rural areas are presented. Section two presents the results of past research, focusing on differences in the availability of broadband between urban and rural areas and Internet access as a business driver in rural areas. This section discusses the benefits of broadband in rural areas and circumstances that make investments in broadband infrastructure less feasible in rural areas. Section three contains methodology, including the test location and object with performance indicators. Section four offers results divided into Lab Data and Field Data. Section five thoroughly discusses the results and attempts to assess the impact of user efforts on the performance of the investigated geoportals. The article ends with a summary and practical implications.

2. Related Work

Authors of most publications on the availability and quality of the Internet in rural areas agree that the improvement in such attributes as better availability and quality of services stimulates rural development [12,17,18]. It is particularly relevant to income opportunities (such as online work) [19], although aspects related to online entertainment are significant as well. Moreover, access to the Internet opens new markets and affects purchasing and communication patterns in rural areas [20]. Internet access further improves the availability of basic healthcare, financial, educational, and other social services [21].

It is not only the Internet availability that counts but also its quality because it affects browsing and user comfort. Some activities can be carried out only through high-quality connections. Furthermore, researchers agree that there are disparities between Internet access and quality in cities and rural areas [22,23]. Nevertheless, it remains a substantial challenge to provide high-quality Internet connection because of the characteristics of rural areas [23,24].

2.1. Broadband Availability in Rural Poland

Even though recent years saw substantial changes in rural broadband availability and use, research shows that there still exist disparities in broadband availability between cities and rural areas [25]. Rural areas still have to catch up with urbanised zones in terms of access and speed of the broadband Internet. The divide is widened by the low use rate of broadband among small and medium enterprises in rural areas [26]. In contrast, research confirms that access to broadband is growing increasingly important for the development of rural entrepreneurship [27]. Despite the dynamic advancements in technology, many developing and developed countries offer only poor-quality connections or no Internet access at all [28]. In Canada, broadband with advertised speeds that meet its basic universal targets (50 Mbps download and 10 Mbps upload) is available to 87.4 percent of households. Still, services that meet the Canadian Radio-television and Telecommunication Commission (CRTC) speed targets are available only to 45.6 percent of households in rural areas [29]. Ochoa and Nonnecke [12] identified a positive relationship between rural Internet access and human development indicators measured by the United Nations’ Human Development Index (HDI). They demonstrated that an increase of 1% in the number of households with Internet access generated a 0.02% increase in the HDI in Mexico. Alternatively, if the number of households with internet access decreased by 1%, the HDI would fall by 0.07%. The authors emphasised that Mexico was the 15th-largest global economy with significant social and economic inequalities, which hindered its development potential. Only 20% of the rural population had access to the Internet compared to 62% in cities. This digital disparity increases the risk of further social, economic, and political differences [12]. Prieger [22] demonstrated that broadband availability was not growing in rural and urban areas at the same rates. Empirical estimates of broadband availability and usage in the US showed that rural areas had fewer high-speed hardwire and mobile ISPs but more slower-speed hardwire providers than urban areas. According to Prieger [22], US availability and adoption of high-speed hardwire broadband were lower in rural than urban areas, and mobile broadband offered great potential for improving economic development in rural areas. Whitacre [24] also investigated the difference between rural and urban Internet availability. He pointed out that rural areas suffered from supply and demand disadvantages regarding Internet access. Telecommunication service providers less often develop the infrastructure in rural areas because of lower population density and demand resulting from income levels, among others [24].

2.2. Advantages of Internet Access in Rural Areas

According to Canfield et al. [30], insufficient Internet access holds back local economies, reduces educational outcomes, and creates health disparities in rural areas. Much of the challenge for rural broadband infrastructure is related to a low return on investment due to high capital costs and low population densities. Internet use directly affects rural residents’ income growth [31], and rural broadband availability drives household consumption [32]. Ma et al. [18] investigated the effects of Internet use on household income and expenditure, with a sample of rural households from China. They demonstrated that the use of the Internet significantly increased household income and spending. Whitacre et al. [19] analysed relationships between the diffusion/availability of broadband and workplaces/income in the rural US using recent data from the Federal Communications Commission and the National Broadband Map. They showed that high levels of broadband adoption in non-metro counties were positively related to the number of firms and total employees in those counties. Moreover, increases in broadband adoption levels were associated with increases in the median household income and the percentage of non-farm proprietors in non-metro counties [19]. Whitacre et al. [17] evaluated how broadband contributed to rural US economic growth over a decade. They demonstrated that high levels of broadband adoption in rural areas positively (and potentially causally) affected income growth from 2001 to 2010, and limited unemployment growth. Similarly, low levels of broadband adoption in rural areas entailed a dwindling number of businesses and total employment [17].

Park [33] established that distance to infrastructure is a strong predictor of household Internet availability, but the digital divide is also aggravated by sociodemographic factors, such as education and employment status. Therefore, digital inclusion strategies in rural areas need to take into account both supply-side factors (availability of infrastructure) and demand characteristics, such as education levels, the industry sector, employment opportunities, and sociodemographics [33]. While Internet access can drive rural household expenditure, it also helps with saving money. Zeng et al. [34] pointed out that Internet use in rural areas significantly promotes rural household savings. Farmers who use the Internet and live close to a bank may exhibit higher saving rates than farmers using the Internet and living far from a bank. Sujarwoto and Tampubolon [35] concluded that the Internet divide may be bridged by improved distribution of telecommunication infrastructure and education facilities, especially in rural areas.

Some studies dispute the effect of Internet access on rural economic growth. According to Aldashev and Batkeyev [36], broadband access does not promote economic growth but stimulates retail, without affecting production and agriculture. Galloway [37] identified limitations of broadband technologies that made all of them unsuitable for rural use. Furthermore, Galloway [37] believed that rural enterprises lacked the urge to grow and diversify. Therefore, Internet availability might be secondary in light of the entrepreneurs’ unwillingness to develop their businesses. As a result, efforts to bring broadband to the countryside might fail to bear fruit [37]. Research by Duvivier and Bussière [38] showed that the positive effects of broadband access in rural areas were limited to municipalities with favourable initial conditions regarding local economic, environmental, and demographic aspects. Duvivier and Bussière [38] did not believe Internet access to be a panacea for socioeconomic problems, and over-focusing on broadband would have little to no effect on structurally weak rural areas. Other studies found a positive relationship between the availability of broadband and local economic growth. The dependence is stronger in those sectors that use ICT more and in areas with lower population density [39]. Still, economic benefits for local communities brought by the growth of Internet infrastructure seem to be limited. Expansion of broadband is connected with population and employment increase. Nevertheless, neither mean salary nor employment rate, that is the percentage of working-age adults with employment, contribute to more investments in broadband. Moreover, better availability of broadband does not affect the percentage of telecommuting or other forms of remote work [39].

According to Schneir and Xiong [23] the installation of high-capacity broadband networks in rural Europe lagged behind the infrastructure in urban and suburban areas. Schneir and Xiong [23] demonstrated that it costs on average 80% more to deploy a network near a town or village in a rural area compared to the cost of deploying the network in the town or village. According to Townsend et al. [40], the value of broadband lies in that it provides access to tools, which are particularly useful to rural businesses in overcoming the challenges associated with remoteness. Townsend et al. [40] concluded that some business sectors might grow more reliant on broadband than others. For example, creative professionals from rural areas are increasingly expected to deliver their products and services online, and this requires higher bandwidth than straightforward applications such as e-mail. The authors believed that to survive, rural enterprises should further their strong online position. It would eventually allow them to reach new markets, take advantage of opportunities and important industry events, and find potential areas for collaboration. Social networks help tap into the existing social capital to build economic capital, which is the primary potential of rural economies [40]. According to a study by Michailidis et al. [41] in three administrative regions of rural Greece, fewer than one in every three village inhabitants used the Internet. The research showed that the respondents most often used the Internet to browse social media and communicate via e-mail. It identified three categories of users: ‘basic users’, ‘socially interactive users’, and ‘farm-oriented users’. Martínez-Domínguez and Mora-Rivera [42] argued that the Internet divide remained wide in rural Mexico. They concluded that a reduction of the digital divide between urban and rural areas required improved Internet access through infrastructure deployment, and high-speed Internet services, a reduction of Internet link costs through more ISP competition, and education and awareness-building regarding the advantages and benefits of the effective use of the Internet. Selected phenomena related to broadband availability in rural areas are summarised in

Table 1. Scope of selected studies on the impact of broadband availability on rural socioeconomic development.

Item	Scope of Study	Selected Keywords	Reference
1.	Demonstration of the influence of broadband on three main rural economy sectors: retail, agriculture, and manufacturing.	broadband infrastructure in rural areas, high-speed Internet, economic growth	[36,37,43]
2.	Profile of rural Internet users. Characterisation of critical challenges linked to broadband projects in rural areas.	rural broadband, broadband investment, rural communities, development of rural areas, digital skills, Internet usage patterns	[29,30,38,41,42,44]
3.	Characterisation of relationships between broadband availability and property value in rural areas.	broadband availability, rural housing values, rural housing prices	[27,43,45]
4.	Analysis of differences in the availability, quality, and use of the Internet in deep rural, shallow rural, and rural/urban areas.	low-speed broadband connection, digital divides, urban-rural digital divide	[7,25,46]
5.	Analysis of the impact of broadband availability on rural socioeconomic development and agricultural production.	electronic commerce, purchasing patterns, broadband technologies, rural businesses, economic development, economic well-being, household income, agricultural productivity	[13,18,20,22,37,44]
6.	Analysis of digital exclusion and social benefits of broadband availability in rural areas.	online service system, digital divide, digital service delivery, social support networks, overall development of rural areas, wireless communication, wireless Internet Access, highly digitalised society, digital exclusion	[21,33,47]

.

Internet availability in rural areas entails specific social and personal outcomes, for example, for older people or regarding relationship fostering. Access to the Internet and webcam helps fend off the feeling of loneliness and isolation, cultivate relationships with friends and family from far away, and makes locally unavailable services available. Information and communication technology can help with the sense of independence and social role [47]. These advantages were found both in the social domain and agricultural production. Sawada et al. [48] demonstrated that Internet use can help control agricultural land abandonment. The results of Sawada et al. [48] suggest a link between ICT and land-use change and offer a new perspective for developing countries to guarantee food security. Moreover, the research implies that Internet use can encourage new farming technologies and affects more smallholders with low education levels, limited training, and high incomes. Hence, local governments should focus on Internet infrastructure growth and promotion of Internet use in rural areas to improve the adoption of agricultural production technology by farmers [49].

2.3. Internet Speed in Rural Areas and Its Implications

According to the Surfshark report [50] and Digital Quality of Life Index (DQL), the Internet is relatively cheap and of poor quality in Poland. The mean mobile speed (Internet Quality attribute) in September 2023 was 59.3 Mbps, which was below the global mean of 74.75 Mbps. In the global broadband category, Poland came in 24th with 186.69 Mbps. The global average was 107.75 Mbps. The Digital Quality of Life Index is calculated based on an analysis of five components, Internet Affordability, Internet Quality, Electronic Infrastructure, Electronic Security, and Electronic Government. In September 2023, Poland was 18th in the global ranking list with a DQL23 of 0.66, which is five places up compared to 2022. The list includes 121 countries with France (DQL23 of 0.79) and Finland (DQL23 of 0.75) leading the pack.

The Speedtest Global Index, which tracks countries’ Internet speeds and the overall global median Internet speeds, reveals that Internet connectivity continues to speed ahead for people around the world, especially as countries prioritize and improve mobile and hardwire broadband networks. Mobile download speed went up nearly 17% over the last year globally (November 2021–November 2022), and fixed broadband increased by at least 28% [51]. The quality of the Internet in Poland is also on the rise, although there is still plenty of room for improvement in terms of infrastructure and service quality. According to Ookla’s report [51], the mean mobile Internet speed in Poland is 43.75 Mbps for download and 8.91 Mbps for upload (Poland Median Country Speeds July 2023). This makes Poland 54th among 143 ranked countries. For comparison, the top-ranking countries offer download speeds over 150 Mbps or even 200 Mbps (United Arab Emirates). Internet speed differs not only by the country, city, level of infrastructure, or socioeconomic development. Other differentiating factors include the natural environment, geography, and the urban-rural divide.

Most inhabited areas of Poland have access to the Internet, but the COVID-19 pandemic and the remote education and work it brought demonstrated that connection speeds are too low. Even though efforts by local governments and ISPs led to an increase in the population with Internet access in remote locations, the Internet traffic demand leads to many of the networks failing to ensure a basic quality of service necessary to run simple applications, not to mention more advanced ones, such as video chats, fintech services [52], or the Internet of Things (IoT) in agricultural production [53]. This is true for Poland and other parts of the world, too [54].

Riddlesden and Singleton [55] noted that the heterogeneous geography of broadband infrastructure and projects resulted in variable service provision, leading to large disparities in access and performance at different spatiotemporal locations. Short-term dynamics reveal that in areas of different densities, speeds can fall dramatically during peak hours, thus influencing the availability of services. Disparities in access and performance pose a significant problem in light of Internet use becoming increasingly ubiquitous in everyday lives. Such inequalities evoke social and economic disadvantages at local and national scales [55]. Farrington et al. [46] found that while overall access to the Internet varied little geographically in rural and urban areas of Britain, very different connection speeds led to consistent differences between urban and deep rural Internet use, which provided evidence of an urban-rural digital divide and ways in which this divide was manifested. According to Whitacre and Mills [56] rural-urban differences in income and network externalities, but not in infrastructure, were the dominant causes of the high-speed gap. The research revealed also that broadband improved business productivity even by 7–10% consistently across urban versus rural locations and across sectors of various knowledge requirements [57]. According to Ioannou et al. [58], the deployment of high-speed broadband access networks in rural Europe lagged far behind the urban and suburban areas, especially due to difficulties with fibre rollout in the final meters. Fixed Wireless Access (FWA) networks based on Long Term Evolution (LTE) technology can be used as a last-mile solution to provide high-speed broadband access to areas where fixed broadband is limited [58]. There is an urgent need to bring rural areas into the mainstream by providing them with last-mile connectivity when other modes of communication are severely hampered [59]. The LTE technology offers high-speed connections capable of supporting Internet browsing and IP services. Thus, it can be considered a viable alternative to other hardwire network solutions [58]. According to Conley and Whitacre [43] not only does the deployment of faster household Internet speeds open new opportunities for entertainment, social interaction, and personal development, but it can also affect the value of the property. However, rural residents face lower availability, slower speeds, and limited provider options, putting them at a disadvantage when compared to urban residents [43]. Conley and Whitacre [43] showed that connected rural households, especially those with higher speeds, might expect a premium on their home value. Deller and Whitacre [45] reached similar conclusions. Townsend et al. [60] noted that creative industries potentially significantly contributed to the social and economic viability of rural regions. Their findings suggested that—at the time of their research—access to broadband of at least 2 Mbps, download speed, had become crucial for those working in the creative sector. Consequently, a lack of adequate access may be detrimental to rural communities by prompting out-migration to areas with better digital connectivity. Khalil et al. [61] argued that cellular technology was mainly for urban areas and would not be effective in rural environments. This is because traditional cellular models are not economically viable in low user–density areas with lower income with which rural areas are associated. The same applies to 5G mobile networks, which may add to the urban–rural digital divide. Research shows that broadband is not available in all rural areas for various reasons. In response, Zhang and Wolff [62] proposed an affordable, quick, and wireless Internet access, which can be deployed in rural and remote areas with innovative, out-of-the-box solutions to bridge the digital divide.

3. Materials and Methods

First, the authors developed the research concept considering the aim and object of the study using design thinking [63]. It was, therefore, possible to select the geoportals for performance testing, select test locations with the right Internet access type, and select performance indices and test tools (Figure 1). Design thinking puts the user at the central point of the design or research process. It is a type of conceptual thinking that pursues solutions that meet user’s needs and expectations. This approach emphasises understanding users’ emotions, thoughts, motives, and problems when using—in this case—a web application [63].

The article presents a case study. It represents the experience of a user in a rural area who has a laptop and a radio link. The connection quality was verified with a web application speedtest.net.pl (

Table 2. Performance test conditions, non-certified measurement.

Test Results	Download Speed		Upload Speed
	Rural Areas Radio Link (Mbps)	City Broadband (Mbps)	Rural Areas Radio link (Mbps)	City Broadband (Mbps)
speedtest.net.pl	10.33	320.10	2.58	134.65
Examples of download speed *
Fibre optic access	243.2		108.9
Household access	129.1		50.2
Mobile access	46.7		11.9

Measured on 12 September 2023. * Illustrative examples of download speeds according to SpeedTest.pl ranking. Source: original work.

). The ‘download’ metric reflects the downloading speed in Mbps. It is a higher-the-better variable. The higher the value, the shorter the user has to wait for a web page to load or a file to download.

Upload speed is the other basic indicator for evaluating Internet quality. The higher the ‘upload’ the faster data are sent from the device to the Internet. Additionally, household Internet access is usually asymmetric. It means that the speed with which data are sent to the user is higher than the upload speed.

3.1. Test Location and Object

The tests were conducted in rural areas around a small village Stara Wieś in Nowosądecki District, Małopolskie Voivodeship, Poland. The local topography is a mix of mountain and upland, households are fragmented, developments scattered, and there is no fibre optic link. Internet access is generally limited (Figure 2). The measurements were performed with a laptop and radio link. They were then repeated with the same computer and a broadband link in a city.

The tests involved selected geoportals offering comprehensive socioeconomic, environmental, and cultural information: (1) Grybów Municipality Geoportal, which is a website for collecting and providing spatial information about the municipality; (2) National Geoportal, which is the central node of the Spatial Information Infrastructure; (3) map portal of the Board of Geodesy, Cartography, and City Cadastre in Wrocław; (4) Open Spatial Data Geoportal; and (5) Lesser Poland Geoportal, part of the spatial information infrastructure of Małopolskie Voivodeship. Municipal portals offer data that local authorities (mayors, heads of communes) are obliged to provide, including addresses and local zoning plans. The data are published overlaid on reference data from district offices (administrative divisions of higher order) and other available resources. The National Geoportal allows users to view spatial data and search spatial databases and services from the National Spatial Information Infrastructure. Lesser Poland Geoportal is part of the spatial information infrastructure of Małopolskie Voivodeship (

Table 3. Profiles of the tested geoportals.

Item	URL*	Technology	Function
W1	https://sip.gison.pl/grybow	GISON	Grybów Municipality Geoportal
W2	https://mapy.geoportal.gov.pl	Viewer for maps of the Geoportal of the Polish Head Office of Geodesy and Cartography (GUGiK)	National Geoportal. Central Node of the Spatial Information Infrastructure
W3	https://wms.zgkikm.wroc.pl	GEO-INFO i.Map	Board of Geodesy, Cartography, and City Cadastre in Wrocław
W4	https://polska.e-mapa.net	GEO-SYSTEM (e-map)	Open Spatial Data Geoportal
W5	https://miip.geomalopolska.pl/imap	Lesser Poland Spatial information Infrastructure (MIIP)	Lesser Poland Geoportal

* accessed on 28 November 2023.

).

Good user experience is not captured at a single point in time. It is composed of a series of key milestones in their journey. Real-world performance is highly variable due to differences in users’ devices, network connections, and other factors. Moreover, the devices and networks used by developers to test load performance often achieve higher performance compared to actual user experience. Therefore, two types of tests were employed, laboratory (synthetic environment) and field. Lab Data are performance data collected within a controlled environment with predefined device and network settings. Field Data are performance data collected from real page loads users experience in real life. Field Data are also commonly referred to as Real User Monitoring (RUM) data; the two terms are interchangeable [65]. Field data cover a wide variety of network and device conditions as well as a myriad of different types of user behaviour. By contrast, the number of variables is purposefully limited for Lab Data [66]. The tests were performed with the PageSpeed Insights (PSI) application, which employs Core Web Vitals: Largest Contentful Paint (LCP), First Input Delay (FID), Cumulative Layout Shift (CLS), First Contentful Paint (FCP). The author further verified the Speed Index (s) and Total Time (s). PSI uses Lighthouse to analyse the URL in a simulated environment for the Performance, Accessibility, Best Practices, and SEO categories. This approach is consistent with the methodologies of other researchers [67,68,69,70]. The results were then verified with Pingdom and Gtmetrix. Gtmetrix tests the performance in terms of page speed metrics. The application measures several performance metrics that reflect what a site visitor would actually experience as the page loads [71].

The measurements followed the so-called regular ad-hoc model (single performance measurement). In principle, website performance is measured over time (monitoring) or ad-hoc (a single measurement). The ad-hoc approach reflects the website’s performance ‘here and now’. It is an illustrative measure. Measurement over time (monitoring) is more complex because it takes into account various conditions that may arise in that time, such as over a week. Monitoring can record periods of intensified traffic (greater server load and potential performance disruptions). This may lead to differences between aggregate monitoring measurement and ad-hoc results [72]. The measured values were juxtaposed with reference values published in the Google Search Console Help Center (

Table 4. Core Web Vitals reference values (performance ranges). Source: own study based on [73].

Core Web Vitals	Good	Needs Improvement	Poor
LCP	≤2.5 s	≤4 s	>4 s
FID	≤100 ms	≤300 ms	>300 ms
INP	≤200 ms	≤500 ms	>500 ms
CLS	≤0.1	≤0.25	>0.25

).

3.2. Performance Indices

A slow web page may have grave consequences. It is detrimental to user ‘satisfaction’ and reduces the chance of them returning to the website (intent to use the website again). Lowered user satisfaction may lead to discontinued use of an application, especially in discretionary applications such as those found on the Internet such as geoportals [74]. In the case of less popular or completely niche websites, a mere two seconds of regular delays cause traffic slumps and bounce-rate peaks. Hoxmeier and DiCesare [74] demonstrated that user satisfaction decreases as system response time grows. Conversely, immediate system response is not perceived as a facilitator of use or operation learning process. Other studies suggest that system response should be comparable with delays occurring in interpersonal interaction. This means that a website should respond to user actions in 1–4 s [75]. Empirical studies suggest that the system load time should not exceed several seconds. Apart from user comfort, website performance affects natural environment protection. Effective design practices (Green Patterns) include such approaches to website and web application design that improve performance and UX while limiting their carbon footprint [76].

Poor performance regarding website usability results in user decline. Performance is a measure of comfort. It means that the website can be browsed smoothly [67]. According to Nielsen [77], there are three primary time limits (defined by human perceptual abilities) that have to be considered when optimising web application performance and are applicable today: (1) 0.1 s: the user feels as if the system responded immediately. They perceive no delays, which means that no extra feedback is necessary in addition to the result. The user also feels that they have directly and with their own action manipulated the user interface instead of ordering the machine, which acts on their behalf; (2) 1.0 s: is the approximate limit for the user’s flow of thought to remain uninterrupted, even though they will notice the delay. A delay of 0.2 to 1.0 s is noticeable, but the user perceives it as the system ‘working’ on the command rather than it being an immediate effect of the user’s activity. If the delay takes more than 1 s, the user has to be notified that the computer is working on the task through a changed cursor, for example; (3) 10 s: this much is about the limit for keeping the user’s attention focused on the dialogue box. Any task that may take longer than 10 s needs a percentage completion indicator and a clearly marked method for the user to interrupt the operation. Still, delays of more than 10 s are only acceptable during natural breaks in the user’s work, for example when switching tasks. According to Nielsen’s [77] response time study, 10 s is the time limit for keeping the user’s attention focused on a dialogue box. When the necessary delay is longer than 10 s, providing feedback about what the system expects to be done can be helpful [78]. If the device is unable to provide close to immediate response, the user should be receiving continuous feedback in the form of a percent-done indicator. Progress indicators offer three main benefits. They inform the user that the system has not failed but working on the task it was given. They provide an approximate estimate of how long the user has to wait so they can switch to another task in the meantime. They also keep the user occupied or can even ‘entertain’ them [79]. These guidelines are universal and apply also to map applications and geoportals used in web browsers.

Google algorithms were updated on 25 June 2021. The Page Experience Update involved key quality indices, Core Web Vitals. Put simply, the changes were intended to take into account ‘the appearance and how websites work’. ‘Appearance’ is not a subjective piece of design here but rather a quantifiable technical attribute. The update was intended to improve the ranking of those websites that offered high usability [80].

Web Vitals is a Google initiative to propose standardised guidelines for website creators to offer high-quality pages. The precondition was that the designers do not have to be performance experts to understand the attributes that affect the user experience quality. It is intended to simplify the assessment of website performance by measuring and analysing three key indices, Largest Contentful Paint (LCP), First Input Delay (FID), and Cumulative Layout Shift (CLS). Each of them represents a different aspect of user experience and reflects the actual comfort of use of the website [81]. LCP measures loading performance. To provide a good user experience, LCP should occur within 2.5 s of when the page first starts loading. FID measures interactivity. To provide a good user experience, pages should have an FID of 100 milliseconds or less. CLS measures visual stability. To provide a good user experience, pages should maintain a CLS of 0.1 or less [14].

Research shows that websites (information and e-commerce) that conform to Core Web Vitals guidelines are 24% less often abandoned by users when content is loading by leaving the page before any content has been painted [82].

3.2.1. Largest Contentful Paint (LCP)

Throughout the history of the Internet, website creators focused mostly on measurements of home page load time in a browser. They even used stopwatches. Still, observations by the W3C Web Performance Working Group and research by Google suggest that performance can be more precisely quantified by the measurement of the time it takes for the largest part of the website to load, which is reflected with Largest Contentful Paint (LCP). The LCP unit is second. It measures the time it takes to render the largest graphic object or block of text in the viewport after the page is first loaded. LCP is user-oriented and describes the result of the measurement of the perceived loading speed. It is the last point on the timeline when the main content is probably loaded. To reach a good user experience, websites should display the ‘largest piece of content’ before 2.5 s [83].

The most common type of interruption experienced by users online is when they wait for the page to load, but the consequences differ depending on the context and situation. As was mentioned before, research suggests that system (website) response should be comparable with delays occurring in interpersonal interaction. Therefore, system response to user activity should not exceed 4 s [75]. These recommendations have been validated over time [77]. The Largest Contentful Paint (LCP) metric measures when a page-to-page navigation appears complete to the user. Sites are recommended to keep LCP under 2.5 s for 75% of their page loads [82].

3.2.2. First Input Delay (FID)

The first impression of a website is usually founded on graphic design and visual appeal. However, it is hard to assess how much users enjoy a website design using algorithmic tools and synthetic quality indices. What is possible is to measure the website’s loading and response time. The first impression regarding page loading is described with First Contentful Paint (FCP). The page’s reaction to the user interacting with its components is just as important. This aspect is quantified with First Input Delay (FID).

Some people are much more sensitive to website delays than others. They can notice even a short delay if their action requires system feedback. Users often expect immediate feedback. Any delays may lead to mistakes. Users can, for example, repeat the action believing the previous one was ineffective. As a consequence, they can click ‘Add to basket’ twice and inadvertently buy two products. The responsiveness related to these experiences is measured by First Input Delay (FID). Sites are recommended to keep FID under 100 milliseconds for 75% of page loads [82]. FID is the time from the first user-page interaction, such as clicking a link, until the browser can actually start running event handlers as a result of the interaction. According to Web Vitals, the time must not exceed 100 milliseconds. Low FID means better website usability [81].

3.2.3. Cumulative Layout Shift (CLS)

Most web pages consist of many components loaded one by one. Some parts of the content are displayed first, which may allow the user to progress towards their objective instead of waiting for everything to load. However, when components loaded later unexpectedly affect the position of those loaded before, user experience can be disturbed.

If a component the user is looking at moves unexpectedly, the user can become temporarily disoriented. Such interruptions slow the user down and can be frustrating. More serious consequences arise when layout shifts cause a mistake. In an attempt to click a specific element, which is moved, the user may click something else. This can lead to clicking an ad or ‘Buy’ button unintentionally. This aspect is measured with Cumulative Layout Shift (CLS). CLS measures how frequent and severe unexpected layout shifts are on a page. Fewer shifts mean less chance for interruption and errors. Sagoo et al. [82] recommend that sites aim for a CLS of less than 0.1 s for 75% of page loads.

Unexpected displacement of page content usually happens when its components are loaded asynchronically, or DOM (Document Object Model) elements are dynamically added to the page above the existing content. This may happen because of a graphic or video object of unknown size or an ad or widget that changes size dynamically. Such experience is usually detrimental to browsing comfort, but sometimes can even hurt the conversion rate. Cumulative Layout Shift (CLS) is a user-oriented metric measuring the visual stability of the page. Low CLS reflects good visual stability. To ensure proper UX, websites should aim for CLS around 0.1 and below [84].

3.2.4. Speed Index and Verification of Measured Values’ Similarity

The geoportal performance was supplemented with measurements of the Speed Index (1) and Total Time (s) followed by attributes Page Weight (KB) and Total Requests (both desktop and mobile).

$$\mathrm{Speed} \mathrm{Index}={\int }_0^{end}1-\frac{VC}{100}$$

(1)

where

end is the end time in milliseconds,

VC is the % visually complete [85].

The Speed Index is the mean time it takes visible parts of the page to be displayed. The unit is millisecond, and the value is associated with the size of the viewport. The Speed Index metric measures the time it takes for page content to be visually populated (the lower the number, the better) [85]. Its value can be interpreted according to the scale in

Table 5. Speed Index reference value. Source: own study based on [85].

Speed Index (s)	Colour Coding
0–3.4	Green (fast)
3.4–5.8	Orange (moderate)
>5.8	Red (slow)

.

Similarity (or degree of discrepancy) between each two measurements was compared using the structural similarity indicator (SI). SI is a statistical measure for assessing the similarity of a feature in two separate populations. SI is calculated from Equation (2):

$$SI={\sum }_{i=1}^{k}min\left(w_{1i}, w_{2i}\right)$$

(2)

where

k is the number of variants,

w_1i is the ith structural indicator for the first population,

w_2i is the ith structural indicator for the second population,

min(w_1i, w_2i) is the smaller structure indicator with the ith variant of the feature or belonging to the ith class.

The value of the structural similarity indicator lies in [0;1]. The closer the SI is to 1, the greater the similarity of the populations. Zero means no similarity and 1 complete similarity.

4. Results

The PSI tests revealed the poor performance of the geoportals both on mobile and desktop devices regardless of the connection type (

Table 6. Evaluation of basic web indices according to PSI * (radio link, Lab Data).

Item	Geoportal ***	Performance **		Overall Performance Test Result
		Mobile	Desktop	Mobile	Desktop
W1	https://sip.gison.pl/grybow	31	63	fail	fail
W2	https://mapy.geoportal.gov.pl	70	25	fail	pass
W3	https://wms.zgkikm.wroc.pl	27	48	fail	fail
W4	https://polska.e-mapa.net	41	47	fail	fail
W5	https://miip.geomalopolska.pl/imap	27	28	fail	fail

■ 0–49 (Poor), ■ 50–89 (Needs improvement), ■ 90–100 (Good). * Measured on 19 September 2023; ‘snapshot’ mode. Desktop emulation with Lighthouse 11.0.0. ** synthetic performance index. The PSI downloads and analyses the web page with Lighthouse, which simulates page loading and then calculates page performance indices, summarising them as a 0–100 score. The results are classified into three tiers. A good result is at least 90. *** accessed on 28 November 2023.

). The Performance index remained below 50 in most cases, which is ‘Poor’ according to the PSI scale. Only in two cases did the tests result in ‘Needs improvement’ on the PSI scale both for radio link and broadband.

Lab Data tests yielded similar results both for radio link and broadband (

Table 7. Evaluation of basic web indices according to PageSpeed Insights * (broadband, Lab Data).

Item	Geoportal ***	Performance **		Overall Performance Test Result
		Mobile	Desktop	Mobile	Desktop
W1	https://sip.gison.pl/grybow	33	51	fail	fail
W2	https://mapy.geoportal.gov.pl	64	25	fail	pass
W3	https://wms.zgkikm.wroc.pl	27	44	fail	fail
W4	https://polska.e-mapa.net	44	45	fail	fail
W5	https://miip.geomalopolska.pl/imap	27	27	fail	fail

■ 0–49 (Poor), ■ 50–89 (Needs improvement), ■ 90–100 (Good). * Measured on 19 September 2023; ‘snapshot’ mode. Desktop emulation with Lighthouse 11.0.0. ** synthetic performance index. The PSI downloads and analyses the web page with Lighthouse, which simulates page loading and then calculates page performance indices, summarising them as a 0–100 score. The results are classified into three tiers. A good result is at least 90. *** accessed on 28 November 2023.

); the performance of the geoportals was inadequate in both cases (PSI Poor score). The SI value is 96.9%, which is close to 1, which means a significant similarity between the results.

No significant differences were found for radio link and broadband with WebPageTest. The fully loaded time exceeded 10 s (

Table 8. Selected performance statistics according to WebPageTest (radio link, mobile).

Geoportal		Quality Indices
Geoportal	LCP (s)	CLS	TBT(s)	Speed Index (s)	Total Time (s)	Page Weight (KB)	Total Requests
W1	11.46	0.092	4.29	10.441	13.134	2.827	101
W2	5.006	0	0	7.995	8.704	702	49
W3	—	—	—	—	—	—	—
W4	—	0	0	10.272	10.300	480	57
W5	24.649	0	3.376	24.263	30.297	1.897	111

From: Virginia USA–EC2–Chrome–Emulated Motorola G (gen 4)—4G. Measured on 21 September 2023.

) for all the geoportals and 20 s for W1 and W3. Considering the reference values for the Speed Index, all the results indicate that the geoportals are ‘slow’ websites for mobile (

Table 9. Selected performance statistics according to WebPageTest (broadband, mobile).

Geoportal		Quality Indices
Geoportal	LCP (s)	CLS	TBT(s)	Speed Index (s)	Total Time (s)	Page Weight (KB)	Total Requests
W1	11.547	0.092	4.439	10.795	22.501	3.430	118
W2	5.707	0	0	8.912	10.059	703	49
W3	—	—	—	—	—	—	—
W4	7.784	0	0.051	10.263	10.313	480	57
W5	24.482	0	3.414	24.284	31.134	1.838	110

From: Virginia USA–EC2–Chrome–Emulated Motorola G (gen 4)—4G. Measured on 21 September 2023.

) and desktop (

Table 10. Selected performance statistics according to WebPageTest (radio link, desktop).

Geoportal		Quality Indices
Geoportal	LCP (s)	CLS	TBT(s)	Speed Index (s)	Total Time (s)	Page Weight (KB)	Total Requests
W1	8.867	0.082	0.375	8.776	18.313	3.767	138
W2	24.193	0.079	12.817	15.051	28.832	6.029	244
W3	—	—	—	—	—	—	—
W4	5.996	0.021	1.775	9.422	11.176	2.273	327
W5	13.204	0	1.339	12.700	15.576	1.952	107

From: Virginia USA–EC2–Chrome–Cable. Measured on 21 September 2023.

) use.

According to WebPageTest, users have to wait relatively long for the main content of the geoportals to be loaded (main page object, LCP) regardless of the Internet connection quality. In the case of W1 and W5, the load time exceeded 10 and 20 s, respectively (Table 9). Despite numerous attempts at different times and test configurations, no results for W3 could be obtained.

Desktop Lab Data tests also yielded under-par performance of the geoportals. Note that the tests revealed a high probability that a different configuration (version) of the geoportals is loaded on mobile and desktop devices. It is suggested by the fact that different file sizes (Page Weight) and numbers of components (Total Requests) were recorded in mobile and desktop tests for W1, W2, and W4. The load time (Total Time) of over a dozen seconds was also unsatisfactory in all cases regardless of the Internet connection quality (Table 10).

All the investigated geoportals exhibited relatively high values of LCP and TBT, which can be indicative of their slow load time in the viewport (

Table 11. Selected performance statistics according to WebPageTest (broadband, desktop).

Geoportal		Quality Indices
Geoportal	LCP (s)	CLS	TBT(s)	Speed Index (s)	Total Time (s)	Page Weight (KB)	Total Requests
W1	6.886	0.082	0.374	6.905	16.890	3.767	138
W2	31.238	0.039	16.959	19.442	36.212	6.443	248
W3	—	—	—	—	—	—	—
W4	5.691	0.021	1.590	9.105	10.908	2.289	330
W5	13.194	0	1.514	12.506	15.532	1.844	108

From: Virginia USA–EC2–Chrome–Cable. Measured on 21 September 2023.

). Still, all boast full visual stability identified with CLS. This means that the portals do not surprise the user with unexpected visual (layout) shifts of such components as maps, menus, etc. The absolute values of the Speed Index for W2 and W5 are significantly higher than the reference value, which is up to 3.4 s for fast websites with 5.8 s being the symbolic performance limit [85].

PSI measurements confirmed that none of the geoportals is burdened with unexpected layout shifts (as measured by CLS). Still, their performance is unsatisfactory, according to the Lab Data analysis (

Table 12. Values of basic web indices (PageSpeed Insights, radio link, Lab Data).

Core Web Vital Metric	First Contentful Paint (FCP)
Measurement Mode	M	D	M	D	M	D	M	D	M	D
Geoportal	W1		W2		W3		W4		W5
Measurement results (s)	3.8	0.9	3.3	3.5	21.6	3.8	3.2	0.9	24.4	9.4
Core Web Vital metric	Largest Contentful Paint (LCP)
Geoportal	W1		W2		W3		W4		W5
Measurement results (s)	19.4	4.7	5.9	33.1	23.9	4.4	4.6	3.3	26.1	9.9
Core Web Vital metric	Cumulative Layout Shift (CLS)
Geoportal	W1		W2		W3		W4		W5
Results for desktop	0	0.001	0	0.064	0.01	0.001	0	0.024	0	0
Core Web Vital metric	Speed Index
Geoportal	W1		W2		W3		W4		W5
Measurement results (s)	11.5	3.0	6.3	13.5	21.6	4.3	5.4	4.9	27.8	9.4

). LCP measurements stand out as particularly poor, especially in the mobile mode. Complete and interactive content is displayed after over a dozen or several dozen seconds. The measured values of the Speed Index are just as distressing regardless of the test conditions (Internet connection speed,

Table 13. Values of basic web indices (PageSpeed Insights, broadband, Lab Data).

Core Web Vital Metric	First Contentful Paint (FCP)
Measurement Mode	M	D	M	D	M	D	M	D	M	D
Geoportal	W1		W2		W3		W4		W5
Measurement results (s)	4.5	0.7	4.1	9.2	21.5	4	3.2	1.1	24	9.8
Core Web Vital metric	Largest Contentful Paint (LCP)
Geoportal	W1		W2		W3		W4		W5
Measurement results (s)	19.2	4.1	4.9	18.1	23.3	4.5	5	3.1	27.2	10.6
Core Web Vital metric	Cumulative Layout Shift (CLS)
Geoportal	W1		W2		W3		W4		W5
Results for desktop	0	0.001	0	0	0.005	0.001	0	0.018	0	0
Core Web Vital metric	Speed Index
Geoportal	W1		W2		W3		W4		W5
Measurement results (s)	11.8	2.7	5.9	9.7	24.6	5.9	5.6	5.9	27.6	10.0

). The usability of the geoportals could be somewhat improved by loading content that will attract the user’s attention first (FCP), but the measured values are not optimistic here as well with over 20 s for W3 and W5.

Field Data performance is slightly better than the results of the Lab Data tests. In a way, it means that the performance of the tested geoportals was mediocre or good for a larger number of users with specific hardware and Internet speed (

Table 14. Values of basic web indices (PageSpeed Insights, radio link, Field Data).

Core Web Vital Metric	Largest Contentful Paint (LCP)
Measurement Mode	M	D	M	D	M	D	M	D	M	D
Geoportal	W1		W2		W3		W4		W5
Measurement results (s)	5.6	4	3.6	0.7	6	4	3.6	3.9	5.8	2.8
Core Web Vital metric	First Input Delay (FID)
Geoportal	W1		W2		W3		W4		W5
Measurement results (ms)	14	2	19	N/A *	17	1	18	2	22	3
Core Web Vital metric	Cumulative Layout Shift (CLS)
Geoportal	W1		W2		W3		W4		W5
Measurement result	0	0	0	0	0.02	0	0.13	0.02	0.06	0.14
Core Web Vital metric	First Contentful Paint (FCP)
Geoportal	W1		W2		W3		W4		W5
Measurement results (s)	1.5	1	1.9	0.8	5.5	3.4	3.4	1.5	4.7	2.3

M—mobile measurement; D—desktop measurement. Measured on 21 September 2023; the result was computed from basic web indices for the last 28-day data collection period. * The usage report in Chrome does not offer a sufficient amount of data on users to propose useful information about the selected device type (desktop or mobile).

). It may also mean that the laboratory setting was more ‘sensitive’ to problems with geoportal content loading than users during actual use. Concerning Field Data tests, results for W1 and W3 stand out as the worst in the population (under the employed research design).

Measurements with GTmetrix confirmed the poor performance of the geoportals. All the geoportals except W3, for which no test could be performed despite numerous attempts with various test configurations, scored ‘F’, which means fail (results below 49%). Relatively low values of the synthetic indices, such as Performance and Structure were consistent with the results for Fully Loaded Time (

Table 15. Performance of the geoportals according to GTmetrix (radio link).

Geoportal		Quality Indices
	Performance (%)	Structure (%)	GTmetrix Grade	Fully Loaded Time (s)
W1	42	47	F	13.7
W2	24	63	F	15.9
W3	—	—	—	—
W4	27	70	F	25.3
W5	15	72	F	16.4

Report generated: 24 September 2023. Test Server Location: Vancouver, Canada. Using: Chrome (Desktop) 103.0.5060.134, Lighthouse 9.6.4.

). Moreover, the load time for the geoportals often exceeded 10 s on a computer with a radio link.

The average similarity between radio link and broadband Lab Data measurements with PageSpeed insight was very large and amounted to 95.44%. The largest discrepancy was identified for W2 in the desktop mode (similarity of 82.11%). The study confirms that the performance measurement results with PSI are only slightly dependent on the Internet connection quality and hardware used by the auditor.

5. Discussion

In many countries, geodetic databases and geographical information systems kept by geodetic and cartographic services are the key sources of reference data on hard infrastructure. Infrastructure spatial databases are interdisciplinary so they can be used by various institutions for organising projects and managing socioeconomic processes. Geoportals facilitate access to spatial data [86]. Poor radio link signal strength and no access to fibre-optic Internet are consequential. Worse geoportal browsing comfort is just one of them. Problems with access to advanced web applications can lead to greater customer numbers in municipal offices. The progress of digitalisation in the form of doing official business via a web browser is limited in rural areas, particularly in mountainous regions and where buildings are scattered, which causes problems with Internet access [26]. Lack of proper infrastructure can even lead to unavailability of digital services. The potential consequences include digital marginalisation and exclusion. Access to satellite telecommunication systems with a significant untapped potential can be a response to the problems with fibre-optic Internet infrastructure in rural areas [87].

Users expect web applications to be constantly available and respond immediately today. This makes performance a key success factor. Engineers come up with newer and newer techniques for improving web application performance to meet these expectations [15]. Most of them focus on optimising the page or application and hosting server [88]. Many researchers investigate the impact of performance on the conversion rate (as a measure of a website’s success), taking into consideration the bounce rate, user satisfaction, and comfort (UX) on desktop and mobile devices. These studies are aimed at identifying critically underperforming components of websites in need of optimisation and offering (technical) guidelines for creators, designers, and publishers of websites. They investigate commercial, e-commerce [89], and public websites, including those of universities [87], public administration, and healthcare institutions [90]. The performance is measured with various tools for test automation (web applications) and through surveys. The results are published raw—as index values available in web applications—or after statistical analysis with various quality models [91]. Some researchers offer case studies on the performance of specific map applications or programming solutions [92]. Król [72] investigated the performance of map applications that used a raster file (raster map) as their component, for example. His objective was to determine the threshold raster file size to breach the map application’s performance, leading to problems with map viewing due to long loading times in the viewport or staggered performance of the application. He analysed values of YSlow and PageSpeed Score and the time it took to load the application in the viewport. Research investigating the user side of things and what they can do to improve website browsing comfort (speed) is not as common.

5.1. Core Web Vitals in Website Quality Tests

Google Core Web Vitals are a relatively new set of indices for evaluating the general quality of websites. They are most often employed in website performance rather than geoportal performance evaluation (

Table 16. Selection of studies on the performance of websites, geoportals, map servers, and spatial network services.

Item	Scope of Study	Selected Keywords	Reference
1	Characterisation of performance testing parameters, performance testing methods, and approaches to improving the performance of web applications.	load speed, load time, interactivity, responsiveness, visual stability	[94]
2	Characterisation of techniques and tools for measuring website performance found in Google for Developers.	web vitals, user experience, web application, Lighthouse, PageSpeed Insights	[95]
3	Assessment of website quality with performance indices with a complementary subjective user assessment.	quality of experience, Core Web Vitals, web technologies	[69]
4	Analysis of benefits of improved website performance focused on Technical SEO.	SEO performance, mobile usability, search engine optimisation	[93]
5	Assessment of website quality using performance indices and user experience.	quality of experience, Core Web Vitals, improving user experience, loading time	[70]
6	Assessment of websites with such indices as Core Web Vitals.	Google Lighthouse, performance, accessibility, search engine optimisation	[96]
7	Website performance analysis with Google PageSpeed Insights.	Core Web Vitals, page loading speed, visual stability	[88]
8	A method for testing and measuring the availability of network viewing services (WMS) for end-users. Latency of application, overall latency (response time), error occurrence, availability, and performance tests.	network services, INSPIRE, map Server, performance testing, benchmarking	[97]
9	Performance testing of web mapping services. The paper describes map service tests in which it is possible to determine the performance characteristics of a map service depending on the location and scale of the map.	Web Map Service, performance testing, map scale, response time	[98]
10	Recommendations regarding efficiency and effectiveness. Emphasis on the need for a buffer mechanism to preload map data to improve geoportal smoothness. Map performance as a major user experience criterion.	map-based geo-portals, user experience, geo-datasets	[99]

). Results of synthetic quality measurements are sometimes juxtaposed with subjective user assessment. Additionally, the fully loaded time is also important in performance evaluation [69]. Research shows that performance is important for user experience as well as search engine optimisation, SEO [93].

Websites are tested to improve conversion rates and minimise users leaving the page (bounce rate) [89] or further optimise the website regarding search engines (SEO) and propose audit recommendations [88,90]. Moreover, performance tests usually employ test applications and performance indicators [72,100] in desktop and mobile modes [88]. A study by Wehner et al. [70] demonstrated that Core Web Vitals were much less predictive for web quality experience than expected and that page loading times remained the main metric and factor in this context. Ogbuju et al. [96] suggested optimising for the quality of user experience and performance to be critical for user satisfaction and the long-term success of any website. Ogbuju et al. [96] evaluated the performance and accessibility of the official websites of 49 accredited Federal Universities in Nigeria using Lighthouse and Core Web Vitals. Their results demonstrated that none of the websites fully conformed to the Core Web Vitals standards, which exposes significant room for improvement. Sumedrea et al. [88] employed Google Page Speed Insight and Google Core Web Vitals to measure academic sites’ page loading times, to identify issues in need of attention so that universities could increase their digital performance, improve candidates’ experience, and achieve sustainable development. The results showed that most of the websites they tested offered good performance on desktop and mobile devices. Nichifor et al. [89] extracted information about the technological performance of e-commerce websites of the most trusted retailers in Romania on smartphones. They also employed Core Web Vitals. Nichifor et al. [89] demonstrated that page speed shaped the customer journey and retailers gained the users’ trust by avoiding a long waiting time between touchpoints. According to their conclusions, every 0.1 s saved can improve the conversion rate by up to 8% due to emotional mitigation with technological performance improvements. Some works offer in-depth case studies. Król [92], for example, noted that the performance map applications can be improved through the compression of raster files or appropriate data server configuration, but also using source code minification, including Cascading Style Sheets and JavaScript. He used automated performance testing applications to measure the impact of JavaScript code minification on the map application’s performance. He demonstrated that the minification of JavaScript code alone may not be sufficient to achieve a noticeable performance improvement. Another study proved it possible to significantly enhance the performance of web applications by using even a small set of performance enhancement techniques [15].

5.2. User-Side Performance Improvement

Most investigations into the quality of websites and web applications yield technical recommendations for administrators and developers. They are concerned with website or web application improvements to make them more usable, better performing, and better quality. This case study attempts to verify possibilities for users to improve geoportal performance. Therefore, it does not focus on the improvement (optimisation) of the application itself (or the developer’s or programmer’s competencies) but on how the user can improve the digital environment and methods of using the applications.

Recommendations for users that may affect the speed of websites and web applications include cache clearing, removal of unused browser extensions (additional components), and employment of plugins that block scripts [101,102]. A cache is a computer storage with selected elements of websites, including JavaScript, CSS, or graphic files. Thanks to the cache, these elements are not downloaded from the server every time the user browses a website but are fetched from the cache to make the process faster.

Web developers use scripts that often slow websites down. Script blocking can speed up loading, but may affect the page’s functionality. Browser configuration is another way to improve the browsing experience. For example, Chrome users can turn on ‘data saving’ and ‘storage saving’ to download much less data when loading a page. A general internet common sense is recommended: closing unused tabs and using applications to block advertisements and other redundant content.

The author’s tests demonstrated that history clearing, and memory saving do improve the general subjective perceived performance of the operating system and/or the browser for some time, but they do not speed up geoportals. Although repeated tests did identify some differences in performance indices, it is not possible to link them to the test device optimisation. What is more, clearing the cache may reduce geoportal’s performance during some first interactions immediately after the operation because components have to be downloaded again as their previous instances were deleted, which slows the interaction down. If scripts are turned off on geoportals, they become dysfunctional. Repeated GTmetrix measurements of basic performance indices following cache clearing and activating the ‘memory saving’ option yielded no results. Sometimes the performance dropped (

Table 17. Performance assessment with GTmetrix (radio link, desktop) after clearing cache and with ‘memory saving’ on.

Item	Geoportal	GTmetrix
		Structure	Increase/Decline	Fully Loaded Time (s)	Increase/Decline
W1	https://sip.gison.pl/grybow	49	↓	16.2	↓
W2	https://mapy.geoportal.gov.pl	65	↑	16.3	↓
W3	https://wms.zgkikm.wroc.pl	—	—	—	—
W4	https://polska.e-mapa.net	71	↑	23.8	↑
W5	https://miip.geomalopolska.pl/imap	73	↑	16.8	↓

Structure: ■ 0–49, ■ 50–89, ■ 90–100. Fully Loaded Time: ■ >10, ■ 4–9.9, ■ 0.1–3.9. ■ ↑ performance increase, ■ ↓ performance decline, ↔ no change. Report generated: 24 September 2023. Test Server Location: Vancouver, Canada. Using: Chrome (Desktop) 103.0.5060.134, Lighthouse 9.6.4. Test date: 26 September 2023.

). Local optimisation efforts are especially futile when the measuring tool reports performance after compiling many measurements and as a result of Lab Data tests.

The measurement results were verified with the Pingdom Website Speed Test, but how its results are estimated is unclear. According to Pingdom, the synthetic performance (Performance Grade) of all the geoportals is relatively high, and their load times are within 3 s. In contrast, the load times according to GiftOfSpeed are much worse (

Table 18. Performance test results from Pingdom and GiftOfSpeed (radio link).

Item	Geoportal	Pingdom		GiftOfSpeed
		Performance Grade	Load Time (s)	Speed Score	Fully Loaded (s)
W1	https://sip.gison.pl/grybow	76	1.07	55	21.27
W2	https://mapy.geoportal.gov.pl	72	3.13	33	29.91
W3	https://wms.zgkikm.wroc.pl	—	—	—	—
W4	https://polska.e-mapa.net	74	0.36	60	6.12
W5	https://miip.geomalopolska.pl/imap	74	3.32	40	15.25

Performance Grade, Speed Score: ■ 0–49, ■ 50–89, ■ 90–100. Load time, Fully Loaded: ■ >10, ■ 4–9.9, ■ 0.1–3.9. Test: USA (New York), desktop (PC). Test date: 26 September 2023.

).

Verification of the results for broadband demonstrated that the test is subjectively performed slightly faster and feels like it, but the results are essentially the same, or even in some cases worse than those for radio links (

Table 19. Performance test results from Pingdom and GiftOfSpeed (broadband).

Item	Geoportal	Pingdom		GiftOfSpeed
		Performance Grade	Load Time (s)	Speed Score	Fully Loaded (s)
W1	https://sip.gison.pl/grybow	76	1.07	53	13.13
W2	https://mapy.geoportal.gov.pl	73	2.12	33	29.36
W3	https://wms.zgkikm.wroc.pl	—	—	—	—
W4	https://polska.e-mapa.net	74	0.46	59	6.12
W5	https://miip.geomalopolska.pl/imap	74	3.83	39	17.46

Performance Grade, Speed Score: ■ 0–49, ■ 50–89, ■ 90–100. Load time, Fully Loaded: ■ >10, ■ 4–9.9, ■ 0.1–3.9. Test: USA (New York), desktop (PC). Test date: 26 September 2023.

). Note that such test tools as PSI, GTmetrix, and GiftOfSpeed employ Google Lighthouse. Lighthouse is an open-source, automated tool for improving the performance, quality, and correctness of web applications. The test tools provide information based on Field Data or Lab Data, which can be aggregated and/or averaged data. Oftentimes, it would be useful to have access to information on the application’s performance here and now, that is when specific Internet infrastructure and devices are used. Therefore a reliable tool is needed for measuring application performance, taking into consideration the measurement environment, i.e., such attributes as Internet connection quality, and test device’s power (speed).

The results of synthetic tests are most suited for comparison and are difficult to interpret unambiguously. Audit recommendations may offer practical (effective) value. Moreover, synthetic results can provide a point of reference for measurements conducted after technical adjustments. The latter, however, are a domain of map application developers.

6. Conclusions

The article reports on geoportal performance tests. The tests were performed with selected web applications. The interface in the web browser is a gateway to the test environment (Lab Data). The website is downloaded and tested by the server, or the result is a compilation of activity of numerous users (Field Data). Test applications usually base their results on Field Data analysis rather than a single measurement. It is a distribution of numbers. This means that a specific geoportal is loaded quickly for one user and slowly for the other.

Test algorithms principally provide insights based on aggregate and converted partial results. All this is to ensure repeatable and objective results. It is the utmost objective of test tools to provide as objective as possible assessment of a website’s performance, not an assessment affected by the tester’s hardware and Internet access. In other words, an objective assessment of performance should not depend on the test environment (configuration). However, such tests could be useful to assess application usability under specific conditions, such as defined geographical location or Internet connection speed.

The study failed to provide unambiguous evidence that radio link users in rural areas could experience problems with the geoportal performance, although PSI Lab Data and Field Data seem to suggest it indirectly. These are usually technically sophisticated portals requiring multiple components, although their mobile versions try to cut out as many of them as possible, which is evident from Total Requests and Page Weight (KB) attributes for mobile and desktop tests.

The user has limited possibilities to speed up map applications. Most recommendations related to performance improvement concern the general performance of the software on the device used to browse the Internet. Hence, the primary recommendation is to obey the online common sense both regarding the operating system and web browser as part of the system. It is possible to slightly improve the geoportal experience through the optimisation of the device locally, but the responsibility to ensure geoportal performance is mainly the publisher’s.

Practical Implications and Future Research

It was possible to use the geoportals with poor Internet access both on desktop and mobile devices, but the interaction comfort was insufficient. It is far too long to wait over a dozen seconds for content rendering on desktop and mobile devices. Although the portal ‘works’ it is too slow, especially on mobile devices. Therefore, many users who are used to fast commercial websites may abandon such geoportals. It will be reflected in traffic statistics.

Applications that offer Lab Data performance measurements represent ‘relatively objective performance’. The tests are conducted in a controlled (synthetic) environment with a predefined set of network and device conditions. The results are independent of the test environment. The purpose of a lab test is to control for as many factors as possible so that the results are consistent and reproducible from run to run. Hence, Lab Data measurements are incapable of identifying performance issues related to the test place and environment. The results are independent of Internet quality (type of connection) and ad-hoc test hardware configuration. Furthermore, the Lab Data tests revealed the relatively poor performance of the geoportals. It means that the perceived performance of the tested geoportals on a radio link in rural areas is most probably even lower. This opens up a space for user-based performance research involving a survey on the perceived performance of the geoportals.

Web pages are usually made up of graphic and text content (front end). They are interactive thanks to script programming languages (back-end). Designers and developers are usually well acquainted with optimisation techniques for these components, such as code minification or file compression. In addition, many other development techniques can improve the performance of websites. Still, not all of them apply to geoportal optimisation because they employ mainly source geodata in a specific format. Geoportals are usually expansive web applications for viewing spatial data and searching spatial databases and related services. It may, therefore, be difficult to ensure geoportal performance comparable to that of a regular website because of the amount of spatial data loaded on the fly as the map view is being configured. Designers often implement preloader progress bars to ease the user’s discomfort caused by waiting for the spatial data to be loaded. This brings up the question of the scale used to evaluate performance. Perhaps geoportal performance should be assessed with a different scale than the universal website performance model. Note also that poor connection can be particularly problematic for more advanced operations than mere geoportal browsing, such as downloading large spatial datasets. All this could be investigated in the future.