A Critical Examination for Widespread Usage of Shipping Big Data Analytics in China

Abstract

Big Data Analytics (BDA) provides valuable opportunities for the optimization of maritime shipping management and operations. This might have a significant and beneficial impact on the Chinese maritime industry, which has recently emerged as a prominent player on the global stage due to the fast development of its maritime infrastructures and economical opportunities. This paper introduces two-field research conducted by a web-based questionnaire survey and semi-structured interviews with a large number of stakeholders in the maritime sector. The analyses show the impact of the development of big data technologies as well as current obstacles which constrain their deployment in the global maritime sector. The paper finally suggests several directions for promoting the wide-scale utilization of BDA in the maritime industry.

1. Introduction

In the contemporary world people, businesses, and devices have all become data factories that are pumping out incredible amounts of information to the internet each day. The number of internet users worldwide grew from 2.4 billion in 2014 to 4.4 billion in 2019, an 83% increase in just five years. It is estimated that 90% of the data on the internet have been created since 2016 [1]. As the quantity of available data grows, policymakers, managers, and researchers increasingly use them to inform their decision making [2]. The development of big data analytics (BDA) has benefitted various sectors, such as retail, healthcare, banking, and education, finding uses in connection, communication, and the provision of far more efficient and customized commercial services [3]. Due to its great value, big data has been essentially changing and transforming the way we live, work, and think [4].

Maritime shipping is, by far, the most cost-effective way to move goods and raw materials around the world [5]. It is essential to the world’s economy, as over 80% of the world’s trade is carried by merchant ships staffed by over 1.5 million seafarers [6]. The maritime industry is a complex system that requires a quick adaptation to changing conditions and in which decision making needs to take into account a large number of parameters [7,8].

The shipping industry has been slow in the adoption of new technologies. Other industries have been leveraging big data for years, whilst the maritime sector has just recently started realizing the importance of data to sustain competitiveness [9]. A traditional mindset, combined with limited knowledge about the cost of investment and true yields of big data, have been identified as some of the barriers to advancing technological change. According to Fields [10], “the future value of big data can only be realized through organizational and cultural change in combination with the acceptance of appropriate analytical tools, skills, and practices”.

In recent years, China has emerged as one of the world’s dominant maritime nations. It has become one of the largest importers of raw materials, one of the largest exporters of many consumables, and one of the largest shipbuilders, seafarer-supplying countries, and ship-owning states. In the meantime, the overwhelming majority of the world’s largest ports are located in China [11]. Seven of the largest ten container ports in the world are located in China, namely Shanghai, Ningbo-Zhoushan, Shenzhen, Guangzhou-Nansha, Qingdao, Tianjin and Hong Kong, accounting for a throughput in 2020 of 182,322,000 TEUs [12]. It could be argued that the development of maritime big data infrastructure would thus have a more significant impact on China than most other countries [13]. With the most abundant information technology as well as maritime resources in the world, China appears to have advantages in establishing the maritime big data infrastructure over other countries [14].

Based on a web questionnaire survey and semi-structured interviews with maritime industry stakeholders, the main purpose of this research is to critically assess how the adoption of BDA can help the marine sector at large. Considering the special role of China in the maritime world, this research will analyze some important questions: How is big data currently being used in the maritime ecosystem in China, and what are the responses of major stakeholders in China to the development of maritime big data infrastructure, and what are the reasons behind these responses? What are the major challenges that constrain the immediate integration of big data into the global maritime sector? The outcome of this research can be harnessed to enhance maritime connectivity, further the discussion on big data in shipping, and promote the development of big data infrastructure through consensus building among key stakeholders in the maritime sector. It will have multiple benefits in solving maritime-related issues, including market prediction, crew management, energy-saving operations, ship status monitoring, and compliance planning.

The remainder of the paper is organized as follows. Section 2 briefly presents related work while Section 3 describes the data and the methods applied. Section 4 reports the results of the integration of BDA in the global maritime industry and applications of Big Data in China’s maritime sector. Section 5 reviews how BDA addresses the key issues facing the maritime sector. Finally, Section 6 concludes the paper by highlighting important findings and a few directions for further work.

2. Literature Review

The primary focus of this research is to establish the financial yields attainable through BDA applications and the associated challenges. There is no intention to explore the highly complex technical aspects of big data. However, to contextualize the findings presented in the study, this section provides an overview of the existing knowledge, including definitions and features of big data, both in a general sense and in the maritime-specific context.

2.1. Big Data Analytics (BDA)

The term ‘big data’ is usually used to describe vast amounts of data that exceed the processing capacity of conventional database systems [15] and can be harnessed for analytics of a very high order, hence the term “big data analytics”. It was first used on a mass scale by Google Flu Trends (GFT) in 2008 to track the annual spread of influenza in the United States, a task that was traditionally undertaken by the Centers for Disease Control (CDC) over a period of several weeks, but took GFT just several days [16]. In the maritime environment, sources of such data can be anything from communication, transport, and GPS signals, sensors that transmit parameters of on-board equipment, including the main engine (ME) and oily water separator (OWS), to social media and search engines [17]. BDA and the Internet of Things (IoT) are emergent paradigms that are slowly becoming integrated parts of our society. However, there are no commonly accepted definitions of these terms [18]. To understand ‘big data’, especially from the point of view of business, one needs to focus on the value that it creates [19,20].

Big data analytics can be defined as the application of advanced analytics techniques to big data. Different analytics methods and tools can be applied to big data, as well as the range of opportunities and applications cover a wide range of applications [21,22,23,24,25]. Mageto [26] identified a few directions and implementation challenges for the successful application of BDA to sustainable supply chain management and manufacturing supply chains. Sann et al. [27] sought to advance the body of published literature on hospitality and tourism (i.e., higher star rating and lower star rating) by using big data analytics and data mining algorithms. The research forecasted tourists’ online complaint attributions to considerably diverse hotel classes. Mustapha [28] aimed to explore the perceptions of the UAE employees on the factors needed to implement sustainable big data analysis and the continuous impact on their organizational performance. The findings indicate that employees believe that the sustainable implementation of big data enhances business performance.

2.2. Big Data in the Maritime Industry

Traditionally, the maritime industry has been considered a low-data industry [29]. However, this has now started to change. Nowadays, a typical large modern container vessel of >8000 TEU can be fitted with as many as over 2000 onboard sensors and can thus capture over 2 GB of data, which becomes available to managers ashore [30]. This level of data generation, remote-access monitoring, carbon-based measurements, preventive and corrective equipment maintenance, data analytics, and forecasting has the potential to significantly improve ship operations and management [31]. There is a wide range of actors sharing a common interest in operational data, from ship managers to machinery manufacturers [32].

Although the implementation of data-driven solutions has a slow uptake in the maritime industry compared to several other industries, shipowners and operators are starting to realize the value big data can bring to their businesses [15]. Other than the liner shipping sector, which has completely different dynamics, research and development regarding the application of big data in the shipboard environment of the maritime industry has until now primarily been concentrated around (i) ship health monitoring and predictive maintenance, (ii) monitoring of energy consumption and efficiency, and (iii) making assessments of environmental impact [33].

A variety of data can be collected from a large network of sensors, devices, tracking equipment, and systems [34]. For example, one of the early ways in which the opportunities of big data were partially utilized in the maritime industry was through AIS (Automatic Identification System) technology [35]. AIS is an automated tracking technology that allows static and dynamic vessel information to be exchanged between terrestrial and on-vessel receiving stations [36]. Since December 2004, Regulation 19 of Chapter V of the Safety of Life at Seas (SOLAS) Convention [37] requires that all ships of 300 GT or above engaged on international voyages, all ships of 500 GT or above not engaged on international voyages, and all passenger ships, irrespective of their size, be fitted with an AIS transponder which both receives and transmits AIS data [35]. The AIS transponder incorporates GPS technology, which records the vessel’s position and movement details, and a VHF transmitter technology, which automatically broadcasts at regular intervals. These data transmitted by AIS, including the position, identification, course, speed, etc., can be used for maritime anomaly detection and vessel route prediction [38].

Although live tracking is a welcome development, AIS technology is still underutilized because data that is collected through it could also be used to analyze global shipping movements and give a better account of how the industry works [39,40]. For example, it can be used to analyze the impact of political events on shipping, such as the effect of lower or higher oil prices on the traffic through the Suez Canal, or for forecasting the impact that lifting the trade embargo on Iran would have on vessel traffic in the Persian Gulf [41].

Furthermore, the availability of maritime data can be identified in a wide range of areas, including ship, cargo, and crew information [42]. For example, ships’ registration, maintenance and upgrade records, logs of machinery events, marine accident investigation reports, electronic chart display and identification system (ECDIS), and weather monitoring systems are all useful for the development of data mining tools in shipping [43]. In the meantime, many other types of data are generated during day-to-day operations. These include the outputs of marine software programs, Key Performance Index (KPI) records, financial reports, safety management system (SMS) records, port state control (PSC) inspection findings, and a record of daily operations and cargo condition [44].

The sheer variety of data sources could be expected to provide complementary data analysis tools for different maritime business solutions. The quality and reliability of data are essential for a dependable BDA. BDA may also reveal inconsistencies and incompleteness on many occasions. The processing and correlation of data and information from different sources and databases can overcome issues relating to inaccuracy and incompleteness of data and can compensate for this sparseness [45].

2.3. The Integration of BDA Technology into the Maritime Business

The integration of BDA into the industry relies on the establishment of big data infrastructure promoting data sharing and consumption. As data has become an integral part of businesses within various industries, stakeholders in the maritime sector are gradually recognizing the benefits of implementing data-driven solutions to optimize operations, improve efficiency and reduce costs. Big data analytics brings many benefits to the logistics and transportation industry [7]. However, a common belief amongst maritime actors is that cost reduction margins of BDA implementations are too insignificant for the investment and the organizational restructuring to be worth it [46]. The existing maritime big data infrastructures are disparate, and there is a lack of standardization and optimization amongst them. Additionally, the availability of a large scale of and varies data sources is crucial for the development of widely applicable infrastructure [47].

In the maritime industry, there are already a variety of systems available in the market that transmit data from onboard sensors to shore-based service centers, especially for parameters related to main engine performance. Within these data centers, analysts can create health asset reports for individual vessels, enabling the detection of maintenance necessities 2–6 months ahead of their current timescale [48]. By investing in systems for predictive analytics, potential equipment breakdowns are detectable and can save owners and operators from the time and costs associated with asset failures. Additionally, environmental regulations and market pressure have driven owners and operators to look for data solutions that can help ensure environmental compliance, and at the same time reduce operational costs. Finally, the shipping industry has seen an emerging trend of data-based procedures for assisting ship safety [17]. Real-time information about vessel positions through onboard equipment, sensor data, and the increasing complexity of satellite communication systems can contribute to enhancing the safety and efficiency of the maritime industry [8].

While big data business maintains speedy growth in many countries, a key difference in China is that it has more people, more data, and more businesses. Indeed, it is far larger in scale than that in any other maritime country [49]. These benefits in research and development, that have already been in practice in domestic information technology (IT) and logistics industries, can have a potential positive spillover effect on other industries. This includes the maritime sector. According to [50], China’s big data development has gone through four stages. The first stage took place before 2012 when there were only several experimental applications of big data in large companies. In the second stage (2012–2014), big data started to gain momentum, led primarily by internet firms, for three purposes. These were targeted advertising, customization, and the placement and delivery of online delivery services. Since 2014, some leading internet firms continued to use machine learning in analyzing their business needs, thus ushering in the third stage of big data development. Companies like JD, Alibaba, and Tencent all played their part in this stage. In the meantime, data governance and middle-office technology gradually became the main focus. The fourth stage witnesses the emergence of big data ecosystems and smart cities which have become a must do project for first-tier cities of China like Shanghai, Guangzhou, Xiamen, and Shenzhen, as well as second-tier cities of China like Tianjin [51]. A notable aspect is that many of these cities are large ports in themselves.

3. Methods

In consideration of the above-mentioned objectives, several issues need to be examined. First of all, it is necessary to investigate the latest developments of BDA in the maritime industry and key trends related to the near future. Furthermore, as stated in the introduction, this study seeks to investigate the responses of the major stakeholders in China to the development of BDA and the major challenges for immediate engagement, to bridge the gap between potential uses that are forecast and the actual on-the-ground scenario. To do this, the investigators needed to not only make sense of the stakeholders’ personal opinions, but also to conduct in-depth interviews to find out the reasons behind their responses.

The literature presented in this paper primarily draws on the existing knowledge and scholarship available in the public domain. Additionally, the empirical data referred to in the discussion was collected by the author during two field research trips in China (2018–2019) where he conducted a web-based questionnaire survey and semi-structured interviews with a large number of stakeholders in the maritime sector, with follow-up data gathering activities being conducted through WeChat and QQ Talk with contacts in China. WeChat and QQ Talk are the two most popular phone-based chat applications in China and are widely used. In 2018, WeChat and QQ Talk had more than 1 billion and 805 million active users, respectively, in China [52]. Semi-structured interviews are exploratory technics that are commonly used in social sciences for qualitative research purposes. They are especially useful when focused, conversational two-way dialogue with the participants can yield valuable information. Indeed, such interviews are appropriately designed and prepared in advance with thorough interview questions to support structured interviews. The use of internet questionnaire surveys were chosen for data collection for their efficiency, flexibility, and relatively low delivery costs. A questionnaire, including 12 structured questions, was delivered through a web-based survey platform WJX. Participants were recording their answers on the survey website using a Universal Resource Link (URL). To increase the response rates, the authors using their social networks available to distribute “invitation to survey” through email, WeChat, QQ talk, and other social media methods, such as Weibo. Akin to a hybrid of Twitter and Facebook. As a Chinese microblogging website, Weibo is one of the most popular social-based communication methods, since the structure of written Chinese enables the transmission relevant data.

3.1. Distribution of Respondents

Within the first week of distributing the invitation, 181 questionnaires were received. The window of the survey was kept open for another three weeks. However, there were only 28 additional submissions received thereafter. Seven key categories of stakeholders were identified: shipowner, ship management company, cargo forwarder, government department, maritime education and research institution, port operator, and information technology (IT) company. Additionally, some respondents fell outside of these categories, such as shipbuilding companies, marine services and consultants, ship agencies, and chandlers. SPSS software was used to analyze the data gathered. Following the sample framing, all respondents were located in China, hence all the below findings refer to stakeholders in China. Indeed, the patterns that emerged are associated to Chinese stakeholders. Figure 1 illustrates the distribution of the stakeholders that answered the questionnaire: amongst the 209 questionnaires received back 32 are from the, 23 from ship management, 27 from cargo logistics, 19 from a government department, 42 from education& research institutions, 26 from the port operator, 25 from information company, and 17 from other sectors such as ship-building company, marine service, and consultant, ship agency and chandlers.

3.2. Attention to Big Data

Through the survey questionnaire and follow-up interviews, this research invited respondents to evaluate the attention they had paid to the development of maritime big data by choosing from the responses: special attention, general attention, low attention, and no interest. The results are shown in Figure 2.

It is noteworthy that the majority of respondents paid special or general attention to maritime big data. However, in the category of IT companies, the respondents paid less attention to the investigated issues than people in other categories. The authors were informed that most IT developers were less interested in the application of BDA to the maritime sector because they knew very little about this industry. It is also interesting to note that ship management companies recorded higher attention levels than shipowners. Some ship managers reported that they were interested in technology development because it helps them deliver better services and products to their clients. Shipowners, on the other hand, are not as sensitive as ship management companies to new technologies and innovations. This could be because the day-to-day management of ships has largely been outsourced by shipowners to ship management companies, and hence the latter could be expected to be far more interested in the subject than the former.

3.3. Integrating BDA into the Maritime Business

This survey also investigated the attitudes of respondents towards the importance of integrating BDA into the maritime business. As shown in Figure 3, while the majority of respondents (over 70%) confirmed the importance, <10% of them claimed that they were highly familiar with the relevant issues, and only three respondents claimed familiarity on a professional level.

In the meantime, it was noted that the respondents who had paid more attention to maritime big data tended to be more familiar with the investigated issues than others. In the same way, the respondents who answered “know nothing” were normally those who had paid low attention or chose “no interest”. This reflects a strong correlation between the lack of awareness and low levels of interest and a potential opportunity for BDA. If greater efforts were made to inform key decision makers in the maritime sector of the uses of BDA, there is a possibility that they might be more amenable and keener on using it to improve their operations.

3.4. Prospects of Using BDA in the Future

The survey investigated the respondents’ opinions on the prospect of integrating BDA into the maritime business in the future. An overwhelming majority of the respondents (84.2%) believed that BDA would be widely commercialized in the maritime industry sooner or later. However, as Figure 4 shows, 32.1% of the respondents also believed that BDA would not be widely commercialized in the next 10 years.

Respondents who are familiar with BDA tended to be more optimistic about its application than those who know very little about it. At the same time, those who disbelieved the realization of BDA in shipping admitted that they had a very low familiarity with or knew nothing about the investigated issues. We are, however, cognizant that there is a possibility of the former being biased towards their views of the future, without being aware of it, due to a confirmation bias, thus leading to ‘motivated beliefs’ [53].

3.5. Identifying the Obstacles to BDA Integration

Various obstacles constrain an immediate engagement with BDA in the global maritime business. As shown in Figure 5, this survey helped to identify some major challenges. These were: technology (inadequate/lack of), standardization of practice (inadequate/lack of), legal framework (inadequate/lack of), security (inadequate/lack of), cost (high), and collaboration challenges. Some other challenges, such as a shortage of expertise, were also identified, but they only represented a very limited portion of the overall answers.

While technology restrictions were acknowledged, they were not considered to be a significant challenge by most stakeholders. Most of the respondents opined that the current communication systems, fitted onboard ships, do not have the data limitations that existed ten years ago. As such, the IT systems onboard do not pose a challenge to BDA. On the other hand, in the eyes of most shipowners, costs were ranked as the biggest challenge. This might be linked to the sluggish market situation, as shipowners are facing extraordinary budgetary pressures. How shipowners and ship managers view costs, perceived and actual, are different, since the latter is far better aware of them from a vessel operations point of view, while the former is far better aware of the cargo-related costs and industry scenario.

Most stakeholders considered that standardization was crucial for the successful integration of BDA, but lacking collaboration was a major challenge to standardization. In shipping, this is especially important, as it is common for ships to be bought and sold often, and thus any application of BDA is only useful if the same standards are used on all ships. However, as of now, there are no such standards in the maritime sector, leading to different software developers developing different methods in isolation from each other. There are no common platforms on which they interact with each other, unlike the mainstream IT sector. This latter constantly and continuously shares its findings and tries to troubleshoot problems collectively, using various platforms like StackExchange. Maritime IT systems (and hence BDA) tend to be made for specific clientele for a given entity’s specific needs. There are few industrywide IT organizations, an additional limitation for superior R&D.

3.6. Personal Experiences

To acquire more detailed accounts and explore the underlying factors, semi-structured interviews were conducted on several field trips for empirical data gathering with the major stakeholders. The field trips covered several major port cities in China as far north as Dalian and as far south as Guangzhou, including the major coastal ports between these. Normally, however, qualitative research does not necessarily involve a large number of cases. Douglas, cited by Maykut and Morehouse [54], estimated that in-depth interviews ‘with twenty-five people’ were necessary before the saturation point could be reached.

The authors conducted 26 semi-structured interviews and 22 informal conversational interviews in this research. Semi-structured interviews normally took place in offices or hotels, with an average length of about 90 min, whereas conversational interviews were conducted in the field, informally, and sometimes over coffee or lunch. Most semi-structured interviews were recorded, while jottings or brief notes were taken during in-the-field conversational interviews. Informants were selected from different categories of professions to represent different stakeholders. The list of participants (respondents) interviewed is shown in

Table 1. List of Participants (N₁: Semi-structured interviews, N₂: Conversational interview).

Sector	N1 = 26	N2 = 22	Identity of Respondents
Shipowners	4	5	Commercial and technical managers
Ship management company	5	3	Ship operators and senior managers
Cargo logistics	5	4	Business managers and document managers
Government departments	3	2	Department heads, policy analyst
Education & Research	4	3	Maritime researchers and scholars
Port operators	2	3	Commercial and technical managers
Information Company	3	2	Information (IT) and business developer

.

Furthermore, in the ‘follow-up’ activities, WeChat group discussions were carried out with several industry groups to ‘catch’ the views and experiences of more individuals who were not part of the interviews but have direct experience in dealing with the related issues. Information derived from the interviews was used to gain greater insights and a deeper understanding of the response of major stakeholders in China to the integration of BDA into the maritime business, and what the are reasons behind these responses. We summarize these findings in the next section.

4. Results

4.1. The Integration of BDA into the Global Maritime Industry

The ability to process large amounts of data and extract useful insights from data has revolutionized a wide assortment of industries, from health care to logistics, from internet searches to social media, and from advertising to weather forecasting [55]. However, while the maritime industry is not exceptional, it has some special characteristics. For example, it has been argued that the maritime industry diverges from other transport industries, such as the aviation and automotive industries, in terms of technological responsiveness. Yap and Munizaga [56] examine the evidence of the use of big data in improving public transport and infer that while it has high potential, its uptake is nowhere close to fulfilling this potential. BDA has, however, proved useful and is used routinely in airline network planning to address major strategic and organizational challenges in this highly competitive industry [57].

In the maritime sector, on the other hand, the uptake of the use of big data has remained niche. The liner trade (container transport and logistics) industry has taken a lead in this, but the bulk carrier and tanker industry not yet has found uses for it to justify the interest from a cargo perspective. The uses of BDA in the maritime industry can be divided into three main aspects based on the perspective of the entity (see

Table 2. Uses of big data analytics in shipping (Source: the authors).

Perspective	Region	Type of Ships	Focus
Operator perspective	International	Container, Ro/Ro/car carriers	Port logistics, space available on ships
Ship manager perspective	International	All	Data from ships, machinery maintenance, and equipment health monitoring
Terminal perspective	Local	Container, roro/car carrier	Port logistics, yard space management, gantry sequencing
Other (government)	Local	Depending on need (all or hazmat carrying ships)	Marine spatial planning of new ports, monitoring of vessel traffic through critical sea lanes

). These are, namely, the use of BDA for cargo planning (the operator perspective), the use of BDA for better ship operations (the ship manager perspective) and the use of BDA for better port and terminal operations (the terminal perspective). Additionally, there does exist a fourth perspective, that of other agencies, including governments, that wish to analyze vessel movements in their coastal waters. However, this tends to be limited to the maritime sector.

Whilst software dependency, research, and technological updates are key drivers in the aforementioned industries, the maritime sector, in general, lacks the required understanding of the implications related to the increasing complexity of software systems. Zaman et al. [15] go further, asserting that the shipping industry will have to develop at a rapid pace to adapt to upcoming regulations, and that ship intelligence inevitably will shape the future of the industry. Hirsch [58] points out that big data has become a significant corporate asset which brings vital economic input and lays down the foundation for new business models, and hence adapting to the technology is crucial for a business to evolve.

On the other hand, it is not the case that the maritime sector has not had any share in the use of BDA. Early movers among shipowners did benefit from data solution adoption, such as concerning fuel efficiency [17]. The study identifies three areas of skills and capabilities which are fundamental for the successful implementation of big data in a business, these being: (i) an organizational culture for data and business development; (ii) skills and knowledge to handle and analyze data, as well as building models from the data; and (iii) the availability of necessary tools and infrastructure. With the right data systems and skills in place, shipowners can not only optimize routes and maintenance schedules, save fuel and mitigate risk, but also develop efficient ways to provide onboard training materials and perform software updates across a whole fleet. However, it could be argued that the drop in bunker prices negates the motivation for any such benefits. Further, cargo charter hire remains unpredictable, with slumps and spikes. This can lead to a conundrum. When charter rates increase, shipowners would be reluctant to invest in BDA, as they see no need to do so. When charter rates fall, shipowners will not have the funds available for such investments, thus leading to a conundrum.

It appears that there is no single dominant force behind the increasing technological developments currently taking place in the maritime industry, but rather a set of various factors contributing to this trend [15]. The authors firstly point towards recent legislation, in particular the EU MRV1, which forces shipowners and operators to quantify emissions for their vessels and hence requires investment in and extraction of data from onboard systems. Contrary to the legally mandated use of big data, several shipowners, primarily from the container shipping sector, were already utilizing big data for cost efficiency reasons. For example, Maersk had achieved route optimizations and significant fuel consumption reductions across its fleet through real-time data analysis [59].

Shipping market pressure, energy price fluctuations, and the slowdown in the world economy are also factors that inevitably have had impacts on the development of shipping technology [15]. To benefit from the advantages of big data, it is essential to embrace its development potential [28]. In addition to using data as a cost-saving measure, manifold organizations have recognized the value of BDA as a strategic decision-making tool. For example, in addition to robotics and autonomous systems onboard vessels, artificial intelligence will assist shoreside processes in terms of strategy making, decision making, and operations management [27].

The shipping industry tends to be reactive rather than proactive, and embraces technology when it provides real-time operational benefits soon (for example, all shipowners installed GPS receivers on their ships in the early 1990s, a significant time before it became mandatory), as compared to perceived benefits via technologies that are either too expensive (ECDIS required IMO legislation for a majority of the shipowners to install, primarily due to its extremely high cost) or has benefits that are not tangible in the shorter time scale (installation of ballast water systems and the use of low sulfur ships have both only occurred when necessitated by legislation). As such, the use of BDA would have to demonstrate its near future benefits convincingly to shipowners, using a time scale of less than one year. Given that the approaches to analyzing the opportunities of big data take into consideration the value for the industry, their potential is measured concerning the utility of big data in the maritime industry.

4.2. Applications of Big Data in the Maritime Industry in China

Big data has been used mainly in China to carry out research concerning maritime traffic and associated data. Yan et al. [46] have adopted distributed parallel k-means clustering algorithms to analyze environmental factors on a big data analytics platform, primarily to establish a ship energy efficiency optimization model. This is validated using a case study on the Yangtze River to show that this can effectively reduce the energy consumption and CO₂ emissions of ships. Peng et al. [60] use a CCPE big data model to determine the effect of the Maritime Silk Road (a project of maritime port development under the umbrella of the large Belt and Road initiative of the Chinese government) on the competitiveness of 99 ports using 15 efficiency measures. Bai et al. [61] use port call data, derived from the Automatic Identification System (AIS) reports from the world’s 30 largest container ports, to quantify both the immediate and longer-term impact of national COVID-19 lockdown policies on global shipping flows. These, however, remain smaller-scale applications of BDA, primarily used for research, and do not operate from a daily-use perspective.

Further, Løvoll and Kadal [62] emphasize how successful big data solutions can only be achieved in business cultures where data is actively pursued, the right business questions are asked, and fact-based answers are sought. A question remains—who determines what is right? It is not the software or IT developers, but rather shipowners, vessel operators, and ship managers (Table 2 one). Moreover, the ability to challenge and change the current practices within organizations will determine whether new business opportunities can be created. In the maritime sector, such change usually comes from a necessity. In conjunction with this, Bollier and Firestone [63] point out that big data has created a demand for adequately skilled workers, able to collect, analyze and communicate the data to ascertain useful knowledge. However, this new set of skills is not conventionally related to seafaring, cargo work, bridge watchkeeping, and marine engineering, which remain the main skill of the 1.5 million seafarers employed in the industry. The questions then remain: does the IT sector have sufficient skilled personnel to do this? Are the skills for BDA in shipping the same as BDA in other sectors? Is there sufficient investment from the perspectives identified in Table 2 to justify the above? These lead to the next section, which reviews some of the key challenges posed in the maritime industry in the era of big data.

5. Discussion

Despite the acclamation of the potential of big data and its benefits, the divergence between traditional data management and BDA imposes new challenges on maritime businesses [64]. The common denominators in most of the literature covering challenges related to BDA in the maritime industry are technology, cost, and the legal framework covering the use of personal data. Our questionnaire surveys and interviews confirmed these challenges, and also suggested the importance of some other aspects. For example, cybersecurity, standardization of data, and collaboration among stakeholders are also vital to address the obstacles of integrating BDA into the maritime industry.

5.1. Cybersecurity

One of the respondents, a ship manager, related the case of one of their ships where a USB drive, infected with a virus, completely stopped the main communication computer from working. This prevented it from sending or receiving any emails, thus necessitating all cargo-related communication for its next potential port of loading to be done either by satellite telephone (too expensive) or telex (too cumbersome). Ships do not have a dedicated IT person onboard, despite a large number of computers being used, thus even simple troubleshooting using a qualified person in a shore-based office is impossible for a ship due to its remoteness in the middle of the sea. Though shipboard computers are usually installed with an anti-virus system, they are rarely updated daily.

It is normal for ship staff to bring useful operational material with them from ship to ship. This creates larger problems from a cybersecurity perspective. Thus, dependence on BDA for shipboard operations remains highly vulnerable to disruption in the current scenario. Despite “The Guidelines on Cybersecurity on-board ships”, published by BIMCO [65] more than a year ago, its implementation on ships remains low. This danger related to cybersecurity increases on container ships, since most terminals bring a BAPLIE file containing all the loading information on a pen drive. Any delay in scanning such USB pen drives delays cargo operations.

5.2. Lack of Legal Framework

As per the views of many respondents, the lack of a legal framework that is uniformly applied in the maritime industry concerning BDA is among the major challenges that prevent it from attaining mainstream usage. The first legal hurdle in the maritime context arises from the fact that shipping companies in many cases are transnational businesses, and there is a national fragmentation in the law. The adoption of the General Data Protection Regulation 2016/679 (GDPR) in 2016, which entered into force in 2018, is expected to harmonize the legal fragmentation for data protection throughout the EU. However, the question remains as to whether that will apply to ships that are owned by a European shipowner located in the EU, but are flagged by any of the largest and most popular non-EU, for example, Panama, Liberia, Hong Kong, or the Marshall Islands [66]. What if the vessel’s day-to-day operations are managed by the same European shipowner’s Singapore-based office? These are normal day-to-day scenarios in the maritime industry that any TBA-related legal framework would need to address.

A single set of rules thus needs to be set out globally in the interest of the future of digital shipping. Additional questions remain: how does one deal with intellectual property, especially when third parties are involved, as they almost always are in the case of IT systems and TBA? How can one provide evidence that data processing is compliant with the required norms and standards? How does one solve disputes? These are among the questions that need immediate and reliable solutions.

5.3. Architecture Required

The large-scale data sets, generated from shipping activity and dealing in terms of navigational and ship performance information, create technical challenges concerning both the quality and quantity of data handling [67]. Nita and Mihailescu [68] support this argument and add that these large and complex data sets require a scalable architecture for efficient storage, manipulation, and analysis. Wojnarowicz and Fagerhus [69] highlight that the Total Cost of Ownership (TCO) associated with acquiring software and electronic systems was a key consideration from a shipowner’s perspective, and must be taken into the equation and cost–benefit analysis concerning big data implementation.

Today, only large organizations with deep pockets can afford to undertake a trial-and-error approach for weeks in failure-prone, resource-intensive BDA projects [70]. As a senior manager in a shipping company commented in an interview in 2019: “We truly believe that BDA can bring a variety of benefits to ship operations, and it is certainly the future trend. However, it is very difficult to integrate it into the daily operation of a small-size company like ours. We have to weigh the costs for the initial setup and maintenance. Additionally, we have concerns regarding data privacy and security, and we are vulnerable to these issues.”

5.4. Data Quality and Accuracy

In the meantime, the lack of structure and systematics in the collection of operational data from various data sources impairs data quality and can cause data inaccuracies. The dependability of TBAs depends on some factors: the cloud network being used; the software used to collect and analyze information; the integrity of physical wiring onboard ships in the harsh, vibration-prone environment of ships, from the equipment (located far away on the deck or in the annals of the engine room) to the transmitter (located on the navigational bridge, high above); the IT personnel who analyze this data ashore, and the analysts who then assess it and provide solutions/decisions related to them. This, in turn, hinders the effective use of the data and hampers the adoption of data-driven applications. The industry needs a standardized way to collect and store data, after which it will be possible to achieve economies of scale and improve quality, safety, and environmental performance.

It is essential to develop matching and structuring systems and create standards that will help to establish a maritime data landscape, enabling the efficient sharing and utilization of data for the benefit of all stakeholders [17]. As a technical manager in an IT company explained in an interview in 2019: “We have been requested to develop data exchange platforms. One of the biggest barriers is the lack of standards. For example, many shipping companies structure the number of Bills of Lading (B/L) in different ways, so it is difficult to export and combine the data between different systems. Standards are particularly effective at facilitating integration and interoperability and at enabling exchange and the sharing of data between different applications and stakeholders.”

However, the lack of collaboration among different players is a major obstacle delaying the process of standardization and the establishment of data-sharing ecosystems and platforms.

5.5. The Need for Collaboration and Resources

Though necessary for TBA, collaboration can sometimes be cumbersome, time-consuming, and even expensive, especially when one considers the vast time zones where people are located, the differences in cultures, and gaps in understanding of the subject matter. Before they can benefit from the standardization activities, many stakeholders lack the incentive to set aside resources and contribute to cost-effective tests and standard making. Furthermore, in addition to concerns about data privacy, many maritime players have different attitudes and views on standards themselves.

While some efforts have been made by some forerunners, such as DNV GL, Maersk, and COSCO Shipping, further consensus building and collaboration among key stakeholders on a larger scale are needed. As a commercial manager explained in an interview in 2019: “The integration of BDA into the maritime business relies on support and collaboration with almost all stakeholders. The improvements made in efficiency, performance, and safety will directly influence the business of shipowners and operators, so they tend to be more active in collaboration activities. However, many other stakeholders are less willing to invest time, resources, and money for the benefit of the industry.”

Investment in big data analytics technologies requires a lot of capital, and in an unsustainable environment with uncertainty about the future, many shipping companies are holding back. The uncertainty about the technology also has to do with the lack of a data-skilled workforce in the industry. Developing the technology and integrating it with existing operations requires skilled human resources. The demand for data scientists is already very high in other industries, and for the shipping industry to get a hold of these people. they will have to offer better terms to attract talent.

5.6. Non-Maritime Experiences of the Use of Big Data in China

In China, big data has been used for the benefit of a vast variety of terrestrial industries. These applications include the use of big data for assessing population flow patterns, air emission reduction measures, routing, and time predictions for shared services companies like Didi Chuxing, and the dominant IT and e-marketing companies, namely JD, Sundin, Alibaba, and Tencent. However, each of them has benefited from (i) a high-quality ground infrastructure required for TBA, (ii) negligible language or cultural barriers, since all players are located within the same country, (iii) the ability and facility to collaborate and (iv) support from local and national players, both private and governmental [71]. Tencent location big data has been used by Zhang et al. [9]. They utilized a Walktrap algorithm to analyze population flows between cities and to determine spatial patterns and determinant factors. This has a real and immediate sociological, as well as developmental, benefit for the government as well as private industries. They are also used to make strategic decisions, for example, on whether a particular high-speed train line must be laid in a certain manner, to develop connectivity solutions and to identify where local and national investment is required.

However, there is a major difference between the interests of local and national businesses and governments, and those of shipowners and managers. The location of the former will never change, and thus they can expect to derive the benefits of any investment for a lifetime. Ships, on the other hand, have highly transitional ownership and management pattern, are bought and sold routinely, see management change hands every few years, and decisions are often not known long into the future by even owners. This instability is due to the as dependence of these companies on factors far beyond their control, including future cash flows, future commodity prices, and future financial markets. The distress sales and closures of large companies like Hanjin shipping following the 2008 economic crisis and the debt crisis that followed in the subsequent years were unpredictable, even for the most astute industry forecasters.

The role of big data in decision making in Chinese firms has also been studied in depth using partial least squares in 108 Chinese firms. The findings of this study suggest that, while challenges related to BDA exist even in Chinese companies, its are vital for decision making [71]. On the other hand, BDA has been used to study air pollution management, a subject that has remained at the forefront of the government focus since 2010 [9]. While the use of big data has been used to assist in the credit assessment of MEs (small and medium enterprises), there is little evidence to suggest that the same is being used for the merchant ship financing industry [72]. It could be argued that merchant ship financing requires a quantum of finance that is far higher (in multiples) as compared to conventional bank loans. However, there is little to show that such methods might have been used for even the purchase or charter of smaller fishing vessels.

6. Conclusions

This article has examined the key opportunities and challenges related to the integration of BDA into the maritime industry using China as a case study.

6.1. Opportunities

Based on the research conducted, it is evident that industry players have significant benefits to gain from implementing BDA into their business processes. To summarize, these benefits include maximizing return on assets (ROA) by utilizing ship health monitoring; compliance with environmental regulations and reducing operational costs by monitoring and analysis of fuel consumption and environmental footprint; and enhancing the safety of masters and seafarers by providing them with the necessary information to support decision making. Furthermore, data sharing in general will contribute positively to the industry by connecting various parties across the supply chain.

6.2. Challenges

Although big data can bring the maritime industry plentiful benefits and opportunities, it also poses certain challenges that will need to be dealt with to encourage players to use BDA and gain from the implementation of data analytics. The research conducted in this paper points towards key challenges such as cybersecurity and technical inefficiencies of current data systems. Additionally, there are significant legal implications concerning big data applications and certain areas of the law that arguably are not sufficiently designed to follow the developments in technology. The biggest issues revolve around the limitations posed by personal data protection law, fragmentation in national applicable law, as well as the industry-specific nature of commercial confidentiality.

6.3. Recommendations

On a concluding note, the possibilities are endless with big data applications in the maritime industry. However, to fully utilize the potential of data analysis, the challenges outlined in this paper will have to be addressed. Therefore, key recommendations as to how to deal with these issues are provided below.

Technical challenges such as data quality, transfer, and integration are issues concerning the current systems available to the industry, and technological developments to enhance the efficiency and quality of such systems are indeed necessary. Cybersecurity is also a challenge that will need to be addressed to protect businesses from cyber-attacks leading to economic losses. Establishing cyber crisis management models is one suggestion as to how to prevent such attacks from occurring. Enhancing information sharing and collaboration between organizations across the industry would also be an appropriate preventative measure.

As for the legal implications concerning big data applications, certain areas of the law arguably are not sufficiently designed to follow the developments in technology. The EU’s decision to adopt the GDPR will certainly deal with the challenges posed by national gaps in legal frameworks governing data protection. However, to address the other two issues, the industry in general would firstly need to be more willing to engage in information and data sharing. Secondly, the borderline issue of the use of personal data will need to be addressed. There is certainly an unlikelihood of amendments to the law being made in this area, as personal data has been, and probably always will be, a matter of protecting citizens. However, European and international legislators should thoroughly consider whether the current data protection law is beneficial to both data subjects and society at large.

The conservative nature of the maritime sector, and the reluctance to embrace digital change, make justifying investment in technology somewhat challenging. However, businesses throughout the industry should evaluate the value such investments collectively can bring, and how data analytical tools can be used to generate strategic insight, which in turn will benefit the sector as a whole. It is however the onus of the developers of BDA tools to first analyze the needs of the industry and then create BDA tools to help solve existing problems identified by the players themselves. This will make it easier to obtain the approval of the industry and help to lowering guards that exist against adopting big data analytics in the maritime industry.

6.4. Significance

Our research findings reflected the opinions of the respondents. A larger or different set of respondents could introduce other findings that have not been uncovered by us. Our research also highlighted the necessity for further research into the application of big data in the Chinese maritime industry, including ship finance, port selection, port development, as well as resilience planning for ports and industries related to them.

Future surveys should consider alternative web platforms to increase participation rates of major maritime stakeholders as well as the design of more qualitative questionnaires. There will be a need to enrich both survey mechanisms and to observe the evolution of the emerging patterns over longer periods of time, the approach might be also applied to different regional contexts. There is also a need to investigate the most appropriate forms of diffusion of the results outputs to the maritime industry at large. So far the study has been limited to major stakeholders and should be extended to a larger maritime community, this favoring for a more refined analysis of different stakeholders groups.