A Platform for Analysing Huge Amounts of Data from Households, Photovoltaics, and Electrical Vehicles: From Data to Information

Round 1

Reviewer 1 Report

The paper presents a platform aimed at analysing data collected from so-called Smart Meters for Households (SMH).

Authors describe the dataflow starting with the collection of data from the field and culminating with its analysis, according to different perspectives, through Web-based dashboards.

 STRONG POINTS

• The paper focuses on a timely topic in the Smart City context.

• The physical implementation of the approach is well documented, with the support of flowcharts and block diagrams.

 WEAK POINTS

• The paper stresses that the huge amount of data is deemed as big data. However, the authors do not properly highlight which challenges of big data (among volume, velocity, variety) are considered and how they are tackled.

• The “big data analytics” essence of the paper, even though is introduced as one of the contributions, boils down to the consultation of Kibana’s generated dashboards.

• The advances with respect to the existing literature are not clearly elicited and the paper appears as a remarkable applicative example, but with a weak research core underneath.

 Apart from the strong and weak points listed above, which give a high level overview of the paper, in-depth comments and suggestions are reported in the following.

 INTRODUCTION

In its current shape, the Introduction is contaminated also with a discussion of state of the art approaches regarding meter data analytics (indeed, the title is repeated in the subsequent section).

Thus, it would be more useful to include this part in the Related Work section.

 A major concern with the alleged contributions of the paper is that all of them do not effectively deliver a shift towards innovation in the research of big data analytics: undoubtedly, behind the paper there is a massive physical development and implementation work, but what about the research core?

 RELATED WORK

While reading this section, the reader does not clearly perceive what are the novel contributions of this paper with respect to the related literature.

To better emphasise them, the discussion should put forth comparison features (with pros and cons) and a table summarising the analysed works: both of them are highly advocated to be included in the Section.

 Additionally, the literature should be enriched with more papers regarding big data approaches (currently, only few references are included).

 SYSTEM ARCHITECTURE - SYSTEM INTEGRATION

These sections present mainly implementation aspects, delving into the description of the physical architecture of the system, data collection process and data presentation techniques.

Hence, the description is tightly coupled with the implementation strategy and fostered technologies (both hardware and software). Therefore, the main flaw regards a lack of generalisability.

A real research contribution would be, for instance, devising a general architecture/methodology, independent from the implementation aspects.

Moreover, as formerly remarked, the big data aspects authors coped with are quite fuzzy; it seems that the adoption of a resilient data acquisition system in synergy with a NoSQL storage system have become the panacea for solving big data issues, neglecting the adoption of techniques/algorithms to reduce the volume of data (e.g., clustering or other data summarisation algorithms), or to favour the exploration for the final users, attracting their attention only on relevant data.

The aforementioned aspects are driving factors to move towards big data analytics and exploration approaches.

In this respect, it is true that dashboards are valuable instruments to ease the exploration of data, but only if exploration use cases are envisaged, otherwise the dashboards as conceived in the paper are useful only for expert users with a full knowledge of the domain and the collected data.

 

Conversely, other types of users (e.g., citizens, energy providers) are more likely to explore aggregated information, more concerning their personal interests, with predefined filters/views over data.

Author Response

REMARKS FOR THE REVIEWER:

We sincerely state our gratitude for invaluable time you allotted to review our paper. We believe that your constructive comments dramatically polished our paper content and conducted us to provide a very organized paper (unlike before) to be read and understand. We have revised and modified our paper generally based on other reviewers and specifically, on your comments as follows.

 

General modifications:

 

• New definitions relating to big data have been added.
• The innovations presented in this paper have been added.
• Comparison between different web portals has been added.
• New references have been added.
• New figures about the architecture and methodology of the system have been added.
• Several typos have been corrected.
• The text has been completely revised by a native speaker.

 

Specific modifications:

 

The paper presents a platform aimed at analysing data collected from so-called Smart Meters for Households (SMH).

 

Authors describe the dataflow starting with the collection of data from the field and culminating with its analysis, according to different perspectives, through Web-based dashboards.

 STRONG POINTS

• The paper focuses on a timely topic in the Smart City context.
• The physical implementation of the approach is well documented, with the support of flowcharts and block diagrams.

The authors would like to thank the reviewer for acknowledging their work.

 WEAK POINTS

• The paper stresses that the huge amount of data is deemed as big data. However, the authors do not properly highlight which challenges of big data (among volume, velocity, variety) are considered and how they are tackled.

Response

The following paragraphs have been included in the paper.

4.2. Characteristics of big data

 

• Volume: big data systems are associated with large volumes of data. Thus, data are created by machines, networks and social media. This means that the volume of data to be processed is high.
• Variety: it is related to data formats and the different sources that generate them. Thus the data format can be structured, semi-structured and unstructured. Since data are generated in formats such as photos, videos, logs, sensor devices, etc., this implies reaching challenges in terms of data storage, mining and analysis.
• Velocity: refers to the pace of data generation and arrival. This section should include the processing and interpretation time of the data received to improve decision making.
• Accuracy: it is essential that the data be reliable and of high quality. To this end, the tools that manage data truthfulness within big data systems must eliminate noise and abnormal data, which are modified to reliable and trustworthy data.

4.3. Big data framework for households

The framework presented comprises the data life cycle of the proposed network. It can be seen from the data generation phase to the analysis phase. Figure 7 shows the framework that serves as the basis for dealing with the big data of the proposed network.

Figure 7. The framework to deal electrical consumption in households with big data.

An integrated architecture based on big data and cloud computing is proposed. The parts of which the system consists are the cloud environment, the big data tools and the database. Fig 1 shows the architecture of the proposed system.

Figure 8. A big data framework for household energy power consumption.

4.3.1. Data generation

The generated data stream comes from Smart meters installed in houses and PV installations, measured every 0.5 s. The belonging of the monitored data is diverse (house, houses with PV, house with PV and EV and houses with EV). It is possible to complete the information of electrical variables generated with sensors that record meteorological variables. In addition to the application to houses, commercial buildings, factories, and other renewable sources, etc. could be included.

 

4.3.2. Data Acquisition

The data acquisition of the designed platform has in three tasks: data collection, data transmission and data pre-processing. The generated data have been explained in the previous point that are collected by the developed Smart Meters.

The transmission of the collected data is done through one or more master nodes that are part of the Hadoop cluster. The collected data is sent to the data storage system where it will be processed in the following phases.

Data integration uses techniques that aim to combine data from different Smart meters in order to unify the information. The files can be transferred in different formats, such as csv files, json files, etc.

The information generated by the Smart meters contains the time stamp, the ID of each house, voltage, current, active power, apparent power and PF. In the data pre-processing phase, erroneous information is modified or removed to improve data quality.

The data acquisition system has to fulfil the collection function. In this sense, it must collect, aggregate and send large volumes of data from the Smart meters to the Hadoop master node. The data is stored in files within an HDFS repository, in the formats used.

 

4.3.3. Data storage and processing

HDFS performs the function of storing data for further processing. HDFS clusters consist of a NameNode with the responsibility of controlling the file system metadata. DataNode lists are used to store the actual data. Hadoop Yarn is the resource manager for data analysis. Yarn runs in conjunction with HDFS on the same node list. This allows processing of nodes with data that are part of the system.

 

4.3.4. Data query

Once the data is stored in Firebase, a Python script is developed, which is executed from time to time, in an automatic and configurable way, by extracting, processing, and filtering all the information. This information is necessary for the next module, the system of analysis and visualization of the information. Subsequently, the information is uploaded to HDFS and the data can be consulted from Kibana.

 

4.3.4. Data analysis

The data of each home is sent to each of the users of the home, who can view it through the web and App. They can also be consulted anonymously by the researchers to analyse load profiles, estimate electricity demand, optimize electricity consumption, etc.

The main objectives of this phase are the improvement and stability of the system. The role of the consumers is fundamental in the system. The data visualization is done with Dashboard, with access through App or web portal.

 

• The “big data analytics” essence of the paper, even though is introduced as one of the contributions, boils down to the consultation of Kibana’s generated dashboards.

Response

Kibana is used for the visualisation of the data, the information is generated and processed on an own server. 

• The advances over existing literature are not clearly spelled out and the paper appears as a remarkable application example, but with a weak core of research underneath. 

Response

As discussed in the existing web and commercial meters, the main innovations that have been realized in this paper are:

• Integration and storage of data from multiple sources. Traditional data usually deals with data from a single domain, it is essential to find a fusion method for the data set from multiple sources, which has different modalities, formats and representations. In terms of big data storage, although some of the systems such as Hadoop (HDFS) seems to be feasible, it still needs to be adapted and modified to fit the big data power grid.
• Real-time data processing technology. For applications such as electricity consumption measurement with resolutions below 0.5 s, demand estimation studies, residential occupancy, etc. Although the cloud system is able to provide a fast calculation service, the network congestion, the complicated algorithm, combined with the massive amount of data, still results in latency.
• Big data visualization technology. Graph and chart visualization can present operators with granular and explicit changes of electrical variables. However, how to effectively find and represent correlations or trends among data from multiple sources is a major challenge.
• Data privacy and security. Data security is provided by 64-bit key encryption. Each user has their own account which makes them independent from others. On the other hand, a user with administrator role manages the system.

INTRODUCTION

• In its current shape, the Introduction is contaminated also with a discussion of state of the art approaches regarding meter data analytics (indeed, the title is repeated in the subsequent section).

Response

The highlights have been modified.

 

2. Design of a web platform that allows to analyse the different electrical magnitudes of the monitored houses, through the data that have been measured by a smart meter developed and sent to the cloud. The design of the system allows for the massive processing of multiple sets of household data, which enables studying the information obtained by applying different algorithms.

• Thus, it would be more useful to include this part in the Related Work section.

Response

Item 1.2 has been moved to the Related Work section and has become item 2.2.

• A major concern with the alleged contributions of the paper is that all of them do not effectively deliver a shift towards innovation in the research of big data analytics: undoubtedly, behind the paper there is a massive physical development and implementation work, but what about the research core?

Response                                  

The following paragraphs have been included in the paper.

 

The websites shown in Table 1 store the recorded data and do not work in real time, the measurements of the electrical variables have a granularity that varies between 1s and 10 min; the granularity times of the websites are less than 1s [1] [30] [31], 1s [1] [30] [32] [33] [34] [35] [36] [37], between 1s and 1min [1] [32] [38] [39] and 10 min. Some allow downloading of the stored data for free and others for a fee. The websites do not display the data of all the monitored variables in real time. Only [32] can display data from the previous day, but this is paid. They do not allow comparisons between different dwellings.

The smart meters used in the websites are commercial devices where the measured data are sent every 1s [30] [32] [33] [34] [35] [37] [38], every 1min [39] [40] and every 10 min. The websites do not indicate the costs associated with commercial Smart Meters.

Due to the existing limitations in the webs that store the data on the variables of electrical consumption homes, this research realizes a platform that allows to visualize the different electrical variables v, i, p, q, s and PF in real time with data upload every 0.5 s, from the smart meters installed in the different homes. In addition, each user has an App that allows to visualize the data in real time. Due to the fact that the measured time series are below 1s, a large amount of data is produced for each of the measured variables, so they have to be treated by NoSQL data management system and structured storage, which reduces the processing time. It has been included in the platform, the comparison between different dwellings, which allows to analyze the dwellings that have been monitored since 2019.

Another advantage that has been realized in this research is design of a measurement equipment and data upload to the cloud every 0.5 s, with data storage both in the cloud and in the equipment made with a data limit of 1 year. The Smart Meter has been designed with a cost of 46.28 €, considered as a low cost equipment. Being an open source equipment and can be programmed according to the needs at any time and for each user.

Table 1. Household Power Open Access Datasets.

Web site	Electrical variables	Time resolution	Number houses	Country
PECANSTREET [32]	v, i, p, q	1 s - 1 min	1,115	USA
ACS –F1 & ACS-F2 [38]	v, i, p, q, f, PF	10 s (0.1 Hz)	225	Switzerland
AMPds [39]	v, i, f, pf, p, q, s, e	1 min	1	CANADA
BLUED [1]	v, i	8.33x10-5 s (12 kHz)	1	USA
DRED [30]	p	1 s (1 Hz)	1	Netherlands
ECO [33]	v, i, p, q, PF	1 s (1 Hz)	6	SUIZA
GREEND [34]	p	1 s (1 Hz)		Austria and Italy
ERC	p	10 min	255	UK
iAWE [35]	v, i, f, p, q, s, e, PF	1 s	1	India
REDD [36]	v, p	6.66x10-5 s	2	USA
REFIT [40]	p	8 s	20	UK
Smart [41]	v, f, p, s	1 min	400	USA
Tracebase [37]	p	1 s	15	Germany
UK-DALE [31]	v, i, p, s	6.25x10-5 s,1 s, 6 s	3	UK

RELATED WORK

• While reading this section, the reader does not clearly perceive what are the novel contributions of this paper with respect to the related literature.

Response

The data measured on the variables of electricity consumption in homes, you can view the different electrical variables v, i, p, q, s and PF in real time with data upload every 0.5 s, from the smart meters installed in different homes. Another novelty is, each user has an App that allows to visualize the data in real time. Due to the fact that the measured time series are below 1s, a large amount of data is produced for each of the measured variables, so they have to be treated by NoSQL data management system and structured storage, which reduces the processing time. It has been included in the platform, the comparison of the measured variables between different dwellings, which allows to analyze them since July 2018.

Another advantage that has been realized in this research is design of a measurement equipment and data upload to the cloud every 0.5 s, with data storage both in the cloud and in the equipment made with a data limit of 1 year. The Smart Meter has been designed with a cost of 46.28 €, considered as a low cost equipment. Being an open source equipment and can be programmed according to the needs at any time and for each user.

• To better emphasise them, the discussion should put forth comparison features (with pros and cons) and a table summarising the analysed works: both of them are highly advocated to be included in the Section.

Response

Table 1 includes the characteristics of the variables measured, resolution time, number of dwellings and country. The existing websites do not visualize the data for each of the dwellings; they only allow downloading the data stored during the measurement time. Some of them are payable.

Most of the websites use commercial Smart Meters that do not allow to adapt the measurement and uploading of data below 1s. The costs associated with Smart Meters are not included in the references used.

The developed platform allows to visualize the different electrical variables v, i, p, q, s and PF in real time with data upload every 0.5 s. from the smart meters installed in the different dwellings. Another novelty is, each user has an App that allows to visualize the data in real time.

Due to the fact that the measured time series are below 1s, a large amount of data is produced for each of the measured variables, so they have to be treated by a NoSQL data management system that performs a structured storage and reduces the processing time.

A comparison of the variables measured between different dwellings has been included in the platform, which makes it possible to analyze them from the year 2019.

• Additionally, the literature should be enriched with more papers regarding big data approaches (currently, only few references are included).

Response

The following bibliography has been included.

[50]	A. A. Munshi and Y. A.-R. I. Mohameda, “Big data framework for analytics in smart grids,” Electric Power Systems Research, no. 151, pp. 369-380, 2017.
[51]	C. Tua, S. He, Z. Shuai and F. Jiang, “Big data issues in smart grid – A review,” Renewable and Sustainable Energy Reviews, no. 79, pp. 1099-1107, 2017.
[52]	A. Kumar, S. A. Alghamdi, A. Mehbodniya, M. a. haq, J. L. Webber and S. N. Shavkatovich, “Smart power consumption management and alert system using IoT on,” Sustainable Energy Technologies and Assessments, no. 53, p. 102555, 2022.
[53]	J. Wang, “A novel oscillation identification method for grid-connected,” 2021 International Conference on New Energy and Power Engineering (ICNEPE 2021), 2021.
[54]	X. Zhao, “Research on management informatization construction of electric,” 2022 International Symposium on New Energy Technology Innovation and Low Carbon, 2022.
[55]	N. Mostafa, H. S. M. Ramadan and O. Elfarouk, “Renewable energy management in smart grids by using big data analytics,” Machine Learning with Applications, no. 9, p. 100363, 2022.

• Big data applied to Smart Grids

Munshi et al. [50] presented innovative research for advancing smart grids through big data. They implemented a secure cloud-based platform. Tu et al. [51] conducted a state-of-the-art review of big data applied to smart grid integration. They reviewed big data applications for smart grids, focusing on the latest applications with the latest big data technologies. Kumar et al. [52] designed a circuit to help users take control of power consumption in their homes, improving energy savings through an intelligent method. The measured information from the monitored homes is stored in a big data server. Wang [53]proposed a localization oscillation scheme based on the theory and support of a vector machine, phase difference oscillation and forced phase difference oscillation. Zang [54] improved the data analysis and data mining tool in energy control and improved the service quality of the electricity market through the computerization of power systems. Mostafa et al. [55] develops a framework for implementing big data analytics for smart grids and renewable energy. Implemented a five-step method to predict the stability of smart grids using five different maching learning methods.

SYSTEM ARCHITECTURE - SYSTEM INTEGRATION

These sections present mainly implementation aspects, delving into the description of the physical architecture of the system, data collection process and data presentation techniques.

• Hence, the description is tightly coupled with the implementation strategy and fostered technologies (both hardware and software). Therefore, the main flaw regards a lack of generalisability.

Response

A device has been designed to measure the electrical variables for dwellings and obtain load profiles. The equipment sends the data through the Internet to the cloud every 0.5 s, due to the amount of data to be processed and analyzed. The data is visualized on the web and App in real time.

The system designed both in software and hardware can be used for general purpose. In software it can be programmed according to the needs of the installation to be monitored. The hardware exposed is for monitoring electrical installations in general, applied to the homes in this research.

• A real research contribution would be, for instance, devising a general architecture/methodology, independent from the implementation aspects.

Response

The following paragraph has been included to explain the proposed architecture and methodology

4.3. Big data framework for households

The framework presented comprises the data life cycle of the proposed network. It can be seen from the data generation phase to the analysis phase. Figure 7 shows the framework that serves as the basis for dealing with the big data of the proposed network.

 

Figure 7. The framework to deal electrical consumption in households with big data.

An integrated architecture based on big data and cloud computing is proposed. The parts of which the system consists are the cloud environment, the big data tools and the database. Fig 1 shows the architecture of the proposed system.

 

Figure 8. A big data framework for household energy power consumption.

• Moreover, as formerly remarked, the big data aspects authors coped with are quite fuzzy; it seems that the adoption of a resilient data acquisition system in synergy with a NoSQL storage system have become the panacea for solving big data issues, neglecting the adoption of techniques/algorithms to reduce the volume of data (e.g., clustering or other data summarisation algorithms), or to favour the exploration for the final users, attracting their attention only on relevant data.

Response

This point has been explained in the weak point.

The aforementioned aspects are driving factors to move towards big data analytics and exploration approaches.

• In this respect, it is true that dashboards are valuable instruments to ease the exploration of data, but only if exploration use cases are envisaged, otherwise the dashboards as conceived in the paper are useful only for expert users with a full knowledge of the domain and the collected data.

Response

The App that each user has is simpler and allows to interpret the information in an easy way, with basic data. The Kibana visualization Dashboards are reserved for more expert users or researchers in the sector.

• Conversely, other types of users (e.g., citizens, energy providers) are more likely to explore aggregated information, more concerning their personal interests, with predefined filters/views over data.

Response

For this type of users, a real time App has been developed.

5.1. App for non-expert users

For less experienced users who have difficulties in using the dashboards created, an app has been created that allows real-time monitoring in a simpler and more interpretable way. Each user can only see their own home in their app, leaving the comparison and visualisation of other data to the dashboards. The App is personalised for each user and only the user can use it.

The app displays real-time data with timestamp of voltage, current and power. In addition, graphs of these three variables can be displayed, allowing the user to observe the evolution of the recorded values over time. Figure X shows the apps for houses 11 and 13.

 

(a)	(b)

Figure 18. App for non-expert users: (a) House 11 and (b) House 13.

 

 

 

 

 

 

 

We believe that your constructive comments dramatically polished our paper content and conducted us to provide a very organized paper (unlike before) to be read and understand. Regarding conclusion, we agree with this assessment, and have modified accordingly, as explained in previous point.

 

 

Sincerely, The authors.

 

 

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments to authors

This paper presents a smart meter for households (SMH) to obtain load profiles and a new platform that allows the innovative analysis of captured Internet of Things data from smart homes, photovoltaic, and electrical vehicles.

However, problems are common and not clear in the abstract as well as in the introduction. It is necessary to clarify the problem for this paper.

Novelty is not clear in the present version manuscript.

The comparison table is also needed in Section 2 with limitations of existing works, and how the authors addressed these limitations with the proposed work. (Missing part)

The results section is totally blurred. (Visible Quality is poor) (Not understanding)

Cite top-rank journal references in the manuscript.

 Not recommended for further steps. Reconsider it. 

 

Author Response

REMARKS FOR THE REVIEWER:

 

We sincerely state our gratitude for invaluable time you allotted to review our paper. We believe that your constructive comments dramatically polished our paper content and conducted us to provide a very organized paper (unlike before) to be read and understand. We have revised and modified our paper generally based on other reviewers and specifically, on your comments as follows.

 

General modifications:

 

• New definitions relating to big data have been added.
• The innovations presented in this paper have been added.
• Comparison between different web portals has been added.
• New references have been added.
• New figures about the architecture and methodology of the system have been added.
• Several typos have been corrected.
• The text has been completely revised by a native speaker.

 

Specific modifications:

 

This paper presents a smart meter for households (SMH) to obtain load profiles and a new platform that allows the innovative analysis of captured Internet of Things data from smart homes, photovoltaic, and electrical vehicles.

 

1. However, problems are common and not clear in the abstract as well as in the introduction. It is necessary to clarify the problem for this paper.

Response

As discussed in the existing web and commercial meters, the main innovations that have been realized in this paper are:

• Integration and storage of data from multiple sources. Traditional data usually deals with data from a single domain, it is essential to find a fusion method for the data set from multiple sources, which has different modalities, formats and representations. In terms of big data storage, although some of the systems such as Hadoop (HDFS) seems to be feasible, it still needs to be adapted and modified to fit the big data power grid.
• Real-time data processing technology. For applications such as electricity consumption measurement with resolutions below 0.5 s, demand estimation studies, residential occupancy, etc. Although the cloud system is able to provide a fast calculation service, the network congestion, the complicated algorithm, combined with the massive amount of data, still results in latency.
• Big data visualization technology. Graph and chart visualization can present operators with granular and explicit changes of electrical variables. However, how to effectively find and represent correlations or trends among data from multiple sources is a major challenge.
• Data privacy and security. Data security is provided by 64-bit key encryption. Each user has their own account which makes them independent from others. On the other hand, a user with administrator role manages the system.

 

On the other hand, the problems encountered on the existing web sites can be read in answer 3.

 

2. Novelty is not clear in the present version manuscript.

Response

In the introduction, the novelties of this paper are cited. They are reproduced below.

 

1. Develop a new prototype SMH for monitoring electrical variables, upload data to cloud using wireless network.
2. Design of a web platform that allows to analyse the different electrical magnitudes of the monitored houses, through the data that have been measured by a smart meter developed and sent to the cloud. The design of the system allows for the massive processing of multiple sets of household data, which enables studying the information obtained by applying different algorithms.
3. The design components of the platform architecture were created. Specifically, the study of the data structure obtained from the households is developed, minimizing the communication overload with the cloud, and the design of the website where the data obtained appears for use by the research community.
4. The necessary technology to obtain data in real time as well as process and store data in the cloud for integrating into the control panels was developed; this shows user data in graphic form and allows downloading the data.

 

 

3. The comparison table is also needed in Section 2 with limitations of existing works, and how the authors addressed these limitations with the proposed work. (Missing part)

Response

The following paragraphs have been included in the paper.

 

The websites shown in Table 1 store the recorded data and do not work in real time, the measurements of the electrical variables have a granularity that varies between 1s and 10 min; the granularity times of the websites are less than 1s [1] [30] [31], 1s [1] [30] [32] [33] [34] [35] [36] [37], between 1s and 1min [1] [32] [38] [39] and 10 min. Some allow downloading of the stored data for free and others for a fee. The websites do not display the data of all the monitored variables in real time. Only [32] can display data from the previous day, but this is paid. They do not allow comparisons between different dwellings.

The smart meters used in the websites are commercial devices where the measured data are sent every 1s [30] [32] [33] [34] [35] [37] [38], every 1min [39] [40] and every 10 min. The websites do not indicate the costs associated with commercial Smart Meters.

Due to the existing limitations in the webs that store the data on the variables of electrical consumption homes, this research realizes a platform that allows to visualize the different electrical variables v, i, p, q, s and PF in real time with data upload every 0.5 s, from the smart meters installed in the different homes. In addition, each user has an App that allows to visualize the data in real time. Due to the fact that the measured time series are below 1s, a large amount of data is produced for each of the measured variables, so they have to be treated by NoSQL data management system and structured storage, which reduces the processing time. It has been included in the platform, the comparison between different dwellings, which allows to analyze the dwellings that have been monitored since 2019.

Another advantage that has been realized in this research is design of a measurement equipment and data upload to the cloud every 0.5 s, with data storage both in the cloud and in the equipment made with a data limit of 1 year. The Smart Meter has been designed with a cost of 46.28 €, considered as a low cost equipment. Being an open source equipment and can be programmed according to the needs at any time and for each user.

Table 1. Household Power Open Access Datasets.

Web site	Electrical variables	Time resolution	Number houses	Country
PECANSTREET [32]	v, i, p, q	1 s - 1 min	1,115	USA
ACS –F1 & ACS-F2 [38]	v, i, p, q, f, PF	10 s (0.1 Hz)	225	Switzerland
AMPds [39]	v, i, f, pf, p, q, s, e	1 min	1	CANADA
BLUED [1]	v, i	8.33x10-5 s (12 kHz)	1	USA
DRED [30]	p	1 s (1 Hz)	1	Netherlands
ECO [33]	v, i, p, q, PF	1 s (1 Hz)	6	SUIZA
GREEND [34]	p	1 s (1 Hz)		Austria and Italy
ERC	p	10 min	255	UK
iAWE [35]	v, i, f, p, q, s, e, PF	1 s	1	India
REDD [36]	v, p	6.66x10-5 s	2	USA
REFIT [40]	p	8 s	20	UK
Smart [41]	v, f, p, s	1 min	400	USA
Tracebase [37]	p	1 s	15	Germany
UK-DALE [31]	v, i, p, s	6.25x10-5 s,1 s, 6 s	3	UK

 

 

4. The results section is totally blurred. (Visible Quality is poor) (Not understanding)

Response

Figures have been improved.

 

5. Cite top-rank journal references in the manuscript.

Response

The following bibliography has been included.

[50]	A. A. Munshi and Y. A.-R. I. Mohameda, “Big data framework for analytics in smart grids,” Electric Power Systems Research, no. 151, pp. 369-380, 2017.
[51]	C. Tua, S. He, Z. Shuai and F. Jiang, “Big data issues in smart grid – A review,” Renewable and Sustainable Energy Reviews, no. 79, pp. 1099-1107, 2017.
[52]	A. Kumar, S. A. Alghamdi, A. Mehbodniya, M. a. haq, J. L. Webber and S. N. Shavkatovich, “Smart power consumption management and alert system using IoT on,” Sustainable Energy Technologies and Assessments, no. 53, p. 102555, 2022.
[53]	J. Wang, “A novel oscillation identification method for grid-connected,” 2021 International Conference on New Energy and Power Engineering (ICNEPE 2021), 2021.
[54]	X. Zhao, “Research on management informatization construction of electric,” 2022 International Symposium on New Energy Technology Innovation and Low Carbon, 2022.
[55]	N. Mostafa, H. S. M. Ramadan and O. Elfarouk, “Renewable energy management in smart grids by using big data analytics,” Machine Learning with Applications, no. 9, p. 100363, 2022.

 

 

We believe that your constructive comments dramatically polished our paper content and conducted us to provide a very organized paper (unlike before) to be read and understand. Regarding conclusion, we agree with this assessment, and have modified accordingly, as explained in previous point.

 

 

Sincerely, The authors.

 

Author Response File: Author Response.pdf

Reviewer 3 Report

The paper presents a smart meter device for households to measure electrical variables, store them locally, and upload them to the cloud. In addition, the authors realized a platform for analyzing the uploaded data.

 

There are some problems related to the language used: missing letters, confusing phrasing, and duplicate text. 

• For example, between lines 330 and 333, there are two sentences with mostly the same content except for the last part: "three types of elements" and "two types of elements". 
• The "KPI" definition from line 338 is a general one with little to do with the current work. Another explanation that suits the implemented platform would be more appropriate. 
• The entire text needs to be revised, maybe proofread by someone with a better understanding of English. Also

 

There are some inconsistencies regarding the figures. 

• For example, in line 221, you mention the WDM1m timeline from Figure 3. However, figure 3 presents the timeline for the measurement process, as stated in the text, line 209. 
• In the "Results" section, you mention multiple times the graph on the "right" (for example, in line 398), but the diagrams are top/bottom placed, not left/right. 
• The values on the Ox/Oy axis for the graphs presenting the electrical parameters take much work to read. 

At the beginning of the "Result" section, a sentence states, "data has been collected from several households". Later, in the same section, data from three households are presented. Only three households were used to collect data to prove the validity of the design? 

 

Figure 12 shows the possibility of filtering the results by device. If only one SMH is in the household, Figure 7, how can the user see electrical data from different devices?

 

Figure 13 and 14 shows graphs from three households, or at least the text on the right corner suggests this. In the text, at line 442, is the following statement "Figure 13 and Figure 14 compares two households, an EV and a PV". Please explain.

 

In line 376 is suggested that the data presented corresponds to October 6 and 7 but later, in line 445, is mentions January. Please explain.

 

Some observations regarding the Conclusions.

• Is there any novelty in the developed SMH compared to the existing products mentioned in the bibliography (for example, reference 28)?
• What "innovative data analysis" is possible with the presented web platform? The presented graphs show only plots of measured/computed electrical values (v, i, p, q).
• "the authors have refined the platform component" - line 481. Please explain.
• "This research use datasets with extensive temporal data with different types of households". The graphs show data collected for two days from three households. Is this enough to validate the platform's applicability and robustness?

Author Response

REMARKS FOR THE REVIEWER:

We sincerely state our gratitude for invaluable time you allotted to review our paper. We believe that your constructive comments dramatically polished our paper content and conducted us to provide a very organized paper (unlike before) to be read and understand. We have revised and modified our paper generally based on other reviewers and specifically, on your comments as follows.

 

General modifications:

 

• New definitions relating to big data have been added.
• The innovations presented in this paper have been added.
• Comparison between different web portals has been added.
• New references have been added.
• New figures about the architecture and methodology of the system have been added.
• Several typos have been corrected.
• The text has been completely revised by a native speaker.

 

Specific modifications:

 

The paper presents a smart meter device for households to measure electrical variables, store them locally, and upload them to the cloud. In addition, the authors realized a platform for analyzing the uploaded data.

 

1. There are some problems related to the language used: missing letters, confusing phrasing, and duplicate text.

 

• For example, between lines 330 and 333, there are two sentences with mostly the same content except for the last part: "three types of elements" and "two types of elements".

Response

The text has been modified.

 

• The "KPI" definition from line 338 is a general one with little to do with the current work. Another explanation that suits the implemented platform would be more appropriate.

Response

 

• A key performance indicator is a measurable value that demonstrates the effectiveness of a company in achieving its key business objectives. The proposed KPI model plays a key role in this work on monitoring residential electricity consumption. The idea is to monitor the electrical variables of homes. Based on our experience in the implementation and creation of smart meter projects, we have defined the most relevant characteristics to know the performance of this type of systems. The proposed KPI allows to measure the quality of the measurements made, the power and energy consumption of the house.

 

• The entire text needs to be revised, maybe proofread by someone with a better understanding of English.

Response

The text has been reviewed by a native speaker.

 

2. There are some inconsistencies regarding the figures.

 

• For example, in line 221, you mention the WDM1m timeline from Figure 3. However, figure 3 presents the timeline for the measurement process, as stated in the text, line 209.

Response

Figure has been modified.

 

• In the "Results" section, you mention multiple times the graph on the "right" (for example, in line 398), but the diagrams are top/bottom placed, not left/right.

Response

The text has been modified.

 

• The values on the Ox/Oy axis for the graphs presenting the electrical parameters take much work to read.

Response

Figures have been modified.

 

3. At the beginning of the "Result" section, a sentence states, "data has been collected from several households". Later, in the same section, data from three households are presented. Only three households were used to collect data to prove the validity of the design?

Response

 

The platform stores data for a total of 20 households since July 2019. In the results section, graphs of three households with different characteristics have been included. Homes with PV generation, EV electric vehicle and with different characteristics in terms of the number of people living in the home are available. New homes are being sought to be added to the grid.

 

4. Figure 12 shows the possibility of filtering the results by device. If only one SMH is in the household, Figure 7, how can the user see electrical data from different devices?

Response

The platform allows us to compare the different variables between the different monitored households. But there is only one Smart Meter in each home. The data is sent to the cloud and through data management all the measured data can be visualized, and the different homes can be compared.

 

5. Figure 13 and 14 shows graphs from three households, or at least the text on the right corner suggests this. In the text, at line 442, is the following statement "Figure 13 and Figure 14 compares two households, an EV and a PV". Please explain.

Response

In the results section, graphs of three households with different characteristics have been included. The households mentioned in question 3 have been monitored.

 

• In line 376 is suggested that the data presented corresponds to October 6 and 7 but later, in line 445, is mentions January. Please explain.

Response

The text has been modified

 

 

6. Some observations regarding the Conclusions.

 

• Is there any novelty in the developed SMH compared to the existing products mentioned in the bibliography (for example, reference 28?

Response

Reference [28] develops a Smart plug, not a Smart meter, although it can measure the electrical parameters of the equipment that is connected to the Smart plug. In addition, in [28] a multiple sensor model PZEM 004t is used that can measure several variables simultaneously. The communication technology used is LoRa and not Wi-Fi as in this research.

 

• What "innovative data analysis" is possible with the presented web platform? The presented graphs show only plots of measured/computed electrical values (v, i, p, q).

Response

As the reviewer indicates, the data of the variables (v, i, p, q, s and PF) are shown. From the measured data, other analyses could be carried out, such as estimation of the electricity demand, housing occupancy rates, etc. The scope of which is beyond the scope of this research.

 

• "the authors have refined the platform component" - line 481. Please explain.

Response

The Smart Meter was first developed in the laboratory. Once the operation was validated, two pilot houses were implemented and operated for 6 months. Once the operation of the two houses was checked, other houses were progressively added until the current 20 houses were reached. The platform has evolved from the initial phase to the current one, improving the visualization of variables and adding new filters for information consultation. Renewable sources and electric vehicles have been added.

 

• "This research use datasets with extensive temporal data with different types of households". The graphs show data collected for two days from three households. Is this enough to validate the platform's applicability and robustness?

Response

The platform has been operating since July 2018 with 2 dwellings. This indicates that it is a solid and valid platform to be used as a web where Smart Meter data obtained from different homes with different characteristics are stored. It has monitored dwellings with electric power generation with Photovoltaic PV, with EV electric vehicles and with a greater or lesser number of inhabitants.

 

 

We believe that your constructive comments dramatically polished our paper content and conducted us to provide a very organized paper (unlike before) to be read and understand. Regarding conclusion, we agree with this assessment, and have modified accordingly, as explained in previous point.

 

 

Sincerely, The authors.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

In the revised version of the papers, authors attempted to solve the concerns that emerged in the previous version.

 

In the following, other suggestions to further improve the readability of the paper are enlisted (note that there are several typos which have to be corrected within the added yellow parts).

 

In the Related Work section, the novel contributions could be grouped into a “Novel Contribution” subsection, in order to find them directly in a unique place in the written text.

Moreover, it seems that the innovation is based solely on improvements made with respect to existing websites, without performing a thorough comparison with the related literature (existing papers). Please, try to address this issue by clarifying what are the enhancements of this paper both with respect to the websites (as it has been done through the table) and the papers from the literature.

 

Figure 8 does not add details with respect to Figure 7 and therefore they can be potentially merged together.

 

Apart from perceiving the temporal evolution of the physical quantity measured (e.g., current, power, …) it would be interesting to provide the end-users with additional information, for instance aggregated information derived from measures, which would be more meaningful, plus instruments to perform multi-dimensional analysis on data.

 

Overall, there are still open issues to be solved as a future research stream.

For instance, improving the usability and the design of the end-user app (which seems to be a replica of Kibana’s dashboards) and performing an in-depth experimentation on time required to process data acquired from the devices.

 

Unfortunately, none of the aforementioned aspects is mentioned in the Conclusions sections, which is a mere repetition of sentences included in the Introduction.

Author Response

Reviewers' comments:

Reviewer: #1

REMARKS FOR THE REVIEWER:

We sincerely state our gratitude for invaluable time you allotted to review our paper. We believe that your constructive comments dramatically polished our paper content and conducted us to provide a very organized paper (unlike before) to be read and understand. We have revised and modified our paper generally based on other reviewers and specifically, on your comments as follows.

General modifications:

• The innovations presented in this paper have been added in point 2.5.
• Figure 7 has been deleted.
• Conclusions have been improved.
• Future lines of work have been included.

Specific modifications:

In the revised version of the papers, authors attempted to solve the concerns that emerged in the previous version.

In the following, other suggestions to further improve the readability of the paper are enlisted (note that there are several typos which have to be corrected within the added yellow parts).

In the Related Work section, the novel contributions could be grouped into a “Novel Contribution” subsection, in order to find them directly in a unique place in the written text.

Item 2.5. Novel Contributions has been added. The novel contributions developed in the article have been included in this point.

 

Moreover, it seems that the innovation is based solely on improvements made with respect to existing websites, without performing a thorough comparison with the related literature (existing papers). Please, try to address this issue by clarifying what are the enhancements of this paper both with respect to the websites (as it has been done through the table) and the papers from the literature.

The authors have carried out a review of the existing websites and have analysed them in table 1. An analysis of the strengths and weaknesses in table 1 has been developed. The other papers related work have not included the analysis of the website.

Figure 8 does not add details with respect to Figure 7 and therefore they can be potentially merged together.

Figure 8 has been deleted.

Apart from perceiving the temporal evolution of the physical quantity measured (e.g., current, power, …) it would be interesting to provide the end-users with additional information, for instance aggregated information derived from measures, which would be more meaningful, plus instruments to perform multi-dimensional analysis on data.

We welcome your comments. In future lines we have included aggregated information derived from measures, plus instruments to perform multi-dimensional analysis on data.

Overall, there are still open issues to be solved as a future research stream.

Future lines of research would include adding datasets from different electrical devices, such as household appliances. In addition, developing a demand response algorithm to be included in the SM to work in conjunction with the proposed methodology. Another possible line would be aggregated information derived from measurements with application of artificial intelligence algorithms, e.g. automatic prediction systems. And development of tools to perform a multidimensional analysis of the data.

For instance, improving the usability and the design of the end-user app (which seems to be a replica of Kibana’s dashboards) and performing an in-depth experimentation on time required to process data acquired from the devices.

We welcome your comments. In the application developed for users, Kibana's dashboards are not used, the data is taken directly from HDFS and Yarn, and the electrical magnitudes are displayed via the App.

 

Unfortunately, none of the aforementioned aspects is mentioned in the Conclusions sections, which is a mere repetition of sentences included in the Introduction.

Conclusions have been improved and added future works.

We believe that your constructive comments dramatically polished our paper content and conducted us to provide a very organized paper (unlike before) to be read and understand. Regarding conclusion, we agree with this assessment, and have modified accordingly, as explained in previous point.

Sincerely, The authors.

Author Response File: Author Response.pdf

Reviewer 3 Report

Thank you for the changes to the paper. They improved the scientific value of the work done.

Author Response

Reviewers' comments:

Reviewer: #3

REMARKS FOR THE REVIEWER:

We sincerely state our gratitude for invaluable time you allotted to review our paper. We believe that your constructive comments dramatically polished our paper content and conducted us to provide a very organized paper (unlike before) to be read and understand. We have revised and modified our paper generally based on other reviewers and specifically, on your comments as follows.

 

General modifications:

• The innovations presented in this paper have been added in point 2.5.
• Figure 7 has been deleted.
• Conclusions have been improved.
• Future lines of work have been included.

Specific modifications:

Thank you for the changes to the paper. They improved the scientific value of the work done.

We welcome your comments

Author Response File: Author Response.pdf