Accuracy and Inter-Unit Reliability of Ultra-Wide-Band Tracking System in Indoor Exercise
and José Pino Ortega 1,*Round 1
Reviewer 1 Report
This paper presents some results on the usage of an UWB positioning system for sport applications. If I am not wrong, the authors exploited a commercial UWB positioning system, hence the contribution of this work is just on the study on this specific kind of application, which is probably new.
In my opinion, the scientific contribution of the paper is minor, however there might be some commercial interest in the results shown in the paper mostly related to the potential of this kind of technology in sport applications.
Some minor observations:
- line 69: clarify what you mean for: "Because of the different architecture and buildings used for different courts."
- line 98: please, specify that you are considering 2D trilateration.
- figure 2: I guess 5 and 7 meters are referred to the distances indicated by the red arrows. If this is the case, "5 m" adn "7 m" indications should be positioned much closer to the red arrows. If this is not the case, authors should provide a clearer explanation of such distances.
Author Response
Response to Reviewer 1 Comments
Point 1: This paper presents some results on the usage of an UWB positioning system for sport applications. If I am not wrong, the authors exploited a commercial UWB positioning system, hence the contribution of this work is just on the study on this specific kind of application, which is probably new.
In my opinion, the scientific contribution of the paper is minor, however there might be some commercial interest in the results shown in the paper mostly related to the potential of this kind of technology in sport applications.
Response 1: Despite that UWB-based positioning system has become a common tool in professional team sport. Accuracy and reliability of this system has never been evaluated in this context. Only distance covered have been assessed, when UWB was development for high accuracy measures. The present study reveals the accuracy and inter-unit reliability of x and y coordinates of player position along the full court in indoor conditions, never revealed in previous studies, and also, about the present model. We think that the high impact of the present research in the scientific literature is in two aspects: the satisfactory results of accuracy and inter-unit reliability of the model evaluated for monitoring of the position of players (time-motion and tactical analyses application), and the innovative, simple and economic methodology used for this. Both aspects would be useful for sports science.
Point 2:- line 69: clarify what you mean for: "Because of the different architecture and buildings used for different courts."
Response 2:Thanks for your comments. We added the sentence: “It refers to the elements and their arrangement around the indoor court as distance from the court to the walls, marker position, etc.”
Point 3:- line 98: please, specify that you are considering 2D trilateration.
Response 3:According to reviewer suggestion 2D trilateration was specified
Point 4:- figure 2: I guess 5 and 7 meters are referred to the distances indicated by the red arrows. If this is the case, "5 m" adn "7 m" indications should be positioned much closer to the red arrows. If this is not the case, authors should provide a clearer explanation of such distances.
Response 4:Thank you very much for the proposal. These indications were positioned closer to the red arrows in Figure 2.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
- The paper aims to assess the accuracy of the positional data and the inter-unit reliability of an ultra-wide-band (UWB) tracking system in indoor exercise.
- The novelty of the paper is quite poor. The UWB technology and the positioning algorithm are not properly explained. Moreover, the experimental setup and the statistical analysis should be more precisely described. The conclusion that the UWB outperforms GNSS was expected. It is not clear the motivation of the paper: if the aim was only to prove that the UWB traking systems have a cm level accuracy and that they can be used for monitoring performance in sport, it is quite poor.
- Below some minor and major comments.
- “Moreover, UWB technology is the ideal … to provide positioning information … especially indoor conditions, which can cause problems in accuracy (e.g., interference with the radio frequency signal). Because of the different architecture and buildings (distance from the court to the walls, marker position, etc.) used for different courts.”
This reasoning is not clear. Please describes the reasons why UWB positioning should be appropriate for indoor environment and why this environment it is not suitable for positioning with other kind of positioning technologies such as GNSS etc.
- Please add references to UWB technology and UWB positioning. Not only referring to sports.
- “This UWB system is designed to alleviate any satellite reference problems by using time-based positioning techniques in which the signal propagates from the transmitter to the receiver.” This definition sounds quite poor and ambiguous. Also GNSSs are positioning systems in which the signal propagates from the transmitter to the receiver. Moreover, the proposed UWB system does not rely on satellites but I suppose it will rely on another infrastructure placed close to the sideline.
- "Thus, if all nodes have a common clock”. If the nodes are synchronized, please describe the considered synchronization process.
- The author describes the positioning system in 7 rows, without references, and just describing a geometric approach. Please, expand this part.
- “If the target receives UWB signals from more than three references, the accuracy of the position measurement increases.” This is generally true, but not always. It is true in ideal conditions.
- Please add pictures referring to the considered devices and the participant wearing the devices.
- The paths travelled by the participants are not clearly described. Are the path travelled in both directions? How many times the participant travels the path for each test? Are there the starting and ending points in the paths?
- “The reference system to compare the results was projected in the software using a GIS mapping application. This application allows representation of geometrical shapes, such as polygons or circles, with millimeter accuracy.” It is not clear what is the reference system. Does the author refer to the paths measured using the trundle wheel?
Author Response
Response to Reviewer 2 Comments
Point 1: - The paper aims to assess the accuracy of the positional data and the inter-unit reliability of an ultra-wide-band (UWB) tracking system in indoor exercise.
- The novelty of the paper is quite poor. The UWB technology and the positioning algorithm are not properly explained. Moreover, the experimental setup and the statistical analysis should be more precisely described. The conclusion that the UWB outperforms GNSS was expected. It is not clear the motivation of the paper: if the aim was only to prove that the UWB traking systems have a cm level accuracy and that they can be used for monitoring performance in sport, it is quite poor.
Response 1: According to reviewer suggestion the UWB technology was more detailed in method sections. However, positioning algorithm is unknown, and is a patent of manufacturers. The algorithm used by the manufacturers does not influence the findings of the study, since no previous study has provided this information. On the other hand, the experimental setup and the statistical analysis was more described in the new version of the manuscript.
On the other hand, we are not agree with the reviewer comment: “The conclusion that the UWB outperforms GNSS was expected. It is not clear the motivation of the paper: if the aim was only to prove that the UWB tracking systems have a cm level accuracy and that they can be used for monitoring performance in sport, it is quite poor.” Previous study established that the presents technology (GPS and UWB) are similar (not significant difference between them) in accuracy and reliability for time-motion analysis. Instead, this comparison in other applications (such positioning of players on the pitch) was necessary and no assessed previously (Bastida-Castillo, Gómez-Carmona, De la Cruz-Sánchez, and Pino-Ortega, 2018). This argument was added in introduction section.
Bastida Castillo, A.; Gómez Carmona, C.D.; De la Cruz Sánchez, E.; Pino Ortega, J. Accuracy, intra- and inter-unit reliability, and comparison between GPS and UWB-based position-tracking systems used for time–motion analyses in soccer. European Journal of Sport Science 2018, 1–8.
Despite that UWB-based positioning system has become a common tool in professional team sport. Accuracy and reliability of this system has never been evaluated in this context. Only distance covered have been assessed, when UWB was development for high accuracy measures. The present study reveals the accuracy and inter-unit reliability of x and y coordinates of player position along the full court in indoor conditions, never revealed in previous studies, and also, about the present model. We think that the high impact of the present research in the scientific literature is in two aspects: the satisfactory results of accuracy and inter-unit reliability of the model evaluated for monitoring of the position of players (time-motion and tactical analyses application), and the innovative, simple and economic methodology used for this. Both aspects would be useful for sports science.
Point 2:- “Moreover, UWB technology is the ideal … to provide positioning information … especially indoor conditions, which can cause problems in accuracy (e.g., interference with the radio frequency signal). Because of the different architecture and buildings (distance from the court to the walls, marker position, etc.) used for different courts.” This reasoning is not clear. Please describes the reasons why UWB positioning should be appropriate for indoor environment and why this environment it is not suitable for positioning with other kind of positioning technologies such as GNSS etc.
Response 2:Thank you for your suggestion. The paragraph was modified as follow: “Moreover, UWB technology is the ideal radio frequency-based candidate to provide positioning information in any environment (indoor and outdoor), but especially indoor conditions, which GPS cannot operate, and others radiofrequency systems have problems in accuracy (e.g., interference with the radio frequency signal). Because of the different architecture and buildings used for different courts. It refers to the elements and their arrangement as distance from the court to the walls, marker position, etc.“
Point 3:- Please add references to UWB technology and UWB positioning. Not only referring to sports.
Response 3:According to reviewer suggestion, references to UWB technology in other areas was added.
23. Pittet, S.; Renaudin, V.; Merminod, B.; Kasser, M. UWB and MEMS Based Indoor Navigation. J. Navig. 2008, 61.
24. Sczyslo, S.; Schroeder, J.; Galler, S.; Kaiser, T. Hybrid localization using UWB and inertial sensors. In Proceedings of the 2008 IEEE International Conference on Ultra-Wideband; IEEE: Hannover, 2008; pp. 89–92.
25. Hirokawa, R.; Ebinuma, T. A Low-Cost Tightly Coupled GPS/INS for Small UAVs Augmented with Multiple GPS Antennas. Navigation 2009, 56, 35–44.
26. Hol, J.D.; Schon, T.B.; Gustafsson, F. Ultra-wideband calibration for indoor positioning. In Proceedings of the 2010 IEEE International Conference on Ultra-Wideband; IEEE: Nanjing, China, 2010; pp. 1–4.
Point 4:- “This UWB system is designed to alleviate any satellite reference problems by using time-based positioning techniques in which the signal propagates from the transmitter to the receiver.” This definition sounds quite poor and ambiguous. Also, GNSSs are positioning systems in which the signal propagates from the transmitter to the receiver. Moreover, the proposed UWB system does not rely on satellites but I suppose it will rely on another infrastructure placed close to the sideline.
Response 4:We are agreed with your comments. The description of UWB system has been improved to make more legible: This UWB system is designed to alleviate any satellite reference problems by using time-based positioning techniques in which the signal propagates from the transmitter (antenna) to the receiver (device). The transmitter in UWB system is a set of antennas (4, 6, 8 or more) that transmit the radiofrequency signal almost under the same principle as the GPS system [24].But in this case, this reference system is fixed, and is not affected by environmental conditions [25].
The UWB system is composed of two sub-systems: (i) the reference system and (ii) the devices tracked (carried by the participants). The reference system is composed of antennas that are transmitters and receivers of the radio frequency signal (see Figure 2). Antennas (mainly the master antenna) perform the computerizing of the position of the devices that are in their area of performance, while the devices receive that calculation.
Point 5:- "Thus, if all nodes have a common clock”. If the nodes are synchronized, please describe the considered synchronization process.
Response 5: The localization technologies can roughly be subdivided into three categories: systems using time, systems using angle-of-arrival and systems using signal strength. UWB system used in this study used time-based method, where position is inferred from the time it takes for a signal to travel from the transmitter to the receiver. That is, the position of the receivers (devices ported by players) is determined from time of arrival (toa) measurement and a common clock in all nodes are necessary. This principle was explained in method section. Then, Once the antennas was installed, they were switched on one-by-one, with the master antenna turned on last, and a process of autocalibration of the antennae was carried out for 5 min, which the master synchronized all antennas with a common clock. Each instant of time that a position coordinates was calculated (18Hz), the master send to the rest of antennas a synchronization in time.
Point 6: - The author describes the positioning system in 7 rows, without references, and just describing a geometric approach. Please, expand this part.
Response 6: According to reviewer suggestion, equipment section was more detailed. See equipment section
Point 7: - “If the target receives UWB signals from more than three references, the accuracy of the position measurement increases.” This is generally true, but not always. It is true in ideal conditions.
Response 7: We are agreed with your comments, the consideration that it is true in ideal condition was specified in the new version of the manuscript.
Point 8:- Please add pictures referring to the considered devices and the participant wearing the devices.
Response 8: According to reviewer suggestion pictures of the device and participant wearing the device was added in the new version of the manuscripts (Figure 1)
Point 9:- The paths travelled by the participants are not clearly described. Are the path travelled in both directions? How many times the participant travels the path for each test? Are there the starting and ending points in the paths?
Response 9: According to reviewer suggestion the paths travelled by the participants was clarify in the new version of the manuscript:
All the tasks started from a standing position. Each task was performed three times for each direction with an interval of at least 5 min of rest between trials. The participants made the different movements shown (Figure 4) according to two criteria: (i) to move only on the lines marked on the basketball court, and (ii) to carry out the movements reaching a speed of >15 km/h when the course allowed it. However, in the center circle (c) and 6.75 m line (e) it was not possible to reach 15 km/h due to the centrifugal force generated by the curvilinear trajectory.
Of all the data entered, only those that corresponded to the execution of the routes were selected, according to registers obtained using ANT+ technology and photocells at the beginning and end of the test. The protocol to register the beginning and the end of the test were described in a previous study (see to more information) [28].
In routes (b), (c) and (d), a number was assigned to each edge of the projected rectangle, so in each test the software automatically calculated the distance of the participant's position between the court line where displacement occurred (y coordinate) and the opposite court line (x coordinate). In lanes (e) and (f), the centroid of the projected circle was assigned, so in each test the software automatically calculated the distance of the position of the participant from the indicated centroid (coordinate x). The calculation of the distance of the participant’s position according to the reference element was made every 0.5 s, obtaining a total of 8344 samples. The reference distances were as follow:
(b) Perimeter markings of court. (1) Baseline: 28 m with respect to the opposite baseline (coordinate x) and 0 m with respect to the displacement baseline (coordinate y); (2) Lateral line: 0 m with respect to the displacement lateral line (coordinate x) and 15 m with respect to the opposite lateral line (coordinate y).
(c) Middle line court. 14 m with respect to the baseline (coordinate x) and 0 m with respect to the center line (coordinate y).
(d) Exterior perimeter of the painted lines. (1) Baseline/front-line of the paint: 5.8 m with respect to the baseline or the front line of the paint (coordinate x) and 0 m with respect to the displacement line of the paint (coordinate y). (2) Lateral line of the paint: 0 m with respect to the displacement line of the paint (coordinate x) and 4.9 m with respect to the opposite lateral line of the paint.
(e) Centre circle. 1.8 m with respect to the centroid of the center circle (coordinate x).
(f) 6.75 m line. 6.75 m with respect to the centroid of the three-point line (coordinate x).
Point 10:- “The reference system to compare the results was projected in the software using a GIS mapping application. This application allows representation of geometrical shapes, such as polygons or circles, with millimeter accuracy.” It is not clear what is the reference system. Does the author refer to the paths measured using the trundle wheel?
Response 10: According to reviewer suggestion the reference system was clarify in the new version of the manuscript:
The reference system to compare the results was projected in the software using a GIS mapping application. To do this, the set of points obtained by the step 4 in procedures were downloaded into the software. Centroid of the each set of point was calculated being the point reference (the four corners) to project the template of the court (Figure 3B). Standard deviation of the centroid calculated in all sets were ± 2.2 cm and ± 2.9 cm for coordinate x and y respectively. GIS mapping application allows representation of geometrical shapes, such as polygons or circles, with millimeter accuracy. In this way, the routes executed with their real measurements were introduced on the template created before. And then, the x and y coordinate data obtained of the UWB system were introduced and compared. Distance errors of each axis (coordinate) were reported. Of all the data entered, only those that corresponded to the execution of the routes were selected, according to registers obtained using ANT+ technology and photocells at the beginning and end of each test. The protocol to register the beginning and the end of the test were described in a previous study (see to more information) [28].
Point 11: Moderate English changes required
Response 11: English editing service by applied science was performed in the previous version of the manuscript.
Author Response File: Author Response.pdf
This manuscript is a resubmission of an earlier submission. The following is a list of the peer review reports and author responses from that submission.
Round 1
Reviewer 1 Report
This work investigates the usage of an UWB-based positioning system for sport applications. Despite this kind of case study is quite new it is not so clear to me what kind of scientific contribute can provide to what already published in the literature.
Even not considering such observation about the poor research contribution, several aspects should be improved on this paper.
- more details should be provided about the considered UWB system and its configuration
- if I am not wrong, the authors considered 6 static UWB devices used as anchors to estimate the positions of just two moving ones. This should be better clarified from the beginning of the paper. Furthermore, this seems to be an almost useless working condition. Most of the times players are much more than two, consequently the work should be either extended to the usage of more moving devices or focused on certain specific sports (e.g. tennis). It is worth to notice that the usage of more (moving) devices would probably reduce the UWB measurement update frequency (actually this depends on the communication protocol of the used UWB system), hence probably worsening the performance of the system.
- authors should also better explain how they computed the position error. It seems like they constrained movements of the players just along certain lines to ease such computation, but it is not clear to me how they checked the exact position of an athlete at a specific time instant.
- Actually, the presented results seem to be a quite poor and insufficient validation for the proposed sport applications: a more detailed analysis of the system performance is needed.
- Geometric problem described in figure 1 (and in the corresponding part of the text) is not triangulation but TRILATERATION.
Other minor observations:
- figure 2 and lines from 96: position of UWB devices is unclear. For instance, are four of them on the corners of the outer rectangle draw in fig 2? Since they form a hexagon probably this is not the case: exact positions should be shown in the figure and properly described in the paper.
- line 100, what is exactly estimated in the UWB system autocalibration step? Positions of all such static devices?
- line 108: it is completely unclear what is n and what kind of samples authors are referring to.
Reviewer 2 Report
The topic of the paper is certainly interesting, but many improvements are needed.
The state of the art is missing.
Three nodes are sufficient for 2d localization. Please comment on this.
The algorithm used to perform localization is not described. Instead, it should be discussed.
The relevance of Fig. 3 is not clear. Please explain.
Section 3 only contains five lines of text. Results definetely need to be commented, even though related data are shown in tables.
Authors state that: “There is limited comparability with the accuracy and reliability studies of other sport related positioning techniques.” Why is this the case?”
Minor: English needs many improvements. In the last few lines only a few number of suggestions is given.
Sentence starting at line 58 needs to be rewritten: it is too long and it contains errors.
Line 64: was development —> was developed; The aim of this study IS (not was)…
There must be an error in the height of people involved in the experiments (+5.56m)
Line 77: Two intertial deviceS WERE
Line 88: allows drawING
