The Design of an Environmental-Noise-Labeling App for Citizen Participation in Smart Cities ^†

Abstract

The urban acoustic environment is composed of a great variety of sounds, whose effects citizens are mainly unaware of, despite medical studies showing that urban noise affects quality of life and health, causing various problems from sleep disturbances to cardiovascular diseases. The development and deployment of wireless acoustic sensor networks (WASNs) presents new ways to meet urban acoustic challenges in the context of a smart city environment. The improvement of the quality of life of the citizens cannot be limited to measuring the equivalent levels of noise in the streets but should also identify the type of noise source and its impact on the overall noise measurement. For this purpose, the collection of noise sources information requires the application of many techniques, including the recording and labeling of noise events. The latter is a task that is mostly performed manually by experts, using certain types of software, and it is very time-consuming. To improve this task and to take advantage of the rise of new technologies, we propose the design of a game using a mobile application to encourage citizens’ participation in research. The goals of the app are two-fold: to raise awareness of the problems generated by noise and to help experts in the work of pre-labeling sounds for later analysis. We detail the envisioned mechanics of the proposed game, its dynamics and its design.

1. Introduction

In our day-to-day life, we are surrounded by a great variety of sounds that we are able to recognize and discern automatically, without paying too much attention to them. This set of data is usually considered irrelevant. However, over the last few years, there has been an increasing interest in the information these sounds can convey, together with an increasing number of studies in the field. Different projects and tools have been created to collect and classify sound events for various purposes [1]. Examples are the study of the behavior patterns of society or cities based on the sound trends [2] or the identification of excess noise for the subsequent study of its impact and the design of improvements [3].

Today, the process of collecting information and subsequently classifying and labeling it is a costly and time-consuming task, because it is carried out manually by experts. This work can also be done in other ways, such as by automating the process with the help of artificial intelligence mechanisms. Nevertheless, depending on how it is done, our team has evidence [4,5] that different results in terms of accuracy are obtained [6], and usually the time exceeds that devoted to precise labelling. For this reason, the manual method is still commonly used as it is more precise [4] and does not require post-supervision by experts as automatic methods do. In this sense, the possibility of giving basic training to citizens in order to make use of their help is a real option, because with the support of technology, citizens can be key actors in distributing the process among more volunteers. Several projects take citizens into account in the process of science, but most do not allow them to set the ground truth of the measurements.

The present work aims to use gaming and new technologies to make the classification and labeling process more enjoyable, taking advantage of the ubiquity of these technologies in today’s society. An example of this is found in schools, secondary schools and universities, which use digital tools for many tasks in the teaching–learning process [7]. Likewise, gamification elements in digital and board games hook the user and create incentives for him or her to continue playing [8]. In addition, mobile phones provide permanent access to games and increase the potential number of players, which in turn can increase the amount of data collected.

2. State of the Art

In order to carry out good design and development of a system aimed at labeling sounds or audio recordings, different knowledge areas are required. These must be intertwined and complementary, so that the best possible experience and results are obtained.

2.1. Gamification

Gamification promotes the best user experience through the use of playful elements—techniques and game mechanics in a mobile application to engage, motivate and encourage its use [8].

When designing a game with gamification in mind, one of the most influential systems is the mechanics–dynamics–aesthetics infrastructure (MDA) [9], in which a framework is created to bridge the gap between game design and development, game criticism and technical research. Specifically, this infrastructure divides the set into rules, system and “fun”, establishing the concepts of:

• Mechanics: describing the game components, rules and concepts that specify the system.
• Dynamics: describing the behavior of the game’s run-time mechanics, which acts on the inputs of the players and the outputs of others over time.
• Aesthetics: describing the desirable emotional responses in the player when interacting with the game system, caused by the dynamics.

Importantly, from the designer’s perspective, mechanics gives rise to behavior in the system, which in turn leads to concrete aesthetic experiences. Likewise, from the player’s perspective, aesthetics sets the standard that arises from the observable dynamics and, finally, from operable mechanics.

2.2. Citizen Science

Citizen science is scientific research carried out with the participation of volunteer citizens, usually not specialists in the subject, who actively contribute to science via their intellectual efforts, tools and resources [10].

On the Observatory of Citizen Science platform in Spain, a compilation of such initiatives, resources and projects can be found. Among these, those that are most related to the topic of this paper are the following:

• Zaragoza Noise-Pollution Map Project [11]: a mobile application to document neighborhood noise. It allows the noise source to be labeled, together with its measurement. Citizens collect data to create an urban map of noise sources harmful to health and to provide the necessary information for administrations’ decision-making.
• Noise App [12]: a mobile application to record and report noise in the user’s area. This has the option of recording noise with the phone itself, in order to report it to a central office registered in the system. In addition, a personal recordings diary can be created in which any noise can be stored and labeled for later use.

2.3. Labeling Sounds

A fundamental part of theprocess of citizen science in this area of study is the proper labeling of sounds; a process that consists of indicating within a recording where a specific sound is located and how it is defined [3]. To carry out this task, first, the characteristics of the sounds that are sought must be precisely defined. The fact of using recordings obtained in a real environment implies that background noise will be present. Consequently, this makes processing difficult, since it is more complex to indicate the beginning and end of the sound or to differentiate the sound event from the ambient sound.

After this definition process, the labeling process is carried out by comparing and analyzing sonograms and $L{}_{A}eq$. The classic method followed by experts is as follows [3]:

• Check the trend changes in the spectrum. For this, different techniques can be used, such as looking for its maxima and analyzing its surroundings. Alternatively, consider a threshold level and look for changes that are above this value.
• Check for level trends in specific noise spectrum bands.
• Analyze at frequencies that are typically found to be abnormal.

Once the sound has been identified, it is labeled with the corresponding word, and the beginning and end of the recording are signaled.

3. Requirements and Design

With the defined objective, the project focuses on the development of a citizen science mobile application to pre-label sound events in a recording. It was decided to use gamification in order to obtain the maximum number of tagged audio files and thus make the project motivating and engaging for users. In addition, the MDA gamification system explained above was applied, as detailed in the following sections.

3.1. Mechanics

For the mechanics, points, ranking and levels were used. The ranking aims to promote positive competitiveness, ordering users according to the points obtained, regardless of level. There are three levels: low, medium and high. All users start at the lowest level and, as the levels increase, the complexity of labeling is increased, with a greater number of sounds present. This is the only difference between the levels. To advance to the next level, the user needs to collect enough points. The application awards points by comparing the labels entered by the user with those previously registered in the database, in order to know how often the sound has been entered, considering whether the beginnings and ends coincide. Coincidence rather than innovation is rewarded, in order to obtain a database of labels concentrating on the terms offered. For this reason, the more times that word has been tagged, the more points the player obtains. The points are awarded as follows:

• Not entered: 0 points
• Entered more than four times: 4 points

This score corresponds to the low level, and is multiplied by 2 at the medium level and by 3 at the high level, as shown in

Table 1. Points obtained according to level.

Level	Never Entered	Entered More Than 4 Times
Low	0 point	4 points
Medium	0 point	8 points
High	0 point	12 points

.

Once the distribution of points is known, the user must obtain 20 points to advance from the low to the medium level and 40 more points to advance from the medium to the high level. It is also worth mentioning that there is no maximum number of points, since the more recordings that are tagged, and the higher the level, the greater the number of points that can be achieved.

Finally, it should be noted that it is important to know the ability of the user to classify and label noise events, both in order to provide a good experience and also to define the level of reliability in the data labeling generated by that user.

3.2. Dynamics

Regarding the dynamics, the functioning that the application follows is to first show a start screen (Figure 1a), where the game begins (Figure 1b) or the frequently-asked-questions area can be accessed (Figure 1c). If the user chooses to play, and it is the first time he or she has accessed the application, a tutorial is shown (Figure 1d) in order to teach the basics of performing good labeling. The tutorial consists of an initial video and later a screen for the user to carry out an example in a practical way and thus verify that they have achieved a suitable level of knowledge.

Once the tutorial is finished, the game starts. It consists of a screen (Figure 1b) where a recording is played. To tag it, a label must be chosen in the upper part from among those shown, and if the sound does not match any of them, there is the possibility of adding a label by entering a name and a description. It is important to prioritize choosing from the available options whenever possible, before introducing new labels. Furthermore, another screen can be opened to see details of the recording (Figure 2a). Likewise, in the lower area, the start and end of the labeled sound should be signaled, adjusting them as precisely as possible. Once both tasks are finished, the label must be saved in order to continue with the recording or, if everything has been labeled, the game is finished and all the information sent to be saved on a central server via the user’s own mobile connection. Once the labels have been submitted, the score obtained is displayed (Figure 2b), followed by a screen showing the user’s profile (Figure 2c), detailing his or her total score, current level, number of tagged audio files with respect to the total and position in the ranking.

Once a recording has been tagged ten times in the same way, it is assumed that the audio file is tagged correctly, and it is no longer displayed in the application for tagging. The decision is compared with that of an expert in sound labeling to decide the range in which this labeling can be considered good.

3.3. Aesthetics

Regarding aesthetics, when making a level change, an informational message is displayed and a different color pattern from the previous one is used.

4. Related Work

There are various different systems and applications related to the labeling of sounds. The most relevant ones are described below:

• MajorMiner [13]: a web application for tagging entire audio files. The process consists of playing offline “against” a database, so that the user tags an audio file and obtains a score according to the number of people who have previously tagged the same audio file in the same way. After each game or audio recording, the application shows a summary of the tags entered by the player, those entered by other people, the score obtained in that round and the total number of points.
• TagATune [14]: an application to tag entire audio recordings in a limited time. This is a game to be played in pairs, or with a bot if no one is online at the same time. The user must describe the sound according to the category indicated, during the set time. Once the round is over, either because the set time has elapsed or because both players have decided to finish the round, the points are obtained according to the matches between the partners. Based on the descriptions received by users, the tag is considered official when a minimum number of people have obtained matched results, which depends on the total number of participants.
• The Listen Game [15]: an application to tag entire audio recordings, playing simultaneously against multiple players. The process begins with all players listening to the audio recording, selecting from the labels shown on the screen whether they are “good” or “bad” sounds and obtaining feedback on their selection compared with the other players, adding points according to the coincidences between them. Firstly, seven rounds are played, followed by a freestyle round where the player enters the word that best matches the sound heard. In the next round, this word appears as an option among those chosen by the system. Finally, every eight rounds a score summary and various statistics are shown.

After researching this field, we observed that all these applications search for techniques to attract the attention of and engage users in a way that is fun, thus generating constancy. For this purpose, most involve competitions or comparisons with other users, with scoring according to the coincidences between them. Likewise, the way to corroborate good labeling is to observe coincidences in labeling between users, so that the labeling can be considered correct in a certain range.

5. Conclusions and Future Work

This paper presented the design of a game using a mobile application to encourage citizens’ participation in research, raising awareness of the problems generated by noise and helping experts in the work of pre-labeling sounds for later analysis. The most important aspects to consider when implementing an application of this type were detailed, such as gamification, citizen science and sound tagging, which all combined generate an attractive prototype for users to perform the task of pre-labeling in a more enjoyable way.

Moreover, we developed a first prototype and tested it with ten researchers from La Salle University. In the test, the criteria assessed were installation, labels, navigation, tutorial, frequently asked questions, operation, design and general aspects. The main points arising during the test were that half of the users did not understand the sounds correctly, some felt that the labels shown were a little confusing and did not always fit with the sounds and most felt that the app was easy to use and useful. Finally, the global valuation of the app was rated as good. From this, we detected the need for some improvements, such as improving the feedback received by the user during the tutorial, reconsidering the labels displayed for labeling and improving the audio controller operation.

In future work, appropriate corrections will be made to the proposed design and implementation to improve the functioning of the app, the app will be tested with users and long-term labeling will be continued.