Data-Driven Knowledge-Based System for Self-Measuring Activities of Daily Living in IoT-Based Test

Abstract

This paper proposes a data-driven knowledge-based system with which aged people can measure the degree of activities of daily living (ADL) by themselves. The proposed system, called E-coach for ADL Test (EAT), provides participants with self-measurement procedures, using e-coaching, which is a guidance mechanism to lead the participants from an initial stage to a target goal. The EAT traces the behavior of the participants to gather ADL data that tell how well they perform the given e-coaching. Driven by the Internet of Things data, the knowledge-based inference of the EAT carries out the e-coaching mechanism that figures out what state the self-measurement procedures stay on and what guidance is necessary for the next state. The EAT ensures that all the procedures for ADL measurement are executed automatically without any help from medical professionals. The experiment described in this paper demonstrates that the EAT distinguishes between dementia patients and normal people. The measurement report assists medical doctors in the diagnosis of certain medical conditions that these people may have.

1. Introduction

Activities of daily living (ADL) consist of basic actions that involve caring for one’s self and body, including grooming/personal hygiene, dressing, toileting/continence, transferring/ambulating, and eating [1]. An ADL assessment is useful in monitoring chronic brain disease progression, such as dementia, and for identifying care needs and risks [2]. A doctor completely depends on an interview with aged people and their family members to diagnose the degree of their ADL. However, the interview relies on the subjective statements of these people. To accurately measure their ADL, the doctor should live with the aged people and observe them. However, this is realistically not possible.

At present, the medical opinion is that the fundamental treatment of dementia is very difficult [3]. Thus, doctors are treating dementia patients with the goal of living with their families in the home environment for as long as possible and safely with their families [4,5]. They are interested in how much the prescriptions they provide improve the ADL of people with dementia.

Doctors use expensive medical equipment to monitor the beta amyloid plaque and tau protein tangles in brain cells, to check the condition of the patients with dementia [6,7]. However, these medical tests are expensive and cannot be performed frequently. In addition, the protein level obtained through this medical examination has a limit to infer the daily living ability of the dementia patients [8].

We collaborated with medical professionals working for a general hospital, to find a solution to supplement the ADL data obtained from the interviews and expensive medical examination. They concluded that it was necessary to measure the ADL of aged people even in a hospital and designed a study to measure the degree of ADL in the hospital environment. The study did not utilize digital automation equipment. In the study, a medical professional verbally presented a task to the participants and then monitored whether the participants performed the given task correctly according to specific criteria. While simulating the experiment in the hospital, we recognized that the participants might feel burdened with the fact that they were the subject of observation. The medical professional might affect the participant’s behavior unconsciously, as well. Thus, an ADL measurement study requires an unmanned system that the participants can use to self-measure the degree of ADL without being disturbed.

The system, called E-coach for ADL Test (EAT), proposed in this paper enables the participants to self-measure the degree of ADL, without the involvement of medical professionals. The e-coaching is tailored coaching to help shift an individual’s behavior. The EAT provides the participants with e-coaching in the measurement procedures. While the participants perform the given e-coaching, the EAT gathers all the Internet of Things (IoT) data related to the degree of the participants’ ADL. Driven by the IoT data, the knowledge-based inference of the EAT executes the e-coaching mechanism that figures out what state the self-measurement procedures stay on and what guidance is necessary for the next state. It generates a report that describes the degree of the participants’ ADL at the end of the procedures. Using the EAT, the participants easily measure the degree of their ADL without any help from medical professionals, and the generated report on the ADL assists doctors in the diagnosis of certain medical conditions that these people may have.

The EAT can execute e-coaching for dementia care even in a smart home, as the EAT has a general-purpose architecture for e-coaching. The inference module of the EAT detects the abnormal behaviors of patients with dementia by comparing the patients’ current state with the expected state of the e-coaching plan and provides e-coaching for improving the abnormal behaviors. If the e-coaching is repeatedly provided to the dementia patients for a specific period of time, their ADL can be improved. The EAT also dynamically changes the e-coaching tactics as the patients’ health conditions change. In other words, the EAT acts as a digital drug.

Those involved in developing and operating the EAT should keep in mind that the EAT is not a patient control, but rather an auxiliary means for healthcare. From the design phase of the EAT, engineers must develop the EAT to provide e-coaching, while taking into account patient emotional aspects. Moreover, collected patient data should not be illegally accessed by others without the consent of the patients.

This paper proposes the EAT, which is a data-driven knowledge-based system with which aged people can measure the degree of activities of daily living (ADL) by themselves. The primary evaluation of the EAT is the patient's ADL measurement through the automated measurement process. The EAT can be used in hospitals and homes. Although this EAT needs to be evaluated in hospitals and homes, the experiments were conducted by first applying it to the hospital, in consideration of realistic situations.

The rest of this paper is organized as follows. The second section introduces the related works. The third section presents a design consideration of the EAT that this paper proposes. A knowledge-based implementation of the EAT is described in the fourth section. The fifth section presents the experimental evaluation. Finally, the sixth section concludes with comments about the contribution of the EAT system and a discussion of future work.

2. Related Research

Cognitive impairment due to Alzheimer’s disease and other forms of dementia affect a patient’s ability to maintain ADL [9]. This has severe implications on the patient’s independence and quality of life [10]. Current scientific evidence indicates that evaluation by ADL measurement helps to identify problems that require treatment or care [11]. For this reason, various studies on ADL measurement have been conducted. Research related to gait ability suggests a variety of measures to score physical function and basic daily living possibilities of the elderly [12]. In order to determine the possibility of independent daily living for the elderly, instrumental ADL for measuring the ability to use tools, appliances, and equipment is also a major issue [13]. Currently, hospitals measure patients’ ADL through complex interviews of elderly people, their relatives, and caregivers. The ADL measured through memory is prone to distortion and is less accurate. In order to accurately measure the patient's ADL, it is necessary to measure the ADL in daily life for a long time [14]. The use of IoT Sensor can be an alternative method for practical ADL measurement [15]. IoT technology enables long-term data collection by installing various sensors in the house [16]. Fukui et al. proposed a promising tool for the daily assessment of walking ability in a home with an easy-to-install distance sensor array [17]. The test bed of this paper collects basic walking ability and instrumental activities of daily living function through an IoT Sensor Network. In addition, we consider helping the patient's daily activity through the EAT system.

The existing studies have attempted to monitor the health of people by using IoT or fog computing. A system called Technology Integrated Health Management was developed to use sensory devices at home to remotely monitor the health of people with dementia [18]. Laplante et al. proposed a structured approach for healthcare systems based on IoT [19]. They illustrated the approach for medical-use cases such as alcoholism, Alzheimer’s disease (AD), and staff/patient safety-use cases. Healthcare systems based on fog computing improve the accuracy, data privacy, communication cost, and response time of the IoT systems. Takayanagi et al. continuously measured 4072 participants’ gait speed at 75 designated places, using fog computing, and found that daily gait speed and in-laboratory gait speed were inconsistent [14]. Fog computing can be designed to control access permissions for non-intrusive sensor data and to protect personal data [15]. Ghayvat et al. proposed a smart-home-based ambient assisted-living framework [16]. The framework has a preprocessing fog layer to acquire real-time data from a heterogeneous sensing system.

Machine learning and rule-based inference technique are used to recognize the changes in a person’s health. The machine learning mainly uses the Bayesian method and statistical models, for example, the probabilistic latent semantic analysis (PLSA) model. The rule-based inference supports a deterministic estimation approach.

As machine learning cases, Martinez-Hernandez et al. used an adaptive Bayesian inference system to predict walking activities and gait events with wearable sensors [20]. The system includes an adaptive action–perception method for the prediction. The experiments show that mean decision times of 240 and 40 ms are needed to achieve mean accuracies of 99.87% and 99.82% for recognition of walking activities and gait events, respectively. Seixas et al. proposed a Bayesian network decision model to diagnose dementia, AD, and mild cognitive impairment (MCI) [21]. The proposed decision model can be used to build clinical decision support systems to diagnose such diseases. For the Consortium to Establish a Registry for Alzheimer’s Disease dataset, the best accuracies of the proposed system are obtained by the multilayer perceptron artificial neural network classifier (84% and 71% for dementia and AD, respectively). Ihianle et al. studied ADL recognition from sensor data using the PLSA topic model [22]. The PLSA was used to discover activity topics and activity–object patterns. The mean accuracy of the proposed algorithm is 72.5%.

Rule-based inference is also used to care for personal health [23]. Urwyler et al. tried to recognize ADL by using a rule-based forward chaining inference engine classifier [24]. They developed two ad hoc ADL classifiers, namely the rule-based forward chaining inference engine (RBI) classifier and the circadian activity rhythm (CAR) classifier. The RBI classifier finds facts in data and matches them against the rules. The CAR classifier works within a framework to automatically rate routine activities to detect regular repeating patterns of behavior. The RBI and CAR classifier recognized activities with an average sensitivity of 91.27 and 94.36%, respectively. Stucki et al. developed a non-intrusive wireless sensor system with a rule-based forward chaining inference engine classifier [25]. The system acquires environmental data essential for the classification of activities of daily living. By analyzing retrieved the data, it is possible to distinguish and assign data patterns to subjects' specific activities and to identify eight different ADLs. The experiments show an overall sensitivity of 91.27%. Anakal et al. studied a clinical decision support system (CDSS), using a rule-based inference [26]. The CDSS was designed for the diagnosis of the various stages of the AD. Its classification accuracy is 93%.

The “C” Language Integrated Production System (CLIPS) was developed to model human knowledge or expertise at NASA’s Johnson Space Center, from 1985 to 1996. CLIPS is a rule-based inference engine useful for creating expert systems and other programs where a heuristic solution is easier to implement and maintain than an algorithmic solution [27]. Rules and objects form an integrated system, as rules can pattern-match on facts and objects.

E-coaching is an emerging computing area in which intelligent systems are used to encourage progress toward specific health-related goals by providing tailored guidance [28]. A mobile application called Lucy infers and counsels people regarding eating behaviors [29]. Pilloni et al. proposed an approach to monitor athletes’ behavioral changes to predict a decline in motivation [30]. This approach encourages these individuals to keep training through more personalized routines and effective strategies. Robillard et al. studied a framework to provide e-coaching that takes into account the feelings of people with dementia [31]. E-coaching systems based on emotion appraisal are more effective and acceptable to people.

Further, Mihailidis et al. reported that participants with moderate-level dementia could complete an average of 11% more handwashing steps independently and required 60% fewer interactions with a human caregiver when the e-coaching system was in use [32]. The existing studies on assistive technology for cognition have indicated that it can increase the efficiency of traditional rehabilitation practices by enhancing a person’s ability to engage in therapeutic tasks independently and by broadening the range of contexts in which these tasks can be exercised [33].

A loosely coupled architecture of an e-coaching system was proposed by Ochoa et al. [34]. The architecture recognizes the four major activities of the e-coaching processes: monitoring, decision-making, persuasion/awareness provision, and diagnosing/learning. Nag and Jain proposed the concept of personal health navigation (PHN) [35]. The purpose of PHN is to help an individual reach and maintain his/her desired health state. PHN starts with a user-specified health goal. From there, the system starts to collect measurements about the person’s health that relate to his/her goal to estimate the current health states. To reach the desired goal optimally, the system breaks the steps from the current state and provides the next step to an actuation mechanism. A change in the health state updates the next action to be executed. A cycle of these actions moves the user’s health state closer to the target health state.

3. Design Consideration of EAT

The EAT consists of IoT terminal devices, sensor gateways, and a supervisor device. The IoT terminal devices send sensor data about the behavior of participants to the sensor gateways. The supervisor device with the data-driven knowledge-based inference module deduces the context of the participants by using the sensor data relayed by the sensor gateways. After the deduction, the supervisor device provides the participants with e-coaching on the self-measurement procedures and then monitors how well the participants perform the given e-coaching. This section describes the design consideration to develop the EAT.

3.1. Service Scenario

This subsection describes an example of a service scenario that shows how the EAT works. An example that consists of two service scenarios is shown in Figure 1. The first is the scenario where the participant uses a toilet, and the second is a scenario where he uses the medicine box.

When the participant enters the starting room, the in/out detector perceives that the participant is in the starting room and sends the supervisor device a message conveying this information. Upon receiving this message, the supervisor device starts the e-coaching in the self-measurement procedures. The supervisor device delivers guidance on the first scenario to the sensor gateway in the starting room. The sensor gateway also has a speaker and delivers an audio message, providing guidance for the first scenario. For instance, the first scenario can be described as follows:

(1)

Go to the bathroom.

(2)

Open and close the toilet lid.

(3)

Flush the toilet.

(4)

Wash hands.

The behavior of the participant is recognized by the IoT terminal devices, which are the in/out detector, toilet, and the faucet. The terminal devices send the detected events to the supervisor through the sensor gateway. When the supervisor device judges that the participant has completed the first scenario or that the predefined time limit has expired, it activates the second scenario.

The supervisor device delivers the second scenario to the sensor gateway in the bathroom because the participant stays in the bathroom. This scenario can be described as follows:

(1)

Go back to the starting room.

(2)

Wait for an alarm for the pillbox.

(3)

Press the button of the pillbox after the alarm.

(4)

Pick up a medicine packet.

When the supervisor device judges that the participant has completed the second scenario or that the predefined time limit has expired, it delivers a measurement report to the doctors upon the participant’s approval.

3.2. E-Coaching Based on Closed-Loop Process

Self-measurement with the EAT requires a stable process to gather and analyze the data about the ADL of the participant. Without a stable process, the participant may get into a state of confusion during the measurement, or the EAT may gather only partial data related to the ADL of the participant.

The EAT supports a closed-loop e-coaching process to measure the ADL. The process consists of two phases, which are the e-coaching plan and e-coaching execution, as shown in Figure 2. The execution phase has four sub-phases, namely monitoring, evaluation, estimation, and actuation.

In the e-coaching plan phase, ADL experts define a goal that the participant has to achieve. The established goal can be decomposed into intermediate goals. The strategy includes a method to achieve each intermediate goal. If the participant executes the strategy, the participant achieves each goal, and the measurement process finally arrives at the final goal. The action refers to the guidance provided to the participant to perform a task, and the rule defines a relationship between the goal, strategy, and action. When the participant executes a strategy, the rule provides the next action.

After the plan phase, the knowledge-based inference module runs the e-coaching execution phase of the closed-loop e-coaching based on the plan. The actuation phase provides the participant with e-coaching actions on how to execute the strategy. The monitoring phase acquires sensor data in the format of the facts from which the knowledge-based inference module deduces the e-coaching actions performed by the participant. The evaluation phase examines an executed action to be detected in the monitoring phase and an expected method of the action given in the actuation phase. If he/she passes this examination, the participant achieves an intermediate goal. The estimation phase decides an e-coaching strategy to achieve the next goal. This loop runs until the estimation phase judges that the participant has arrived at the final goal state or cannot perform any other task, which implies that the participant fails all the procedures for self-measuring ADL.

3.3. State Machine for E-Coaching Plan

The e-coaching plan of the EAT is designed by using the Unified Modeling Language (UML) state machine. The knowledge-based inference module carries out the closed-loop process according to the e-coaching plan state machine. Each intermediate goal of the plan is a state of the state machine. The strategy is presented by a transition event within the state machine. The action of the plan is a transition action. The rule triggers a transition to define a relationship between the goal, strategy, and action. The fact that the participant executes a given action means that a transition event occurs. The rule examines a condition to trigger a transition. If the condition is satisfied, the rule fires the state transition. The participant achieves an intermediate goal and is given the next action by the rule.

A state machine of the e-coaching plan is shown in Figure 3. If the participant starts self-measuring ADL on the state of Ready, the actuation phase provides guidance of the Coach(“Enter_Bathroom”) action. The participant state is changed from the state of Ready to the state of Bath_Idle. An expected fact of the Coach(“Enter_Bathroom”) action is presented by <Enter_Bathroom>. The monitoring phase observes the IoT data in the format of the facts that the knowledge-based inference of the EAT can use. The evaluation phase examines whether the observed facts are equal to the expected fact. If the examination is passed, the transition from the state of Bath_Idle to the state of In_Bathroom is fired. During the firing of the transition, the inference module records the success of the participant executing the guidance of Coach(“Enter_Bathroom”). After the firing of the transition, the estimation phase decides an e-coaching strategy of Coach(“Wash_Hand”) to achieve the next goal of Wash_Hand.

If the timer expires three times on the state of Bath_Idle or Wash_Idle, the transition from the state of Bath_Idle or Wash_Idle to the composite state of Second_Scenario is fired. If the actuation phase provides guidance of the Coach(“Second_Scenario”) action on the state of Hand_Washed, the participant state is changed from the state of Hand_Washed to the state of Second_Scenario.

3.4. Architecture for Data-Driven Knowledge-Based Inference

The knowledge-based inference of the EAT is driven by the data that the IoT terminal devices produce. The mechanism of the data-driven knowledge-based inference is shown in Figure 4.

The IoT devices produce data that describe the participant’s behavior. All of the produced data imply the ADL information of the participant. They deliver the data to the EAT through the sensor gateways. The inference module of the EAT changes the data to a fact with a time stamp, under real-time constraints. If the data processing violates the real-time constraints, the inference module may understand the faulty context of the participant status and provide a wrong e-coaching action. The facts are stored in the real-time fact base. The rule base of the inference module has rules that decide the e-coaching actions. Experts in the ADL e-coaching domain define the rules. Rules whose antecedents match the specific facts are fired. The fired rules assert new facts and provide the e-coaching actions. The participant operates the IoT devices to perform the given e-coaching actions. The operations also produce data that imply the ADL information. The data are changed to a fact circularly.

The core of the EAT is the data-driven knowledge-based inference module. The supervisor device has the inference module inside. The software architecture of the supervisor is shown in Figure 5. The supervisor consists of modules, such as the e-coaching inference, external service, and CLIPS java native interface.

The e-coaching inference module deduces the e-coaching actions by using the IoT data and the inference engine of CLIPS. The rule base defines the inference rules. CLIPS activates the rules that match the IoT facts. The fact base stores the facts related to the IoT data. The terminal-device instance stores the last status of the terminal devices. The target-user instance contains the current status of the participant. The timer invocation requires the timer service of the external service to generate an event of time expiration. The TTS invocation calls for the TTS service of the external service to turn text into lifelike speech.

The fact injector external service injects facts into the fact base. The external service provides the e-coaching inference module with the timer and the TTS service. The sensor gateway manager contains the space information of the sensor gateway. This space information is used for the TTS service to send the synthesized speech to a specific sensor gateway that takes charge of the speech play. The inference instantiation initializes and starts the e-coaching inference. An ADL report is produced by the report generator. The personal health database stores the ADL reports. The inference update manager alters the rules of the rule base when the e-coaching application domain is changed. The CLIPS java native inference module is in charge of the message communication between the e-coaching inference and the external service.

The submodules of the rule base shown in Figure 5 are described in terms of their relationship with rule activation, as shown in Figure 6. When the participant executes the e-coaching action, the fact injector infuses the facts of the IoT terminal data into the fact base in real time. The monitor module in the e-coaching inference updates the last status of the terminal device and the target user instance by using the instance update rules that match the facts in the fact base. The target user is the participant.

The evaluation rules examine whether the expected fact is observed within a predefined time period on the current state of the e-coaching plan state machine. The expected fact is retrieved from the strategy. The observed fact refers to the terminal device and the target user instance. The evaluation rules call the timer invocation served by the timer service of the external service. After the predefined time duration, the timer service generates a time expiration fact that is infused into the fact base by the fact injector. If the evaluation result is true, a state transition of the e-coaching plan state machine occurs from the current state to the next intermediate state by using the goal and the strategy.

The estimation rules decide the e-coaching actions on the basis of the current state of the plan state machine. The e-coaching action is retrieved from the strategy and referred to by the expected fact of the evaluation rules. The estimation rules activate the actuation rules to provide the participant with the e-coaching action. The actuation rules invoke the TTS service of the external service. If the fact injector infuses a fact that the participant executes the given e-coaching action, the rule chaining is activated again. If the e-coaching plan state machine reaches the final goal state, the estimation rule activates the measurement analysis. The target user history rule records all of the rule activations. The measurement analysis rule calculates the degree of the participant ADL by using the target user records.

Medical systems that deal with patient data may be threatened with illegal data access. The EAT stores patient data on the supervisor device located in a local environment and does not store data on a public cloud system. The supervisor performs a data access permission control function to prevent illegal access to patient data from outside, without the consent of the patient.

4. Knowledge-Based Implementation

4.1. Transformation from State Machine to CLIP

E-coaching plans are designed by using the state machine of the UML 2.0 superstructure specification [36]. We chose the knowledge-based programming language of CLIPS to develop the e-coaching inference rules that execute the state machine. The rules reflect the concept of state-based programming by using the knowledge-based programming language.

A transition of UML is a directed relationship between a source state and a destination state, as shown in Figure 7. If an event occurs on the source state and the guard condition is satisfied, the transition is fired, and the exit activities of the source state, effect, and the entry activities of the destination state are executed in sequence. Finally, the current state of the e-coaching plan changes from the source state to the destination state.

We defined the rule convention that supports the state transition concept of the UML. The rule convention is shown in Figure 8. The lines labeled (1) are the events of the transition. The line labeled (2) matches the current state of the e-coaching plan state machine for the target user. Line (3) is for the guard condition of the transition. Line (4) finds the target user history module to record the state transition. If all the lines for the pattern match are satisfied, the transition is fired. Line (5) corresponds to the exit activities of the source state. Line (6) shows the effects of the transition. Line (7) corresponds to the entry activities of the destination state. Line (8) signifies the arrival at the destination state finally. Line (9) records this state transition in the target user history.

The e-coaching plan state machine that we implemented has four types of state transitions related to the e-coaching action provision, timer service request, target user movement, and terminal device operation. The rule convention is applied to implement these four types of state transitions.

The two transitions related to the e-coaching action provision and the timer service request are shown in Figure 9. The transition from the state Ready to the state Bath_Idle has no trigger but effects. As the destination state Bath_Idle has an outgoing transition with the timer expiration event, the state Bath_Idle implicitly has an entry activity to request the timer service to the external service module. The implementation of the transition from the state Ready to the state Bath_Idle is shown in Figure 10a. The line labels are the ones shown in Figure 8. The line labeled (7) denotes the entry activity of the state. The CLIPS code in Figure 10b shows the self-loop of the state Bath_Idle.

4.2. IoT Sensor Devices and User Activity Recognition

In this study, we developed the smart band and sensor boards for the terminal devices, including a faucet, toilet, pillbox, sphygmomanometer, coffee machine, microwave, gas oven, and an in/out detector. The sensor boards were attached to the terminal device. The smart band and some of the terminal devices are shown in Figure 11.

All of the smart band and sensor boards run on Ubinos [37], which is an operating system designed and implemented to fit the ultra-small and ultra-low-power devices of ubiquitous computing networks. The wearable smart band communicates with the terminal devices via Bluetooth and a low frequency (LF), to announce its presence [38]. The proximity-based neighbor identification protocol (PNIP) box also has the Bluetooth Low Energy (BLE) and LF communication function.

First of all, the PNIP box broadcasts a BLE advertising packet. The sensor gateway recognizes the PNIP box close to itself through the received signal strength indicator of the BLE advertising packet and registers it through the BLE connection. The Sensor Gateway delivers the current time information to the registered PNIP box. After that, the PNIP box stores the time information and delivers it, along with the user event results. Through this, the PNIP box stores user activity results with time information even if a network problem occurs. The PNIP box retransmits the stored result information when performing the reconnection procedure.

Next, to recognize user activity events, the PNIP box and smart band use low-frequency communication. The low-frequency communication has high-transmittance and low-power characteristics. For this reason, it is possible to accurately recognize the communication distance by using the received signal strength indicator (RSSI) of the LF signal. When a user activity event is recognized through the PNIP box, the LF signal is transmitted to the surrounding area through the LF transmitter of the PNIP box. The smart band receiving the LF signal converts the RSSI value of the LF signal into a distance value, and the smart band transmits the converted distance value to the surrounding PNIP box through the Bluetooth advertising packet. The PNIP box recognizes the distance value of the smart band through the received BLE advertising packet. Through this, the PNIP box collects the user ID of the nearest smart band and estimates it as the user of the activity event. Finally, the PNIP box sends the user ID and activity event result to the Sensor Gateway to which it is registered.

The PNIP box detects user activity events through various methods. The PNIP box basically has an external interface that can receive digital and analog signals. Equipment such as a coffee machine is connected to the external interface of the PNIP box through the button input of the coffee machine. Through this, the PNIP box receives the user button input of the coffee machine directly as an activity event. Like the in/out detector, user events can be recognized by the communication function of the PNIP box without additional connection. The in/out detector for detecting a user's movement event is installed on all doors, to detect the user's access [39]. The in/out detector was developed by a PNIP box with two LF transmitters. Each LF transmitter is installed inside and outside of a door and transmits the LF signal at a fixed interval. The smart band compares the RSSI of the two LF signals and recognizes the high side signal as its location. The smart band transmits this access information through BLE advertising packet. The PNIP box collects BLE advertising packets of this smart band. Through this, the PNIP box recognizes a change in the user's location information and sends an activity event to the sensor gateway. Finally, the user's water-use detection is recognized through the faucet-sensor module. The faucet-sensor module generates a user activity event by recognizing the movement of a faucet, using an acceleration sensor. The PNIP box is connected to the faucet sensor module by BLE and delivers the activity result to the sensor gateway.

4.3. IoT Sensor Network for EAT

The EAT consists of the supervisor, sensor gateways, and IoT terminal devices. The IoT device configuration for the EAT is shown in Figure 12. A type of terminal device, the smart band, contains the identity of the participant. When the participant with the smart band operates a terminal device, the smart band sends the identity of the participant to the terminal device. The terminal device uses the identity to detect its operator. The terminal device sends the sensor gateway its operation data with the participant identity. Diverse terminal devices may join the EAT according to the e-coaching application domain.

The sensor gateway relays the IoT data to the supervisor. The number of sensor gateways also increases depending on the system installation environment. The supervisor has the knowledge-based inference capability using the IoT data. The knowledge-based inference finds e-coaching actions that fit the participant’s context. The e-coaching action is given to the participant in the form of a speech. The supervisor uses Amazon Polly for the TTS service to synthesize speech that sounds like a human voice. Some software development kits (SDKs) for the TTS already exist, and most of these SDKs support English with high quality. The TTS quality of other languages is not sufficiently good to hear. We wanted to provide a TTS service that supports many languages of the world with high quality. Thus, we chose Amazon Polly, which is a service that turns text into lifelike speech and is realized on the Amazon platform [40].

The software of the supervisor and gateway was implemented on Jetson AGX Xavier of NVIDIA running Ubuntu Linux [41]. The message frame that each terminal device transmits is heterogeneous. The sensor gateway transforms the heterogeneous message frame of each terminal device into the uniform JavaScript Object Notation frame. The fact injector module of the supervisor can easily read the transformed message frame by using the uniform communication interface that is independent of the heterogeneous message frame of the terminal devices. The inference module of the supervisor is a CPU-consuming job. The uniform communication interface helps the supervisor save CPU resources, which is required to transform and read the heterogeneous message frames of each terminal device.

5. Experimental Evaluation

5.1. ADL Self-Measurement Experiment

The EAT conducted an ADL self-measurement experiment with 17 participants at a general hospital. This experiment was approved by the Institutional Review Board and was therefore performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments. Written informed consent was received from participants prior to inclusion in the experiment. The equipment included a supervisor, three sensor gateways, and ten types of terminal devices. The equipment was installed in three rooms.

Each of the 17 participants belonged to one of the following three groups: normal group, MCI group, and dementia group. The normal group had five normal people. The MCI group and the dementia group consisted of four people with MCI and eight people with dementia, respectively. The e-coaching plan for the experiment was that the participants moved between the three rooms and operated the seven terminal devices, namely the sphygmomanometer, water tap, pillbox, toilet, microwave, coffee machine, and oven. Each participant executed all the nine e-coaching actions given by the supervisor of the EAT. The supervisor measured the elapsed time that the participants spent to execute the e-coaching actions.

The failure and the mean time ratios of the e-coaching execution of the participants are shown in Figure 13. The failure ratio is the percentage of failures in the total number of the given e-coaching actions. The mean time ratio of movement is the proportion of time spent on moving between the three rooms to the total time allowed for movement. The mean time ratio of operation is the proportion of time spent on operating the terminal devices to the total time allowed for operation. A significant difference was not observed between the normal group and the MCI group. The dementia group had a significant difference in terms of the failure and the time spent on movement and operation.

The execution time of the nine e-coaching actions given to each participant was represented by a nine-dimensional vector. The distance between these vectors was used to compare the similarity of the participant characteristic. The distance heat map between the participant vectors is shown in Figure 14. The number on the horizontal and the vertical axes of the map is the participant number. Numbers 0 to 4 denote the normal group, numbers 5 to 8 represent the MCI group, and numbers 9 and 10 and 11 to 16 indicate the dementia-β group and the dementia-α group, respectively. The dementia-α and β groups are the subgroups of the dementia group. The map shows a symmetric color matrix. The similarity between the normal group and the MCI group was very high. The dementia-α subgroup of the dementia group was dissimilar to the normal and the MCI groups. However, the dementia-β subgroup was in the same dementia group as the dementia-α subgroup and appeared to be slightly biased to the normal and the MCI groups.

The geometric median of the set of the participant vectors in a Euclidean space was used to compare the similarities among the groups. The function dn(x) denotes the distance between the geometric median vectors of the normal group and the x group. The function dβ(x) calculates the distance between the geometric median vectors of the dementia-β group and the x group. The function σ(x) represents the standard deviation of the x group. The feature comparison among the groups is shown in

Table 1. Feature comparison among the groups.

x	Normal	MCI	Dementia	Dementia-β	Dementia-α
dn(x)	0	0.553475	2.183423	-	-
dβ(x)	1.166706	1.148768	-	0	1.279989
σ(x)	0.415242	0.41574	0.875101	-	-

. The value of dn(x) shows that the MCI group was more dissimilar to the dementia group than to the normal group. The value of dβ(x) denotes that the similarity of the dementia-β group was slightly more biased to the MCI group than to the dementia group. The value of σ(Dementia) is bigger than that of σ(Normal) and σ(MCI) because the dementia-β group was slightly more biased to the MCI group.

The k-means clustering method was used to find the number of clusters k of the participants. The change in the inertia value according to the number of clusters is shown in Figure 15. The inertia value is the sum of the distances between the center point and the data of each cluster after clustering. The inertia value is a value indicating the cohesion of clusters. The smaller inertia values mean better clustering.

The k-means clustering was performed on the 17 participant vectors with the k-values of 2, 3, and 4. The results of the k-means clustering are presented in

Table 2. K-means clustering results.

Participant Number	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16
Group	Normal	Normal	Normal	Normal	Normal	MCI	MCI	MCI	MCI	Dementia-β	Dementia-β	Dementia-α	Dementia-α	Dementia-α	Dementia-α	Dementia-α	Dementia-α
k = 2	A	A	A	A	A	A	A	A	A	A	A	B	B	B	B	B	B
k = 3	A	A	A	A	A	A	A	A	A	B	B	C	C	C	C	C	C
k = 4	A	A	A	A	A	A	A	A	A	B	B	C	D	D	C	C	D

. If each participant had the same cluster label, they belonged to the same cluster. When k = 2, two clusters were generated. The A cluster had the normal, MCI, and dementia-β participants. The B cluster had only the dementia-α participants. When k = 3, the dementia-β group formed a separated cluster with the normal group labeled A and the dementia-α group labeled C. When k = 4, the dementia-α group was divided into two clusters labeled C and D. These k-means clustering results implied that the ADL self-measurement experiment with the EAT could distinguish between the dementia group and the other groups when k = 3.

5.2. Response Time of EAT

The EAT should provide the target user with a correct e-coaching action within an appropriate time duration. To validate the time requirement, we measured the time performance of the EAT. The measurement was conducted 100 times. The average response time is shown in Figure 16.

After the target user operates a terminal device, the notification of the terminal device operation spends 1864 ms to be delivered to the supervisor through the sensor gateway. The supervisor needs 2 ms to infer an e-coaching action. The supervisor waits 511 ms until AWS Polly returns a synthesized audio file of the e-coaching action text. The EAT needs a total of 2377 ms to provide the target user with the correct e-coaching action. This means that the target user receives the next e-coaching action 2.4 s after operating the terminal devices. The EAT has the appropriate time performance for the ADL measurement.

6. Conclusions

In this paper, we proposed an IoT data-driven knowledge-based system for self-measuring the degree of ADL in a real hospital. The system, called EAT, provides participants with the e-coaching on self-measuring ADL. To develop the EAT, we designed a closed-loop e-coaching process, software architecture, state machine of the e-coaching plan, and inference rule convention. The closed-loop e-coaching process could provide tailored feedback depending on whether or not the participants executed the given e-coaching action. The software architecture of the supervisor device was designed to support the closed-loop e-coaching process. We proposed the formal representation of the e-coaching plan by using the UML state machine. The inference rule convention of CLIPS was proposed to support the state-based programming. Inference rules based on the rule convention were implemented to execute the state machine of the e-coaching plan.

We conducted the experiment with 17 participants, in a general hospital, by using the EAT. Although the experiment had the disadvantage of a small number of participants, it demonstrated a significant difference between the group of dementia participants and the other groups of normal and MCI participants. Because of the relatively small number of MCI participants, the difference between the normal and the MCI participants was not apparent. In this experiment, the EAT could classify participants with dementia and participants without dementia. Thus, the EAT can assist medical doctors in the diagnosis of certain medical conditions.

The EAT has a general-purpose mechanism and architecture for e-coaching. Diverse types of IoT devices can join the EAT. After defining an e-coaching plan for dementia care in a smart home, the EAT can detect the abnormal behaviors of people with dementia and provide the relevant e-coaching. If such e-coaching is repeatedly provided to people with dementia for a specific period of time, their ADL will improve.

In a hospital environment, the patient can perform the examination on their own, without the help of medical staff, so the EAT can serve as a medical-staff assistant. The EAT allows patients to continue to receive care in a home environment, without frequent doctor visits. The patient’s family and doctors can remotely monitor the patient's condition. Doctors can improve patient care strategies in a remote environment. Thus, the EAT provides personalized healthcare.

The EAT can be universally applied to patients in need of care in a home environment, e.g., autism patients. The EAT can also be extended to personal nutrition management and assistive systems for athletes. It helps balance nutrition by monitoring an individuals’ food intake and weight, and guides them through the necessary exercise. The EAT analyzes the training data of the athletes to motivate higher training performance, as well.

In the future, we intend to continue to experiment with the EAT in a general hospital, to collect big data related to ADL. We plan to develop an automatic generator for the CLIPS inference rule generator on the basis of the state machine of the e-coaching plan. Finally, we will attempt to enhance the EAT to maintain the personal health state of aged people in real homes. The enhanced EAT will be a digital drug expected to help with the early diagnosis and symptom improvement of dementia.