Next Article in Journal
WE3DS: An RGB-D Image Dataset for Semantic Segmentation in Agriculture
Previous Article in Journal
Modeling Topics in DFA-Based Lemmatized Gujarati Text
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sensors
Volume 23
Issue 5
10.3390/s23052709
sensors-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

The Design and Development of Instrumented Toys for the Assessment of Infant Cognitive Flexibility

by
Vishal Ramanathan
1,
Mohammad Zaidi Ariffin
1,
Guo Dong Goh
2,
Guo Liang Goh
3,
Mohammad Adhimas Rikat
3,
Xing Xi Tan
4,
Wai Yee Yeong
2,
Juan-Pablo Ortega
5,
Victoria Leong
4,6 and
Domenico Campolo
1,*
1
Robotics Research Center, School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798, Singapore
2
Singapore Centre for 3D Printing, School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798, Singapore
3
School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798, Singapore
4
Division of Psychology, School of Social Sciences, Nanyang Technological University, Singapore 639798, Singapore
5
Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore 637371, Singapore
6
Department of Pediatrics, University of Cambridge, Cambridge CB2 1TN, UK
*
Author to whom correspondence should be addressed.
Sensors 2023, 23(5), 2709; https://doi.org/10.3390/s23052709
Submission received: 27 December 2022 / Revised: 14 February 2023 / Accepted: 23 February 2023 / Published: 1 March 2023
(This article belongs to the Section Wearables)
Browse Figures
Review Reports Versions Notes

Abstract

:
The first years of an infant’s life represent a sensitive period for neurodevelopment where one can see the emergence of nascent forms of executive function (EF), which are required to support complex cognition. Few tests exist for measuring EF during infancy, and the available tests require painstaking manual coding of infant behaviour. In modern clinical and research practice, human coders collect data on EF performance by manually labelling video recordings of infant behaviour during toy or social interaction. Besides being extremely time-consuming, video annotation is known to be rater-dependent and subjective. To address these issues, starting from existing cognitive flexibility research protocols, we developed a set of instrumented toys to serve as a new type of task instrumentation and data collection tool suitable for infant use. A commercially available device comprising a barometer and an inertial measurement unit (IMU) embedded in a 3D-printed lattice structure was used to detect when and how the infant interacts with the toy. The data collected using the instrumented toys provided a rich dataset that described the sequence of toy interaction and individual toy interaction patterns, from which EF-relevant aspects of infant cognition can be inferred. Such a tool could provide an objective, reliable, and scalable method of collecting early developmental data in socially interactive contexts.

1. Introduction

Executive functions (EFs) are higher-order cognitive control mechanisms that are commonly conceptualised as a triad of mental skills comprising inhibitory control, working memory, and cognitive flexibility [1]. These mental abilities support complex thinking skills, such as reasoning and creative problem-solving [2], and influence the development of socioemotional competencies, such as the Theory of Mind [3]. They are considered essential for mental and physical health [4,5,6].
Cognitive flexibility refers to switching between tasks, rules or dimensions and adapting one’s behaviour to a changing environment [7,8]. Categorisation tasks test an infant’s ability to flexibly categorise objects based on different dimensional features (or attentional sets), such as shape versus material [9,10]. An infant’s mental categorisation of objects can be inferred from a behavioural measure of sequential touching. If infants sequentially touch objects from the same category (e.g., balls) more often than expected by chance, it is inferred that they are doing so because they perceive these objects to belong to the same category [11,12,13]. Horst and colleagues (2009) [14] found that 14–18-month-old infants could flexibly adapt their categorisation of objects by either a perceptually salient dimension of taxonomic distinction (e.g., shape) or a less salient dimension (e.g., deformability). An example study protocol for this task is available at this URL: https://psyarxiv.com/dejns.
Changes in the pattern of an infant’s object touch sequences can index mental set-shifting, i.e., shifts in the mental dimensional set that infants use for categorisation, such as shape or compressibility. One common performance measure is mean run length (MRL), where run lengths are the number of touches in a row to objects from the same category (i.e., shape). MRLs are calculated by dividing the total number of touches by the total number of runs across all categories [15]. The calculated MRLs are then compared against a Monte Carlo simulation’s average “random” sequence lengths to assess performance against chance [14]. In addition to these classic measures, newer indices based on the conditional probability of infant touch sequences may provide further insight into the cognitive strategies adopted by infants during this task.
However, to detect sequences of object touches made by infants, these events must first be manually extracted and coded from video footage by a trained human experimenter. Furthermore, it is possible to measure the effect of maternal scaffolding on infant mental set-shifting via the introduction of a brief period of maternal social interaction during which the mother demonstrates object compressibility to the infant. The quantification of maternal behaviour during this period of social interaction similarly relies on the hand-coding of relevant behaviour.
Accordingly, this paper presents the design and validation of a set of instrumented toys as an objective, reliable, and scalable form of cognitive flexibility task instrumentation and a behaviour measurement tool that can complement and accelerate manual means of data coding (Figure 1). It can extract typically coded measures, such as when a toy is touched, how long the infant interacts with it, and the overall touching sequence. We can additionally measure squeezing patterns that the infants exhibit to validate mental set-shifting from shape-based to material-based categorisation. Such instrumented toys may, in the future, inform the development of scalable quantitative measurement tools and screening methods for the assessment of the early development of executive function skills in young children.

2. Functional, Technical, and Physical Specifications

2.1. Requirements and Existing Setup

To evaluate mental set-shifting in infants during the object categorisation task, the number and sequence of touches of each object by the infant need to be identified. The current methodology relies on human coders manually extracting these touch sequences by watching video recordings of infants playing with toys. The set of eight toys comprises two hard balls (79 g), two soft balls (23 g), two hard cubes (80 g), and two soft cubes (5 g) (Figure 2). All the cubes have sides measuring 50 mm, and all the spheres have diameters measuring 55 mm and are all different colours. A latent factor currently not measured, which could be useful in reinforcing the identification of mental set-shifting in infants based on the less salient dimension of deformability, would be the squeezing patterns of the infant’s grasp on the toys.

2.2. Detecting Toy Interaction

To scale up the object categorisation task for lab-based and ecological environments, we need to automatically detect when and how an infant interacts with the toys. Motion tracking enables the desired automatic extraction of interaction and motion patterns. This can be implemented using various technological solutions, as shown in [16]; however, not all solutions are suitable for use in ecological environments. We also need to keep costs low and ensure the system is easy to set up and manage. Based on technological assessments to identify suitable techniques for motion tracking, optical and inertial sensing seem to be the most favourable and widely used [17,18].
Optical markerless methods can be used on recorded videos that are part of the existing object categorisation task paradigm. Combining human pose detection, object detection, and multi-object tracking algorithms, key points can be detected and tracked throughout the video (Figure 2) [18,19,20]. However, the accuracy of the underlying detection algorithms determines the performance, which suffers from the issue of occlusions. It also requires a structured environment restricted to the area within the camera’s field of view.
Alternatively, inertial measurement unit (IMU)-based motion and orientation tracking eliminate line of sight problems and structured environment requirements, making it more appealing from an ecological perspective. These methods are a compact, low-cost, and robust way to detect the motion and orientation of objects. They have been extensively used in lab-based and ecological studies of infant motor development [21,22,23] and are the ideal method to detect motion and interaction for our application.

2.3. Detecting Toy Squeezing

To detect the squeezing of the toys, we need to detect the forces applied to them using force, pressure, or tactile sensors. Several different working principles for such sensors were explored by [24,25,26] and have been used in previous infant-related research. A sensorised ball designed by Campolo et al. [27] used quantum tunnelling composites (QTC), which change its electrical resistance based on changes in applied force [28] in order to detect grasping patterns during manipulation. Cecchi et al. [29] incorporated piezoresistive pressure sensors and flexible force sensing resistor (FSR) sensors in sensorised toys to measure infants’ reaching and grasping. Serio et al. [30] used pressure sensors connected to air chambers to measure the amplitude of the force applied for quantitative monitoring and measuring infants’ motor development.
Tenzer and Jentfot [31,32] developed a versatile, low-cost, and sensitive tactile sensor using commercial off-the-shelf MEMS barometers, and they commercialized it as TakkTile [33]. It has been used in robotics by Ades et al. [34] and Koiva et al. [35] to sense grasping events using robotic grippers. The working principle of the MEMS barometer-based tactile sensor is the communication of surface contact pressure within a layer of rubber to the ventilation hole of the sensor and, thus, to the MEMS transducer. Similarly, Takada et al. [36] and Quinn et al. [37] have used a waterproof mobile device’s built-in barometer to measure touch force, which works on a similar principle. When an airtight or waterproof device is touched, the distorted surface changes the air pressure inside that device and, thus, changes the built-in barometer value [36,37]. This ability to detect forces through changes in internal pressure makes barometric tactile sensing suit our requirements for a low-cost, versatile, and sensitive way to detect squeezing forces on the toys.

2.4. Proposed Platform

For the object categorisation task, balls and cubes help assess an infant’s cognitive flexibility. The selected sensors need to be integrated into a platform that can be used with minimal alterations to existing paradigms. We proposed using instrumented toys to measure the development of EF in infants in a scalable manner in lab-based and ecological environments.
Campolo et al. [27] designed a sensorised ball to analyse the development of perceptual and motor skills in ecological environments. IMUs have been embedded in toys to assess spatial cognition [21,22,38] and detect possible autism spectrum disorders (ASD) at an early stage [23]. Pressure and force sensors were used to study infants’ grasping actions [29,30,39]. A whole suite of instrumented toys was developed to provide early intervention for infants at risk for neurodevelopmental disorders and reduce parental stress [40,41].
These examples demonstrate the viability of instrumented toys for integrating sensors to assess infant development. However, instrumented toys targeted explicitly at measuring the development of EF in infants are yet to be developed, and a need exists for such tools.

3. Instrumented Toy Design and Fabrication

3.1. Sensor Core

Using commercially available sensors, particularly in infant behavioural research tool development, has the advantage of being certificated for public use while ensuring high quality and safety standards are met. These certifications reduce the potential risk of harm when using instrumented toys that contain sensors and batteries in particular. It allows us to leverage existing expertise in sensor development while focusing on designing, developing, and deploying instrumented toys at scale without compromising accuracy.
We used Physilog 6® (P6), a commercial off-the-shelf sixth-generation wearable motion sensor platform produced by MindMaze Assessments (Figure 3). It comprises an inertial measurement unit (IMU) with nine degrees of freedom (DoF) and a barometer that is typically used for human motion and gait analysis [42,43]. Its versatility allowed us to capture, measure, and analyse the necessary parameters for detecting the touching and squeezing of the toys during the object categorisation task.
Each instrumented toy had a P6 sensor embedded into it to detect the touching and squeezing of the toys. The specifications of the P6 sensor are summarised in the user manual document provided by the manufacturer [44]. The IMU and barometer were configured to sample at 64 Hz for easy data synchronisation and maximising battery life while striking a suitable temporal resolution. The P6 sensor has an internal clock that can be synchronised to a PC’s clock, which allows for easy synchronisation with other sensors, opening up the possibility of multi-modal data analysis for studying EF development. Multiple P6 sensors can communicate and synchronise with each other using the built-in BLE communication protocol. One sensor acts as the master, wirelessly broadcasting the clock signal on a particular channel, and the others act as the client listening to this signal. Data stored onboard as “.BIN” files are sorted into timestamped folders accessed by connecting the sensor to a PC via the USB-C interface and downloading it.
Dragon Skin™ 20, a high-performance platinum-cured liquid silicone compound, was used to seal the hole of the P6 sensor’s barometer to use it as a tactile sensor. The silicone will transduce the squeezing forces on the surface to changes in internal pressure that the barometer can detect. A custom 3D-printed mould (Figure 4a) was used to enclose the sensor and shape the silicone layer surrounding it. The silicone was cured overnight (Figure 4b,c) and unmoulded to create the final sensor core for the instrumented toys (Figure 4d). The manufacturer, Smooth-On, Inc., Macungie, PA, USA, provides a detailed material safety data sheet and skin-safe certification, making Dragon Skin™ 20 a suitable nontoxic silicone material for use in our application where infants might come in direct contact with the material [45,46].

3.2. Physical Structure

The physical structure of the instrumented toy encloses the sensors, creates the final shape, and controls the rigidity. The weight and dimensions of the physical structures of the instrumented toys were defined by the anthropometry of an infant’s hand, the existing regular toys used, and the physical dimensions of the sensor (44 mm × 36 mm × 17 mm) to be embedded. The cube has a length of 60 mm, and the sphere has a diameter of 65 mm.
Modern 3D printing technologies and advances in material science have enabled the fabrication of complex structures using materials of varying rigidity [47,48,49,50]. Ansys SpaceClaim’s Faceted Shell and Infill tool was used to design an octahedral lattice that minimises mass, ensures uniform stiffness, and avoids needing a support structure during fabrication. The lattice structure was designed with a wall thickness of 1 mm, a unit cell size of 10 mm, and a strut thickness of 1 mm, which translates to 11% infill density (Figure 5).
Fused deposition modelling (FDM) 3D printing using thermoplastic polyurethane (TPU) filament was used to fabricate the octahedral lattice physical structure (Figure 6). The hard toys were printed using a 95A shore hardness TPU filament, while the soft toys were printed using an 85A shore hardness TPU filament. The print parameters used to print the lattice structure were as shown in Table 1. The 3D-printed TPU parts are inert and nontoxic, making them suitable for use in our application where infants will come into direct contact with the material [51,52].

3.3. Integration

The instrumented toys were designed using a modular multilayered approach to decouple the sensing capabilities from the physical properties. The sensing core determines the modality of the data that can be captured, while the physical structure determines the real and perceived affordances of the toy by the infant. The risk of harm and injury to the infant is further minimised by placing the sensors and battery at the toy’s core. The physical structure, fabricated using nontoxic and noncombustible materials, acts as a barrier. This instrumented toy design paradigm can be expanded to utilise other sensors within the sensing core that can be enclosed by different physical structures based on the desired play and interaction style. For the set of instrumented toys to be used in the object categorisation task to assess cognitive flexibility in infants, the sensor core records motion and pressure data from which we infer the toys’ sequence of touching and squeezing. The 3D-printed lattice structure defines the toy’s affordance by varying the ball and cubes’ colour, size, and rigidity.

4. Squeezing Detection

4.1. Experimental Setup

A preliminary quantitative experiment to verify if the barometer of the P6 sensor embedded in each toy could detect squeezing was performed using a Kinova Gen3 7 DoF robotic arm (Figure 7) to repeatedly and consistently squeeze each toy ten times. An ATI Industrial Automation Net Force/Torque Sensor Mini40 measured the reaction forces ( F s e n s e d ). A 3D-printed holder was fastened to the robot end effector and force sensor to hold the toys in place and control the toy’s contact area. The cubes had a contact area of 72.00 cm 2 , and the balls had a contact area of 64.84 cm 2 between the robot end effector and the force sensor. It was noted that studies on infant grip force within the first 12 months had measured the range to be between 5 to 35 kPa [29,30,39,40,53]. As such, to validate performance in a minimal force application condition, the robot was programmed in position control mode to move 10 mm vertically from z 0 to z 1 with a velocity of 65 mm/s to apply a force F r o b o t = 20 N on the toys. The expected pressure applied on the cubes was 2.77 kPa and on the balls it was 3.08 kPa, well under the typical grip strength range. Before beginning the experiment, each object was placed on the force sensor and its weight was zeroed out. Data from the robot and the force sensor were timestamped and logged to a PC. The internal time of the P6 sensor was synchronised with the PC clock.

4.2. Results and Discussion

The robot consistently moved 10 mm to squeeze the toys 10 times with a regular force of F r o b o t (Figure 8). The compliance of the instrumented toy’s structure produced a difference between the force applied by the robot and the force measured by the force sensor (Table 2). The soft cube absorbed 12%, the hard cube absorbed 10%, the soft ball absorbed 18%, and the hard ball absorbed 14% of the applied force. Based on the force applied by the robot and the area of contact of the 3D-printed holders, we estimated the pressure applied to each toy when squeezing. As expected, the pressure exerted ranges from 2.77 to 3.39 kPa. This did not saturate the barometer as its specified dynamic range is 100 kPa (26 to 260 kPa).
In Table 2, we can see the summary of the barometer readings across 10 squeezes for all the different toys. Noise in the barometer signal is smoothed out using a 5 Hz low-pass filter. The squeezing force applied to the toys produced a change in internal pressure that was successfully recorded by the barometer (Figure 8). The soft toys recorded a smaller change in pressure than the hard ones due to the compliance of the physical structure absorbing some of the force applied.
A drift in the baseline pressure reading of the barometer in the soft cube (Figure 8c) was observed. A drift of approximately ±20 Pa was present across all sensors either due to air leaking or getting trapped within the 3D-printed structure or the sensor core through the USB-C connector port. This drift was not of particular concern as the sensor could still consistently pick up on the dynamic changes in pressure caused by the actual squeezing of the toy. However, applying a 0.5 Hz high-pass filter helped filter out such a drift and the baseline offset of the ambient room pressure as well (Figure 9).
Furthermore, our barometric squeezing detection validation was based on 10% of a typical infant’s grip pressure. The softer toys detected approximately 1% of the applied pressure, and the hard toys detected approximately 11% of the applied pressure. Therefore, we can be confident that under more representative conditions, where infants may apply 5 to 35 kPa of grip pressure, our novel implementation of detecting squeezing using a barometric tactile sensor will be able to detect the squeezing of the toys.

5. Sequence of Touching

5.1. Experimental Setup

A preliminary quantitative experiment was performed using the instrumented toys to detect the sequence of touching. A participant was presented with 4 instrumented toys on a table and asked to touch and play with them (Figure 10). The IMU onboard the P6 sensor embedded in each toy recorded the motion, while a camera simultaneously filmed the toy interaction from an overhead angle to minimise occlusions. The 4 IMUs and the camera were synchronised using the timestamped data from both sensors.
The signals from the IMUs were processed as in Figure 11 to obtain the sequence of touching. First, the signal from the 3-channel accelerometer was combined by calculating the Euclidean norm ∥ a ∥ 2 = a x 2 + a y 2 + a z 2 . Then, the DC-offset and noise were filtered using a band-pass filter ( f L , 1 and f H ), and the signal was passed through a full-wave rectifier to obtain only the positive magnitude of the signal. Finally, the linear envelope was calculated using a low-pass filter ( f L , 2 ). Similarly, the signal from the 3-channel gyroscope was combined by calculating the Euclidean norm ∥ ω ∥ 2 = ω x 2 + ω y 2 + ω z 2 , and the linear envelope was calculated using a low-pass filter ( f L , 3 ).

5.2. Results and Discussion

From the video, an independent rater manually coded for the sequence of touches of each toy. These results were our ground truth data for evaluating the performance of the instrumented toys (Table 3). The soft and hard cubes were touched two times, and the soft and hard balls were touched three times.
For the accelerometer, the DC-offset and noise were filtered using f L , 1 = 0.5 Hz and f H = 10 Hz band-pass filters. The linear envelope for the accelerometer and gyroscope were calculated using f L , 2 = 0.3 Hz and f L , 3 = 0.2 Hz low-pass filters, respectively. Threshold values of 0.046 g for the accelerometer and 65 deg/s for the gyroscope were used to identify the initial instance of touching.
From the IMU data (Figure 12), the touching sequence corresponded with the video’s sequence. The timings of the touch from the video were extracted by interpolating from the frame timings, as touch sometimes occurs between frames. In contrast, the timing for the IMU came directly from the data logged with precise discrete timestamping at a high sampling frequency of 64 Hz.
To quantify the accuracy of the IMU results, we computed the timing error and root mean squared error (RMSE) (Table 4). The accelerometer tended to estimate the touch time to be earlier than it was and had an RMSE of 0.32 s, and the gyroscope did a better job at estimating the touch timing with an RMSE of 0.23 s. We could further improve the IMU-based touch detection accuracy by taking the average touch time from the accelerometer and gyroscope, resulting in an RMSE of just 0.19 s.
Therefore, IMU-based touch detection is an accurate way to detect touching and interacting with toys, removing the subjectivity of manual human coders. However, to ensure maximum accuracy and robustness, rather than entirely replacing manually coded data with IMU-based touch detection, the human coders can leverage the IMU data to speed up the manual coding process. In a video recorded at 30 frames per second (fps), the instrumented toys could help narrow the video to a segment spanning approximately 6 to 10 frames to confirm the exact touch time rather than go through the entire video.

6. Future Work and Conclusions

In this paper, we presented the initial design and validation of the sensing capabilities of instrumented toys that can be used in lab-based and ecological environments, allowing for a more robust and naturalistic assessment of the development of cognitive flexibility, an aspect of EF, in infants. The results of our study suggested that the instrumented toys were effective at detecting cognitively relevant aspects of infants’ behaviour.
The toy can detect periods of motion to determine when they are touched. From this, the overall sequence of touching can be inferred to calculate MRL and the conditional probability of infant touch sequence to identify mental set-shifting. The toys can also detect when and how much they have been squeezed to further validate infants’ change in mental classification from shape-based to material-based classification.
Such results confirmed the hypothesis that these instrumented toys could enable more quantitative monitoring and measurement of an infant’s EF development. The next step of our iterative design philosophy is to evaluate the instrumented toy’s performance while considering factors such as robustness, reliability, practicality, and ease of use through appropriate studies with mothers and infants in lab-based and home-based environments.

Author Contributions

Conceptualisation, V.R., M.Z.A., G.D.G., G.L.G., M.A.R., X.X.T., W.Y.Y., J.-P.O., V.L. and D.C.; data curation, V.R. and M.Z.A.; formal analysis, V.R.; methodology, V.R., M.Z.A., G.D.G., G.L.G., M.A.R., X.X.T., W.Y.Y., V.L. and D.C.; resources, V.R., M.Z.A., M.A.R., G.D.G., G.L.G. and X.X.T.; software, V.R.; supervision, W.Y.Y., V.L. and D.C.; validation, V.R., M.Z.A. and D.C.; visualisation, V.R.; writing—original draft, V.R., V.L. and D.C.; writing—review and editing, V.R., M.Z.A., G.D.G., G.L.G., M.A.R., X.X.T., W.Y.Y., J.-P.O., V.L. and D.C. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by funding from the Wellcome Leap 1kD Program.

Institutional Review Board Statement

Ethical approval for this work was obtained from the Nanyang Technological University Institutional Review Board (IRB-2022-294).

Informed Consent Statement

Appropriate consent was obtained from participants.

Data Availability Statement

The data presented in this study are available on request.

Acknowledgments

The authors would like to acknowledge the support of the broader Wellcome LEAP Programme, members of the Robotics Research Center, Baby-LINC Lab, and Sri Harsha Turlapati. This work was also supported by the Singapore Centre for 3D Printing, Nanyang Technological University, Singapore, through the use of its additive manufacturing facilities.

Conflicts of Interest

The authors declare no conflict of interest. The funder had no role in the design of the study; in the collection, analysis, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Diamond, A. Executive Functions. Annu. Rev. Psychol. 2013, 64, 135–168. [Google Scholar] [CrossRef] [Green Version]
  2. Deak, G.O. The Growth of Flexible Problem Solving: Preschool Children Use Changing Verbal Cues to Infer Multiple Word Meanings. J. Cogn. Dev. 2000, 1, 157–191. [Google Scholar] [CrossRef]
  3. Hughes, C.; Ensor, R. Executive Function and Theory of Mind: Predictive Relations From Ages 2 to 4. Dev. Psychol. 2007, 43, 1447–1459. [Google Scholar] [CrossRef] [PubMed]
  4. Anderson, P. Assessment and Development of Executive Function (EF) During Childhood. Child Neuropsychol. 2002, 8, 71–82. [Google Scholar] [CrossRef] [PubMed]
  5. Gündüz, B. Emotional intelligence, cognitive flexibility and psychological symptoms in pre-service teachers. Educ. Res. Rev. 2013, 8, 1048–1056. [Google Scholar] [CrossRef]
  6. Stahl, L.; Pry, R. Attentional flexibility and perseveration: Developmental aspects in young children. Child Neuropsychol. 2005, 11, 175–189. [Google Scholar] [CrossRef]
  7. Buttelmann, F.; Karbach, J. Development and Plasticity of Cognitive Flexibility in Early and Middle Childhood. Front. Psychol. 2017, 8, 1040. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Miyake, A.; Friedman, N. The Nature and Organization of Individual Differences in Executive Functions: Four General Conclusions. Curr. Dir. Psychol. Sci. 2012, 21, 8–14. [Google Scholar] [CrossRef] [PubMed]
  9. Ellis, A.; Oakes, L. Infants flexibly use different dimensions to categorize objects. Dev. Psychol. 2006, 42, 1000–1011. [Google Scholar] [CrossRef]
  10. Mareschal, D.; Quinn, P.C. Categorization in infancy. Trends Cogn. Sci. 2001, 5, 443–450. [Google Scholar] [CrossRef]
  11. Mandler, J.M.; Fivush, R.; Reznick, J.S. The development of contextual categories. Cogn. Dev. 1987, 2, 339–354. [Google Scholar] [CrossRef]
  12. Separating the sheep from the goats: Differentiating global categories. Cogn. Psychol. 1991, 23, 263–298. [CrossRef]
  13. Ricciuti, H.N. Object Grouping and Selective Ordering Behavior in Infants 12 to 24 months old. Merrill-Palmer Q. Behav. Dev. 1965, 11, 129–148. [Google Scholar]
  14. Horst, J.; Ellis, A.; Samuelson, L.; Trejo, E.; Worzalla, S.; Peltan, J.; Oakes, L. Toddlers Can Adaptively Change How They Categorize: Same Objects, Same Session, Two Different Categorical Distinctions. Dev. Sci. 2009, 12, 96–105. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Rakison, D.; Butterworth, G. Infants’ attention to object structure in early categorization. Dev. Psychol. 1998, 34, 1310–1325. [Google Scholar] [CrossRef] [PubMed]
  16. Welch, G.; Foxlin, E. Motion tracking: No silver bullet, but a respectable arsenal. IEEE Comput. Graph. Appl. 2002, 22, 24–38. [Google Scholar] [CrossRef] [Green Version]
  17. Campolo, D.; Taffoni, F.; Schiavone, G.; Formica, D.; Guglielmelli, E.; Keller, F. Neuro-Developmental Engineering: Towards Early Diagnosis of Neuro-Developmental Disorders. In New Developments in Biomedical Engineering; Campolo, D., Ed.; IntechOpen: Rijeka, Croatia, 2010; Chapter 35. [Google Scholar] [CrossRef] [Green Version]
  18. Luo, W.; Xing, J.; Milan, A.; Zhang, X.; Liu, W.; Kim, T.K. Multiple object tracking: A literature review. Artif. Intell. 2021, 293, 103448. [Google Scholar] [CrossRef]
  19. Cao, Z.; Hidalgo, G.; Simon, T.; Wei, S.E.; Sheikh, Y. OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 43, 172–186. [Google Scholar] [CrossRef] [Green Version]
  20. Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 779–788. [Google Scholar]
  21. Campolo, D.; Taffoni, F.; Formica, D.; Schiavone, G.; Keller, F.; Guglielmelli, E. Inertial-magnetic sensors for assessing spatial cognition in infants. IEEE Trans. Biomed. Eng. 2011, 58, 1499–1503. [Google Scholar] [CrossRef] [Green Version]
  22. Campolo, D.; Taffoni, F.; Formica, D.; Iverson, J.; Sparaci, L.; Keller, F.; Guglielmelli, E. Embedding inertial-magnetic sensors in everyday objects: Assessing spatial cognition in children. J. Integr. Neurosci. 2012, 11, 103–116. [Google Scholar] [CrossRef]
  23. Bondioli, M.; Chessa, S.; Narzisi, A.; Pelagatti, S.; Zoncheddu, M. Towards motor-based early detection of autism red flags: Enabling technology and exploratory study protocol. Sensors 2021, 21, 1971. [Google Scholar] [CrossRef] [PubMed]
  24. Chi, C.; Sun, X.; Xue, N.; Li, T.; Liu, C. Recent progress in technologies for tactile sensors. Sensors 2018, 18, 948. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Peng, Y.; Yang, N.; Xu, Q.; Dai, Y.; Wang, Z. Recent advances in flexible tactile sensors for intelligent systems. Sensors 2021, 21, 5392. [Google Scholar] [CrossRef] [PubMed]
  26. Anwer, A.H.; Khan, N.; Ansari, M.Z.; Baek, S.S.; Yi, H.; Kim, S.; Noh, S.M.; Jeong, C. Recent advances in touch sensors for flexible wearable devices. Sensors 2022, 22, 4460. [Google Scholar] [CrossRef]
  27. Campolo, D.; Maini, E.S.; Patane, F.; Laschi, C.; Dario, P.; Keller, F.; Guglielmelli, E. Design of a sensorized ball for ecological behavioral analysis of infants. In Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Roma, Italy, 10–14 April 2007; pp. 1529–1534. [Google Scholar]
  28. Bloor, D.; Graham, A.; Williams, E.; Laughlin, P.; Lussey, D. Metal–polymer composite with nanostructured filler particles and amplified physical properties. Appl. Phys. Lett. 2006, 88, 102103. [Google Scholar] [CrossRef] [Green Version]
  29. Cecchi, F.; Serio, S.; Del Maestro, M.; Laschi, C.; Dario, P. Design and development of sensorized toys for monitoring infants’ grasping actions. In Proceedings of the 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics, Tokyo, Japan, 26–29 September 2010; pp. 247–252. [Google Scholar]
  30. Serio, S.M.; Cecchi, F.; Assaf, T.; Laschi, C.; Dario, P. Design and development of a sensorized wireless toy for measuring infants’ manual actions. IEEE Trans. Neural Syst. Rehabil. Eng. 2013, 21, 444–453. [Google Scholar] [CrossRef]
  31. Tenzer, Y.; Jentoft, L.P.; Howe, R.D. Inexpensive and easily customized tactile array sensors using MEMS barometers chips. IEEE R&A Mag. 2012, 21, 2013. [Google Scholar]
  32. Jentoft, L.P.; Tenzer, Y.; Vogt, D.; Liu, J.; Wood, R.J.; Howe, R.D. Flexible, stretchable tactile arrays from MEMS barometers. In Proceedings of the 2013 16th International Conference on Advanced Robotics (ICAR), Montevideo, Uruguay, 25–29 November 2013; pp. 1–6. [Google Scholar]
  33. Takktile Sensors, Righthand Labs. Available online: https://www.labs.righthandrobotics.com/takktile-sensors (accessed on 15 October 2022).
  34. Ades, C.; Gonzalez, I.; AlSaidi, M.; Nojoumian, M.; Bai, O.; Aravelli, A.; Lagos, L.; Engeberg, E.D. Robotic Finger Force Sensor Fabrication and Evaluation Through a Glove. In Proceedings of the Florida Conference on Recent Advances in Robotics 2018, Orlando, FL, USA, 10–11 May 2018; Volume 2018, pp. 60–65. [Google Scholar]
  35. Kõiva, R.; Schwank, T.; Walck, G.; Meier, M.; Haschke, R.; Ritter, H. Barometer-based tactile skin for anthropomorphic robot hand. In Proceedings of the 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 25–29 October 2020; pp. 9821–9826. [Google Scholar]
  36. Takada, R.; Lin, W.; Ando, T.; Shizuki, B.; Takahashi, S. A Technique for Touch Force Sensing using a Waterproof Device’s Built-in Barometer. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, Denver, CO, USA, 6–11 May 2017; pp. 2140–2146. [Google Scholar]
  37. Quinn, P. Estimating touch force with barometric pressure sensors. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Scotland, UK, 4–9 May 2019; pp. 1–7. [Google Scholar]
  38. Taffoni, F.; Formica, D.; Campolo, D.; Keller, F.; Guglielmelli, E. Block-box instrumented toy: A new platform for assessing spatial cognition in infants. In Proceedings of the 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Minneapolis, MN, USA, 3–6 September 2009; pp. 210–213. [Google Scholar]
  39. Serio, S.; Cecchi, F.; Boldrini, E.; Laschi, C.; Sgandurra, G.; Cioni, G.; Dario, P. Instrumented toys for studying power and precision grasp forces in infants. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA, 30 August–3 September 2011; pp. 2017–2020. [Google Scholar]
  40. Cecchi, F.; Sgandurra, G.; Mihelj, M.; Mici, L.; Zhang, J.; Munih, M.; Cioni, G.; Laschi, C.; Dario, P. CareToy: An intelligent baby gym: Home-based intervention for infants at risk for neurodevelopmental disorders. IEEE Robot. Autom. Mag. 2016, 23, 63–72. [Google Scholar] [CrossRef]
  41. Sgandurra, G.; Beani, E.; Inguaggiato, E.; Lorentzen, J.; Nielsen, J.B.; Cioni, G. Effects on parental stress of early home-based caretoy intervention in low-risk preterm infants. Neural Plast. 2019, 2019, 7517351. [Google Scholar] [CrossRef]
  42. Digital Assessments for Movement and Cognition. Available online: https://www.mindmaze.com/digital-assessments-for-movement-cognition/#physilog (accessed on 5 October 2022).
  43. Physilog—Gait Up | Make Sense of Motion. Available online: https://physilog.com/ (accessed on 5 October 2022).
  44. User Manual—Gait Up Research. Available online: https://research.gaitup.com/wp-content/uploads/2021/01/P6S_Instructions-for-Use_V1.0.0.pdf (accessed on 5 October 2022).
  45. Dragon Skin 20 Material Safety Data Sheet. Available online: https://www.smooth-on.com/msds/files/BD_DS_Eco_Equ_EZB_EZS_Psy_MS_OOMOO_Reb_ST_SS_Soma_Sol_Sorta.pdf (accessed on 10 October 2022).
  46. Dragon Skin 20 Skin Safety Test Certification. Available online: https://www.smooth-on.com/assets/pdf/DRAGONSKIN_SKIN_SENSITIVITY_TEST_RESULTS.pdf (accessed on 10 October 2022).
  47. Goh, G.L.; Agarwala, S.; Yeong, W.Y. 3D printing of microfludic sensor for soft robots: A preliminary study in design and fabrication. In Proceedings of the 2nd International Conference on Progress in Additive Manufacturing (Pro-AM 2016), Singapore, 16–19 May 2016; pp. 177–181. [Google Scholar]
  48. Luis, E.; Pan, H.M.; Sing, S.L.; Bastola, A.K.; Goh, G.D.; Goh, G.L.; Tan, H.K.J.; Bajpai, R.; Song, J.; Yeong, W.Y. Silicone 3D printing: Process optimization, product biocompatibility, and reliability of silicone meniscus implants. 3D Print. Addit. Manuf. 2019, 6, 319–332. [Google Scholar] [CrossRef]
  49. Goh, G.D.; Sing, S.L.; Lim, Y.F.; Thong, J.L.J.; Peh, Z.K.; Mogali, S.R.; Yeong, W.Y. Machine learning for 3D printed multi-materials tissue-mimicking anatomical models. Mater. Des. 2021, 211, 110125. [Google Scholar] [CrossRef]
  50. Goh, G.D.; Goh, G.L.; Lyu, Z.; Ariffin, M.Z.; Yeong, W.Y.; Lum, G.Z.; Campolo, D.; Han, B.S.; Wong, H.Y.A. 3D Printing of Robotic Soft Grippers: Toward Smart Actuation and Sensing. Adv. Mater. Technol. 2022, 7, 2101672. [Google Scholar] [CrossRef]
  51. Esmaeili, S.; Shahali, M.; Kordjamshidi, A.; Torkpoor, Z.; Namdari, F.; Samandari, S.S.; Khandan, A. An artificial blood vessel fabricated by 3D printing for pharmaceutical application. Nanomed. J. 2019, 6, 183–194. [Google Scholar]
  52. Abdul Samat, A.; Abdul Hamid, Z.A.; Jaafar, M.; Yahaya, B.H. Mechanical properties and in vitro evaluation of thermoplastic polyurethane and polylactic acid blend for fabrication of 3D filaments for tracheal tissue engineering. Polymers 2021, 13, 3087. [Google Scholar] [CrossRef] [PubMed]
  53. Upshaw, M.B.; Bernier, R.A.; Sommerville, J.A. Infants’ grip strength predicts mu rhythm attenuation during observation of lifting actions with weighted blocks. Dev. Sci. 2016, 19, 195–207. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Sensors 23 02709 g001 550
Figure 1. Set of instrumented toy prototypes.
Figure 1. Set of instrumented toy prototypes.
Sensors 23 02709 g001
Sensors 23 02709 g002 550
Figure 2. Results of human pose tracking and object detection in object categorisation task videos. (Image used with specific parental consent).
Figure 2. Results of human pose tracking and object detection in object categorisation task videos. (Image used with specific parental consent).
Sensors 23 02709 g002
Sensors 23 02709 g003 550
Figure 3. MindMaze Physilog 6® (P6) Sensor.
Figure 3. MindMaze Physilog 6® (P6) Sensor.
Sensors 23 02709 g003
Sensors 23 02709 g004 550
Figure 4. (a) 3D Printed mould. (b,c) P6 sensor enclosed in silicon. (d) Fully cured sensor core.
Figure 4. (a) 3D Printed mould. (b,c) P6 sensor enclosed in silicon. (d) Fully cured sensor core.
Sensors 23 02709 g004
Sensors 23 02709 g005 550
Figure 5. Lattice structure with different outer geometries: (a) sphere (b) cube.
Figure 5. Lattice structure with different outer geometries: (a) sphere (b) cube.
Sensors 23 02709 g005
Sensors 23 02709 g006 550
Figure 6. 3D printed lattice structure (left) and instrumented toy assembly (right).
Figure 6. 3D printed lattice structure (left) and instrumented toy assembly (right).
Sensors 23 02709 g006
Sensors 23 02709 g007 550
Figure 7. Kinova Gen3 7 DoF Robot applying a squeezing force on instrumented toys.
Figure 7. Kinova Gen3 7 DoF Robot applying a squeezing force on instrumented toys.
Sensors 23 02709 g007
Sensors 23 02709 g008 550
Figure 8. Barometric squeezing detection: (a) soft cube, (b) soft ball, (c) hard cube, (d) hard ball.
Figure 8. Barometric squeezing detection: (a) soft cube, (b) soft ball, (c) hard cube, (d) hard ball.
Sensors 23 02709 g008
Sensors 23 02709 g009 550
Figure 9. Filtered barometric signals.
Figure 9. Filtered barometric signals.
Sensors 23 02709 g009
Sensors 23 02709 g010 550
Figure 10. Sequential touching experiment.
Figure 10. Sequential touching experiment.
Sensors 23 02709 g010
Sensors 23 02709 g011 550
Figure 11. Accelerometer and gyroscope signal processing.
Figure 11. Accelerometer and gyroscope signal processing.
Sensors 23 02709 g011
Sensors 23 02709 g012 550
Figure 12. Accelerometer and gyroscope data indicating the sequence of touches.
Figure 12. Accelerometer and gyroscope data indicating the sequence of touches.
Sensors 23 02709 g012
Table
Table 1. Print parameters for the fabrication of lattice structures.
Table 1. Print parameters for the fabrication of lattice structures.
Nozzle TemperatureBed TemperaturePrint SpeedLayer Thickness
TPU (85A/95A) 235 ∘ C 50 ∘ C30 mm/s0.2 mm
Table
Table 2. Squeezing Results.
Table 2. Squeezing Results.
ToyRobot Applied Force (N)Applied Pressure (Pa)Measured Force (N)Barometer Pressure (Pa)% Pressure Detected
Soft cube.20.02777.7817.5 (88%)16.04 ± 2.640.58%
Hard cube.21.02916.6719.0 (90%)324.60 ± 3.6211.13%
Soft ball.22.03392.9718.0 (82%)51.75 ± 4.531.53%
Hard ball.21.03238.7418.0 (86%)371.25 ± 5.1211.46%
Table
Table 3. Ground truth touch timing and sequence from video coded by a human rater.
Table 3. Ground truth touch timing and sequence from video coded by a human rater.
Instrumented ToyTouch Timing (s)
Soft cube.1.60
Hard cube.5.30
Hard ball.9.30
Soft ball.12.20
Hard ball.16.50
Hard cube.16.60
Soft ball.20.10
Soft cube.20.30
Hard ball.23.70
Soft ball.28.20
Table
Table 4. Touch sequence timing comparison.
Table 4. Touch sequence timing comparison.
ToyGround Truth (s)Accelerometer (s)Accelerometer ErrorGyroscope (s)Gyroscope ErrorAverage (s)Average Error
Soft cube.1.601.50−0.101.670.071.59−0.01
Hard cube.5.305.16−0.145.420.125.29−0.01
Hard ball.9.309.06−0.248.87−0.438.97−0.33
Soft ball.12.2012.200.0011.91−0.2912.06−0.14
Hard ball.16.5016.25−0.2516.35−0.1516.3−0.20
Hard cube.16.6016.30−0.3016.50−0.1016.4−0.20
Soft ball.20.1019.75−0.3520.180.0819.97−0.13
Soft cube.20.3019.92−0.3820.350.0520.14−0.16
Hard ball.23.7023.30−0.4023.50−0.2023.4−0.30
Soft ball.28.2028.800.6027.83−0.3728.320.12
RMSE——0.32—0.23—0.19
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Ramanathan, V.; Ariffin, M.Z.; Goh, G.D.; Goh, G.L.; Rikat, M.A.; Tan, X.X.; Yeong, W.Y.; Ortega, J.-P.; Leong, V.; Campolo, D. The Design and Development of Instrumented Toys for the Assessment of Infant Cognitive Flexibility. Sensors 2023, 23, 2709. https://doi.org/10.3390/s23052709

AMA Style

Ramanathan V, Ariffin MZ, Goh GD, Goh GL, Rikat MA, Tan XX, Yeong WY, Ortega J-P, Leong V, Campolo D. The Design and Development of Instrumented Toys for the Assessment of Infant Cognitive Flexibility. Sensors. 2023; 23(5):2709. https://doi.org/10.3390/s23052709

Chicago/Turabian Style

Ramanathan, Vishal, Mohammad Zaidi Ariffin, Guo Dong Goh, Guo Liang Goh, Mohammad Adhimas Rikat, Xing Xi Tan, Wai Yee Yeong, Juan-Pablo Ortega, Victoria Leong, and Domenico Campolo. 2023. "The Design and Development of Instrumented Toys for the Assessment of Infant Cognitive Flexibility" Sensors 23, no. 5: 2709. https://doi.org/10.3390/s23052709

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop