Next Article in Journal
A Literature Survey of How to Convey Transparency in Co-Located Human–Robot Interaction
Previous Article in Journal
Comparison of Using an Augmented Reality Learning Tool at Home and in a Classroom Regarding Motivation and Learning Outcomes
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Can AI-Oriented Requirements Enhance Human-Centered Design of Intelligent Interactive Systems? Results from a Workshop with Young HCI Designers

1
Department of Computer Science, University of Salerno, 84084 Fisciano, Italy
2
Faculty of Political Science and Psychosocial Studies, Università degli Studi Internazionali di Roma—UNINT, 00147 Roma, Italy
*
Author to whom correspondence should be addressed.
Multimodal Technol. Interact. 2023, 7(3), 24; https://doi.org/10.3390/mti7030024
Submission received: 11 December 2022 / Revised: 14 February 2023 / Accepted: 23 February 2023 / Published: 25 February 2023

Abstract

:
In this paper, we show that the evolution of artificial intelligence (AI) and its increased presence within an interactive system pushes designers to rethink the way in which AI and its users interact and to highlight users’ feelings towards AI. For novice designers, it is crucial to acknowledge that both the user and artificial intelligence possess decision-making capabilities. Such a process may involve mediation between humans and artificial intelligence. This process should also consider the mutual learning that can occur between the two entities over time. Therefore, we explain how to adapt the Human-Centered Design (HCD) process to give centrality to AI as the user, further empowering the interactive system, and to adapt the interaction design to the actual capabilities, limitations, and potentialities of AI. This is to encourage designers to explore the interactions between AI and humans and focus on the potential user experience. We achieve such centrality by extracting and formalizing a new category of AI requirements. We have provocatively named this extension: “Intelligence-Centered”. A design workshop with MsC HCI students was carried out as a case study supporting this change of perspective in design.

1. Introduction

In our previous works [1,2], we explored the role of artificial intelligence (AI) based on neural networks in sign language learning. Specifically, we employed AI to recognize user errors when performing sign language gestures and designed the system to help users learn from their mistakes by adapting to the AI assessments. Through user experience studies and observations, we found that users willingly adapted their learning process to incorporate AI feedback, establishing a disciple–mentor relationship with the system. However, the design of the system had to account for variations in gestures and the user’s hands, requiring the AI to undergo a process of inference to recognize gesture variants and align with each user’s unique characteristics. This collaboration between humans and AI resulted in a positive relationship of alignment. Nonetheless, such positive relationships only sometimes become established.
Stanton and Jensen [3] argue that the perceived trustworthiness of an AI system is a critical factor in determining its influence on users. User perceptions of technical trustworthiness characteristics significantly shape their level of trust in a system. Sneiderman [4] also highlights the issue of trust in the context of artificial intelligence and autonomous systems. He notes that, in general, humans tend to need more trust in such systems and expend more effort monitoring and attempting to control them, rather than using them to their full potential. This lack of trust is pervasive across different levels of autonomous systems.
In this study, we delve into the various elements that contribute to shaping the user experience in AI-based systems, thus reinforcing our viewpoint. In particular, we concentrate on users’ feelings, cognitive approaches to decision making, and means of interaction, without focusing on specific types of AI systems.
We contend that a redesign of the relationship between humans and artificial intelligence is necessary to instill trust in users toward these systems. The importance of designing physical objects that can evoke positive feelings in users, thereby generating satisfaction, is emphasized by the authors of [5] in the context of the Internet of Things. Unlike a software interface that can be turned off when not needed, a physical object persists in the physical world and must be dealt with in space, cared for, and seen daily. Similarly, commercial voice assistants, although not visible, remain active by monitoring and listening to the context and are always ready to assist the user, resulting in a passive yet continuous interaction between the user and AI.
Thus, it is important to establish a positive relationship between users and AI to promote positive emotional responses toward the technology. Moreover, positive feelings can be obtained, for example, by providing a sense of trust, understanding and transparency of the system, and an interaction perceived as more natural to humans, as with conversational interfaces [6]. Indeed, such systems imitate human interactions by communicating through a natural spoken language. However, this may be not sufficient, because often the decision-making power is an exclusive responsibility of the human part, while in a natural human–human interaction both parts are able to make a decision for which the final decision is made by mediation.
Regarding the decision-making approach, recently, the cooperative approach [7] was introduced as a new perspective. The authors assert that machines must learn to align with human expectations and collaborate effectively with them. They outline four key components of cooperative intelligence: Understanding, Communication, Commitment, and Norms, highlighting the similarities with human cooperative activities. We concur that a shift in the current paradigm is necessary to foster deeper collaboration between humans and AI, which has the potential to bring numerous benefits, similar to when individuals with diverse skills collaborate to solve problems or make improvements. These changes can be reduced to two main objectives: enhancing communication for improved understanding and supporting the user experience. A similar paradigm shift can be seen in the progression from the six levels of driving autonomy. However, differently from our position, cars are expected to take progressively complete control of the decision-making process.
Finally, regarding the kind of interaction, the communication between humans and AI heavily relies on interfaces [8]. Relationships often involve a mixture of shared and conflicting interests in the real world. To reinforce trust in the human–AI collaboration, finding a common vocabulary to communicate intentions is an effective solution [9].
Starting from all the considerations mentioned previously, Figure 1 illustrates the factors that can impact the user experience in an artificial intelligence (AI) system. While some factors, such as usability, task completion satisfaction, interface ergonomics, and comfort, fall under the typical umbrella of user experience (UX), others are more closely tied to AI. Specifically, these factors include the type of decision-making process, the level of interaction, and the feelings elicited by the system in users.
In this work, we introduce a modification to the Human-Centered Design (HCD) approach with the aim of incorporating artificial intelligence (AI) considerations from the inception of the design process for interactive systems. Our proposed approach, named Intelligence-Centered Design (ICD), is intended to provide support and guidance to novice designers, particularly in educational settings. It emphasizes the interaction between AI and humans and its impact on the user experience, thereby assisting novice designers in navigating the intricate design of AI-based systems. The centrality of AI in ICD is established by focusing on AI and formalizing a novel set of AI requirements through their extraction besides user requirements, understanding the context, identifying the AI use case, and conducting user research. This shift in perspective places AI beside the user at the forefront of the design process in a manner equivalent to the traditional focus on the user. In this paper, it is essential to note that our argument is not centered on the notion that designers should cater to the interests of AI, as AI lacks the capacity for self-interest. Instead, our stance is that the interaction design should be aligned with the actual capabilities, limitations, and potentialities of the specific neural network model in question. For instance, similar to avoiding visual interfaces in designs intended for the visually impaired, designers should steer clear of interactions that involve temporal sequences if the neural network lacks the capability to recognize them.
Additionally, we present a case study supporting the focus on AI performed through a workshop with undergraduate HCI MsC students who were invited to use the new approach for a project of their course. The analysis brings out emerging themes and some initial clues, such as the experienced difficulties, identified advantages, potential adoption, and the quality of students’ projects developed through the ICD design process.
The organization of the paper is as follows: Section 2 presents an overview of related works in the literature. Section 3 introduces the Decision Perception Model and elaborates on the shift in perspective, including an extension of the Human-Centered Design (HCD) approach and the specific requirements for AI. Section 4 presents an illustration of the design process through a scenario based on our design experience. Section 5 presents the outcomes of a case study conducted with graduate students in computer science with a background in HCD design aimed at testing this new approach. Finally, in Section 6, the paper concludes with a summary of findings and suggestions for future work.

2. Related Work

Given the rapid evolution of artificial intelligence and its increasingly constant presence in our daily life, the last decade has seen many researchers and IT companies working in the field of Human-Centered Machine Learning (HCML) [11]. HCML is an area of research that studies how to align ML-based interactive systems with the goals, needs, and concerns of humans.
In [11], the authors highlight that, in general, developers focus especially on problems related to the algorithms to be trained and underestimate the needs and the goals of their stakeholders. Indeed, studies published on users using AI systems highlighted various concerns and problems related to the explainability of the results, reliability, and the user experience in general [12,13]. The problems related to the explainability of AI results stem from the black-box nature of many AI models, where the internal workings and decision-making processes are complex for humans to understand. This lack of transparency can lead to difficulties in building trust and interpreting the outcomes of AI systems, which can impact the reliability and user experience of these systems. Regarding the reliability of AI systems, some concerns have arisen due to the potential for AI systems to make incorrect or biased decisions. This can occur when AI systems are trained on partial data or their decision-making processes are not adequately validated. Regarding the user experience, some challenges have arisen from the complexity of AI systems, which can make it difficult for users to interact with and understand the outcomes of these systems.
Yang and colleagues [14] provide designers and researchers with insights to tackle challenges in human–AI interactions, highlighting that uncertainty about the capacity and complexity of AI system outputs is a significant factor that negatively impacts user experience. Furthermore, Chancellor and co-authors [15] analyzed the literature on mental health and AI to identify priorities and compile guidelines that prioritize the human element.
Others in the community have investigated how to design specific human–AI interaction scenarios. For example, researchers have been studying how to effectively interact with intelligent agents for many years, particularly focusing on voice user interfaces and conversational interfaces in general. This area has seen a recent resurgence of interest, given advances in natural language processing and embedded devices that drive the proliferation of conversational agents [16].
On the other hand, companies such as Google (https://pair.withgoogle.com/, accessed on 24 February 2023), Apple (https://developer.apple.com/design/human-interface-guidelines/technologies/machine-learning/introduction/, accessed on 24 February 2023), and Microsoft [8] encourage their developers to utilize AI and ML to enhance user experience, providing guidelines to create human-centered ML systems that prioritize explainability, usability, and understandability in system design.
In this paper, we propose a new approach to designing AI-based systems that considers the mutual understanding required between AI and its users for a seamless and positive experience. Previous works and recommendations have primarily focused on improving user understanding of AI and how it can serve their goals. However, it is important to acknowledge that AI also needs to understand the users [17]. A design that only focuses on either AI or the user can lead to challenges in their interaction and negatively impact the user experience.
Our approach considers both the needs, skills, and goals of the user and the capabilities and limitations of AI, and places equal emphasis on both in the design process. By highlighting the importance of mutual understanding between AI and humans, we aim to develop further the principles of HCML and provide a solution to the issues of the interrelationship between AI and humans.

3. A Change in Perspective: Intelligence-Centered Design

In this work, given the revised outlook on the role of AI in an interactive system, we propose a reorganization of the design process that takes into account AI as a proactive participant in the interaction, possessing unique qualities and requirements, capable of making decisions independently or in collaboration with the user. For this, the decision-making process should evolve towards a cognitive process mediated by two intelligences, much like the cooperative balancing game. This game, played by two individuals, involves working together to solve a challenge rather than competing. The objective is to guide a ball through a labyrinth placed on a wooden board, where the corners are supported by four nylon threads held by both players. The actions of one player inevitably impact the other, leading to initial misunderstandings, but, with time, the two players learn to understand each other’s decisions, imitate them, predict them, and develop a common strategy.
To further clarify this approach, we present the Decision Perception Model, depicted in Figure 2. This model updates the Context Perception Model introduced by [18], which was utilized to demonstrate the mismatch in awareness between the user and a context-aware system. Our model demonstrates that the user’s anticipation of AI decisions is based on the current context and previous experience. Likewise, AI perceives the current context through data collected by environmental sensors and processed by its neural network, which has been trained for specific scenarios and can continue to evolve and improve through continuous learning. Given the present context, AI can make decisions and anticipate the user’s decisions. A decision made by AI may not align with the user’s expectations due to varying perceptions of the context and previous experiences. Nevertheless, this does not necessarily pose a problem. The user may find AI’s decision to be better than expected and adjust their perceptions accordingly for future interactions. Conversely, if AI’s decision is not appreciated, it can learn from the experience and align with the user’s expectations. In practice, the decisions made by the two intelligences represent a constant alignment of expectations that allows them to grow as users of the same interactive system.
A revised approach is necessary to reflect the changing perspective in interactive system design. Conventionally, design considerations for interactive systems focus on the needs and objectives of the users. However, with the integration of artificial intelligence, it must now be viewed as a crucial and active component in the design process. This approach may seem biased toward technology-driven design, but it is not intended to override user-centered design. Instead, it aims to give equal consideration to both AI and the user during the design phase.
Just as humans have requirements for effective communication with AI systems, artificial intelligence also has specific needs that must be met to ensure accurate and efficient interaction with humans. These requirements, viewed from the perspective of AI, may include factors such as usability, interaction design, and cognitive decision-making processes. In Table 1, we analyze these last three factors, presenting a list of possible AI requirements. Each requirement is associated with its rationale. The list is not to be considered exhaustive, but it is a starting point when analyzing this category of requirements. The proposed list of requirements was formalized and discussed by the authors and a team of academic HCI experts and AI engineers.
The participants started by reviewing a set of interactive AI-based systems to gain insight and inspiration for their list. The criteria for inclusion in the list were based on factors such as the importance of the requirement for a human-centered AI system, the frequency with which the requirement was observed in the reviewed systems, and the potential impact of the requirement on the user experience. The process of constructing the list involved discussions and debates among the experts to determine which requirements should be included. The goal was to reach a consensus, with more than 80% of the experts agreeing on each requirement. In the case of disagreements, the experts engaged in further discussion and negotiation until a resolution was reached that satisfied a majority of the team. This process was repeated until the team had agreed upon a comprehensive list of AI requirements that reflected the collective wisdom of the experts and their collective understanding of the principles of AI and Human-Centered Design.
Recognizing the importance of meeting the needs of both users and AI to support the user experience, it is necessary to extend the classic Human-Centered Design process outlined in ISO 9241-210:2019. In Figure 3, we present the Intelligence-Centered Design (ICD) process, which involves analyzing the context of use to identify user and AI requirements. From this, designers can develop a solution that meets the needs of both sets of requirements, as its evaluation should be performed against both user and AI requirements.
The purpose of the following process is to extract and formalize AI requirements within the context of the ICD approach:
  • Understand the context: The first step is to understand the context in which the product or service will be used. This includes:
    • Understanding the user’s environment, necessities, tasks, goals, and motivations through conducting user research;
    • Understanding the AI use case, the problem it is trying to solve, and the benefits it is expected to bring to users.
  • Extract user requirements: Based on the AI use case and user research defined, extract the specific user requirements.
  • Extract AI requirements: Based on the AI use case and user research defined, extract the specific AI requirements. Such requirements are, for example, related to interaction, usability, decision-making approach, performance, accuracy, explainability, transparency, and privacy.
  • Formalize the requirements: This step will formalize the requirements into a structured format. This structured format helps develop AI systems that meet the user’s expectations and requirements.
  • Design a solution: Produce a design solution addressing user and AI requirements. This allows focusing on the user experience through a solution taking advantage of AI and user characteristics so that they do not conflict but support each other.
  • Evaluate the solution: Evaluate the design solution against the requirements; if it does not meet them, consider modifying the solution, reviewing the requirements, or exploring the context again.
In summary, the process of extracting and formalizing AI requirements within the context of the ICD approach involves understanding the context, identifying the AI use case and conducting user research, and extracting and formalizing AI and user requirements into a structured format.

4. A Scenario Based on Our Experience

We observed the following scenario during our experimentation with the tangible interface proposed in [19,20]. We designed a mobile controller prototype that enables users to interact with and receive feedback from different devices in an ambient intelligence-controlled environment. The controller incorporates a wide range of gestures based on combinations of touch and rotations to provide users with diverse options for interacting with the environment.
Upon entering a dimly lit room managed by ambient intelligence, a user operates an interactive device to request more light. While the user anticipated that a lamp would turn on to brighten the space, the AI system chose to open the curtains, allowing natural light to enter instead. This decision was based on undisclosed data regarding energy consumption and a preference for energy conservation when possible. Although the user was surprised by the AI’s decision, they may have accepted it if they had been informed of the reasoning. However, if the user had concerns about privacy, they might not have agreed with the AI’s decision to open the curtains. In this situation, AI should reinforce its learning to consider such possibilities in the future.
In this example, artificial intelligence is based on an ML model that has been initially trained for basic actions, while the ML’s reactions to the specific profiled user are customized by a Continuous Machine Learning (CML) methodology. The ML model will fine-tune the information continuously, learning which of the learned basic actions require alternative outputs because of the user’s profile.
The example highlights a design flaw in the tangible interface, resulting from a need for more consideration of the necessity for artificial intelligence to communicate effectively with humans. Although the design followed the principles of Human-Centered Design for interactive systems (ISO 9241-210:2019) [21], the focus was primarily on providing a rich set of input commands for the user, resulting in poor haptic feedback. As a result, the conversation between AI and the user was limited.
Taking into account the previously described AI scenario, we will formalize the high-level requirements of the AI system using Table 1:
  • Interaction must be easy to learn by AI. Since user needs and expectations can vary from one user to another, AI needs to be able to easily evolve and adapt to the user and context. For example, asking for more light in an office could generate the expectation of natural light, as the user goes to that room to work for several hours and natural light can be more relaxing. However, in situations where a user enters a bedroom, they may prefer a more intimate environment and may choose to have the blinds closed. The ratio of the time required for learning to the amount of information correctly learned by the AI should produce a rapid learning curve.
  • Interaction must be easy to interpret by AI. In unambiguous cases, AI should not learn all the time, for example, when the user asks it to perform a specific task, such as turning on the TV. In this case, the interaction must be immediately interpretable.
  • Interaction must have a recognizable context. If the system environment is frequented by multiple users, then audio interactions can generate privacy problems or confusion. A multimodal interaction able to give enough expressiveness to the communication must be selected while respecting the limitations of the environment.
  • The AI decision-making approach must be clearly established. In our case, the type of decision-making process is primarily cooperative. For example, the user expresses the necessity to have more light and the artificial intelligence offers the most appropriate solution according to the context it perceives and its previous “training” with the user. The user can agree or not, can ask to modify the decision, or even be convinced.
  • The interaction must be correctly shaped to support single or multiple users. AI must relate to different users in the same space. Therefore, each user must be easily distinguishable by AI to allow it to correctly pursue its decision-making process.
Incorporating solely AI requirements into the design process may result in the creation of interfaces that are difficult for users to comprehend. It is crucial for designers to balance the AI requirements with the needs of the users to ensure a seamless and meaningful interaction. The designer must consider both the technical and privacy limitations of the AI, as well as the preferences of the user when determining the appropriate mode of interaction. In some cases, a verbal interaction may be preferred by both AI and users, while, in other cases, this may not be feasible due to technical or privacy restrictions.

5. A Case Study Based on an Educational Experience Workshop

We designed a case study involving young students of HCI as a means to collect evidence and identify emerging themes deriving from the usage of the proposed approach. The case study was held at the University of Salerno in the form of a design workshop during regular lessons of an advanced HCI course.
The case study allowed us to examine the ICD process involving a restricted group of participants sufficient to analyze the design experience, encourage discussion, and collect points of view and reflections on it. This kind of approach is not to be considered exhaustive; however, its analysis offers some initial clues regarding the experienced difficulties, identified advantages, potential adoption, and the quality of the students’ projects developed through the ICD design process.

5.1. Procedure

The case study was conducted in the following steps:
  • The researchers equipped the students with a comprehensive list of ML models sourced from [22], which had been previously discussed during the course. The students were then encouraged to evaluate the ML models by considering practical-use cases and to identify specific examples of how a particular ML inference model can enhance business operations, decision-making processes, and user experiences.
  • A three-day workshop, in which students learned and applied the ICD process in a project. This was to allow the researchers to observe the students’ difficulties and dynamics and collect doubts.
  • A group discussion of the project and of the ICD. This was to allow the researchers to examine the projects and explore the ICD effects on the design activity.
  • A design experience survey with the aim of evaluating the ease of the ICD adoption.
Three researchers participated in the experiment to explain the workshop, tutor the participants, and facilitate the discussion.
The workshop lasted a total of 3 nonconsecutive days for a total of 9 hours, during which there were three didactic activities in blended mode to present the needed knowledge, discuss the difference between a common interactive system and an AI-based interactive system, introduce the ICD process, and tutor participants. The projects were finally delivered as a course exercise and reviewed by the teacher to evaluate the overall quality.
As it was an advanced course, we emphasized independent exploration and problem solving, with 20–30% of the time dedicated to explicit instruction and guidance, 40–50% to exercise and design challenges, and the remaining time devoted to discussion and feedback.

5.2. Participants

The participants were 39 computer science students (25 males, 14 females) of the “Human-Computer Interaction and Software Usability” master’s course at the University of Salerno. Participant ages ranged from 22 to 25 (mean = 23.8). The course is an advanced course, and all the participants had already taken a foundational HCI course in their bachelor’s studies, maturing initial experience in interface design and the HCD process. The participants were distributed into 11 groups, each of which dealt with a topic chosen autonomously.
Finally, the students had at least a basic knowledge of AI-based systems. They also had previous experience carrying out HCD exercises to design AI-based interactive systems explicitly.

5.3. Data Collection and Analysis

During the workshop, the researchers observed the groups at work, taking notes on the dynamics and doubts.
The discussion with the groups was recorded using Microsoft Teams for later transcription. The transcripts were then analyzed using a thematic analysis method.
At the end of the workshop, the students filled out a questionnaire on their design experience. The questionnaire was distributed through Microsoft Teams, and it was formed of 11 items evaluated on a five-point Likert scale. The collected answers were then analyzed, first, through a reliability test and then through the analysis of the means and coefficients of variation.
We also collected evidence on the quality of the student’s projects by interviewing the teacher, as the projects were delivered as a course exercise. The collected projects were also analyzed to make it possible for the researchers to explore the role of AI envisioned by the students.

5.4. Workshop Execution

At the beginning of the workshop, the participants were invited to apply the new approach to extract requirements and generate ideas based on AI in the context of smart cities with the aim of improving the quality of life of communities in problematic situations to be identified.
More precisely, after choosing a “big idea”, the students investigated the context, identified problems working with personas and problem scenarios, extracted functional and non-functional requirements, and presented an idea of a basic solution that met the requirements. The design process ended here to allow researchers to collect data for the analysis.
During the first workshop day, to facilitate the definition of the problems to solve, one researcher illustrated to the participants the Sustainable Development Goals of the 2030 agenda of the United Nations organization, with particular reference to objectives 11 and 13, which are: “Make cities inclusive, safe, resilient and sustainable” and “Take urgent action to combat climate change and its impacts”.
Groups worked autonomously on the problem definition before the second blended activity. In this activity, the participants presented the selected working context, the challenges they identified, and their objectives, and discussed them with the researchers and classmates.
The rest of the time was reserved for the work of the groups to define the personas and the problem scenarios. Here, the participants had the opportunity to ask questions and to receive support from researchers when needed.
The days before the last blended activity were dedicated to the extraction of the requirements, with particular attention to the AI requirements, and to the proposal of a design idea to be exhibited through storyboards and paper prototypes.
On the last workshop day, the participants were asked to present their projects and discuss them with the researchers and the other participants. In particular, they discussed the extracted requirements and the solutions adopted. Furthermore, they highlighted their own view of designing an AI-based interactive system using the new approach instead of the traditional one.
At the end of the discussion, the researchers provided all participants with questionnaires to analyze their experience with the Intelligence-Centered Design process.

5.5. The Students’ Projects

Each group proposed an original design solution to make cities inclusive, safe, resilient, and sustainable. In the following, we describe the distribution of the projects across different domains.
A. Recycling and littering
Three groups dealt with recycling and littering problems, adopting similar solutions. Littering occurs due to difficulty in identifying bins and lack of waste-separation knowledge. The solutions geolocated bins to make waste disposal easier. The AI decision-making process was crucial, and two approaches were used: collaborative approaches, where users may disagree with the AI conclusion regarding the selected waste bin and can help it to improve its decision making, and autonomous approaches, where AI makes the decision to avoid human error. In the autonomous approach, the explainability of the decision is important to inform users.
B. Public transportation
One project addressed the problem of making public transportation more accessible, as traffic is responsible for 49% of polluting emissions. The aim of the project was to try to promote the use of public transportation. AI was used to offer predictions on vehicle crowding, delays, and to estimate the best routes for the users’ necessities. Again, artificial intelligence plays a collaborative role in the decision-making process, indeed the choice of the most appropriate route is given by a continuous collaboration of both the human and AI, which can consider both previous users’ choices and behaviors.
C. Eco-shopping
One project focused on this application area, with the goal of reducing consumption and preserving the planet’s resources. For these reasons, the project aimed to support users during shopping by assisting them in choosing more eco-sustainable products. AI was utilized by observing the users’ behavior during their purchasing decisions and tracing their evolution, allowing the system to better respond to their needs and offer results or suggestions that align with their goals, trying to be as empathetic as humans would be.
D. Smart parking
Three teams tackled the issue of smart parking, proposing an intelligent parking system to address such issues as finding a parking space, time, money, sustainability, and traffic. AI was used to select parking spaces based on collected data and adapt to the user’s preferences. The decision-making process and interaction between users and AI were identified as crucial. The process must be collaborative and continuous, where AI suggests a parking space, but the user may choose a different one. The interaction between AI and the user must be easy and interpretable by AI. The decision-making process ends when the user finally parks, and if the user does not stop at the indicated parking space, this is interpreted as the AI decision being rejected.
E. Anxiety
One project focused on the electric car market and on the problem that often leads people to be skeptical about buying one, i.e., “battery anxiety”. The proposed solution provided functionalities to allow the localization of charging stations and determination of their availability to mitigate drivers’ battery anxiety. The AI requirements identified were the analysis of the user’s position and the ability to analyze the driver’s stress level. User interaction must be easy to learn from an AI point of view. Indeed, in the solution idea, AI was also used to better interpret the user’s facial expressions over time and understand their stress and to offer reassurance and mitigate anxiety.
F. Food waste
One project addressed the problem of food waste, as 17% of the world’s food is thrown away. The proposed solution was designed to interpret the ingredients entered by the user in order to suggest possible recipes and, thanks to the support of AI, to learn the user’s tastes in order to optimize suggestions and reduce waste as much as possible. A characteristic of the project idea was linked to the decision-making process. The AI chooses more suitable recipes based on available ingredients and expiration dates; however, as the user becomes more experienced, they can take more control than the AI over the decisions about which ingredients to use.
While applying the ICD design methodology in which they were instructed, all the teams found it useful to categorize as AI requirements also the capabilities of AI, such as the prediction model, the type of evolutionary process of the AI understanding, the information needed to make AI work, and the goals of AI within the system.

5.6. Group Discussion

At the end of the workshop, in addition to the questionnaire, the groups of students were invited to present their work and were asked about their experiences to understand from their point of view how the use of AI was guided by the formalization of AI requirements and whether this made a significant contribution to their work and how.
For this, they were asked the following questions:
  • What role does AI play in your solution?
  • Did AI give a significant contribution to your solution?
  • Is the use of AI requirements helping you to make the most of AI in your project or not?
Furthermore, each student had the opportunity to freely express their opinions on any aspect of the workshop.
The participants explained that giving artificial intelligence centrality in their work, as well as the user, made it possible to develop more powerful ideas to solve problems or mitigate situations better than they would otherwise have done.
In most cases, the participants explained that their ideas would have developed even without AI but that its use made it possible to understand the context, the mood, or the physical states of the users and learn from their behaviors to offer them better experiences and to make the solutions more effective. Then, the students felt encouraged to work during the design process at studying the interactions among AI and humans and focusing on the potential user experience.
Regarding the use of AI requirements, in general, the participants emphasized that considering AI requirements forced them to focus more on AI and the role it plays in the system. This opened up to new possibilities that were not contemplated in their initial ideas. One said: “Defining the AI requirements guided me through a more detailed definition of how to apply AI to the system”.
Some participants also highlighted that, initially, they considered AI in their project only for exercise but that after working on the list of AI requirements they discovered the centrality of its role within their project. A student explained: “Although I am not familiar with the AI area, using the AI requirements made it relatively easy to identify its role within my design” and another: “Exploring AI requirements, I understood how AI can be fully integrated into the project”.
Another interesting prevalent consideration was related to the opportunity to discover new requirements in other categories by focusing on AI, as one student explained: “Analyzing at the beginning of the project this category of requirements helped me to better understand and detail other categories of requirements”. Most of such considerations were related to the category of functional requirements: “I find that defining requirements involving AI is an effective method to extrapolate functional requirements not yet identified”, or “Once one or more metrics of AI interest have been identified, it is easier to extrapolate the associated functional requirement”.
Other common considerations were related to the potential effects on the user experience; the participants considered that focusing on the AI characteristics from the beginning of the project allowed them to provide better solutions that were able to engage people and satisfy their needs. One participant said: “I realize that by focusing more on AI, my solution can be more engaging and interesting to the user”. Another one highlighted: “Working with AI is something new for me, but I found this approach very interesting and above all useful for creating applications that will improve the user experience.”

5.7. Findings from the Group Discussion

To analyze the discussion results, we adopted a thematic analysis method [23]. In [24], the authors state that such an approach is suitable to analyze qualitative data for a wide variety of research questions, including those about people’s experiences, for example, students’ experiences [25]. As explained in [24], the method can be used with both large and small datasets. This method is mainly composed of data coding and searching for themes.
Therefore, according to the group data, we formalized three main themes concerning our approach, described in Table 2, which are: focus on the AI role, benefits for functional requirements, and a way to empower the user experience.
The study validity was achieved through cross-checking of the transcribed discussions. Moreover, to make sure that the researchers coded the text in the same way, we adopted a reliability analysis approach based on Cohen’s Kappa presented in [26]. Cohen’s Kappa rates interrater reliability from 0 to 1, where 1 means perfect reliability between the coders. Two researchers were in charge of coding the transcripts. Before starting the code activity, the research group developed a set of coding instructions to align the coders. After training, the coded data were measured using Cohen’s Kappa analysis. The Kappa coefficient is calculated as follows:
K = (Pa − Pc)/(1 − Pc).
where Pa represents the percentage of cases in which the coders agree and Pc represents the percentage of agreed cases when the data is coded by chance. A coefficient bigger than 0.6 is considered satisfactory, and a coefficient bigger than 0.8 is considered near-perfect agreement. After iterating the coding, when the Kappa result was satisfactory (above 0.6) the coders started the formal coding.
In our case, at the final iteration, the coefficient K was 0.79, which was considered definitively as a good agreement result.
From the discussion with the students, it emerged that the new approach to design was perceived as a useful way of working with interactive systems, both if AI is foreseen from the beginning of the project and if it is not initially contemplated.
Focusing on the possible role of AI, indeed, carries immediate benefits to the visions that students have of their projects. In this way, it was possible to either consider it as an essential element of the system or as an element capable of raising the user experience.
The AI requirements made it possible to capture functional requirements that escaped the students’ initial ideas for their projects; at the same time, however, they guided them to define the way in which AI should interact with users, having clear experience goals, such as users’ trust in the system, empathy, etc.

5.8. Students’ Work Assessment

As an initial assessment of the validity of the proposed framework, the teacher evaluated the paper prototypes and documentation provided by the students against the identified requirements. All of the delivered projects were judged satisfactory and able to provide users with an effective and efficient interaction experience through AI.
The teacher was asked about the quality of the projects. She observed that, thanks to the framework, the students exploited the potential of AI more than they would usually do. She said: “Focusing on AI from the very beginning they were able to properly consider the relationship between AI and system stakeholders”. She supported these considerations by also comparing the projects her students produced in this workshop with those created in the workshops of the previous course edition.

5.9. Design Experience Survey and Result Analysis

The survey goal was to evaluate the students’ perceptions and the ease of adopting the new approach and, more specifically, its usefulness with respect to previous design experiences and the intention of using it in future projects. The choice to use a questionnaire instead of carrying out further activities, such as a focus group or several interviews, was due to the didactic nature of the workshop performed during the regular term. Indeed, a questionnaire is a practical way of minimizing intrusiveness with respect to student activities.
The questionnaire was based on the technology acceptance model (TAM) [27,28,29] and was modified for evaluating the user experience while practicing with the new AI requirements category.
Adapted versions of the TAM have already been used in the literature to explore students’ experience when using new methods or tools during regular classes [25,30].
The questionnaire was formed of 11 items divided into three scales, which were: the perceived ease of use (EU) of the AI requirements category in the design of an interactive system, the perceived usefulness (PU) of the requirements in the design, and the intention to use (IU).
PU, in particular, leveraging the previous experience of designing interactive AI-based systems among the students, explores the utility of such requirements to help designers to focus more on the role of AI in the interactive system (PU1), to better understand the interaction among AI and stakeholders, and to make it easier to generate ideas of how to empower interactive systems using AI (PU3, PU4).
The items of the questionnaire that were answered using a five-point Likert (1 = strongly disagree, 5 = strongly agree) are summarized in Table 3. According to the literature, this method seems to increase the response rate and quality and reduce the “frustration level” of respondents [31,32].
Finally, the students had at least a basic knowledge of AI-based systems. They also had previous experience carrying out HCD exercises to design AI-based interactive systems explicitly.
We collected 429 responses from the 39 participants. Before analyzing the results, we evaluated the reliability of the questionnaire, following the recommendations in [33] for questionnaires with a sample size between 30 and 100 participants. We ran the reliability test based on Cronbach’s Alpha coefficient [34] on each scale. In our case, running the test, we obtained values above 0.6, as recommended in [35,36]. More precisely: EU → α = 0.6, N = 3; PU → α = 0.8, N = 6; and IU → α = 0.7, N = 2. Thus, the preliminary results of this questionnaire were deemed reliable and were confirmed also by the researchers’ observations.
Table 3 shows the results of the questionnaire. The first column reports the item ID, the second the description, the third one the number of collected answers for each item, and the two last columns the mean and the coefficient of variation.
In general, all the items were scored with a high enough mean above 3, and, as Table 3 shows, all the coefficients of variation (CVs) were below 1. This means a relatively low variation for the distribution of the means.
The lowest item scores were given entirely for the EU scale and were closest to 3. This means that the participants generally considered extracting and using AI requirements neither easy nor difficult. In fact, at the end of the workshop, some students explained that they had initial difficulties in understanding the new approach. However, by collaborating with their team members and asking the teacher for explanations, they eventually were able to understand the right focus to adopt and use it profitably.
Indeed, the PU and IU scales show scores close to or higher than 4. They highlight that, although the new approach may have required an initial effort, the participants clearly saw its usefulness for their goals and would be willing to use it in future projects.

6. Conclusions and Final Remarks

In this work, we have presented an extension of the HCD approach to be adopted as a way to teach design students how to engage in the complexities of designing for human–AI interactions. In this paper, we have identified the need to rethink both the relationship between artificial intelligence and users and to consider the Human-Centered Design approach to focus from the beginning of design processes on the effects of AI on the user experience.
We also set up a case study supporting this focus based on a workshop that involved 39 HCI students on the MsC course in computer science at the University of Salerno.
The workshop highlighted how the change in perspective may require some minimum initial effort by designers, but that, at the same time, it may help them to focus on issues related to the relationship between AI and users and identify the desired user experience.
One of the issues that emerged during the workshop activities was how to evaluate a design against AI requirements without having developed a working prototype.
Therefore, we aim to establish a comprehensive evaluation approach that considers a system’s usability from the perspective of both human users and AI. To this end, we are developing a set of heuristics to evaluate the system’s usability, as well as creating scenarios to represent the viewpoint of AI within the system. Additionally, we are exploring the possibility of using static or scripted methods to represent the system’s interface and demonstrate its potential functionality, thus facilitating its evaluation.

Author Contributions

P.B., M.D.G., M.R., M.S. and G.V. contributed to the design and implementation of the research, to the analysis of the results and to the writing of the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by MIUR, PRIN 2017 grant number 2017JMHK4F 004.

Institutional Review Board Statement

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Battistoni, P.; Di Gregorio, M.; Romano, M.; Sebillo, M.; Vitiello, G. AI at the edge for sign language learning support. In The International Journal of Humanized Computing and Communication; IJHCC: Singapore; KS Press: Lawrence, KS, USA, 2020; Volume 1, pp. 23–42. [Google Scholar]
  2. Battistoni, P.; Di Gregorio, M.; Romano, M.; Sebillo, M.; Vitiello, G.; Solimando, G. Sign Language Interactive Learning-Measuring the User Engagement. In International Conference on Human-Computer Interaction; Springer: Cham, Switzerland, 2020; pp. 3–12. [Google Scholar]
  3. Stanton, B.; Jensen, T. Trust and Artificial Intelligence; Technical Report; NIST: Gaithersburg, ML, USA, 2020. [Google Scholar]
  4. Shneiderman, B. Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy. Int. J. Hum.–Comput. Interact. 2020, 36, 495–504. [Google Scholar] [CrossRef] [Green Version]
  5. Rowland, C.; Goodman, E.; Charlier, M.; Light, A.; Lui, A. Designing Connected Products: UX for the Consumer Internet of Things; O’Reilly Media, Inc.: Sebastopol, CA, USA, 2015. [Google Scholar]
  6. Jentzsch, S.F.; Höhn, S.; Hochgeschwender, N. Conversational interfaces for explainable AI: A human-centred approach. In International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems; Springer: Cham, Switzerland, 2019; pp. 77–92. [Google Scholar]
  7. Dafoe, A.; Bachrach, Y.; Hadfield, G.; Horvitz, E.; Larson, K.; Graepel, T. Cooperative AI: Machines must learn to find common ground. Nature 2021, 593, 33–36. [Google Scholar] [CrossRef] [PubMed]
  8. Amershi, S.; Weld, D.; Vorvoreanu, M.; Fourney, A.; Nushi, B.; Col-lisson, P.; Teevan, J. Guidelines for human-AI interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK, 4–9 May 2019; pp. 1–13. [Google Scholar]
  9. Wang, D.; Churchill, E.; Maes, P.; Fan, X.; Shneiderman, B.; Shi, Y.; Wang, Q. From human-human collaboration to Human-AI collaboration: Designing AI systems that can work together with people. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25–30 April 2020; pp. 1–6. [Google Scholar]
  10. Battistoni, P.; Romano, M.; Sebillo, M.; Vitiello, G. A Change in Perspective about Artificial Intelligence Interactive Systems Design: Human Centric, Yes, But Not Limited to. In HCI International 2021—Late Breaking Papers: Multimodality, Extended Reality, and Artificial Intelligence HCII 2021; Lecture Notes in Computer Science 2021; Springer: Cham, Switzerland, 2021; Volume 13095. [Google Scholar] [CrossRef]
  11. Kaluarachchi, T.; Reis, A.; Nanayakkara, S. A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning. Sensors 2021, 21, 2514. [Google Scholar] [CrossRef] [PubMed]
  12. Manikonda, L.; Deotale, A.; Kambhampati, S. What’s up with privacy? User preferences and privacy concerns in intelligent personal assistants. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, New Orleans, LA, USA, 2–3 February 2018; pp. 229–235. [Google Scholar]
  13. Ras, G.; van Gerven, M.; Haselager, P. Explanation methods in deep learning: Users, values, concerns and challenges. In Explainable and Interpretable Models in Computer Vision and Machine Learning; Springer: Cham, Switzerland, 2018; pp. 19–36. [Google Scholar]
  14. Yang, Q.; Steinfeld, A.; Rosé, C.; Zimmerman, J. Re-examining whether, why, and how human-AI interaction is uniquely difficult to design. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 25–30 April 2020; pp. 1–13. [Google Scholar]
  15. Eric, S.C.; Baumer, P.S.; De Choudhury, M. Who is the “Human” in Human-Centered Machine Learning: The Case of Predicting Mental Health from Social Media. Proc. ACM Hum.-Comput. Interact. 2019, 3, 147. [Google Scholar] [CrossRef] [Green Version]
  16. Myers, C.; Furqan, A.; Nebolsky, J.; Caro, K.; Zhu, J. Patterns for How Users Overcome Obstacles in Voice User Interfaces. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems; Association for Computing Machinery: New York, NY, USA, 2018; Volume 6, pp. 1–7. [Google Scholar] [CrossRef]
  17. Riedl, M.O. Human-centered artificial intelligence and machine learning. Hum. Behav. Emerg. Technol. 2019, 1, 33–36. [Google Scholar] [CrossRef] [Green Version]
  18. Musumba, G.W.; Nyongesa, H.O. Context awareness in mobile computing: A review. Int. J. Mach. Learn. Appl. 2013, 2, 5. [Google Scholar] [CrossRef]
  19. Battistoni, P.; Di Gregorio, M.; Romano, M.; Sebillo, M.; Vitiello, G. TactCube: Designing mobile interactions with Ambient Intelligence. In IFIP Conference on Human-Computer Interaction; LNCS; Springer: Berlin/Heidelberg, Germany, 2021; in press. [Google Scholar]
  20. Battistoni, P.; Sebillo, M. A tactile user device to interact with smart environments. In International Conference on Human-Computer Interaction; Springer: Berlin/Heidelberg, Germany, 2021; pp. 461–471. [Google Scholar]
  21. Norman, D.A.; Draper, S.W. User-Centered System Design: New Perspectives on Human-Computer Interaction; Lawrence Erlbaum Associates: Hillsdale, NJ, USA, 1986. [Google Scholar]
  22. Reddi, V.J.; Cheng, C.; Kanter, D.; Mattson, P.; Schmuelling, G.; Wu, C.-J.; Anderson, B.; Breughe, M.; Charlebois, M.; Chou, W.; et al. Mlperf inference benchmark. In Proceedings of the 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), Valencia, Spain, 30 May 2020–3 June 2020. [Google Scholar]
  23. Merton, R.K. Thematic analysis in science: Notes on Holton’s concept. Sci. Cult. 1975, 188, 335–338. [Google Scholar] [CrossRef] [PubMed]
  24. Clarke, V.; Braun, V. Teaching thematic analysis: Overcoming challenges and developing strategies for effective learning. Psychologist 2013, 26, 120–123. [Google Scholar]
  25. Romano, M.; Díaz, P.; Aedo, I. Empowering teachers to create augmented reality experiences: The effects on the educational experience. Interact. Learn. Environ. 2020, 1–18. [Google Scholar] [CrossRef]
  26. Cohen, J. A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 1960, 20, 37–46. [Google Scholar] [CrossRef]
  27. Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989, 13, 319–340. [Google Scholar] [CrossRef] [Green Version]
  28. Davis, F.D.; Bagozzi, R.P.; Warshaw, P.R. User acceptance of computer technology: A comparison of two theoretical models. Manag. Sci. 1989, 35, 982–1003. [Google Scholar] [CrossRef] [Green Version]
  29. Venkatesh, V. Creation of favorable user perceptions: Exploring the role of intrinsic motivation. MIS Q. 1999, 23, 239–260. [Google Scholar] [CrossRef] [Green Version]
  30. Van De Bogart, W.; Wichadee, S. Exploring students’ intention to use LINE for academic purposes based on technology acceptance model. Int. Rev. Res. Open Distrib. Learn. 2015, 16, 65–85. [Google Scholar] [CrossRef] [Green Version]
  31. Babakus, E.; Mangold, W.G. Adapting the SERVQUAL scale to hospital services: An empirical investigation. Health Serv. Res. 1992, 26, 767–786. [Google Scholar] [PubMed]
  32. Buttle, F. (Ed.) Relationship Marketing: Theory and Practice; Sage: Thousand Oaks, CA, USA, 1996. [Google Scholar]
  33. Samuels, P. Advice on Reliability Analysis with Small Samples; Birmingham City University: Birmingham, UK, 2015. [Google Scholar] [CrossRef]
  34. Cronbach, L.J. Coefficient alpha and the internal structure of tests. Psychometrika 1951, 16, 297–334. [Google Scholar] [CrossRef] [Green Version]
  35. Field, A. Discovering Statistics Using SPSS; Sage Publications Inc.: Thousand Oaks, CA, USA, 2009. [Google Scholar]
  36. Hertzog, M.A. Considerations in determining sample size for pilot studies. Res. Nurs. Health 2008, 31, 180–191. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Factors influencing the UX in an AI-based system. Elaborated from [10].
Figure 1. Factors influencing the UX in an AI-based system. Elaborated from [10].
Mti 07 00024 g001
Figure 2. The Decision Perception Model elucidates the process by which both human and artificial intelligences form their expectations regarding the decisions made by the other party. Elaborated from [10].
Figure 2. The Decision Perception Model elucidates the process by which both human and artificial intelligences form their expectations regarding the decisions made by the other party. Elaborated from [10].
Mti 07 00024 g002
Figure 3. Intelligence-Centered Design (ICD) extends the traditional Human-Centered Design process to incorporate AI requirements at the same level as users’ requirements. Elaborated from [10].
Figure 3. Intelligence-Centered Design (ICD) extends the traditional Human-Centered Design process to incorporate AI requirements at the same level as users’ requirements. Elaborated from [10].
Mti 07 00024 g003
Table 1. Some possible AI requirements and their rationales based on our research and analysis.
Table 1. Some possible AI requirements and their rationales based on our research and analysis.
AI RequirementsRationale
Interaction must be easy to learn by AIUser interaction must be adequate to facilitate the AI continuous learning process. This is to allow the network to improve by learning from mistakes, user requests, and context.
Interaction must be easy to be interpreted by AIThe user’s interaction with the AI must be designed so that it can be correctly interpreted by the AI, which can act in a way that best meets the user’s intent, rarely requiring a new learning process.
Interaction must have a recognizable contextDepending on the type of environmental and technological context, AI may require a specific type of interaction, such as voice, gesture, or text. This includes the way in which users can give feedback to AI.
The interaction must be correctly shaped to support single or multiple users Artificial intelligence may have to interact with a single user or with multiple distinct users at the same time or at different times.
The AI decision-making approach must be clearly establishedThe decision-making process can be:
  • Collaborative, that is, the intelligence and the user make a shared decision;
  • Fully automated, when the decision is made only by the AI;
  • User-dependent, when the decision is made by the user independently;
  • A mix of the previous options.
Table 2. The three main themes brought out by the student project discussion.
Table 2. The three main themes brought out by the student project discussion.
CategoryRationale
Focus on the AI roleThe category of AI requirements allows designers to focus on the role that AI can play within the project from the first stages of the design process. It may enable solutions with features that often are underrated at the initial stages.
It can also lead beginner designers with only a little knowledge of AI to work in this topic easily.
Benefits for functional requirementsThe formalization of AI requirements benefits the extraction of functional requirements.
Indeed, these requirements, describing the behavior and capabilities of AI, lead designers in the formalization of functionalities not considered before or in detailing them more.
A way to empower the user experienceBy focusing on the role of AI, its capabilities, and the way it relates to users, designers are guided to produce solutions that best meet user experience needs. This was considered by the participants to be the greatest contribution of this category of requirements.
Table 3. The results of the workshop questionnaire.
Table 3. The results of the workshop questionnaire.
ItemDescriptionN.M.C.V.
EU1Focusing on AI requirements is easy393.180.27
EU2Recognize the AI requirements among others is clear393.490.24
EU3Overall, I find the AI requirements easy to formalize393.210.27
PU1Formalizing AI requirements increases my attention towards the AI’s role.394.100.21
PU2Formalizing AI requirements helped me to deeper understand the interaction among users and AI.394.080.21
PU3Formalizing AI requirements makes generating ideas of AI-based interactive systems easier.394.050.21
PU4Using AI requirements, I think I made better use of artificial intelligence than I would have done otherwise.393.950.22
PU5Using the AI requirements, I believe I will get better designs of AI-based interactive systems.393.950.22
PU6Overall, I find the AI requirements formalization useful for designing AI-based interactive systems.393.950.22
IU1I would use the AI requirements in my designing activities.393.900.22
IU2I would recommend using the AI requirements to my colleagues.394.050.21
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Battistoni, P.; Di Gregorio, M.; Romano, M.; Sebillo, M.; Vitiello, G. Can AI-Oriented Requirements Enhance Human-Centered Design of Intelligent Interactive Systems? Results from a Workshop with Young HCI Designers. Multimodal Technol. Interact. 2023, 7, 24. https://doi.org/10.3390/mti7030024

AMA Style

Battistoni P, Di Gregorio M, Romano M, Sebillo M, Vitiello G. Can AI-Oriented Requirements Enhance Human-Centered Design of Intelligent Interactive Systems? Results from a Workshop with Young HCI Designers. Multimodal Technologies and Interaction. 2023; 7(3):24. https://doi.org/10.3390/mti7030024

Chicago/Turabian Style

Battistoni, Pietro, Marianna Di Gregorio, Marco Romano, Monica Sebillo, and Giuliana Vitiello. 2023. "Can AI-Oriented Requirements Enhance Human-Centered Design of Intelligent Interactive Systems? Results from a Workshop with Young HCI Designers" Multimodal Technologies and Interaction 7, no. 3: 24. https://doi.org/10.3390/mti7030024

Article Metrics

Back to TopTop