Smartphone Moves: How Changes in Embodied Configuration with One’s Smartphone Adjust Conversational Engagement

Abstract

Smartphones are often spontaneously used for personal purposes and during face-to-face gatherings. New terms like “phubbing” and “technoference” describe negative consequences of this behavior, but analysis of the actual everyday social situations where smartphones feature has largely been neglected. This article shows how simultaneous smartphone and conversational engagements are shaped by participants’ embodied conduct. A naturally occurring three-party conversation in a Finnish café is analyzed in detail to show how changes in embodied user–smartphone configuration impact ongoing conversation. User–smartphone configuration consists of the smartphone’s location, its physical relation to its user’s hands, and its screen direction in relation to the user’s head. User-smartphone configuration can manifest a change in an interactive footing in conversation, function as a turn-holding device, and organize a change in the conversational state. New methods and concepts for studying smartphone use in social situations are introduced. “Smartphone positions” refers to the embodied user–smartphone configurations that are oriented as manifestations of degrees of user–device engagement. “Smartphone moves” are the changes in smartphone positions, and they carry sequential relevance. Increased smartphone engagement is seen as decreased conversational engagement and vice versa. Making interactive resources available for one engagement manifests as an accountable event of disengagement from another. Engagement and disengagement are argued to be a continuum rather than a contrast pair.

1. Introduction

Smartphones are some of the most central objects in people’s lives today. They are used to communicate with others, to get where one needs to be, and to shop for other items and services. These, and a plethora of other smartphone activities, are the reason behind over 200 billion application downloads in Google Play and Apple’s App Store in a year (Perez 2020). Is it a surprise, then, that the average U.S. adult spends three hours daily on their smartphone, a 47% rise in five years (eMarketer 2019)? Much of this use takes place in the presence of others, and questions about the politeness of smartphone use in social situations have gotten a lot of media attention (Ducharme 2018; Molina 2017). Research and publicity around terms like “phubbing” and “technoference” are examples of academic attention on the topic, but they are limited by presumably unintended negative consequences of smartphone use at the statistical level (Dwyer et al. 2018; e.g., Molina 2017; Roberts and David 2016; Rotondi et al. 2017), therefore neglecting, as Goffman (1964) would have put it, the social situation in which the phenomenon actually resides. Ayaß (2014) adapts Goffman’s (1963, pp. 39–40) concept of “involvement shield” in suggesting that mobile phone users may embody cues that convey their unavailability for interaction (cf. also Hampton et al. 2010). The capacity of smartphone use to modulate interactional availability is further supported by Pew Research Center’s survey (Pew Research Center 2015), which found that most Americans have used their phones specifically to avoid face-to-face interaction, and a third have done so during their most recent social gathering. If smartphones are able to serve such purposes, there must be specific embodied user–smartphone configurations affording this. That is to say, some specific ways of engaging with a smartphone must embody stronger engagement with the device than with others (cf. Figure 1).

1.1. Engaging with a Smartphone vs. a Human Participant

Objects may feature in interaction as a means to personal aims or as semiotic fields in constructing the contextual configuration of that moment (Goodwin 2000). Smartphones, however, are not only objects of individual engagement (Roberts and David 2016) nor semiotic fields (cf. Raclaw et al. 2016), but can form with their users interactive trajectories with specific actions and reactions, becoming relevant at specific moments (cf. Arminen 2007; Figeac and Chaulet 2018; Mantere and Raudaskoski 2017). Therefore, smartphones afford interactive involvement that, to a degree, resembles human-to-human interaction. Smartphone use taking place in a face-to-face gathering has even been likened to Goffmanian cross-talk (Ictech 2019), “a conversation or conversation-like activity maintained by persons who differentially share other interaction capacities” (Goffman 1971, p. 25). By “differentially shared” interaction capacities, Goffman meant that a member of the cross-talk interaction is bystander to a primary face-to-face engagement the other member is involved in. In other words, cross-talk means being part of a face-to-face engagement and temporarily interacting with some co-present other who is not part of that same encounter. Ictech’s concept of digital cross-talk builds on Humphreys’ (2005) “cellphone crosstalk” and relies on Goffman’s inclusion of “conversation-like activity”1 in the original definition (Ictech 2019, p. 28). Humphreys and Hardeman (2021, p. 112) later do the same with the concept of “mediated cross-talk”, all then overlooking the fact that in cross-talk the bystander, acting as a cross-talk partner, is still member of the same social situation as the participants of the primary face engagement. The presence of the bystander, and at least some aspects of the cross-talk interaction, are therefore perceivable to everyone present (Goffman 1964, p. 135). Smartphone activities, however, can even consist of hidden interactions between participants in the same co-present group (Tjora 2011), and can also be something completely different than social interaction (adjoe GmbH 2021). Unlike non-digital objects, which rarely afford more than one type of activity to be done with them, smartphone is a metamedium that works as a gateway for other media (Jensen 2016), typically not conveying the exact nature of its use to others (Mantere et al. 2021). This makes smartphones in face-to-face encounters a special research topic, not fully explained by previous research on other objects in face-to-face encounters and not optimally conceptualized, with too much reliance on concepts developed before the invention of smartphones.

1.2. Smartphones and Multiple Involvements

Main and side involvements were defined by Goffman (1963, pp. 83–148) using an very different example from that of a smartphone situation: talking (side involvement) while waiting in a line at the bank (main involvement). Involvement as a category supersedes engagement, like encounter supersedes conversation. Analyzing conversations and engagement requires access to the interactive resources used by the interactants themselves, but neither digital cross-talk (Ictech 2019) nor mediated cross-talk (Humphreys and Hardeman 2021) were developed with video or transcriptions of the actual social situations as their data. They therefore have limited capacity to take into account the minute details of smartphone engagement in social situations or to offer tools for analyzing actual moments of smartphone engagement during face-to-face encounters. Engaging in face-to-face interaction with a bystander during another face-to-face involvement, i.e., Goffmanian cross-talk, differs from engaging with a smartphone in face-to-face settings. Bystander inaccessibility (Mantere et al. 2021) shrouds the nature of face-to-screen interaction from others, at times shrouding even whether the smartphone is used for interaction at all, or just for personal information seeking or entertainment. Concepts of multiple involvements and varying intensities of engagement are therefore more useful than cross-talk. Smartphone involvement taking place within a face-to-face encounter gives birth to a simultaneous but separate digital participation frameworks (cf. Hampton et al. 2010). It is not an overlap of two participation frameworks like cross-talk is. It is separate, because without a specific account, the members of the face-to-face participation framework do not know whether the smartphone involvement they witness even constitutes a social interaction, and previous research suggests it to be relevant to them (Mantere et al. 2021). Likewise, the possible other members of the digital participation framework do not have awareness of the face-to-face participation framework unless specifically made aware of it (cf. Weilenmann 2003). This amounts to a type of interaction somewhere in between Goffmanian categories of focused interaction, unfocused interaction, and multi-focused interaction (Goffman 1963, pp. 83–148), but not entirely in any of them. The ambiguous presence of separate participation frameworks in smartphone situations further calls for new research that does rely not too much on pre-existing conceptualizations.

1.3. Interacting with Smartphones

Walsh and Clark (2019) argue that smartphones disrupt what they call the “socialized trance” of face-to-face interaction, whereas Licoppe (2004) argues that already traditional mobile phones brought about a “‘connected’ management of relationships” where the expectation of continuous mediated availability brings distant participants present through assumed connectivity in a way not possible before the proliferation of such technology (cf. also Ling 2008, 2012). Henriksen et al. (2020), through observations and interviews in cafés, found smartphones to be mainly present at the beginning of the encounter, e.g., for posting photos of food on social media, and at the end of the encounter, e.g., for searching for transport options to the next destination. Checking the phone for urgent messages or calls, showing photos to others, disengaging from someone who used their phone too much, and contagious using of one’s phone because others were using theirs, were also found to be common. Some previous research suggests that the opportunities for disengaging from smartphone use are at least partly related to the phase of the course of activity being advanced with the device. Laurier et al. (2016) studied collaborative navigation with map applications and found user–smartphone interaction to have its own sequential progression, which was interleaved with that of the social interaction. Mantere and Raudaskoski (2017) coined the concept of “sticky media device” to depict situations where interleaving user–smartphone interaction and social interaction lacks fluidity and results in silences, hesitation markers, and other dispreferred turn signs, even during preferred sequences. User–smartphone interaction and face-to-face interaction may progress in separate sequential rhythms, and the sequential organization of one may at times encumber on the other. Licoppe and Figeac (2018) described how smartphone use while driving is interleaved with traffic light stops. They found user–smartphone interaction to be structured in transition relevance places (TRPs), similar to human interaction. TRPs are the moments delineating turn constructional units (TCUs). In human interaction, a TRP is a moment in which the current speaker may change without it being interpreted as an interruption. Similarly, Figeac and Chaulet (2018), using eye-tracking technology to study smartphone use in public transport, found that the gaze shifts away from the smartphone to be organized in relation to the sequential progression of the user–smartphone interaction, as well as the events in the physical environment. These studies support the view that engaging with one’s smartphone may form interactive projects whose expectations for user participation do not always temporally align with the expectations of participation in the current social situation. This does not, however, mean that user–smartphone activity during face-to-face gatherings would necessarily at any point take all of one’s interactive capacities. Tjora’s (2011) interviewees used text messages for multiple purposes to discreetly interact with others in the same physical space while keeping this interaction hidden to the rest of the encounter. DiDomenico et al. (2018) described how a participant in three-party face-to-face conversation can attend to an incoming text message while remaining sufficiently aware of the unfolding storytelling to later produce a relevant smile or utterance (cf. also Didomenico and Boase 2013). Human capacity to attend to several simultaneous courses of action is also depicted in Haddington et al.’s (2014) concept of “multiactivity.” On the other hand, over 100 years of experimental research point towards our inability—without a significant reduction in efficacy—to attend to two or more simultaneous activities necessitating conscious evaluation (Wu and Liu 2008). Therefore, social engagement or availability for interaction should not be assumed to be binary categories, either. It is much more likely they manifest in degrees.

1.4. Embodied Engagement in Social Interaction

Displays of engagement have long been recognized to play a crucial role in manifesting one’s relation to the ongoing talk, and through this, also to be constitutive features in the realization of speakership and hearership (Goodwin 1981, p. 125). Goodwin and Goodwin (2005) propose the perspective of ongoing participation, including non-vocal participation, to revise Goffman’s (1981) classic concept of footing. Goffman focused on different persons and their roles within the categories of speaker and hearer through linguistic alignments and forms such as reported speech. In interactive footing, the different roles in a framework of participation “are constituted not only through talk, but also through participation as a dynamically unfolding process” (Goodwin 2006, p. 37). Embodied participation through different forms of engagement has also been classified. Schegloff (1998) conceptualized the social implications of physical configurations in simultaneously attending to multiple activities. Whether the activities were exclusively physical, or also social in nature, there were recognizable and socially influential, qualitatively different forms of orientation toward those engagements. Though the number of ways one may position their body in relation to multiple engagements is surely unlimited, Schegloff recognized three rotational axes to have distinctive relevance: the hips, the shoulders, and the neck (see

Table 1. Configurations of embodied engagement in Schegloff’s (1998) body torque classification.

Direction 1	Direction 2	Direction 3

Note: the feet icon refers to the orientational axis of the waist, the body icon to the shoulders, and the smiley face to the head.

). Body directions below and above these axes embody projected durations and hierarchy of the involvements and the axes afford three possible directions for simultaneous orientation.

Use of one’s body and objects, just like the use of eye gaze, may organize several aspects of social interaction, like the practices of suspending or alternating between activities (Haddington et al. 2014; Nevile et al. 2014; Rossano 2012). Engagement and action are not separate, but reflexively construct one another (Goodwin 2006). A general classification of forms of smartphone engagement in relation to conversational engagement and conversational activity has not previously been attempted. Today, smartphones afford nearly ubiquitous opportunities for engaging in activities and participation frameworks separated from the ongoing face-to-face situation (cf. Mantere et al. 2021). This article analyzes spontaneous smartphone engagement in naturally occurring interaction to show how it is timed to not overlap prominent positions in face-to-face conversation, how different positions of user–smartphone configuration manifest different levels of engagement, and how moves from one position to another are oriented to as adjustments in the degree of face-to-face engagement. It answers a need for more interaction research on smartphone use in social situations in general (cf. Walsh and Clark 2019) and conversation analytics research on smartphone engagement in face-to-face settings in particular, and opens new pathways for future research on such mobile digital metamedia devices in collocated encounters.

2. Materials and Methods

2.1. Conversation Analysis

Data collection and analysis followed established procedures in ethnomethodological conversation analysis (CA). CA is a collection of methods and cumulative findings in the study of conversational organization, also encompassing embodied interactions not constituting a conversation per se. CA seeks to reveal and describe the methods participants themselves use in producing and understanding social realities in naturally occurring everyday encounters. CA therefore responds to the challenge of ecological validity by only studying non-motivated social situations directly, not organizing enactments of social behavior in a laboratory or prompting participants to imagine social situations while answering a survey. Participants’ own orientations towards the events taking place in the encounters are described through turn-by-turn analysis, paying attention to what is said and how it is said, as well as all of the embodied non-vocal behavior. A detailed transcript of the interaction is prepared to aid the analysis. A fundamental observation guiding the analysis is that talk organizes into turns. In everyday interactions, turns are not predestined, but are negotiated through vocal and non-vocal conduct as the encounter unfolds (Have 2011; Heritage 1984; Sacks et al. 1974). Typically, only one person speaks at a time, and both overlapping speech and silences are minimized. When arriving to a moment of possible turn completion, a transition relevance place (TRP) and another speaker may take the turn, and if that does not happen, the speaker may continue (Sacks et al. 1974).

Turns in interaction also organize into recognizable sequences. Encounters often begin with greetings, may move into exchange of news, and end with a sequence of goodbyes. Sequences often further organize into pairs of turn-types, or adjacency pairs, where a certain type of first turn produces an expectation for a certain type of second turn. A greeting is expected to be followed by another greeting, not a farewell or a request. Moreover, there are recognizable preferences related to turns. An invitation is typically expected to be followed by an answer and not a greeting, and moreover, it is expected to be accepted rather than rejected. The “expectation” here, however, does not reflect any assumptions of private cognitive states of participants. It merely points out that interaction is typically produced “as if” acceptation was expected to follow an invitation. This is an institutionalized aspect of interaction affording concerted and mutually intelligible social action, not a psychological claim. Preference organization may be observed, for instance, in acceptations of invitations typically not being accompanied by delays and explanations like rejections are (Schegloff 2007).

2.1.1. Storytelling Sequence

The typical distribution of turns in interaction creates the expectation that exceptionally long turns require some special work. Sequences of spoken narrative form this kind of exception. In a storytelling sequence, the typical turn-taking organization is relinquished and the teller of the story may continue through multiple TRPs without orienting to a possibility of a speaker change. This continues until the story is completed. Storytelling sequence affords the recipients to the story to give their consent in entering this special arrangement by including a slot for a story preface that precedes the story. Here, some form of “go ahead” signal is expected by the story recipients and hints are usually provided about the nature of the story, enabling them to easily give appropriately affiliative responses as the story starts to unfold. The “punch line” or climax of the story is made to stand out enough for the recipients to offer their reactions at the right time. The stance the teller themself has on the climax is also often displayed in the way they produce it (Mandelbaum 2003).

2.1.2. Transcription in Conversation Analysis

CA prides itself as a highly empirical enterprise, and the recordings of social encounters used as data and evidence are accurately transcribed. Any vocal or non-vocal resources available to the interactants in the production and understanding of interactive events should ideally also be available to the analyst and their audience through the transcripts. Accurate transcripts therefore play a crucial role in deciphering and showcasing the methods the participants use in the construction and interpretation of these events. CA transcripts aim to accurately represent not only the words that are spoken, but also the way they are spoken, as well as all other uttered sounds, like coughing and audible inbreaths and outbreaths. Similar accuracy is aimed at transcription of non-vocal conduct. Gail Jefferson (cf. Jefferson 2004) developed the first transcription system in CA during the 1960s and 1970s, at a time when most data extracts were of English-language phone calls. This excluded the need for transcription of non-vocal conduct and translations. In Jeffersonian transcription, each line of transcribed vocal conduct is designated with its own line number, as well as when it overlaps with vocal conduct in the previous or following line. The transcripts in this article, however, assign numbers to transcribed segments2 of co-operative multimodal conduct. It reflects the author’s Goodwinian stance against logocentricism in interaction research (Goodwin 2006, pp. 25–28), and also makes representation and referring to the transcripts more efficient and intuitive. The transcribed vocal conduct, non-vocal conduct, translations, and overlapping conduct of other participants are all organized and referred to by the same segment number whenever they represent concurrent events. When drawings of the encounter are provided, they always occupy the first line of the segment.

2.2. Data

A corpus of 13 video recordings with a total length of 11 h was collected from naturally occurring everyday encounters in Finland, France, and California, U.S., between 2017 and 2019. Participants were approached in cafés, parks, campus dining halls, bars, terraces, and in a line for an outdoor public bathroom. The participants’ ages varied from 17 to their late 20s. Ten out of 13 encounters involved smartphone use. After the participants agreed to partake in the study, the researcher departed and left behind a camera on a small tripod. The recordings were viewed repeatedly while writing down preliminary observations on forms of smartphone engagement. These viewings and notes formed the basis for the classification of commonly occurring embodied user–smartphone configurations, i.e., the smartphone positions (Figure 2). Initial observations hinted towards changes in these configurations not appearing randomly. A 97 min encounter among three Finnish high-school students was then chosen for more precise analysis in order to study the potential roles these changes might have in the organization of interaction. The encounter was chosen for its ample conversations, several changes in the themes of conversation and states of interaction, varying ways of smartphone use by its participants, and the relative ease of distinguishing from the recording what was said and who said it.

The 97 min recording was transcribed in two levels of accuracy: (1) simplified transcripts for sections where the role of smartphones was clearly limited, simple, and straightforward, and (2) accurate transcripts for sections where smartphones were present in more relevant and varied ways (see Appendix A). The simplified transcripts were organized into 40 intervals of two and half minutes. Smartphones were engaged with, by looking at and touching the screen, in 38 out of the 40 two-and-half-minute intervals. The only intervals where smartphones were not engaged with were the five minutes soon after the beginning of the encounter, during which the participants focused on eating and talking. Transcribed data excerpts were viewed in several data sessions with colleagues in the U.S. and Finland.

2.3. Ethical Considerations

The research did not include harming the participants in any way. Visual representations of the data, participant names, and other recognizable details were anonymized. The national laws and ethical guidelines regarding underage participants were followed. According to the Finnish National Board on Research Integrity (2019), participants aged above 15 are given the right to independently decide on participation in non-dangerous research. The participants between ages 15 and 18 were all in Finland. The participants were informed that the video recordings are part of a data collection on forms of social life in today’s societies. Participants who chose to partake in the study were informed of the more precise research topic, i.e., smartphone use during face-to-face encounters, after the recording. Participants were at this point offered the option to retract their participation without having to give any explanation. Participants were also given the researcher’s contact information to retract their participation later. Participants in all groups expressed understanding as to why the exact topic was not revealed beforehand. No participant retracted their participation. The data were kept in a secure locked location and protected by a password.

2.4. New Classification and Transcription System for Smartphone Use in Social Situations

To accurately capture the role of smartphones in face-to-face encounters, a new transcription system was developed. This system takes into account how smartphones differ from other everyday objects through the interactive nature of their use (Figeac and Chaulet 2018; Licoppe and Figeac 2018; Mantere and Raudaskoski 2017) and the bystander inaccessibility they incite (Mantere et al. 2021). The system affords an accurate recognition and study of the levels of embodied smartphone engagement during face-to-face gatherings. Smartphone positions describe the user–smartphone configurations that embody the physical relationship between a smartphone and its user. The concept has kinship with Schegloff’s “body torque” (Schegloff 1998), which depicts levels of embodied engagement towards two or more simultaneous activities. Though defining embodiment of engagement in multiple involvements through the three axes of the head, shoulders, and hips, Schegloff did not suggest the feet, hands, or eyes to be interactionally irrelevant. He merely described body torque at the head, shoulders, and hips as a level of embodied organization of interactive involvement that has distinct functional unity and coherence. Smartphone positions are similar. Both body torque and other embodied interactional conduct, such as eye gaze, should be transcribed and analyzed in any episodes where they feature as sequentially important. The same applies for smartphone positions. They do not in any way make eye gaze irrelevant in relation to the smartphone screen. Nor do they make the swipes and taps done on the screen’s surface irrelevant. Meanwhile, no transcript or analysis can ever report to an equal degree on all of the embodied and vocal conduct that have a role in the construction of social realities. This does not, however, mean that new systems of grouping and classification of interactionally relevant conduct could not facilitate analytical understanding. After repeatedly viewing and taking notes on the data, with the focus on how smartphones were engaged with, three parameters of embodied smartphone engagement were recognized as central4:

• Location of the phone (e.g., pocket, bag, hands, table, or another surface);
• Direction of the phone’s screen in relation to the user’s head or the surface the phone is resting on;
• Number of hands holding the phone or held in front of the phone.

The parameters form 13 typical smartphone positions (Figure 2). Occasionally, the data also feature phones in empty beer glasses, inside a bra, resting on a leg, or being held by one’s feet. Those positions may also manifest as interactionally distinct in the episodes they feature in, but due to their rarity, it would be impractical to include them in any standard system. They may be added when needed.

The majority of the 13 positions are self-explanatory (Figure 2). In TableUp (TU) the phone rests on the table, with the screen pointing upwards. In TableDown (TD) the screen points down. In the HandFace (HF) position the user is holding the phone with one hand while the screen points towards the user’s head—whether the user’s head points towards the screen or not. The phone may therefore be moved in and out of the HF position without the user ever gazing anywhere near the phone. In HF, the screen “gazes” at the user, whether the user gazes back or not.

Even small changes to the screen’s direction might be sequentially relevant. Therefore, when the user relaxes their hand and the screen no longer points directly towards the user’s head, the phone is said to be in a HandAway (HA) position. The difference between HandFace (HF) and HandAway (HA) is at times hard to tell. In some HA, the screen may still point towards the general direction of the user’s head, and the user may continue to have visual access to the screen. The border cases are solved by looking at the smartphone position through time, and in relation to using the phone. If the smartphone’s screen is operated upon and looked at and the user then turns away and then turns back towards the screen in order to re-engage with the device without needing to adjust the screen’s direction, the phone is said to have been in the HandFace (HF) position the whole time. This kind of episode, often occurring in the data, also makes the HandFace (HF) position a temporary “home position,” a position from which bodily behavior is begun and to which it returns (Sacks and Schegloff 2002).

The BothHands (BH) position has the most interactive resources allocated to be available for user–smartphone interaction (cf. DenBleyker 2010). The phone points towards the user’s head and is held by both hands while both hands are in an equal position to manipulate the screen. When the screen is offered to be viewed by someone else, it usually is a case of 1-handedShow (1S). It is also possible to show the screen to others with both hands (2-handedShow = 2S). When both hands are relaxed in the BothHands (BH) position and the screen is allowed to point away from the user’s head, it is called Both-handsAway (BA). This, like 2S, appears to be rare. Another less common position is the Chest (CH) position. In CH the screen is made inaccessible to both the self and others by burying it into one’s chest. In TableManipulate (TM) the phone is on the table, screen up, and a hand is placed in front of the screen, in a position to manipulate it.

Sometimes the positioning and the use of the second hand made demarcation between BothHands (BH) and HandFace (HF) difficult. This was the case especially when users touched the screen intermittently with both hands, but the hands still also clearly had different main roles. For consistency, HandFace (HF) is defined to cover only those cases in which the same hand that holds the phone is the only hand used to manipulate the screen. Ambiguities between BothHands (BH) and HandFace (HF) were solved with the recognition of HoldManipulate (HM) as a separate category, similar to that of TableManipulate (TM). HoldManipulate (HM) covers the cases in which one hand is dedicated to holding the phone and the other to manipulating it by positioning it in front of the screen, with the palm towards the screen. HoldManipulate (HM) differs from BothHands (BH), which enables both hands to equally, though at times alternatingly, manipulate the screen and hold the phone.

Some smartphone positions are not qualitatively comparable with others. They do not always differ from each other only in the degree of engagement they manifest on a qualitatively coherent axis. Positions like 1S and 2S invite others to participate in viewing the screen. In the PC position, the presence of the smartphone may be completely hidden from others, but it may still silently communicate with its user through vibration. The level of engagement embodied by each smartphone position in relation to other positions is not yet comprehensively known.

2.5. Smartphone Moves

Changing a smartphone position is called a smartphone move. Turning around a phone lying on the table is a TableDown (TD)–TableUp (TU) move. Releasing one hand from a smartphone while still engaging with it is a BothHands (BH)–HandFace (HF) move. Grabbing a phone that is lying on the table screen downwards and pointing the screen towards one’s face is a TableDown (TD)–HandFace (HF) move. If each smartphone move should be studied separately, the sheer number of them would pose a challenge. With the 13 smartphone positions defined above, the sum of possible smartphone moves adds up to 156 (see Appendix B). If inverse moves (e.g., TD–HF and HF–TD) were not counted separately, 78 moves would still remain. These large numbers also discourage including rarer positions in the standard classification. Adding just one more position would increase the sum of possible smartphone moves by 26, or by 13 if inverse moves were not included. These large numbers also make studying smartphone engagement differ drastically from the embodied multiactivity depicted by Schegloff (1998). His embodied orientational configurations resulted in only 20 possible moves between the three possible directions (see Appendix B). This is far fewer than the 182 smartphone moves proposed in this article. Other differences also apply. The hierarchy of Schegloff’s bodily axes is important for his analysis: Waist direction indicates the engagement treated as the most permanent, shoulders indicate engagements of mediocre duration, and head direction marks the most passing involvement. Smartphone moves only depict the regulation of two engagements: smartphone interaction and face-to-face interaction. In addition, the availability of body torque as an interactive resource is likely even more ubiquitous than that of smartphone moves, and the use of body torque does not necessitate an object like smartphone moves do. Body torque nevertheless sets precedence in the classification of embodied multiple involvements. The above numbers also speak to the challenge of studying each of the smartphone moves individually before creating a transcription system for them. No social significance should be projected to any smartphone move before empirical research, but the data as a whole suggest that smartphone moves in general do not occur as sequentially random or inconsequential. This article begins the work of studying smartphone moves by examining sequential implicativeness of a few of them and developing a transcription system to facilitate these analyses.

3. Results

The results explore three ways smartphone moves feature in conversation. Smartphone moves may construct shifts in the participation framework towards interactionally fewer involved roles, or they may work as turn-holding devices and take part in organizing a shift in the conversational state of the encounter. Many other ways of using smartphone moves surely exist. What they seem to have in common, however, is that they embody adjustments in engagement. The results depict how smartphone engagement is treated as conversational disengagement, and how the adjustment of smartphone engagement by a smartphone move may impact ongoing conversation.

3.1. Smartphone Move as a Change of Footing

Clo, Liz, and Deb (names changed) meet in a café shortly after the end of the semester. Throughout the afternoon they engage both with each other and with their smartphones. Sometimes the phone use is connected to what they are talking about and sometimes not. They occasionally disclose to each other the reason for using the phone but not always. The first extract takes place 15 minutes after arriving at the café. They have just finished eating their cakes and are talking about different kinds of being drunk. They each continue the previous speaker’s story with one of their own. Clo is telling about an experience she calls “the night of the lizards.”

Extract 1

Throughout segments 1–3 Clo is telling a story while holding her phone in the HandFace (HF) position. At the end of the turn, while uttering “shaking and I’m all like-,” she produces a corporal description of her past shaky state. The cut off last vowel (“like-”) guides others to seek the completion of her turn from the semiotic field of her body rather than that of the verbal modality (cf. Goodwin 2000). The extreme case formulation “all like-” (Pomerantz 1986) and the enacted shaking hint to this possibly being the climax of the story (cf. Mandelbaum 2003). A mere 0.1 s later Clo begins a HandFace (HF)–BothHands (BH) move, making both her hands available to interaction with the smartphone. Similarly to how withdrawing the gaze away from one’s interlocuters may participate in forming an end to a course of action (cf. Rossano 2012), Clo’s HandFace (HF)–BothHands (BH) move shifts her multimodal participatory ensemble towards less engagement in face-to-face interaction, and at a moment of possible storytelling completion, reflexively constructs her face-to-face activity of drunkenness-describing as finished. The move therefore contributes to the construction of the very same activity completion it partly derives its meaning from. With the HandFace (HF)–BothHands (BH) move in this particular sequential environment, Clo leaves behind the status and any claim to the position of principal speaker, releasing it to be available to others without the risk of being seen to interrupt.

3.2. Smartphone Move as a Turn-Holding Device

The next extract takes place 43 s after the previous one. Clo has been in the BothHands (BH) position the whole time. Liz has been telling about a humorous encounter with her neighbor. When Liz finishes, Clo starts a story of a ride home from a high-school party.

Extract 2

At segment 1, Clo begins a storytelling sequence with a typical story initiator: “I remember.” She assumes the role of the principal speaker also by decreasing her smartphone engagement with a BothHands (BH)–HandFace (HF) move at the end of her story preface (“the ((anonymized)) par^BH-ty?^HF”). The move releases some of her interactive resources, namely, one of her hands, from being assigned available for face-to-screen engagement to being assigned available for face-to-face interaction. Liz and Deb take positions as the story recipients by silently gazing at Clo and letting her speak.

Clo tells about the alcohol she drank before being picked up from the party, setting the scene for the encounter with her father. Deb provides typical minimal input with repeated nodding nearing the TRP at segment 3. Liz acts differently. Previously in the encounter she had produced long overlapping turns on top of Clo and Deb, making the speech already projected by them left unproduced. Now Liz produces a snapping sound with her mouth (segment 3), followed by an animated “@Ye:ah@” at segment 4. This differs from Deb’s nodding or other alternatives Liz would have had, like “Yeah?” or “Mhm?” Liz’s “@Ye:ah@” is highly animated. It is accompanied by her eyes gazing up and performing a roll from right to left, her facial muscles tensing into a mouth-open smile, and her head moving rapidly from side to side. Her back and neck straighten to a more alert posture. Her utterance is also vocally striking. Though lasting only 0.3 s, it sounds long in comparison to the surrounding speech. The pitch starts at 221 hertz, drops rapidly after 0.04 s to a relatively long 0.14 s “oo” (in the original Finnish), steadily descends from 173 Hz to 151 Hz, rises back to 200 Hz for 0.03 s, and finishes as a steadily arced 0.09-s “o” at around 236 Hz.

Clo’s “from-somewhere” (jostai in the original Finnish) at segment 3 sounded much like “s’m’thing like” (just j’tai). It is possible that Liz misheard it. As response to “s’m’thing like” (just j’tai), Liz’s “@Ye:ah@” would verify the specialness of the describable hinted by Clo’s usage of the term just (the colloquial version of the Finnish juuri, which translates to “just”, “right”, “exactly”, or “very” in English). Whatever the reason, Liz’s “@Ye:ah@” treats the setting of the scene as having already introduced an element of interest inviting commentary from the story recipients—a treatment starkly different from that of Deb’s.

Clo begins a HandFace (HF)–Chest (CH) move overlapping Liz’s “@Ye:ah@” and rapidly rushes through the words “>and then went into] the car<” to continue the story. Quickly initiating the turn constructional unit with an “and” connector marks the just-preceding TRP as an unsuitable moment for a speaker change. This is made evident also by the speed of Clo’s utterance. She spent 1.3 s on the eight syllables of “from-somewhere like cider and such” before Liz’s animated “@Ye:ah@,” but compresses the following eight (in the original Finnish) syllables of “and then went into the car“ to a mere 0.5 s. Concurrent with the rush through (Walker 2010) the HandFace (HF)–Chest (CH) move, which reduces the allocation of interactive resources available to user–smartphone involvement, constitutes a multimodal turn-holding device intensifying Clo’s engagement in face-to-face interaction, thus helping to hold on to speakership. In the HandFace (HF) position the screen of the device is pointed towards Clo’s head, making the visual modality clearly available as a resource in the smartphone involvement. In the Chest (CH) position the screen is hidden in her chest. The interactive resources available for face-to-screen interaction are thus diminished by the HandFace (HF)–Chest (CH) move. Simultaneously, the move achieves an increase in the interactive resources available exclusively in the face-to-face involvement. Disengagement from the smartphone is, however, not total. It need not to be in order to be sequentially significant. A smartphone in the Chest (CH) position still remains in a closer relationship with its user than a smartphone in the TableDown (TD) position would, let alone a smartphone in the Pocket (PC) or Bag (BG) positions. Smartphone engagement is thus adjusted by the move, but not completely transformed.

Through Clo’s rushed speech a space is created wherein Liz, were she to continue to vocalize, would do so in overlap to Clo’s turn. This would take place far from a TRP and could in a storytelling context be easily seen as an interruption. The smartphone move, for its part, participated in the construction of Clo as the current speaker due to the accountability and intelligibility of the decrease in the interactive involvement with one’s personal device in such a sequential environment. The position of a hearer, rather than a speaker, is manifestly proffered to Liz, who, directly following the enactment of Clo’s turn-holding device, assumes it with silence, fixes her gaze towards Clo, and takes on a facial expression of focused seriousness (cf. Schegloff 1987; Walker 2010).

3.3. Smartphone Moves and Organizing the Conversational State

The next extract takes place soon after the previous one. Clo has just finished her story about the ride home from the high-school party. Liz now comments on the story.

Extract 3

At segments 1–2, Liz abstracts a generalized lesson from Clo’s story and Deb voices her agreement (“so true,” segment 1). Clo then affirms Liz’s interpretation at segment 2 (“[>exactly<] and”) by overlapping Liz at the earliest opportunity, just before the end of Liz’s turn. By adding “and,” she projects to continue speaking, which she then does with “then also that the thing that my-” in moment 3, producing indexical expressions pointing to referrals yet to come. This projects further continuation of her turn. Considering the prevailing sequential environment, i.e., telling stories, Clo seems to be embarking on a spinoff story to her previous one, suggesting a conversational organization giving her the right to hold the floor through multiple TRPs. If affirmed by others, they will normatively be restricted to positions of story recipients, expected to give Clo their attention, but to produce little speech themselves (Mandelbaum 2003; Stivers 2008). Gaining a right to tell a story and have others position themselves as story recipients also includes an expectation to tell it. A story prefaced, gaining an audience but then not told, would go against the preferred sequential unfolding of the situation (Mandelbaum 2003). However, the progression of Clo’s vocal conduct is much less nimble than projected by her rapid and overlapping turn beginning. It includes hesitations, a cut-off phrase, a long pause, filler words, and a smacking of the mouth and a hesitating “umm” sound in the middle of the turn, in places not forming TRPs (segment 3).

3.3.1. Sticky Media Device and Lack of Teller Engagement

These delays in progressivity in Clo’s turn share characteristics common with dispreferred turns, but nothing in the turn’s sequential placement or content suggests it to be so (Pillet-Shore 2017). Rather than relating to the verbal environment, these elements seem to form a case of sticky media device (Mantere and Raudaskoski 2017), i.e., dispreferred turn signs connected to advancing interactive projects in two simultaneous but separate participation frameworks: (1) the face-to-face interaction with Liz and Deb, and (2) the face-to-screen interaction with her smartphone (cf. Hampton et al. 2010). The stickiness may manifest her work in getting the private face-to-screen interaction to a sequential place where it more readily proffers an opportunity to re-allocate interactive resources to face-to-face interaction (cf. Figeac and Chaulet 2018; Licoppe and Figeac 2018; Mantere and Raudaskoski 2017). Engagement in two concurrent but separate participation frameworks may challenge the temporal organizing of interactive conduct, as one may find it difficult to (1) read their smartphone screen, (2) decide on how to relate to what they are reading, and (3) tell an unrelated story to one’s face-to-face interlocutors all at the same time, without at least some of these activities being hindered (cf. Wu and Liu 2008).

However, unlike in a solitary multiactivity situation where others’ interpretation of one’s conduct does not play a role in enactment of the activities engaged in, here, engaging with one’s co-participants includes within this activity a witnessing of Clo’s distributed engagement. This forms a reflexive context for others to understand her conduct, including the delayed progressivity in her turn. Clo’s hearers see her maintaining the BothHands (BH) position throughout the rushed turn beginning (segment 2) and the majority of her turn’s delayed proceedings (segment 3). They see that only after a cutoff and a pause at segment 3 (“my- (0.8)”) does Clo finally decrease her smartphone engagement with a BothHands (BH)–HandFace (HF) move. Even then, she maintains engagement with her phone in the HandFace (HF) position, in which the phone’s screen points towards her face and one of her hands remains dedicated to be available for user–smartphone interaction. Her continuing smartphone engagement also manifests in her head remaining oriented towards the device. The cut off “my-” could, in another context, be a Goodwinian (Goodwin 1980, pp. 55–94) device for achieving mutual gaze at the turn beginning. Here, however, Clo already has the gazes of both her recipients. She herself is the only one not orienting to the face-to-face framework as her primary engagement, neither with head orientation nor embodied user–smartphone configuration. Clo’s delayed progressivity therefore suggests face-to-face engagement at this point to be subordinate to her face-to-screen engagement. Contrary to acquiring hearers’ gazes, the cut off “my-” in this context seems rather to repel them. Deb and Liz respond to Clo’s conduct as a proposed re-organization of situated priorities and enact a symmetrical retrieval from face-to-face involvement on their own part. Their means of disengagement, however, differ.

3.3.2. Differing Disengagements

Deb is the first to disengage, torquing her head away from Clo in segment 3 before Clo’s BothHands (BH)–HandFace (HF) move and just after the cut off “my-”. At this time, Liz continues to keep her head torqued towards Clo, silently smiling with her mouth frozen open, as if waiting to react to a humorous spinoff story soon to follow. However, when Clo, regardless of her partially disengaging BothHands (BH)–HandFace (HF) move, continues to keep her head oriented towards the screen of her smartphone, and her storytelling simultaneously includes pauses, smacking of the lips (”.mt”), and an “umm” sound, delaying its advancement, Liz finds herself in a peculiar situation. Though Clo lessened her phone engagement, her HandFace (HF) position still affords the smartphone screen to be her primary focus of attention. Her head direction and the delayed story production also suggests this to be so. Hence, neither the person with the floor to speak and therefore responsible for producing the object of joint attention (Clo) nor Liz’s fellow story recipient (Deb) are orienting to the story as the primary involvement in the situation. This represents an emerging shift in the co-operatively constructed hierarchy of involvements in the encounter: a downgrade in the primacy of joint social projects, giving space and opportunity for individual pursuits like personal projects with one’s smartphone. At segment 4, Liz also disengages, ironically starting to reach for her phone exactly at the same moment Clo finally disengages from hers.

When Liz starts to move her body in order to make the TableDown (TD)–HandFace (HF) move, she produces an explanation (“I’ll check the-”). This echoes explanations often given to dispreferred actions (cf. Pomerantz 1985). Even though Liz’s utterance ended with a cut off “the-” and the explaining object was never produced, the turn still marked the accompanying TableDown (TD)–HandFace (HF) move as something to be explained. Previously in the encounter, participants had occasionally checked their phones for time or notifications, e.g., quickly moved in and out of the TableManipulate (TM) position, while never accounting for these moves. Those checks, which can be done without unlocking the phone’s lock screen, are so quick that accounting for them would take more time than doing them. Accompanying an activity-occupied disengagement (cf. Goodwin 1981, 1987) with an explanation projects an activity long enough to merit such an account.

3.3.3. Smartphone Engagement as Conversational Solidarity

Liz’s TableDown (TD)–HandFace (HF) move retroactively reframes Clo’s prior stumbling by further developing the emergent shift of the conversational state towards a format of lesser gravity for Clo’s interactive responsibilities. The TableDown (TD)–HandFace (HF) move participates in transforming the setting into such, where the story recipients may also engage in other activities, not just listening to the story. Similarly to how performing in a karaoke bar, where people also engage in conversation and drinking, is not the same as performing in a concert hall, Liz’s co-occurring activity reframes Clo’s storytelling. In telling a story, one is given special right to continue through several TRPs, and others are expected to listen until the end, similar to an audience at a concert. This entitlement is supposed to be taken responsibly by investing oneself in the storytelling and producing the story parts with contiguity. When Clo visibly engages in another activity and observably struggles with storytelling at the same time, it speaks to her assigned priorities of the storytelling activity in relation to that other activity. Liz’s TableDown (TD)–HandFace (HF) move then invites others to take the perspective that Clo’s stumbling would already have taken place in an environment in which the audience was also free to do other things as well, not just listen to the story. Just like in a karaoke bar one might start to pay more attention to talking with their friends when the singer is in trouble, in a way, by avoiding attention to a struggling performance and decreasing publicly assigned importance to it (cf. Goffman 1955), Liz diverted focus from Clo’s lack of disengagement from her phone and the retardant storytelling accompanying it with her TableDown (TD)–HandFace (HF) move.

3.3.4. Harmonizing Engagements

Liz’s TableDown (TD)–HandFace (HF) move also portrays her understanding of her diminished interactive obligations in the situation. She treats Clo’s continued gaze towards her phone, coupled with the lack of progressivity in the storytelling, as an opening for her also to engage in her own individual smartphone activities. Liz’s TableDown (TD)–HandFace (HF) move this way also harmonizes the conversational engagements of the situation by making them more symmetrical by recognizing and aligning with the implicated reorganizing of shared situated priorities of the moment (cf. Stivers 2008; Tiilikainen and Arminen 2017). As people generally conduct themselves by principles of solidarity in interaction (Pomerantz and Heritage 2012), maintaining symmetry between levels of engagement follows the norm of aligning with another’s projected line (Goffman 1955, p. 5) unless there is a “‘visibly-rational-and-reportable-for-all-practical-purposes,’ i.e., ‘accountable’” reason not to (Garfinkel 1967, p. vii). Clo’s storytelling might be seen to have faltered when she did not orient to the storytelling as the main involvement of the moment, while Deb and Liz were clearly doing so. However, Liz’s TableDown (TD)–HandFace (HF) move shifts the shared reality towards a situation where a face-to-face framework, and therefore also the storytelling taking place in it, is not necessarily the participants’ main involvement. This harmonizes the engagements in the situation.

3.3.5. Possibilities for Retracting Smartphone Engagement during Storytelling

Due to holding her phone in the same general direction as her interlocutors, both Deb’s disengagement by turning away and Liz’s continued engagement until segment 4 were observable to Clo in her peripheral field of vision. Clo’s turn having started with a continuation (“[>exactly<] and”) overlapping Liz’s turn and Deb’s disengagement at segment 3 makes Liz the expected primary recipient to Clo’s story. When Liz then relinquishes her smartphone engagement by torquing her head at segment 4, after a fleeting glance at a table-staring Deb, she continues to torque her head towards Liz. Ironically, she now finds Liz already engaging with her smartphone, as Liz has just started to move her body in order to begin the TableDown (TD)–HandFace (HF) move exactly at the moment Clo began to disengage from her smartphone.

Liz’s smartphone-engaging TableDown (TD)–HandFace (HF) move utilized the opportunity of lessened conversational obligation and an emerging shift in the shared situated priorities. Prioritizing an individual smartphone project over the shared project of the face-to-face interaction (cf. Tiilikainen and Arminen 2017) had become normatively a more viable option by Clo’s continued smartphone engagement and its accompanied non-progressing storytelling. However, Clo has now disengaged with her phone and is suddenly progressing with the storytelling and is directing the telling directly to Liz. However, if Liz were to end her just-begun smartphone activity immediately after Clo’s lack of progressivity ended, it would inevitably suggest causality (cf. Hume 1739) between the two. Having just explained the engagement with her phone by checking something on the device, it would be contradictory to the line of action established for her (cf. Goffman 1955, p. 213) to immediately retract engaging with the phone. The just-produced explanation would become questionable and the real reason for Liz’s smartphone engagement might very much look like it was connected to the way Clo was performing her storytelling. Liz’s continued smartphone engagement, even after Clo’s conversational engagement, therefore in a way retroactively protects Clo’s “face” (Goffman 1955) in relation to the lack of progressivity in her storytelling, and also protects the consistency of Liz’s own account-accompanied smartphone engagement.

3.3.6. Two Perspectives

Two perspectives to Excerpt 3 have emerged. Clo’s smartphone engagement and lack of progressivity in storytelling might be considered a shortcoming in conversation. Her story was prefaced and initiated, but the story recipients were left hanging. One might think that engaging with her smartphone caused her to underperform as a storyteller. Previous research on multitasking and the psychological refractory period (Wu and Liu 2008) might support this view. A cognitive bottleneck might be posited to have made the conversation spiral towards diminishing intersubjectivity and a degradation in conversing due to distributed attention.

However, the conversationalists themselves did not orient to the goings-on of the episode as something explicitly problematic. This points towards another possibility: The participants reflexively enacted, in concert, a shift in the shared priorities of the situation. The relative ranking of face-to-face interaction was lowered and individual smartphone use moved up (cf. Tiilikainen and Arminen 2017). This nudges the conversational state towards a continuing state of incipient talk (Berger et al. 2016; Goodwin 1981, p. 23; Schegloff and Sacks 1973, pp. 324–25) where the primary involvements of participants may be individual activities, with face-to-face interaction being subordinate to them. Furthermore, even if a cognitive bottleneck (Wu and Liu 2008) were hypothesized to be impacting Clo’s lack of progressivity in storytelling at segment 3, the unfolding of the events could still be seen as a co-operative construction of an incremental shift in the conversational state (cf. Ergul 2016; Goffman 1963, pp. 58–59). Until segment 3, participants’ orientations structured the “working consensus” (Goffman 1963, p. 96) of the hierarchy of the involvements as:

• Main involvement: Face-to-face interaction serving collective projects;
• Side involvement: Face-to-screen interaction serving private individual projects.

When Clo prioritized her private face-to-screen interaction over shared face-to-face interaction, it implied a suggested shift in this order. Liz’s smartphone engagement took up this suggestion. However contradictory these perspectives seem prima vista, it is also possible that the reality lies in between and in both. Meaning should not be ascribed to interactive phenomena without evidence of the interactants themselves orienting to them accordingly, but it also should not be assumed that every interactive event must achieve a completely unambiguous shared meaning between the participants in order to be sequentially relevant.

4. Discussion

The article introduced a new system for transcribing and analyzing smartphone use in face-to-face social situations. Adjusting one’s smartphone engagement with a smartphone move may alter one’s interactive footing in conversation. Smartphone moves or the lack of them may also participate in a co-operative move toward a change in conversational state (cf. Henriksen et al. 2020). Smartphone moves do not occur randomly. They have recognizable implications for conversation. They adjust the degree of engagements between face-to-face and face-to-screen interactions. Simply talking about “smartphone use,” without paying attention to the plurality and nuances of smartphone engagement, would lead to important aspects of common contemporary interactions being missed. The affordances, and consequently the social practices constructed with them, differ in smartphones and traditional mobile phones (Mantere et al. 2021). This must be considered when analyzing smartphone use in social situations. Engaging with a smartphone is engaging in an activity whose meaning is typically hidden from others. This makes accounting for the activity more relevant than is the case with traditional mobile phones.

The article proposed improvements in transcription. Replacing lines with multimodal segments may have benefits. Numbered lines emerged early in EMCA’s history, when the field was still heavily focused on speech and used mainly phone calls as data. Traditional transcripts served as written representations of speech. Organizing them into “lines”, as is typical with text, made sense. Face-to-face interaction is a multimodal enterprise. Numbered lines of speech are less optimal when needed to be supplemented with representations of the visual aspects of the situation like gaze, posture, and other embodied conduct. Organizing transcripts into segments that gather each interlocutors’ relevant verbal and corporal activity into one numbered temporal chunk, a multimodal segment, is more suitable with multimodal data than the use of numbered lines. It eliminates some redundancy that takes place in referring to talk and occasionally adds heuristic value (when referring to talk by two individuals at the same segment number, the reader immediately knows their turns to be at least partly in overlap). Transcription for smartphone moves, like for any embodied conduct, is better organized in multimodal segments than lines.

4.1. Generalizability

The 13 smartphone positions classified here manifest different levels of engagement, but to what degree is the embodiment of engagement in smartphone positions objective and generalizable? Conceptualizing engagement as an allocation of the availability of interactive resources to an interactive involvement offers some insight. A TableDown (TD) position has fewer interactive resources dedicated to being available for an involvement with one’s smartphone than a BothHands (BH) position does. It is therefore considered a position of lesser engagement with the smartphone. This principle was clearly seen in the data as a whole, as well as in the extracts analyzed in this article. The conceptualization does not, however, reveal the kind of adjustments of engagement each of the possible 156 smartphone moves, resulting from the 13 smartphone positions, would represent in each of their naturally occurring contexts. These types of questions on generalizability are, however, common to the field of CA in general.

4.2. Smartphone Moves and Schegloff’s Body Torque

This study yields a new possible definition for the infamously ambiguous concept of the continuing state of incipient talk (Berger et al. 2016; Schegloff and Sacks 1973, pp. 324–25). The continuing state of incipient talk could be categorized as a social situation where face-to-face talk is collectively produced as secondary to some other involvement and may be oriented as a passing subordinate engagement (cf. Berger et al. 2016; Didomenico and Boase 2013). This subordination may typically be marked by embodied manifestations of engagement, like body torque or smartphone positions.

Body torque manifests engagement in multiple concurrent involvements. Smartphone engagement may be one of the involvements towards which one’s body is torqued. However, smartphones are typically engaged with by positioning the smartphone to align with the position the body already occupies. This follows Schegloff’s principle of minimizing unnecessary torque, and also makes smartphone engagement in social situations to corporally manifest more like, for instance, cigarette smoking (Goodwin 1981, pp. 104–6) than cross-talk (cf. Ictech 2019).

Schegloff (1998) referred to Fox’s (1993) data, where Grace (Figure 3, on the right) engaged in interaction with the research assistant Marjorie, who was standing on the doorway (out of the picture). Grace’s shoulders and hands remained positioned towards the assignment soon to begin with the notebook on the table in front of her. It was unfeasible for Grace to change Marjorie’s position in the doorway or of the table in front of her. Her body torque manifested this physical reality.

Smartphone engagement is different. Smartphone moves can typically be easily enacted without body torque, though a slight head torque is common in some, and a slight shoulder torque in other positions. The adjustment of engagement through smartphone use would not be revealed by mere analysis of body torque. The same is true with gazing at the smartphone and manipulating the smartphone screen. They are part of engaging with the device but are not enough without the inclusion of smartphone positions. Smartphones require their own system of representing engagement. Smartphone moves may reveal, akin to body torque, the projected duration and priority of smartphone engagement in relation to face-to-face involvement.

People have countless ways to continue and extend sequences basically indefinitely. They must therefore also have ways to arrive to mutual understanding on when sequences end. Schegloff and Sacks (1973) described the use of vocal conduct in the production of sequences ending a phone call, Schegloff (1998) described the use of body torque in regulating sequence length, and Rossano (2012) described how gaze shifts away from a conversational partner to contribute to constructing the end of an activity sequence in face-to-face conversation. Smartphone moves describe a new and, in today’s world, virtually ubiquitous method for enacting endings for activities. However, like other resources of interaction, smartphone moves are not significant in and of themselves, but through the recognizable events in interaction they participate in constituting them (cf. Schegloff 2010). Although smartphone moves may adjust engagement in face-to-face talk, it is not the moves themselves that connect or disconnect people (cf. Steiner-Adair and Barker 2014; cf. Turkle 2015). The moves function in their local contexts, participating in the construction of specific recognizable series of events. Through a smartphone-engaging TableUp (TU)–HandFace (HF) move the aims of face-to-face conversation may also be advanced (Raclaw et al. 2016). The shared project of the face-to-face interaction may, for instance, be contributed to by looking up relevant information online. However, due to bystander inaccessibility (Mantere et al. 2021), it can be hard for bystanders to know what is being done with the phone. Because the context of the action itself is determined and renewed by the action that takes place in the situation (Clayman 2015; Garfinkel 1967; Heritage 1984), smartphone moves also get their meaning partly from what is being done with the phone. Engaging in interaction with a non-present other, looking up information to benefit the ongoing face-to-face conversation, or engaging in a private smartphone project are far from interchangeable activities (Mantere et al. 2021).

4.3. Limitations of the Study

The format of the data posed some limitations. Occasionally the eyes of some participants were out of frame. Spherical cameras like GoPro Fusion could be used in the future. The study is also partly limited in generalizability. The type of encounter may impact how smartphone engagement emerges and is interpreted by other participants (cf. Humphreys and Hardeman 2021). For instance, people in cafés, work meetings, and at home may behave differently in relation to smartphones. A collection-based study should be done to extract the most general aspects of different smartphone moves across varied contexts.

4.4. Further Research

Some smartphones’ sizes make them difficult to be used with one hand. The impact of this on smartphone moves, like the use of smartphone wallets, should be studied. Future research has much to cover in addressing the role of smartphone moves in constructing social realities. For instance, none of the participants in this study accounted for checking their phone for the time or notifications. Further research might also try to find generalities in how smartphone moves function in turn-initial, mid-turn, and turn-final positions in different sequences. Smartphone moves at activity borders should also be studied more widely, similar to Rossano’s (2012) study on gaze behavior. The spread of emerging technology like augmented reality goggles will soon introduce even more unknowns. Smartphones and future forms of ICT, though forming interactive relationships with their users, also work as technological extension of the user’s agency. Smartphones might be the most important feature of the human cyborg today (cf. Gray et al. 2020). An object described both as an extension of agency and an agent itself is surely worth studying more.

5. Conclusions

The article introduced a vocabulary and classification of 13 forms of embodied smartphone involvement based on the smartphone’s location (pocket, table, hands, etc.), its relation to its user’s hands, and the screen direction in relation to its user’s head or the surface the device is resting on. Smartphones can be engaged and disengaged with in different degrees through moves from one smartphone position to another. When individual smartphone engagement is not connected to advancement of the face-to-face social project, engaging smartphone moves are seen as disengagement from face-to-face interaction, and vice versa. Engaging and disengaging moves may be used to end conversational activities and relinquish speakership, to work as a turn-holding device to hold onto speakership, and to suggest and enact changes in conversational state. Multiple other interactive functions of smartphone moves should be explored by future research, both in general and more context-specific settings.