Multimodal Transduction and Translanguaging in Deaf Pedagogy

Abstract

Multimodal transduction is an interaction of teaching and learning. It traverses changes in epistemology and ontology through judgements about axiology. Using multimodal transduction (MT), students and teachers transcend languages and employ nonlanguage and quasi-language modes (e.g., drawing, color, line, math, infographics, and even sculptures). This study uses qualitative empirical data via grounded theory and case study designs to make theoretical claims about MT in a deaf higher-educational context. The data for this multi-year project were sourced through interviews, document analysis, observations, and stimulated recall with six university professors who are deaf. My analysis shows that these deaf faculty-members employ MT to convert inaccessible modes to become accessible for deaf learners. By changing modalities through MT, deaf faculty enhance comprehensibility and equity for deaf learners. This theoretical account of MT contends, extends, and clarifies aspects of translanguaging theory. As I argue, both operations transform power relations in the classroom by addressing ethics through deaf-centric aesthetics. In deaf education, MT is equally important for faculty in teaching and students’ learning. MT is widely and creatively used, owing to its flexibility and adaptivity. MT is useful for all deaf agents, regardless of additional disabilities, language competencies, or language deprivation. The MT process is inexplicit and undertheorized in the literature about deaf pedagogy and in translanguaging research. My study provides empirical support for theoretical claims about underlying mechanisms of translanguaging. One focus is to explore how MT and translanguaging (and similar theories) align or diverge. I argue that MT is a core mechanism that supports changes between all modes of discourse that enable information exchange, including but surpassing languages and translanguaging. In sum, MT is an interaction whereby deaf agents change the forms of knowledge; meanwhile, new realities and new power relations are manifested.

1. Introduction

Multimodal transduction—What is it good for?

This article summarizes components of a larger research project (Skyer 2021) where I examined deaf pedagogical praxis using a multimethod (grounded theory and case-study) design (Easterbrooks 2017) involving six deaf educators who are themselves deaf and teach deaf students in higher education, who I refer to in collective as “deaf faculty”. While researching alongside these deaf faculty, I found copious evidence of multimodal transduction (MT) (Kress 2010). The dominance of MT in my data extends current research about how lecturers use translanguaging and multimodality in deaf higher education (Holmström and Schönström 2018). Throughout this article, I juxtapose the theories and practicalities of MT against translanguaging to explore my introductory question about utility. The main contribution is that MT is the mode-transcendent mechanism that enables translanguaging. Furthermore, my empirical analysis of deaf pedagogic MT clarifies, complements, and closes gaps that exist in translanguaging theory and applications.

Presently, I explore a data-grounded theoretical account of MT using qualitative case study evidence, which is analyzed, interpreted, and contextualized against deaf research highlighting the role of multimodality in translanguaging (De Meulder et al. 2019; Swanwick 2017b). I begin by sketching out the context of my study and the dilemmas it addresses. In the body of the text, I describe my study’s methodology and showcase MT’s theoretical and practical features, meanwhile illustrating (and seeking to close) gaps and dilemmas about the application of translanguaging theories.

My analysis of MT uses data to explain gaps in the research on translanguaging in deaf pedagogy; I do so by decentering language-based changes (Wei 2022) and by exploring the pivotal role of multimodal semiotics (Kress 2010) in deaf pedagogic interactions. This shift in emphasis is congruent with some translanguaging theory that highlights multimodality but does not usually explore transduction specifically. Likewise, my examination of MT is contextualized within extensive bodies of knowledge about the crucial role of sign languages in general deaf pedagogy (Kurz et al. 2021) and in deaf translanguaging pedagogy research (e.g., Swanwick 2017b; Swanwick et al. 2022). The following summary should orient the reader but not exhaust the analysis—each subsequent section deepens these basic claims.

MT interactions occurred extensively across and within all six cases I studied. This study occurred in a US context. It was approved by two IRBs. All participant names are pseudonyms. All subjects underwent informed consent. All data were member-checked by participants during data analysis, who also had pre-print access to the final report. Throughout this article, I focus on three cases to describe useful data that illustrate my claims about the limitations of language in deaf pedagogy and the assets of MT that transcend them. In the first case, Astoria converted a textual narrative into a drawing, as supported by exchanges in American Sign Language (ASL) alongside embodied modes such as gesture and eye gaze. Next, Howard constructed a digital image also embedded in a video featuring ASL and English alongside whimsical toy props including hula-hoops and foam swords. Finally, Tessa Rose explained how MT occurs while deconstructing her students’ process of building graphical memoirs about food using creative comic-book style illustrations that sublimate texts into sophisticated multimodal formats.

Throughout, MT is explored empirically using the supporting data and by juxtaposing translanguaging and multimodal theories. My basic theory is summarized as follows: MT describes all interactions where the form (e.g., mode) of knowledge changes, regardless if that change uses or does not use language (Kress 2010; Skyer 2021). My definition is orientated toward multimodal discourse analysis (Kress 2011), which differs in key ways from linguistics-informed research on multimodality in translanguaging theories (Wei 2022). The primary issue is the theoretical structure of how “language” and “multimodality” relate. We might ask: which category subsumes the other? In contrast to Wei’s (2022) definition of translanguaging, which subsumes multimodality under a rubric of translanguaging, I claim the reverse and share empirical verifications for Kress’s (2010) theories. In this, MT is an “umbrella” encapsulating all changes to modes including those not using language.

If MT includes all interactions where modes (e.g., knowledge forms) change, then, in consequence, the theory of MT is necessary to fully explore the complex information exchange ecologies of deaf pedagogy and to extend the theory of translanguaging, which lacks a mechanistic explanation (Holmström and Schönström 2018; Swanwick et al. 2022). What I attempt to resolve and describe is that MT is the mechanism that allows translanguaging (and similar language-based changes) to occur, while also constructing a primary space where changes to modes that are not language may also co-occur. My data challenge the primacy of language in deaf pedagogy and translanguaging and show that MT is a “bridge” capable of linking changes both to and among language and communication modes (nonlanguage and quasi-language modes). In contrast with translanguaging theory, my analysis suggests that MT (not translanguaging) is the larger category in the same way that multimodality includes but is broader than language.

2. Methodological Framework

2.1. Impetus

Across a three-year time period, I observed and theorized deaf pedagogic interactions. These were partly defined as reciprocal exchanges involving deaf collegiate students’ learning in relation to deaf faculty who used pedagogic methods and curricular materials they designed for deaf learners at the research site. My sample included six faculty members, all of whom are deaf and also teach deaf students in a higher education setting, traversing STEM and Humanities disciplines. I delimited my initial focus by identifying visual pedagogic theories, tools, and methods (Easterbrooks and Stoner 2006; Rose 2012). These, I speculated, might resolve dilemmas in deaf education teaching and research, including a paucity of empirically-substantiated teaching methods and theories (Cawthon and Garberoglio 2017; Kusters et al. 2017; Swanwick and Marschark 2010).

My findings were interpreted through a conceptual framework about dissensus and (Rancière 2010) axiological conflicts of deaf pedagogy (Kress 2010; Vygotsky 1993). It is beyond the scope of this article to fully discuss the history of dissensual conflict in deaf education (See: Skyer 2021), particularly in relation to sign languages, but it should be noted that several dilemmas exist about the epistemic worth of deaf people’s learning and teaching using visual modes (Skyer 2020). One element is an ongoing devaluation and subsequent marginalization of multimodal discourses in deaf education (Thoutenhoofd 2010). As my study developed, I moved from an initial focus on visuality toward a stance that centered multimodal theories of education (Hodge and Kress 1988; Jewitt 2008; Kress 2010). These, I reasoned, were a pragmatic means to understand the role of values in deaf educational axiology (Skyer 2021) along two linked axes: the aesthetic values embedded in (deaf) educational changes (Cherryholmes 1999) and the ethical values that configure the choices made by deaf pedagogics in classrooms (Christensen 2010a, 2010b).

2.2. Questions and Design

The overall study addressed two broad research questions: (1) What modes, tools, discourses, and interactions are present in deaf multimodal-visual pedagogy (DMVP)? And, (2) How do deaf faculty understand, construct, apply, and evaluate DMVP?

To adequately address the queries, I used a multimethod (Easterbrooks 2017) approach with two mutually supporting qualitative designs in tandem. The first was collective case study (Enns 2017; Stake 1995, 2005). I used this method mainly to construct my sample and devise and plan data collection procedures. The second design was grounded theory (Charmaz 2014; Konecki 2011; Timmermans and Tavory 2012). Grounded theory methods provided a formal approach to data analysis that I used in my quest to construct explanatory theories about the data. Ultimately, I constructed five major theories about the forms and functions of visual and multimodal pedagogy in deaf higher education contexts by using the case data. The theories ranged in content and scale from descriptions of teaching methods to detailed analyses of operations of power. In the context of this paper, my focus is on one data-grounded theory (about MT). Overall, I work to show that MT is a core interaction that links other modes and tools in deaf pedagogy. My theory of MT addresses the first research question in the sense that MT is a primary interaction of DMVP. I addressed the second question by exploring my MT theory with my participants who confirmed the accuracy of my analysis then offered new insights.

2.3. Population, Site, Sample

I identified participants by using a sampling plan derived at the point of overlap between case study and grounded theory (Charmaz 2014; Mertens 2020). By using theoretical sampling, I purposefully oversampled for deafness and other relevant markers of social difference (Enns 2017; Stake 1995). The six cases represent the diversity of the research site and wider deaf populations, well-known for fractal heterogeneity (Luckner 2018).

I support this claim with the following narrative of how the site, sample, and population relate. First, all participants met the following baseline criteria. All participants were credentialed faculty in deaf education with five or more years teaching experience in deaf higher education. Two were from STEM disciplines and four were from Humanities fields. Four participants self-identified as “d/Deaf” and two as “hard of hearing” (DHH). All were ASL–English bilinguals. All overtly professed to use some form of “multimodal-visual pedagogy” specific to deaf education. All had earned master’s degrees, and one participant had earned two. Another participant was working toward a terminal degree during data collection. Yet another had earned an EdD. The remaining three had earned PhDs prior to data collection.

Numerous individual differences between participants were noted. My participants were aged between 30 and 70. One participant was a child of deaf adults/parents (CODA). Two were siblings of another DHH person. Three were parents of DHH children. Where data could be located, demographic statistics for the site are included and compared with descriptions of the participant sample. At the time of data collection, published institutional data about site personnel were limited. Readers may note that my sample is representative of the site in some ways but not in others. For example, my sample had three women and three men, which closely matches the site’s reported gender distribution (51% female to 49% male). At the site, only 47% of site faculty are DHH, whereas my sample is entirely comprised of DHH faculty. Other categories lacked institutional data for comparisons. For example, the site does not subdivide deaf and hard of hearing groups but my participants did. Alternately, whereas the site does not publish these data, two participants identified with underrepresented sexual orientations and gender expressions. The reported demographic data from the site about race and ethnicity explicitly subdivide faculty into some racial and ethnic minority groups (8% Black, 4% Hispanic) but either neglect to include others (e.g., Asian) or the statistics show that there were no faculty identifying with another group (e.g., Indigenous or Pacific Islanders). Two of my participants explicitly identified with one or more of these named minoritized racial or ethnic groups.

This study was conducted in the US against a sociopolitical backdrop of increasing conflicts between (a) White supremacy and anti-minority fascism and (b) pro-diversity movements, such as Black Lives Matter and disability rights campaigns. As a result, I encountered a particularly challenging choice about either highlighting or masking social categories of difference (e.g., race, gender, and disability) as there are good reasons ethically and methodologically for revealing or concealing the identities of individual deaf people who share membership with another minoritized group. First, there is the issue of increased recognizability that is concomitant with increased description—meaning, in small deaf communities, the act of indicating any one visible social trait such as race rapidly increases the risk for a loss of anonymity (Young and Temple 2014).

In another instance, research shows that it is problematic to “Whitewash” the contributions of Black deaf educators and scholars (Moges 2020); however, in contrast, there is a contemporaneous discourse about how communities and researchers must protect vulnerable deaf people who face intersectional oppression; that is, people who face multiple and interlocking forms of oppression (Moges-Riedel et al. 2020). Deaf-forward forms of intersectionality include persons from any of the following groups: indigenous people who are deaf, deaf people who are queer, deafblind, deaf-immigrants, deaf-and-disabled persons, or any other deaf person who also belongs to one or more marginalized groups, including any other disability. My final choice erred toward protecting confidentiality at the expense of complete description. By describing racial differences and gender diversity at the sample-level, I am able to avoid describing people at the participant-level, which could raise the probability of harm. In the end, I elevated some aspects of social difference out of obscurity but not so much as it could appreciably elevate the danger of subsequent racially-motivated targeting or discriminatory reprisal, including bullying based on ableism, harassment based on gender-expression, or anti-Indigenous xenophobia.

2.4. Data Collection

Four methods were paramount: (1) observations of deaf faculty teaching deaf students, (2) interviews with deaf faculty about pedagogic processes, (3) document analysis, sourced from curriculum artifacts and objects built by deaf learners, and, (4) stimulated recall. In grounded theory and case study, observations and interviews (Miles et al. 2020; Corbin and Strauss 2008), and document analyses (Phillips and Carr 2014) are well-known data sources, widely used by general qualitative researchers (Charmaz 2014), multimodal discourse analysts (Kress 2011), and deaf translanguaging researchers (Holmström and Schönström 2018; Swanwick et al. 2022). Each “raw” type of data was documented with still-images and video recordings. Quantities and data-capture methods of my primary dataset included 51h of video (captured using a GoPro 5 and processed on a MacBook) and 162 documents and images (captured using digital photography and scanning technologies). The data are stored in external hard-drives using encrypted digital processing. They are archived for purposes including data audits and member-checking.

Stimulated recall (SR) is not widely cited in the deaf pedagogy research literature (Perniss 2015). SR proved both complex and valuable. SR is an elicited-response task where a segment of one participant’s data becomes a prompt for subsequent data from the same participant. As Perniss (2015) relates, researchers may begin by showing images or videos to deaf participants, who then describe what they see using sign language, gestures, or other relevant modes. In SR, a participant and I would examine their data, which included video-clips from recordings of classroom observations, transcribed segments from their interviews about teaching, or documents they or their students produced. With my supporting questions, participants offered insights about the data that were previously inexplicit. SR tasks were dialogic in the sense that they were a conversation among near-colleagues and were multimodal in the sense that they produced new data that combined language, images, gestures, etc. I describe two SR examples in the present article, one from Howard’s data, including a co-analysis of a multimodal video he produced, and another from Tessa-Rose, where she and I co-analyze a multimodal transcript I built.

2.5. Data Analysis

This study generated 1.38 terabytes of multimodal data, which I analyzed to form taxonomies, narrative case studies (Stake 2005), and explanatory grounded theories using visual representations (Konecki 2011). Primary data analysis included writing observational field notes and analytic memos about data collection events. Raw data were coded and analyzed using MAXQDA digital software (Version 2018.1). This program allows the researcher to directly code (analyze) texts, images, and videos. In so doing, I produced 18K specific codes, organized in an extensive codebook, color-categorized into hundreds of code families. The data were compiled (built) using abductive reasoning, constant comparisons, and analytic memos (Charmaz 2014; Saldaña 2012; Timmermans and Tavory 2012). This trio of methods supports ongoing analysis, which juxtaposes insights stemming from the data alongside insights sourced from the researcher. All findings were interpreted within a superstructure of multimodal educational discourse analysis (Kress 2011). Theories of multimodality were paramount in my quest for meaning-making, yet this theoretical bias may be thought as a limitation for some readers.

Through my research, theory and data were always juxtaposed. No claims were made that lacked empirical support. All findings were either triangulated (good) or crystalized (better) and verified in member-checking activities. Triangulation and crystallization are comparable metaphors that communicate the basic idea that multiple converging data sources make for more-robust claims (Charmaz 2014). During focused data analysis, I used constant comparisons (Corbin and Strauss 2008) to problematize my ideas. I actively sought out disconfirming data and explored contrary explanations. If I had a hunch, I would verify it in the data or with participants and then modified my ideas based on these interactions. If a hunch lacked data, it was discarded.

I used multiple stages of member-checking (Marshall and Rossman 2016) where I invited my participants to read or view selections of my analysis and offer competing or complementary explanations. In the end, I deemed that the theories I produced were saturated (Charmaz 2014). Member-checking is a process recommended for both case study and grounded theory, but saturation is unique to grounded theory. Both techniques were vital to ensure that my research was deemed trustworthy by the people it describes (Enns 2017; Charmaz 2014). Saturation is a main criterion of value in grounded theory construction, which shows that theories are robust and potentially transferable to other sites. I discuss transferability in my limitations section, below. All participants were supplied with the full report upon its completion and had ample time to reflect on it prior to publication. No major refutations have been proposed by my participants in the 2 years since I supplied my initial report to them.

That is to say: my theory of MT was conceived as suppositions borne out of my interpretations of the data, but it was also confirmed by each participant who evaluated my theories and found them adequate. The data not only support my theoretical claims but comprise them.

3. A Theory of Multimodal Transduction in Deaf Pedagogy

3.1. Metatext

MT was present in all six cases. Section 3 complies general features of my theory of MT, which I present as a class-wide phenomenon of deaf pedagogy, expertly used by the deaf faculty I studied. This overview presages an examination of empirical evidence from three cases, which I explore after (Section 4). The same pattern continues within the three case descriptions that follow—a broad claim is made and then data that support the claim come afterward, with increasing particularization and granularity (Stake 2005). In this section and in the Discussion (Section 5), I consciously work to construct and review a bricolage of theories with the express purpose of exploring what my theory of deaf pedagogic MT contributes to the literature about deaf translanguaging studies and deaf pedagogy broadly. In so doing, I critique the literature as I review it; meanwhile, I construct new theories using data that fit into noted gaps or that address specific problems.

To orient my research, I cite three definitions. First is Kress’s (2010) definition of multimodal transduction—a change “moving across modes” (p. 124), which includes but transcends languages. Notably, this definition predates translanguaging (c.f., Lewis et al. 2012). Second, Holmström and Schönström (2018) describe translanguaging in deaf education as “a process in which two languages are used together in meaning-making [across] the whole language repertoire” (pp. 90–91). This definition offers a broad definition of multimodal language, which is in line with most current translanguaging research. Finally, Kusters et al. (2017) acknowledge a key problem in deaf research between methods focused on (1) changes to languages modes and (2) changes via multimodal semiosis extending beyond language. Exploring the second category is my main interest.

In the context of this Special Issue of Languages about translanguaging in deaf education, it is important to distinguish how and why I use the terms transduction and multimodality in the context of translanguaging. First, multimodality is a characteristic of discourse. Second, transduction represents all changes to knowledge and reality. Overall, my claims converge on the following idea: MT is a broadly useful mechanism that supports and enables translanguaging. Specifically, MT allows for the transcendence of language. This is the most important function of MT. I explore this claim in many ways in this article. In part or in sum, MT need not rely on, reference, or use any language modes at all. This shift supplants the focus of language and subverts several original definitions of translanguaging, including those adjacent to deaf pedagogy theory, which attempt to subsume multimodality in a language-based conceptual frame (Garcia 2009; Wei 2022). My account of MT reverses this operation and aptly places language as one classification of modes within a plurality of discourses (Bezemer and Jewitt 2010; Kress 2010).

3.2. Multimodal Transduction in Deaf Pedagogy

As I theorize it, MT is an interactive, intercreative process. MT mainly occurs between agents (deaf faculty and students) and curriculum content. It is and requires epistemic operations that have corresponding ontological functions. As deaf agents change the forms of knowledge, they concurrently restructure reality. These changes generally increased accessibility and simultaneously improved the aesthetics and ethics of deaf pedagogy. Deaf students and deaf faculty use MT for general and particular purposes. When deaf faculty use MT in teaching, deaf students often emulate the process, and MT reappears in their learning products, which are both visual and multimodal in form. I also found that deaf agents use MT for creative aesthetic experimentation. Not only is MT useful for teaching and learning, it is also fun and deeply enjoyable. Because deaf faculty change knowledge modes to enhance accessibility, MT is an ethical process of change (Kress 2010). Deaf faculty mostly use MT to construct accessible paths to curriculum content, to make knowledge comprehensible for deaf learners, or to enhance educational interactivity. MT’s mechanism is also its purpose—to change the forms of knowledge without substantially changing content (Kress 2010). This is what Kress (2010) describes as a “(usually total) rearticulation” of modality (pp. 124–25, parens. original).

The most basic form of MT entails two sets of modes, two stages, and one major change. Usually, mode/s present in stage one change into new mode/s in the second stage, but many stages are possible, and many changes to modes are not only feasible but desirable. When using MT, deaf faculty aim to maintain or expand epistemological concepts while dramatically changing the form of knowledge—in one case, this involved literal sculptures. Via MT, epistemic forms are constructed, deconstructed, then reconstructed; meanwhile, meaning is preserved or even enhanced across the stages of change. In this study, MT occurred in teaching and learning and in curriculum and assessment. MT was present across all six cases. Its ubiquity suggests its importance. Yet, transduction (the verb and the theory) is not widely known. It is not a term widely used among multimodalists, translanguaging theorists, or deaf pedagogics, for that matter. Likewise, when pressed, each of my participants, who are all highly-credentialled deaf faculty, lacked research-based theoretical or pedagogical terminology to describe what they were doing when using MT, even though they reasoned (and in two cases empirically proved) that such changes were both ethically necessary and beneficial for deaf students.

3.3. Contrasting Multimodal Transduction and Translanguaging

Theorizing MT in historical and current translanguaging theory is necessary to fully explore the discursive complexities of these subjects alone and in relation to deaf pedagogy, which all-but requires multiple language modes, yet, also readily transcends changes to language forms (Swanwick et al. 2022). Another area where MT and translanguaging differ is that MT does not originate in a language-based framework. It is not limited to, focused on, or first defined by language—sign, speech, text, etc. MT is the under-acknowledged “bridge” that links changes among language-based, non-language, and quasi-language modes. This gap is acknowledged (but not closed) by the leading minds of translanguaging theory. Wei (2022) explains, the new focus of translanguaging is “a shift away from language [to] attend to a wider range of multi-semiotic resources” (p. 2). Furthermore, while new translanguaging research emphasizes “going beyond languages” (Wei and Garcia 2022, italics original), “intersemiotic correspondences” (Swanwick et al. 2022), and “intramodal translanguaging” (Holmström and Schönström 2018), there is no clear agreement about a core mechanism that collocates these changes.

The most important features of MT are its independence and transcendence of language. Likewise, my core emphasis on multimodality is purposeful, as multimodality is often subordinated in translanguaging (c.f., Baker 2011; Swanwick 2017a, 2017b; Swanwick et al. 2022; Thoutenhoofd 2010). In my view, Kress’s (2010) transduction offers a more compelling understanding of how humans (deaf or otherwise) use a plurality of discourse forms (modes) in teaching and learning and other sociocultural interactions.

While it is definitely an oversimplification to suggest that translanguaging is “only” concerned with how language changes form, it is not too much to note that the theoretical genealogy of translanguaging is overwhelmingly preoccupied with language. The name—translanguaging—readily shows this bias. As one specific example, Wei (2022) writes,

Translanguaging reconceptualizes language as a multilingual, multisemiotic, multisensory, and multimodal resource for sense- and meaning-making… It has the capacity to enable us to explore the [holistic] human mind… and rethink some of the bigger, theoretical issues in linguistics.

(n.p., emphases added)

Wei’s multimodality is a facet of language. Multimodality is used as an adjective for language. The cited definition is primarily concerned with characterizing languages, which Wei achieves by enlarging a boundary as an answer to the question: What is language? It is true that Wei (2022) leads the recent charge to enhance multimodality in translanguaging; however, only four years prior, Wei (2018) defined “Translanguaging as a practical theory of language” (emphasis added). This bias is evident not only in the title of Wei’s paper but also the journal in which it was published. This language-preoccupied approach contravenes Kress’s (2010) definition of multimodality and confuses the link between multimodality and language. Kress asks a different basic question: What is discourse? His answer inverts the translanguaging stance: Kress states that multimodality is the larger descriptor of human knowledge forms, in which language (as one major class of modes) resides. In Kress’s view, multimodality always already transcends language.

Like Kress, my theory of transduction concerns all modalities in which information is exchanged. I focus on how MT happens in deaf classrooms, in particular, on how deaf faculty redress inequities by converting modes to be fully perceptible and mutually comprehensible for all deaf students. Throughout my dataset, I found that MT occurred at one scale or another in almost every interaction, whereas changes centered on language occurred much less frequently (frequency counts are supplied in the Discussion). According to my participants and published accounts in the literature, translanguaging theory is not well understood by practitioners and researchers, which leads to misapplications in classrooms (Swanwick 2017b). Confusion has several sources. Some stem from rapidly accruing changes to basic tenets of translanguaging, including revisions about multimodality (Garcia and Lin 2016; Wei and Garcia 2022). Likewise, confusion stems from fundamental differences between how researchers define “language” and “multimodality” and how they relate these two terms to each other. The noted differences between Kress (2010) and Wei (2022) shows this ambiguity. Confusion is also a result from translanguaging theorists who offer inexplicit analysis of transduction, which is the gap I intend to close presently.

Many new deaf studies of translanguaging are nominally inclusive to images, yet non/quasi-language modes such as laughter, color, line, and graphic design are afforded less status compared to language-based modes such as text and sign language. Most importantly, empirical data (Holmström and Schönström 2018; Swanwick et al. 2022) are essential but often lacking in studies of deaf translanguaging (Kusters et al. 2017). Whereas language is exalted in these studies, non/quasi-language modes such as mathematics, gesture, and body movement occupy subordinate positions. To use a colloquial phrase, semiotic multimodality appears to be a “bolt-on” element to the original translanguaging theory (Baker 2011), which focused largely on two or more language modes in bilingual learning. Thoutenhoofd (2010) speculates that the marginalization of multimodality in deaf studies is one element of a “persistent motivated alignment” to subordinate deaf ways of being in modern societies (p. 217). In the discussion section, I explore my contentions about the ample theoretical daylight between translanguaging and MT in much more detail. Furthermore, I suggest that dissensus in theorizing MT and translanguaging is far from harmful and is instead productive.

3.4. Interpreting Data on Multimodal Transduction

An overview of how data were interpreted by myself and my participants may clarify some of my bold claims. All six participants were familiar with specific theories and methods that describe changes from one language mode into another language mode. They gave me examples, including the role of ASL interpreters who change speech into sign (and the reverse), and noted classroom practices, where they and their students translated texts into sign language (and the reverse). They used explicit theoretical terms to describe multiple modes of language, including sign languages, in their pedagogy; including bilingualism, plurilingualism, multilingualism, and yes, translanguaging. This interpretation of the central role of language is consistent with definitions of translanguaging from deaf pedagogy literatures—including Holmström and Schönström’s (2018) definition, which, at the time of data collection, was only a year old.

However, in sharp contrast, none of the six highly credentialed participants used the term translanguaging to describe the equally-important and equally-common changes of modality going beyond language modes. Not one participant, for example, called it translanguaging when an utterance in sign language was represented as a mathematical equation using numbers, symbols, and letterforms. Nor did they call it translanguaging if a word or phrase was transformed into a drawing or when a text was constructed into a sculpture or if an English text was represented as an image or as an ASL poem. They did, however, characterize this kind of deaf pedagogy as multimodal.

Once I defined MT for them, my participants and I realized that it occurred in almost every educational interaction. Overall, we learned that MT is both extremely simple and ineffably complex. First, a simple concept: MT is (and requires) a change from one mode or set of modes into another mode or set of modes. At face value, deaf-centric MT usually results in increased visuality, as noted by Holmström and Schönström (2018), Christensen (2010a), and Smith (2010); however, and more to the point, I found that MT results in increased multimodality. MT both supports and demands an understanding of how inter-mode and trans-mode changes occur. While the purpose of operations constitutive of MT is acknowledged by Raike et al. (2014) and Swanwick et al. (2022), the terms MT and transduction are not used or theorized in the research I reviewed, aside from Kress (2010), who appears to have coined the term and whose work I used most to advance my interpretations.

The lack of theoretical specificity is not altogether surprising as the literature about deaf pedagogy and the literature about deaf translanguaging both lack mechanistic descriptions of MT, including what it is, why it is applied, and how it works, let alone how MT relates to literal translation and transcription, though they share a common prefix—trans—meaning: across, beyond, or through. Based on this disparity, I decided to dig deeper. Interestingly, there were an abundance of data from my study that repeated the basic misunderstanding that translanguaging is about changes to languages. It is hard to fault classroom teachers who use translanguaging theory incorrectly when the theory itself is constantly in flux. For example, in my dataset, Howard—who is a sign language researcher—used translanguaging following an observation. He wrote “I compared English and ASL to see similarities and differences in the linguistic structure of both languages” (OBS. 1, DEBRIEF, p. 16)1. This statement characterizes a typical “translanguaging” code from my data analysis sessions; it also reflects the deaf translanguaging literature in vogue at that time, which posits that translanguaging is an applied bilingual theory of language learning not one focused on multimodal transduction or the transcendence of language.

The more I examined MT—theoretically, in the context of data and in comparison to the literature—the more I understood that MT interactions were not adequately represented in any knowledge base known to me, including in the literature intersecting deaf pedagogy and translanguaging, precisely where it is so sorely needed. While the MT phenomenon is widespread, it is also elusive—theoretically, conceptually, and practically. While each of them used MT, none of these experienced, widely-read, well-credentialed deaf faculty had a name for MT or could point me to precise research to support their thinking and actions. When I presented one of my participants with my tentative theory, they became visibly excited. They seemed to say, Yes!—THAT! Someone finally understands!

3.5. Multimodal Transduction: Power and Axiology in Deaf Education

One extremely important reason that the MT perspective is needed in deaf pedagogy research is one that is seldom acknowledged: The discursive repertories of deaf students and deaf educators differ. Alongside differences of discursive capability are concomitant differentials in power (Bourdieu 1991; Kress 2010). Educators, including deaf educators, nearly always enter pedagogic relations with some superior language skills and with other advanced semiotic repertories. Concordantly, students including those who are deaf usually enter these interactions with fewer skills and abilities. With this imbalance of knowledge there is an implied and preexisting power imbalance. In nearly all interactions of language, teachers hold linguistic power (Bourdieu 1991) over deaf students. This is always problematic, but especially so with deaf students who are language deprived or possess language dysfluencies, specific language disabilities, or learning disabilities affecting their ability to use language in educational or social tasks. As my participants showed me, MT is also an effective and ethical pedagogical strategy for these students; MT mitigates differentials in power and knowledge and dis/ability.

Class-wide data showed that deaf faculty and deaf students used MT for general and particular purposes. The deaf faculty I studied professed to use MT for one or more of the following dialectic reasons, each of which is linked to discursive power: first, to increase accessibility and decrease inaccessibility; second, to enhance interactivity and reduce passivity; third, to decrease communication breakdowns and improve the exchange of information through mutual comprehensibility; and finally, to make educational interactions more ethical by reducing pre-existing inequalities of power in deaf education. Because of its dominance in my dataset, I came to understand that MT is like an umbrella subsuming a range of other changes to modalities, including translanguaging and several other theories focused on language, like codeswitching. To reiterate; MT unites all changes to all modes under one common rubric linked to power. Disambiguation among these ideas, therefore, is the focus of my Discussion.

Via axiological judgements—that is, the choices that are made by deaf faculty about deaf-positive values—MT and translanguaging share a goal of “fundamentally [transforming] power relations” (Wei and Garcia 2022, p. 322), but there are key differences. This is another main reason why a theory of MT is so sorely needed: to flesh-out the axiological values and aesthetic processes about multimodal components of translanguaging theory, which are inexplicit, misunderstood, ignored, implied, or latterly added. While language is almost always relevant when theorizing power in deaf pedagogy, the active suppression or marginalization of multimodality is a specific source of harm that reifies and reinforces discursive inequities in deaf education (Thoutenhoofd 2010).

Generally, while observing my participants in deaf educational interactions and again when co-analyzing the data with them, I saw that the changes they made to discourses via MT enabled mutual information exchanges, which were principally performed to redress preexisting imbalances of power. To do this, deaf faculty worked to change modes and support not only mutual perception but also mutual comprehension for all curricular and pedagogic content and for all deaf agents involved. As my participants and I defined it, MT involves the ethical manipulation of aesthetics in deaf pedagogy; it involves both domains of axiology (ethics and aesthetics), including their operationalization in value-based decision making in teaching and in response to specific learning events. In the discussion section, I enlarge on this definition and further explore how and why MT differs from translanguaging and other similar linguistic theories that share a common genealogy in the social uses of language by learners, including languaging and chaining, among others. Said differently, the origin of MT is the pedagogic act of leveraging of multimodality, and it finds focus and direction in the desire to enhance equity and redress imbalances of power through ethics and aesthetics.

4. Data—Multimodal Transduction in Use: Three Case Analyses

4.1. Introduction to Data about MT

Overall, the data taught me that MT is the core mechanism enabling epistemological operations and corresponding ontological functions that are manifested when deaf agents apply positive values about deafness in education contexts. What I mean is that transduction is a positive change to improve the form of modes in which knowledge is shared to be more amenable to the deaf learner’s cognitive and social characteristics. The MT mechanism requires two simultaneous operations: the first is the change in knowledge itself (e.g., the epistemology), and the second is a change in the form of reality (e.g., the ontology). When these cooccurring changes happen, deaf faculty members work to keep the epistemic content the same or highly similar, meanwhile, radically changing the form, thus, creating a new reality that is more just because it is more aesthetically aligned and situated. These cooccurring shifts often required visual modes in conjunction with embodied or kinetic modes, or all three, and are, therefore, multimodal in character and transcendent of language, as the evidence I will explore demonstrates.

The data showed that in the context of deaf pedagogy, MT was necessary, beneficial, and even beautiful. Before showing a close analysis of three cases, I provide a short introduction to MT in the full dataset. This is warranted to illustrate the depth and breadth of MT’s applications within not only the three cases but in the entire class. The data-based list (below) shows the broad applicability of MT in diverse disciplines of deaf higher education and shows how deaf agents concurrently change forms of knowledge or reality using the entirety of their multimodal semiotic toolkits in ways that readily surpass language repertoires (Kusters et al. 2017). Most examples entered my corpus as observations. A few were synthesized or adduced (TERTIARY ANALYSIS MEMO SEPT. 1 2020, pp. 158–59).2

Examples of MT in the disciplines of deaf higher education include when:

• a print word fixes the meaning of an image in an English class on new media forms
• two mathematical formulae are drawn on the board prior to a chemistry experiment
• a gesture explicates the connotation of a print word in a composition/rhetoric course
• an ASL narrative is metonymized with imagery in a science laboratory infographic
• an image captures “the feeling” or ethos of an era in a history or philosophy lecture
• a diagram is explained in detail using a descriptive text in the science lecture hall
• a drawing provides contextual cues for locating keywords in the computer lab
• a visual tool simplifies commonly used ASL phrases in the linguistics classroom
• a photograph documents a correct result in a science laboratory procedure
• a Google image search illustrates concepts in developmental writing classes
• a gesture links a textual definition with its applications in math word problem sets
• an ASL poem is “back-translated” from ASL into English in a deaf literature class

Because MT is best understood in context, I now present an analysis of specific case-based empirical data. Each example begins with a main claim, which is accompanied by evidence and the corresponding theoretical implications.

4.2. Case-Analysis 1—Astoria

4.2.1. Main Claim

Multimodal transduction is mode-independent and language-independent.

4.2.2. Evidence

In the observation that follows, Astoria used MT in conjunction with her deaf students’ social critical thinking. As we discussed in her SR task, deaf students are more capable of sophisticated critical analysis when it is presented as a social rather than individual task. In this example, Astoria co-constructed a hand-drawn, multimodal image based partly on a text and partly on an extended discussion in ASL. MT, therefore, is cognitive but involves aesthetic, cultural, and interpersonal components. Astoria’s data support aspects of Holmström and Schönström’s (2018) theory of translanguaging: “Deaf lecturers in higher education settings create a visually based learning environment…by using their whole repertoire of semiotic resources (e.g., different languages, gestures, pointing, pictures, etc.)… to create an accessible learning environment” (p. 90).

Astoria began by displaying a textual narrative about a Bento Box lunch to a developmental writing class, where many students, she told me, have diagnoses of language deprivation. Next, she added semiotic overlays, including circles and yellow highlighter, to draw focus to terms about spatial orientation, the focus of her lesson. Next, she sketched elements on the whiteboard, which included ovals, circles, line drawings, and text. Then, she asked questions in ASL to her students. The students read the text and described their interpretations in ASL as well as fingerspelled words. They also used their own gestures, such as pointing and eye-gaze. Astoria changed their form by using transduction, where she converted English and ASL into new hand-drawn images. Below, two selections of my data from Astoria’s case are shown. One is an excerpt from a field note; then, two stages of their collaborative process, including the multimodal text and multimodal drawing are depicted in Figure 1 (below).

Astoria told me she wanted to explore spatiality. She used language as data to describe spatial arrangements. She creatively uses ASL classifiers, the in-progress diagram, and complex gestures to ask questions about the particularities of spatial arrangements. She asked: “How is a bento box arranged? How is it eaten, from the center to outside or outside to center? Can circles represent fried zucchini? May small ovals characterize onigiri? Can triangles show winter squash?” In addition, Astoria used other tools from her semiotic toolkit: arrows, shapes, typewritten digital text, handwritten English, and images. Notably, she used ASL classifiers to illustrate spatial concepts from the text and ask questions. The English word “perched,” was a focus as she asks: “How might we depict a ‘flower perching on an egg’?” Her query was accompanied by an ASL classifier skit, showing how birds alight on a wire.
As the class closely analyzed and unpacked the text, it became an act of social critical thinking. Together, they signed, drew, wrote, and gestured until the spatial arrangements and drawn elements generated a cohesive image. By using close reading and unpacking the image and the text, members of the class determined the placement and arrangement of items and answered the following questions: “What goes on top, to the left, right, center?” Astoria and her students collectively decided where to add an element to the diagram. Astoria physically drew the visual representations but under guidance from her students. Hypothetical arrangements were discussed, tested, and agreed upon (or changed) before setting them down in ink. Spatial arrangements were discussed with English, ASL with classifiers, and gestures but they were affixed by the visual image. Options were shown with language, but meaning was determined by drawing.
Astoria’s Data End	(OBSERVATION 1, FIELD NOTE, pp. 5, 9).

  

In this example, text (a language mode) is basal, adjoined by ASL (another language mode), then the text is overlaid with color, line, and shapes (nonlanguage modes) to alert students to relevant words about spatial orientation (another nonlanguage mode). Then, segments of the text are translated into ASL (another language mode) as augmented with other language modes including classifiers and fingerspelling, co-presented with non-language modes such as color, the Google Image search results, and gestures—to depict objects in relation to other objects (e.g., “a flower perches on each egg”). Then, Astoria added more questions and statements in ASL, which were also modified by using gestures or facial expressions (quasi-language modes). Then, she reverted to illustration.

There were over a hundred isolated instances of MT in her 90-min lesson. Her pedagogy explored language modes in dialogic interactions. However, the centerpiece was the image. It contained multiple visual design modes, sketched in colorful markers. During points of transition, Astoria constructed joint visual attention and deaf mutual gaze interactions. Her teaching uses translanguaging, assuredly, but it supersedes a linguistic focus—in fact, her lesson’s objective was about visualizing spatial arrangements, not about any language.

4.2.3. Implications

Astoria’s MT reverses modal constancy. Modal constancy is the tendency to meet an utterance in a first mode with a response in the same mode. For instance, if a student asks a question in ASL, the teacher generally answers in ASL (not speech or writing). While Astoria often used modal constancy, multimodal inconstancy was observed with two key variations: (1) within-language inconstancy and (2) trans-mode inconstancy. Within-language inconstancy occurs when changing from text to ASL (translation) or ASL into written text (transcription). In another observation with Astoria, an “oral” deaf student replied to Astoria’s ASL using speech, which was interpreted by another agent back into ASL. These are within-language forms of modal inconstancy while also being clear examples of translanguaging as it was commonly understood by my participants.

Trans-mode inconstancy gives MT its most potent powers. It can traverse all modes quickly and effectively. It can enhance meaning and make knowledge more accessible and, thus, supports a pedagogy that is equitable for students lacking fluency in language, such as Astoria’s language deprived students. This description of trans-mode inconstancy supports and enlarges the original definition of translanguaging, something that is commonly misunderstood by teachers (Wei and Garcia 2022). However, the multitude of trans-mode changes in Astoria’s pedagogy shows that MT is not restricted to language-based changes. Trans-mode changes and modal inconstancy give MT its most awesome powers. This finding shores up weak aspects of general translanguaging theory, which deemphasizes non/quasi-language modes in theory or neglects them empirically. Notably, MT traverses all modes—MT is what happens when any one mode is converted into any other mode, regardless of the inclusion of language. For example, Astoria watched her students’ gestures, then, her pedagogic response was to construct a ovoid drawing. There were many other examples of trans-mode inconstancy in Astoria’s Bento Box lecture that eschewed language: Astoria changed a photograph into a drawing, a gesture into a line, and a digital color swatch into an analog marker line using a similar color. Ultimately, she transduced a multimodal assemblage from myriad parts, many of which had nothing to do with language or did not go through a language process. In Astoria’s pedagogic MT, a language-based question was often answered using a non/quasi-language mode. I asked her why. In reply, she wrote:

I am a visual person and do a lot of drawing myself. I often create concept maps, charts, graphs, or other images to make sense of new content. So, this [change] is natural for me. I just go through my toolbox… and try different methods—drawing, images from Google… There are instances where I ask students to do this (draw, find images/content, etc.)—to depict their understanding and how they would apply the content they learned. This helps me gauge their learning process (MEMBER CHECKING, p. 4, brackets added, parentheses original).

Astoria’s MT allows her to traverse the entirety of her own semiotic toolkit (Kusters et al. 2017) and to assess her students’ ability to do the same. She can teach, they can learn, and she can assess their learning using MT. To guide her students, she relies on her own experiences as a deaf learner. This shows that MT is a compelling form of “deaf-same” (Kusters and Friedner 2015) and intergenerational deaf pedagogy (Kusters 2017; Holmström and Schönström 2018). She used MT to affix meaning visually, multimodally, and experientially; in a word, via aesthetics. Astoria’s pedagogy is characteristic of deaf educational aesthetics, which are artful forms of knowledge achieved by way of centering deaf ethics. Astoria’s actions and designs redress inequity. Thus, throughout a range of intermediary steps, new forms of knowledge and reality and power are generated. A whole deaf-centric axiology is constructed in the process of MT.

MT is of utmost importance in deaf education, especially for deaf students with language deprivation, such as Astoria’s developmental composition students, who, I observed, lacked for some language-based semiotic resources but were able to compensate with a wealth of other modes at their disposal. This phenomena is only marginally explored in deaf research (Pollard and Fox 2019). Sacks (1990), for example, describes a young deaf boy who has no language whatsoever but is a drawing virtuoso. Vygotsky (1993) also shows that modes other than language can be used to great effect in deaf pedagogy, including to enhance sociocultural development in multiply disabled deaf youth. To neglect intact non-language and quasi-language semiotic abilities in deaf students is to simultaneously imperil the efficacy of educational interactions and disempower deaf learners.

4.3. Case-Analysis 2—Howard

4.3.1. Main Claim

Deaf pedagogy is multimodal and requires multimodal semiotics.

4.3.2. Evidence

During SR, I asked participants to bring examples of deaf-centric visual tools (Easterbrooks and Stoner 2006) that they had designed for instructional purposes. Then, we co-analyzed them. Howard brought two diagrams and a video. One of the diagrams reappeared in the video, suggesting its importance. I analyze both the visual tool and the video to illuminates features of MT. By analyzing these data, I learned that MT allows deaf educators to construct multiple forms of representation that extend embodied learning through non/quasi-language means, including visuality, spatiality, and temporality.

First, I describe the diagram, which I also call a visual tool. It is shown in Figure 2 (below). The diagram is multimodal. The most immediate component is an icon representing a human body. It is surrounded by rings much like the planet Saturn. The rings are labeled with letters (P, M, D, E) and numbers (0, 1, 2, 3). There are four text boxes, one per quadrant, explaining technical terms that Howard derived from ASL linguistics research. Howard’s diagram encodes morphemes of ASL graphically and in text, alongside design elements such as boxes, brackets, color, layout, and the deft use of negative space.

Howard designed the diagram to solve a specific problem: his ASL linguistics students (who were both deaf and hearing) could not “visualize” how sign languages used embodied space. This lack of understanding frustrated classroom progress. The diagram solved the problem by helping students think visually (Howard’s term). I argue its efficacy is dependent on the affordances of MT. The diagram helps them think multimodally. Deaf faculty who use MT maintain or enhance content knowledge across changes in modality. Doing so positively affects learning by increasing interactivity and accessibility. Both goals are clearly seen in Howard’s diagram and an excerpt from a field note (below), which also use multiple modes in support of my claims.

Figure 2. Howard’s Visual Tool.

Figure 2. Howard’s Visual Tool.

Field Note:

I asked Howard: “What is the visual tool? What is its purpose?”

Howard commented: The diagram represents ASL’s embodied spatial locations and body production sites. I designed it to visualize ASL morphemes like palm orientation, touch sites, and physical movements. The diagram represents spatial location in 2D. In addition to changing the form of knowledge, I simplified and reduced complexity and simplified the dimensions.

I considered Howard’s diagram and narrative, then, constructed a drawing of my own (Figure 3, below). After, I wrote a memo to illustrate what I thought to be three primary kinds of knowledge: spatial, visual, and multimodal for two deaf agents engaged in a simple sign language educational interaction. These comprise an early draft of my theory of multimodal transduction, which explores four distinct forms of knowledge, each discussed in turn.

1.   SPATIAL knowledge

Like Howard’s tool, my sketch has icons representing people. On the left is a person signing in ASL, which is known for its embodied use of space as a form of knowledge. To know and use ASL is to describe signs in four spacetime dimensions. While ASL is temporal, here (as with Howard’s diagram), we pause time to focus on 3D axes: X, Y, and Z (height, width, depth). Simply, to sign in ASL is to use SPATIAL and embodied knowledge in language production.

2.   VISUAL knowledge

To the right is a second person, whose gaze rests on the signer at left. The icon is labeled VISUAL for its use of visual sensory systems to learn from spatial-embodied knowledge. The optic system “flattens” 3D signs into 2D neurological visual images. If signing production (“describing”) is spatial, then reading signs (“descrying”) by other agents is primarily a visual phenomenological event. Howard’s tool is also VISUAL. It reduces complexity by flattening 3D ASL production to a 2D surface. The visual tool eliminates the dimension of “time”, further simplifying the information exchange.

Figure 3. Multimodal Transduction Diagram.

Figure 3. Multimodal Transduction Diagram.

3.   MULTIMODAL knowledge
A bracket connects SPATIAL and VISUAL. The bracket classifies the total interaction as MULTIMODAL. Multimodality combines spatiality and visuality and much more. Multimodality refers to the supersystem in which spatial and visual interactions co-occur. More explicitly, deaf multimodality is social and cognitive and dependent on human perceptual and cultural relationships and uses for specific modes (epistemology) that frequently undergo states of change affecting reality (ontology), in which forms are changed but meaning is conserved. This requires multimodal transduction.
4.   Knowledge via MULTIMODAL TRANSDUCTION
For an observing teacher, student, or researcher, spatial-embodied ASL knowledge is perceived as visual, whereas for the signer, it is perceived as spatial/embodied. Both are correct, relative to the point of view. This shows that Howard’s pedagogy is multimodal overall and requires numerous forms of transduction that supersede language-based changes. MT is an interaction; it makes explicit linkages both across and between sets of modes; this distinction obviates the change of mode beyond language. By establishing relationships among discourse forms in pedagogical interactions, both knowledge and reality changes. How it changes is different for Howard and for his students as relative to perspective and agent position. Likewise, educational forces, such as power and self-determination, are relevant in spurring the operation. This is what prompted Howard to make the tool—an ethical urge to support equity and redress inequality.
Howard’s Data 1 End	(ELICITATION TASK, FIELD NOTE, p. 8)

 

Because ASL signs are built on and off the body, Howard determined there was a need to create a multimodal tool to externalize, apportion, and effectively disembody ASL. To properly teach ASL, Howard needed to supersede language. When students encounter the Saturnine diagram, Howard reflects, they say: “Oh! This is math!” (ELICITATION TASK, p. 8)—referring to the three-dimensional axes. As we discussed it, the diagram provides concrete referents that abstract the spatial body. This occurs by reducing complexity and eliminating a dimension. Both are achieved with MT as the broad mechanism and are supported by translanguaging, including Howard’s explicit analysis between ASL and English.

Beyond these two named languages, Howard used aesthetic design elements, such as: line, icon, number, color, layout, font, and so on, that exist alongside language modes such as text and sign. Howard’s tool uses MT to represent embodied sites into a concrete form that is quite different from ASL. Where ASL is embodied, the diagram is disembodied. Where ASL is temporal and fleeting, the visual tool is static and durable. When his students use the tool, they reverse the MT changes and construct their own meaning, from image to sign. These MT changes support Howard’s goals: to visualize embodied sign production, maintain content knowledge, and increase accessibility and interactivity.

4.3.3. Implications

MT reduces cognitive load (Christensen 2010b; Smith 2010) for deaf learners by effectively focusing visual attention then dividing it. Howard’s pedagogy reduces the volume of data by abstracting some aspects and concretizing others. Concurrently, he slows down the delivery of information, disaggregates complexity, and reduces dimensionality. His students gain conceptual understanding with increasing sophistication over time as they use his tools and pedagogy in piecemeal fashion. This is a general feature of MT; embedded knowledge becomes overt to novices through adding, subtracting, dividing, or multiplying layers of representation (See the Discussion for more on these four modal logics). As Howard used it, MT reduces complexity by abstracting and reducing dimensionality, then, it rebuilds complexity back over time. Howard told me he did not adapt the visual tool from elsewhere but created it independently based on his knowledge of the research and understanding of what his students needed to learn.

Howard embedded the same “Rings of Saturn” diagram into a video we co-analyzed in the SR task. He used the video-based pedagogy to effectively unpack the diagram using another layer of meta-pedagogical expository ASL. Together, our co-analysis constituted a complex form of methodological MT featuring scores of additional transductions. Howard leverages the affordances of video technologies to frame and analyze knowledge forms that are mutually sensible, interpretable, and comprehensible for deaf learners and deaf faculty alike. The video extends MT’s additive logic to multiply and increase forms of multimodality. Howard re-introduces the fourth dimension: time, which he had previously removed. Howard’s original video is 35 min long. Juxtaposed against the depth and breadth of his ASL linguistics research and knowledge, it is notable for its sense of whimsy, and fanciful toy props.

Howard demonstrates ASL morphology by juxtaposing erudite terms with concrete aspects of the visual tool using tables and charts, his ASL description, English text, and of course, a foam sword and three hula-hoops for good measure. In the video, Howard wears these variously-sized hula-hoops suspended off of his body that function identically to the “The Rings of Saturn” in the diagram. Both the toys and the visual tool are juxtaposed in the video. The precise parallel between both sets of rings, one graphic and one physical, emphasizes the role of MT in the interaction. The encoded labels (P, M, D, E, 0–3) on the rings are identical and further assist students to link meaning through MT. By dividing aspects of visual complexity in piecemeal fashion, MT reduces cognitive burdens and increases accessibility while also making the content relatable, fun, and even funny.

Wei and Garcia (2022) emphasize that “bilingualism is a lot more complex than simply having a first language and a second language” (n.p.). In kind, transduction is more complex than just interactions between two language modes or a language mode and a nonlanguage mode. The literature readily supports this claim; when one looks past language, there are wider discursive networks at play—First, both Howard and deaf math pedagogics use numbers and dimensions which are not languages but function with rule-bound structures and require the ability to navigate a complex intersemiotic circuit (Pagliaro and Kurz 2021). Second, deaf artists and educators use visual tools, information behaviors, and aesthetic epistemologies that supplant language yet remain bound by rules and conventions and result in infinitely creative artistic forms (Lupi 2016; Schif 2010).

4.4. Case-Analysis 3—Tessa Rose

4.4.1. Main Claim

Multimodal transduction constructs, simulates, and augments reality.

4.4.2. Evidence

Tessa Rose’s teaching was astoundingly multimodal. While we discussed pedagogy in several ways, in this section, I pivot my analysis to focus on the interactive impacts of teaching on deaf students’ learning. The data demonstrate how deaf students capably use MT when they are prompted and supported in doing so. In one SR session, Tessa Rose and I focused on how and why MT appeared in her deaf student’s multimodal learning artifacts. In one assignment, Tessa Rose directed her students to write about a food with special meaning; one that evoked a childhood memory, was culturally important, or was otherwise significant for them in terms of family or place. Two selections of data are shown. Both describe the preparation of bahn tet—a Vietnamese rice dish. The first comes from an act of document analysis involving her student’s work. In Figure 4, both compositions were created by the same deaf student, whose family emigrated from Vietnam. The left panel of Figure 4 (below) showcases a recipe–memoir. The right panel shows a graphic-novel form of the same memoir.

Tessa Rose explained, as students illustrated and wrote about the foods that represented parts of their cultural identities, the first text’s design also mimicked the visual grammar of a recipe (Kress and Van Leeuwen 2006). This complex design was only stage one. Next, her students used additional forms of MT to create another newly derived composition, this one using the visual grammar of comic books and graphic novel genres. In very different ways, these students used the same basic content but diverging assemblage processes to represent their cultural identities. On the whole, this process of learning is multimodal, yet it also requires many intermediary stages of MT. As shown in Data 1 and 2, Tessa Rose’s students write and draw concurrently and sequentially. Each mode informs the other. When the stages are complete, Tessa Rose asks: What changed? Was anything gained or lost?

Figure 4. Tessa Rose’s “Student Translations”.

Figure 4. Tessa Rose’s “Student Translations”.

In the SR session, I asked Tessa Rose, How do you and your students change between modes? I wanted to clarify her metaphorical term “translate”, which she used to describe an analytic process of breaking texts down and reconstructing them as images. Tessa Rose was shy about watching herself on video and asked that instead of co-analyzing video data, as I did with other participants, that we analyze a transcript instead. To do so, I needed to first produce a multimodal–multilingual transcript. These were the most complex data from my entire study—in terms of production and analysis. Some of what follows stands on its own (e.g., Tessa Rose’s description of the power of multimodality for rich, multisensory learning), while other areas need explicit interpretation (e.g., her comments about what MT is or does).

The data analyzed next were gathered in the SR session, where I asked Tessa Rose questions about the transcript I built that depicts her classroom interactions. The transcript was intended to use language modes; however, my transcript shows the limits of language to represent multimodal data; it also shows why MT is useful and at times necessary in data collection. While analyzing the data, I found it necessary to devise my own forms of MT. In the SR session, Tessa Rose also used simultaneous communication—this is an idiosyncratic mix of ASL and speech. Furthermore, she used gestures that are simultaneous with the two named languages so she can refer to non-present objects and images such as papers and comic book panels. To represent these as text, I used color, arrows, and other lines. I also used italics to describe her facial expressions, prosody, and other extra-lingual information. Midway through our discussion, I describe my tentative theory of MT. In response, Tessa Rose readily agreed that yes, this is what occurs. After, she offered her own new insights about how MT is an interactive phenomenon.

 

 

The remaining data on pages 22 and 23 are all sourced from Tessa Rose’s member-checking document. In it, Tessa Rose’s words and my own are juxtaposed in an asynchronous conversation. Member checking data interrelate information from several sources located in Tessa Rose’s case study, which are interspliced with my commentary and new questions, resulting from the interactive member-checking process. This set of highly-complex data perhaps suggests why MT is inexplicitly and insufficiently described in the deaf translanguaging research, as a research-based methodology or as a theoretical term of pedagogy.

After I described my theory of MT to her in the SR session, in the member checking section, I asked Tessa Rose point blank: How do you transduce? She explained that in her use of MT, cognitive and poetic processes co-occur. These intercreative processes affect deaf students and deaf faculty in different ways, depending on the modes used, the purpose for the MT, and the deaf agent’s prior experiences with MT. Often, deaf faculty have wider and deeper semiotic toolkits, and more ways to use the tools within them. This allows them to support a range of differentiated tasks and processes for specific learners and content areas. Following my request to explicate the processes embedded in multimodal transduction, her reply describes MT using metaphors, in ineffable and evocative ways.

She wrote:

I see this [MT] change as an expansion/distillation dynamic—breathing in and out, almost—[because] often ASL into English requires added language—expansion—English into ASL requires a distillation of meaning—almost like going from prose to poetry—keeping the essential oil if you will—ASL into drawing could be a much closer adaptation/translation = from poem to poem, almost—from ideogrammatic language to ideogrammatic language—so more “word for word” in a way [it requires a] cinematic grammar [in the way that H.D-L.] Bauman [writes about].

When I asked why?, she linked multimodality and multisensory data to deaf cognition and memory-building. As an example, she cited annotating as a form of notetaking that physically uses handwriting as a semiotic overlay atop another printed text. She claimed that deaf agents who manually interact with texts create deeper, more lasting memories because of the uniquely multimodal and embodied dimensions of multimodal deaf learning. “This is a motor action that triggers [processes in] the brain.” As she theorizes it, deaf students’ composition process is multimodal; likewise, the product requires multimodality. Images and texts are enmeshed in a common matrix. Her pedagogy, likewise, shows and tells. Tessa Rose notes what she did not do:

I could have scanned the materials into a PowerPoint, but I didn’t for at least two reasons: First that it was just not doable in a way that would preserve the sensory information that I wanted to share with them, and second, it would be too time consuming and [would] not [be] worth the effort.

4.4.3. Implications

For Tessa Rose, multimodal transduction is the alpha and omega of deaf pedagogy. Not language, not sign language, languaging, or translanguaging. “[They are] not rich enough,” she says. They lack modes. Images alone, Tessa Rose judged, would not suffice. Words alone do not suffice. Even sign language pedagogies alone are insufficient. Likewise, static reproductions or single modes that lack valuable information. Images of books lack the rich smells and the texture of paper. During one of Tessa Rose observations, I saw a deaf student scraping a fingernail against a swatch of India ink on vellum. Even the tactile qualities of ink were a source of fascination. This, I observed, was noted by many of her deaf students, who touched, smelled, closely inspected, were absorbed with, and artfully interacted with Tessa Rose’s MT. Watching her students’ keen interest directly supported this view. For Tessa Rose, visuality in pedagogy is a bare minimum but not an end goal.

Changes to epistemological forms and ontological realities are enmeshed in MT. In MT, meaning is “carried across” the change, just like in metaphors. This abstraction affects deaf agents, who often increase visuality as only one stage toward more sophisticated multimodal simulacra. In one of Tessa Rose’s lectures, her MT process was prompted with the phrase, “look, look, look!” Tessa Rose used this interjection to guide her students’ gaze. The gaze was only the beginning of their interactions with the materials on the table, which contained books, final images, sketches, and written drafts of the memoir project. When I again asked Why did you use this phrase?, she countered that it was about focusing all of their attention. She recognized that the approach is based on enhancing the students’ understanding through multiple, converging forms of knowledge, in multiple modes, and through multiple senses, which are evoked by the materials themselves, along with the ideas they represent. This demands that non-language and quasi-language modes such as sculptures and odors take on equal epistemic weight relative to language modes.

For Tessa Rose, deaf pedagogic MT interactions are multisensory and multimodal, they require tactile, kinetic, spatial, visual—even olfactory knowledge: “Sometimes I am old fashioned,” she says, “I like old books, real papers, and writing with pen. I like the smell of old books,” she told me. For Tessa Rose, tangible materials offer fully immersive sensory experiences, which include visuality but goes well beyond visuality or language. Notable here is that Tessa Rose’s class is ostensibly a writing class. However, as they write, Tessa Rose invites her students to draw, to sculpt, to Tweet, to perform, and to be fully-engaged deaf learners—to “see that” or “do that” for themselves. She commented: “I’d like my students to experience the real thing, too”. When I yet again asked Why?, she expanded on the idea of deaf ontologies in deaf pedagogy and the concept of the real. Teaching and learning in deaf education can be more real or “closer to reality”. According to Tessa Rose, MT can even supersede the limits of raw perceptual reality. Tessa Rose wanted to build simulacra that are close to the real, perhaps better than the real. These make for more rich—but carefully delimited—sensory environments.

5. Discussion

5.1. Theorizing Multimodal Transduction

MT is a mechanism and process involving changes between forms of knowledge that supersede language. MT is at once epistemic and ontological—these co-occur. Kress (2010) states that MT results in a complete “rearticulation of [epistemic] meaning [and a] near total…change in ontological orientation” (pp. 124–25). When knowledge changes, new realities are built and socially shared; importantly, doing so changes the dynamics of power in deaf classrooms. Etymology demonstrates that transduction is a movement across or through stages. Because of this, MT uses the same underlying logic as scientific and poetic metaphors (Lakoff and Johnson 1980; Sfard 1998). This lends MT its ineffable characteristics. Tessa Rose explained, “something happens” between the phase-change. In this section of my article, I attempt to explain what happens, why, and how. Likewise, I work to explore MT in the context of several more-or-less similar theories and methods.

All six deaf faculty in my study were observed using MT flexibly. When deaf faculty used MT, they aspired to retain meaning, enhance accessibility, and extract or amplify the most salient features of knowledge to benefit learning in deaf students. MT supports the transfer of knowledge between modes (or assemblages of modes) via intermediary forms, processes, operations, and functions that reconfigure deaf students’ sensory access to reality; meanwhile, the essentials of meaning are “carried across” the stages of change. MT is not only an educational interaction between agents and knowledge but an aesthetic form of cognition endemic to deaf pedagogy. MT is creative, artful, and artistic; those who use MT act as artists and poets do.

In deaf pedagogy, MT supports flexible changes from any one mode or set of modes into any other mode or set of modes. MT can happen all at once in one major change or reoccur across a sequence of linked changes. MT can use language at the start, intermediate, or end stages of change; however, MT need not use any language modes at all. When deaf faculty use MT in teaching, it usually results in end-products or by-products that are appreciably visual; however, on the whole, MT is multimodal in character (Kusters et al. 2017; Tapio 2013).

Observations of deaf pedagogic MT nearly always emphasized visuality over orality, bilingualism over monolingualism, and multimodality over monomodality. As the dea faculty and students I interacted with showed me and told me, MT may occur in conjunction with translanguaging but is separate from it. MT is capable of interactive pedagogies and learning events that are not just bilingual or bimodal–bilingual but, overall, use multimodal-multilingualism (Kurz et al. 2021). Here, the term multimodality refers to three things: multimodal forms of language (Wei 2022), multimodal semiotics (Kress 2010), and multimodal discourse analysis (Kress 2011).

5.2. Disambiguating Multimodal Transduction

This discussion section aims to disambiguate MT with several more-or-less similar theories and forms of epistemic and ontological change already present in deaf education. I mainly analyze similarities and differences between the theories of languaging, translanguaging, code-switching, chaining, and bi/multilingualism. I also construct new comparable ideas that extend the theory of MT as I have posited it thus far. As a central pedagogical method observed in all cases of my study, MT is a class-wide phenomena. As I observed it, MT is used to construct or reconstruct knowledge toward deaf ways of being. MT represents an “umbrella,” that can organize related changes dominated by language and those driven by communication modes (e.g., non/quasi-language modes). Below (Figure 5), I propose a nested diagram to depict how these varied terms might be organized. In my diagram, the largest category is MT. Under the umbrella, I subdivide two large categories, one focused on changes using language and the other focused on changes with communication modes.

Given the context of this Special Issue about translanguaging in deaf spaces, I give special attention to exploring disambiguating and contrasting MT and translanguaging in deaf pedagogy research contexts. My rationale is enlarged in a narrative that comprises a synthetic and critical literature review (Boote and Beile 2005), which highlights the need for a theory of MT in general research on deaf pedagogy. Because of widespread confusion, disambiguation is necessary to describe how MT and translanguaging are similar but different from each other, and how language-dominant and communication-driven forms of MT differ but also relate. Doing so fulfills a practical need to describe teaching methods for preservice and practicing deaf educators who may find similar or often-changing theories hard to distinguish, especially as their basic definitions shift (Wei 2022). It should also assist researchers of deaf education for similar reasons (Swanwick and Marschark 2010), including a need for empirically-grounded teaching theories. As I argue, the unique aspects of my theory of deaf pedagogic MT are assets for general translanguaging and multimodal theorists also, who may explore MT as a mechanism and means to correlate the distinct concepts of: language, communication, and discourse, more broadly.

Figure 5. The MT Umbrella.

Figure 5. The MT Umbrella.

Seen in this way, MT is a theoretical concept that acts as a basic substrate to enable other kinds of educational interactions, as represented across the major theories and paradigms already commonly used in deaf education. My taxonomy lists four forms of language-dominant modal transductions. In the image, they are presented from least to most complex or demanding for semiotic resources. In order, they are listed as: codeswitching, languaging, chaining, and translanguaging. Thereafter, I describe four comparable forms of MT dominated by communication modes. In the diagram, communication modal logics also track from least complex to the most complex and borrow mathematics terms. They are: additive, subtractive, multiplicative, and divided visual attention. I call all eight of these approaches “modal logics,” borrowing phrasing from Kress (2010).

Generally, all eight modal logics are distinguished based on the number of modes and intensity of their use. Furthermore, language transductions must begin or end or begin and end with language modes but sometimes involve digressions into communication modes as supports for language interactions. Language transductions retain an emphasis on language, and usually use within-language transductions or modal constancy, where the mode never changes—even if the language does. On the other side, using intentional designs, communication transductions decenter or entirely eschew language. Communication transductions usually highlight trans-language changes and require modal inconstancy. In communication transductions, language may be employed as a support, but need not be the focus. It should also be noted that the relatively-rigid classifications I propose are data-based theoretical categories. They are artifacts of the constant comparison method, and are also designed to highlight the differences between the modal logics within and across categories; however, in actual practice, deaf faculty were observed combining and remixing these approaches with aplomb and joy.

5.3. Language Transduction: Four Modal Logics

Language often changes form in deaf education. This operation defines much of what is known in deaf research (Young and Temple 2014). As I see them, language transductions are a major subset of MT and have been the primary preoccupation of deaf research since time immemorial (Rée 1999; Sacks 1990). Language transductions entail within-language changes to modes but support limited trans-mode changes (e.g., shifting from text to speech, in which both italicized modes are forms of language). I discuss four examples of language dominant transductions, called modal logics, in bold, italic typefaces below. I begin with languaging to emphasize its formative role in the theories of translanguaging; thereafter, I explain how codeswitching and chaining fit between these bookends in the classification scheme. Readers may note this narrative diverges from the ordering listed in Figure 5. Whereas the figure illustrates a sequencing based on relative complexity, the text that follows begins with contemporary theories and then tracks backwards to examine theories that comprise important historical precedents.

 

Languaging is the theory that language is not something that is but rather something one does (Bagga-Gupta 2019; Swanwick 2017a). Languaging theory is rooted in the idea that deaf students need to actively use multiple languages in several language modes within social contexts. However, the idea that originally described social habits was quickly adapted to describe classroom-based learning; thereafter, it was again adapted and applied as a teaching framework (Swanwick 2017a). The theory of languaging in deaf education is originated in and articulated through linguistics research focused on a social framework for language-based interactions (Ortega 2009). Languaging theory emphasizes a multiplicity of uses and contexts for language. Its origins are rooted in bilingual traditions, but it has been since used to support bimodal–bi/multilingualism in deaf education (Kurz et al. 2021; Swanwick 2017a). While the theoretical stance about social language interactions may appear relatively benign, it represents a significant departure from the popular cognitivist theories of language that continue to hold sway in deaf research (Knoors and Marschark 2014). It is also notable that the languaging theory was only in vogue for a short time before being replaced by translanguaging.

 

Translanguaging extends languaging but retains many of its key characteristics. In deaf pedagogy contexts, Swanwick’s (2017b) initial definition states that “translanguaging represents an additive view of bilingualism and multilingualism of deaf learners [that] focus on language as a social phenomenon” (p. 233). Although other differences exist, the chief one to consider here is that languaging and translanguaging differ in the number of languages and language modes and the intensity of language changes that occur. Both theories emphasize actions with language modes. Translanguaging takes a broader view of linguistic repertories of deaf and hard of hearing people and aims to support a positive and heteroglossic ideology of multilingualism (Garcia 2009; Kusters et al. 2017).

Translanguaging is and requires competence in two (or more) languages (Baker 2011; Holmström and Schönström 2018). However, these languages need not be known equally, nor does it require different modes—for example the rapid intertwining of spoken English and spoken Spanish may comprise one form of translanguaging. Often this implies persons from two (or more) sociocultural groups, which may be unequal in status or power (Garcia and Wei 2014). Translanguaging theory offers inborn support for changes among modes of language, as with the change between written English and spoken English. In deaf education, this offers wide utility, as deaf students and deaf educators often make changes not only between two languages but also two language modes. For example, moving between signed ASL and printed English, or Swedish Sign Language and written Swedish (Holmström and Schönström 2018). Translanguaging is useful for teachers in deaf education because it may support the overt change between two oral modes (e.g., English to Dutch), two sign language modes (e.g., ASL to Langue des Signes Québécois), or between one sign mode and one oral mode (e.g., ASL to spoken English).

Swanwick (2017b) shows that translanguaging occurs as an output of socialization or learning, but more recently, the theory is supported for use by teachers in pedagogy or as curriculum (Garcia and Lin 2016; Swanwick 2017b; Swanwick et al. 2022). Swanwick (2017a) states: “Translanguaging [is the] active and purposeful use of language for learning. From this theoretical perspective, language is transformed from a noun into a verb. The ideological roots [are] located in the shift to a more positive view of bilingualism” (pp. 82–83, emphasis added). Teachers in deaf education may exploit these features by calling attention to the changes between language modes. At the outset, it is important to note that translanguaging theory is rapidly changing, especially in the years since my data were collected. More recently, general translanguaging theorists such as Wei, Garcia and Lin and theorists of deaf translanguaging such as Kusters, Tapio, Swanwick and others have explored the role of non-language modes and take a broader view of multimodality; however, in general, language remains the major focus. I discuss the differences between these ideas and my theory of MT in the final section of this article.

 

Codeswitching follows the language-dominant pattern of changes to modes but on a much smaller scale than languaging or translanguaging. Codeswitching occurs when an utterance made in language one is briefly interrupted by an utterance made in language two; thereafter, the conversation returns to language one, where it generally remains. In most nondeaf cases, the language modes also remain the same; for example, an utterance in English is interrupted by one word or phrase in Spanish, but both English and Spanish are spoken throughout. This kind of code switching is within-mode and within-language transduction. Codeswitching has social, learning, and pedagogic value, but codeswitching is generally functional not transformative of power relations. Codeswitching is rooted in monoglossic, monolingual norms that may tacitly or explicitly support subtractive bilingualism (Garcia 2009; Garcia and Lin 2016; Swanwick 2017b). For an extended account of differences between codeswitching and translanguaging, see Garcia and Lin (2016), who briefly discuss deaf educational codeswitching. This gap provides impetus to discuss these ideas at some length.

Generic code switching by nondeaf bi/multilinguals is seldom mode-switching; in contrast, for deaf people, code switching is very often mode-switching (Kuntze 2016). In deaf education, codeswitching usually involves trans-mode changes that are (at minimum) bilingual and bimodal (Garcia 2009). Kuntze (2016) writes that while generic code switching “retain[s] phonological and morphological structure, [deaf code switchers do] not conform to characteristics of spoken-language code switching” (n.p.). Like languaging and translanguaging, codeswitching is derived from social language uses. Its origins are social and functional; while it may be used subversively, it is not generally transformative of power relations or inequities thereof.

While it may be pedagogically utilized (this term is used deliberately)—codeswitching is not always valued similarly by students, teachers, or institutions (Garcia and Lin 2016). As Guardino et al. (2018) explain, codeswitching is a phenomenon of learning, of sociocultural exchanges that can be exploited by deaf educators. It is “an instructional technique [to] utilize ASL where [deaf agents] alternate between English and ASL [to] illustrate the differences between the languages” (Guardino et al. 2018, p. 227). Codeswitching has a limited use-value in deaf pedagogy. I found only nine events of clear codeswitching in my data corpus. For example, Louis used one or two English words in the middle of an ASL utterance to teach a student with a cochlear implant who preferred speech modes. Elsewhere, Sarah Jo used codeswitching to emphasize grammatical differences between suffixes in English by using Signed English.

 

Finally, Chaining deliberately links sign language modes with fingerspelling alongside written texts. Overall, the method supports print literacy as the primary goal. The aim is important here: signacy is valued as a means to an end, not as an end itself. Guardino et al. (2018) define chaining as “an instructional strategy [that maps] ASL to English print by fingerspelling a word, pointing to it in written form, and showing the ASL sign for the word” (p. 226). The method was coined by Humphries and McDougall (2000) as a way to pedagogically link visual language modes, including text, sign, and fingerspelling. Per their empirical description, chaining may use undefined “media” as auxiliary supports, but these are deemphasized theoretically and empirically. A variant form is called “Sandwiching.” Sandwiching is even more limited—it focuses on changes that begin and end in the same mode (“the bread”), with one change of mode in-between (“the filling”). A “sandwich” may look like this—sign, image, sign, or text, sign, text, etc.).

In Chaining, multimodality is optional to a pedagogy dominated by and focused on language. As Humphries and McDougall (2000) write:

[Chaining identifies] the ways that ASL and English interact with each other in various forms. Specifically:

• how teachers make connections between signing and print
• how teachers introduce/talk about English words
• how teachers use fingerspelling and initialized signs
• how teacher[s] introduce new words/concepts
• how teachers use different media to [connect ASL] with print
• other types of language interplay that teachers use (p. 87).

Chaining is dominated by language. Language modes are mentioned 13 times in this definition and “media” just once, and in relation to languages. Chaining is linear changes to language modes. Chaining’s inputs and outputs are language modes and move in a specific sequence. Chaining supports literacy. “Media” are not required, nor do they appear to be valorized or thoroughly described in the original article, nor do Humphries and McDougall deign to detail their forms, functions, or aesthetics. There is a conspicuous lack of detail about “media”, suggesting that researchers and teachers may not value them. In this way, chaining may also tacitly be a subtractive bilingual process (Garcia 2009) that inadvertently maintains imbalances of power. I identified 17 instances of chaining, suggesting it has some use-value in deaf higher education, but it remains fundamentally limited by its overreliance on and fealty to language modes and transductions.

5.4. Communication Transduction: Four Modal Logics

Communication transduction is different. These modal logics deploy modes flexibly in as-needed, dynamic combinations to create complex web-like representations. There is no upper limit to the number or kind of modes used. There are no restrictions for which mode begin or ends, how many changes can or should occur, or what discipline it can be used with. Modes overlap in transit, work together, or work at cross-purposes (Smith 2010). The data show intermediary stages, events, digressions, and additional processes. Like the four language logics, communication transduction has four independent modal logics (below,

Table 1. Communication Transduction: Four Modal Logics.

Modal Logic:	Additive	Subtractive	Multiplicative	Divided Visual Attention
Definition	Deaf agents add one or more new mode/s or multimodal assemblages in a slow, methodical fashion.	Deaf agents purposefully remove single or multiple modes, or reduce the intensity or volume of multimodal assemblages.	Deaf agents deploy multiple modes or several assemblages quickly by means of simultaneous or near-simultaneous events.	Deaf agents divorce or disaggregate complex multimodal assemblages into components or split their gaze resources purposefully.
Purpose	…to increase explanatory power with the addition of new modes or to conserve time or effort.	…to augment reality or change knowledge forms by reducing dimensions or to clarify a salient concept using fewer modes.	…to simulate the depth, breadth, and intensity of multiple senses acting in concert or to simulate a complex, multimodal reality.	…to emphasize relationships between plural entities, including as an overt analysis of language-based or communication-based multimodal assemblages.
Corpus Examples	1. Louis draws an arrow between two columns of biochemical data, drawing his students’ focus to links between data in one column and an applied mathematical formula in the second column. 2. Through elaborate, multimodal dialogues and social critical thinking, Astoria and her students create a new drawing using visual design principles to depict spatial arrangements, sourced from a text.	1. Sarah Jo comments: English is 2D and linear, whereas ASL is 3D and spatial. She explains, textual language imposes limits on expansive discursive dimensions that are latent in ASL. 2. Howard disembodies and schematizes ASL modes using visual tools he designed to reduce the number and intensity of embodied modes students encounter at one time.	1. Edward’s video lecture uses text, sign, image, speech, layout, the movement of the body in physical space [MOTBIPS], gesture, visual tools, and other modes in concert. 2. Tessa Rose’s table lecture deploys numerous graphic memoirs in various stages of completion. She prompts students to “look, look, look,” then, guides the students in multimodal analysis.	1. Sarah Jo’s student uses a smartphone app to translate and decode her English instructions by using traditional Chinese pictographic characters. 2. Edward’s student sits with a notebook and his textbook to the left of his computer monitor; as he manipulates the visual tool software suite, he also refers back to his hand-constructed visual tools and textual notes in a lengthy cycle of learning.

). There are at least these; other datasets would likely reveal more nuance and variation. Each differently alters the forms and dimensions of knowledge modes that configure reality for the deaf agents who interact with them.

MT seeks mutually comprehensible information exchanges and is broadly in support of multimodal educational interactions. Languaging, translanguaging, codeswitching, and chaining originate as techniques or theories about learning and originate from the use of social languages. MT is pedagogical in origin and involves the widest-range of semiotic resources (Kusters et al. 2017; Tapio 2013). MT readily extends into other domains such as curriculum, assessment, feedback, and so on. In MT and communication transduction, modes very often interpose, are indeterminate, or are purposefully juxtaposed within or atop one another. In reverse (as it is commonly applied), translanguaging is language transduction that focuses on language modes with other modes in subordinate positions. The results and processes of MT are multimodal clusters of meaning, where hard distinctions between modes, be they language or communication, are neither needed nor sought out.

5.5. Situating MT and Translanguaging in Deaf Pedagogies

It is important not to be overly reductive. In this section, I attempt to describe how MT and translanguaging are similar to and yet still very different from one another. The main issue at hand is: which modes of discourse that enable information exchange are (tacitly or overtly) the focus and which modes are considered subordinate? Deaf translanguaging pedagogy, theoretically and practically, mostly deals with language transductions; however, deaf educational translanguaging studies are increasingly written to be nominally inclusive to multimodality, including the “modes of image, sound…gesture…gaze, body posture” (Swanwick et al. 2022, p. 1). Changes of modalities from language-to-language are relatively common and are assuredly important in deaf pedagogy. To my knowledge, no transduction mechanism has been described in this literature to date, and the theoretical focus of translanguaging remains on language. It is not a coincidence that Swanwick et al. (2022) and Holmström and Schönström’s research (2018), the objective of translanguaging is learning language.

However, language transduction, including modal constancy and within-language transduction is only a fraction of the total forms of MT that occur in deaf pedagogy. When I coded my data in 2019, the most popular theory of deaf pedagogic translanguaging available was Swanwick (2017b), whose theory advances the claim that translanguaging requires teachers to match the language repertories of their deaf students. In my total dataset, I identified just 32 instances of language transductions; and yet only 6 fit this popular definition of translanguaging. Importantly, this signals a problem that has been unaddressed in translanguaging research, which I discuss next.

5.5.1. Linguistic Overdetermination and Multimodal Superfusion

From the theory’s inception, translanguaging is keenly focused on language in (deaf) students learning (Swanwick 2017b). A canonical, foundational text begins this way: “Translanguaging is the process of making meaning, shaping experiences, gaining understanding and knowledge through the use of two languages” (Baker 2011, p. 288, italics added). Lewis et al. (2012) find it necessary to disambiguate translanguaging with two not-quite-synonyms, including codeswitching and bilingual education. These show the genealogical roots of translanguaging, which place it squarely in the context of language research. Wei and Garcia (2022) express consternation about the ongoing confusion about translanguaging theory—“Translanguaging is often interpreted simply as enabling students to go across the two languages of instruction” (p. 314). The idea that translanguaging transcends language is relatively recent (Wei 2018, 2022), and is only partly analyzed by current empirical research in deaf studies (Holmström and Schönström 2018; Swanwick et al. 2022). While translanguaging is “moving away from a focus on linguistic behavior to embrace the multimodal and multisensorial aspects of communication” (Swanwick et al. 2022), translanguaging in and outside deaf education remains dominated by references into, out of, or in near proximity to languages.

The translanguaging generalist’s focus on language interactions affects deaf research in specific ways. While most deaf translanguaging research rightly champions sign language modes and their important role in development and cognition and to redress imbalances and asymmetries of power in teaching (De Meulder et al. 2019; Kusters et al. 2017), excusing select hyper-current studies (Swanwick et al. 2022), most current research (De Meulder et al. 2019; Garcia 2009; Swanwick 2017b) about deaf translanguaging exhibits linguistic overdetermination. Overdetermination is the notion that within a given set of co-existing and plural causes and effects, contradictions appear because causes and effects are restricted by a given frame of reference (Althusser 1962; de Alba et al. 2000). By linguistic overdetermination, I mean that (at least some foundational) translanguaging theorists have (perhaps inadvertently) constructed a “round peg and square hole” dilemma, whose methodology generally attempts to force communication modes (that are not language modes) into a framework expressly built to theorize language. Doing so distorts reality, subordinates multimodality and, reinforces inequities of power in deaf education (Thoutenhoofd 2010).

Linguistic overdetermination is an unacknowledged limitation to translanguaging theory. Perpetuating linguistic overdetermination is perhaps unwise. Here are three other examples from foundational deaf translanguaging research that show why linguistic overdetermination is a practical dilemma. First, Garcia (2009) writes, “Deaf people in the United States translanguage between American Sign Language, mime, written English, and often International Sign Language” (p. 135). This example names specific languages then vaguely refers to “mime” to fill the gap where multimodal aspects like gesture might fit. Second, Garcia and Lin (2016) write, “bimodal bilingual translanguaging of deaf children [is a] useful means of conceptualizing their language practices and the ways in which they use their language repertories” (p. 9, emphasis added). This characterization affords language privileged status. Garcia does excellent other work in many other volumes related to deaf bilingualism, but her theory of deaf translanguaging in teaching is limited by its dedication to languages.

Likewise, as a final example, De Meulder et al. (2019) carefully include multimodal semiotics and gesture studies in their theoretical frame, but, in the following quote—“translanguaging and (multimodal) language repertories” (p. 898)—they literally bracket multimodality, which is subordinated by parenthesis. Elsewhere in the same study, they offer a strong critique, which gives another view of linguistic overdetermination, which I quantify next. Their passage is lengthy, but instructive:

Originally, translanguaging described [how] a minority language was used in the classroom along with a majority language (Lewis et al. 2012), but since then, it has become a ‘terminological house with many rooms;’ (Jaspers 2018, p. 2). An often-cited recent definition is that translanguaging is ‘the deployment of a speaker’s full linguistic repertoire without regard for the watchful adherence to the socially and politically defined boundaries of named…languages (Otheguy et al. 2015). Garcia and Lin (2016, p. 19) suggested that we are witnessing a ‘translanguaging turn’ with the term now referring to both the complex language practices of plurilingual individuals and communities, as well as the pedagogical approaches that use those complex practices…Translanguaging is currently used in both descriptive and prescriptive ways. It can be used to refer to a bilingual pedagogy, multilingual spontaneous language practices, everyday cognitive processes, a theory of language in education, as well as a process of personal and social transformation” (p. 893).

This lengthy excerpt surveys a decade of studies, yet in it, there is no mention of modes other than language or citations related to multimodality. Instead, 14 references are specific to language or language modes. Other examples of linguistic overdetermination in translanguaging could be given, but largely, the pattern holds. While Jaspers (cited above) describes translanguaging as a house with many rooms, to me, it appears that the house is overfull and its contents are spilling into the yard.

Linguistic overdetermination reinforces discursive inequity and contributes to the marginalization of deaf ways of being and knowing by minimizing the role of non/quasi-language modes, including images and artforms, mathematical modes, and embodied modes such as kinetic discourses, proxemics and eye gaze, which are critically important in deaf pedagogic contexts. Affording multiple language modes privileged status above multimodality, including gestures or drawings—discounts or denigrates intact abilities in deaf learners (Skyer and Cochell 2020). This reinforces problematic power relationships and results in subpar information exchange, further impairing educational processes. The MT perspective could support and clarify examples present but undertheorized in studies of deaf pedagogy performed by Swanwick et al. (2022) and Holmström and Schönström (2018). In Swanwick’s British study, the authors note “language becomes dominant” and overtakes in importance “the use of pointing, gaze, and body orientation” (p. 7). The British team also notes that pedagogic “decisions [are] dominated by a reliance on spoken language” (pp. 12–13). Echoing this completely, the Swedish team writes, “our analysis focuses on languages” (p. 98). Later, the Swedes reveal, “translanguaging in general is only possible if the interlocutors share two or more languages” (p. 109).

This is the key to understanding why linguistic overdetermination is so problematic—which is perhaps only revealed by the inclusion of deaf perspectives in translanguaging research. What I mean is that the persistence of language deprivation is widespread in deaf populations; this total or partial lack of language fluency is not adequately dealt with in translanguaging theory or practice. At least according to the Swedes above, translanguaging should be conceptually impossible with language deprived deaf learners. In actual practice, this means that deaf students, such as students in Astoria’s class do not have equal access to the two or more languages that translanguaging requires. This disempowers deaf learners in immediate classroom interactions and marginalizes their intact abilities in research and teaching. Linguistic overdetermination is a round peg and square hole dilemma that must be resolved.

The incoherence of the round peg and square hole method is problematic, because deaf education as a whole is probably best characterized not as a phenomena of bimodality in languages but as a multimodal superfusion of discourses. My research showed this multimodal superfusion as 500 discrete modes, which I classified into 11 categories, alongside 6 specific sensory systems (Skyer 2021). It is perhaps understandable that my participants used and cited multimodality much more often than translanguaging. Empirically, the data I collected are univocal—MT traverses the entirety of the multimodal superfusion to transcend languages entirely.

5.5.2. Reclassifying Translanguaging as Multimodal Transduction

In addition to a longstanding emphasis on multiple modes of language, recently, translanguaging has embraced a semiotic approach to multimodality. First, Wei (2018) writes “Human beings think beyond language, and thinking requires the use of a variety of cognitive, semiotic, and modal resources of which language [is] only one.” (p. 18). Second, Wei and Garcia (2022) have emphasized that translanguaging research is a “transcendence of named languages” (p. 313). Finally, Swanwick et al. (2022) claim, “a lot could be learnt from the close observation of even short interactional episodes to map out the classroom layout, positions and resources of the participants, analyze the auditory and visual attention demands of the setting and the coordination possibilities of these intersecting [multimodal] resources” (p. 14).

As a result of the data I have shown and in response to the literature and theories I have reviewed, I propose two major taxonomic reclassifications: first, that translanguaging be understood as a subset of MT that is mainly defined by language transductions with multimodal semiotics playing a supporting role; second, that there are (at least) two major sub-classifications of MT—those that are focused on language transductions (including traditional translanguaging) and those inclusive to communication modes such as non- and quasi-language semiotic forms. Neither language transductions or communication transductions are better than the other; both have value in the deaf classroom. There are too few empirical studies examining how translanguaging and multimodality are used in deaf classrooms. Those available focus on language in learning (Holmström and Schönström 2018; Swanwick et al. 2022), not communication transduction, multimodal pedagogy, multimodal discourses, or MT mechanisms. Importantly, my study empirically addresses each of these gaps. The two strongest differences between the new classifications and those explored in traditional translanguaging theory are that (1) MT and communication transduction are not restricted to or dominated by language, and (2) communication transduction and MT are not originated in social learning theories or principally about or defined by linguistics concepts. Instead, communication transduction and MT are pedagogic in origin and orientation and readily bypass the limits of languages, specifically the limits of language in deaf pedagogy and its research traditions. While multimodal discourses can be mutually comprehensible for all deaf agents (Skyer 2023), if they are devalued in research, they will also be devalued in pedagogy, curriculum, assessment, learning, and in the administration of deaf education.

My stance about reclassifications is an empirical affirmation of Wei’s (2022) claim that researchers and teachers must “refus[e] to privilege particular modes and methods of meaning making over others” (p. 1). My proposed shift is foundationally oriented toward multimodal discourse analysis, as opposed to linguistics, and configures multimodality (not language) as the fundamental basis for ethical pedagogy and curriculum (Kress 2010) in deaf education (Skyer 2020, 2022). My study of MT empirically describes what much of translanguaging theory only suggests. It assists in theorizing with increasing precision how language relates to communication via MT and shows both as equally important parts of a superstructure of multimodal discourses. My study is one small contribution, but as my participants told me often and showed me in many ways: the MT stance is critically needed, it is novel and represented something they wanted but could not find—even when they looked to deaf pedagogic translanguaging.

6. Limitations and Applications

6.1. Limitations and Transferability of MT

As with all research, this study was limited. Mainly, it was limited by data, by available participants, and my own biases and interests (Timmermans and Tavory 2012). A key area of interest to grounded theory and case study researchers is the probability of transfer. The transfer of findings based on my theories to other sites and populations must be carefully considered. Ultimately, decisions about transference fall under the purview of consumers (not producers) of research. I have attempted to extract useful data and construct useful theories, but my analysis and data are partial. It is worth noting again, this article excerpts data from a three-year study (Skyer 2021). Interested readers are encouraged to see how my findings of MT fit the larger study, which includes other forms of multimodal data and an expanded discussion of methodology, theory, and interpretation related to multimodality and visuality.

More empirical-based theorizing, such as other grounded theory studies, is necessary, along with new datasets and other participants, researchers, and research teams. These should be lead by researchers and teachers who are deaf. Doing so could make both theories, MT and translanguaging more comprehensible and applicable for researchers and teachers in the classroom or lab. My research setting was deaf higher education—this may not be the same as yours. Although not observed directly, deaf pedagogies based on MT may have transferable use-value for deaf educators and stakeholders in early childhood education, such as parent–infant programs (PIPs), or sites of early-detection and early-intervention relative to deafness and other disabilities. MT may support deaf andragogy, that is, studies of teaching and learning with adults, or deaf gerontology, defined as studies of living related to elder deaf persons. MT could link domains including: Graham’s (2015) work on early-childhood deaf gaze interactions, Simms et al.’s (2013) deaf early-childhood visual communication competencies, Kuntze et al.’s (2014) and Kuntze and Golos’s (2021) visual literacies, and Raike et al.’s (2014) work on deaf adult uses of multimodality in deaf education teacher preparation. MT could be useful for deaf mental health interventions with language deprived persons (Glickman and Hall 2019; Pollard and Fox 2019). MT might fill the gap where “deaf dual coding theory” should reside, where “the verbal” and “the visual” coincide, something current cognitivist theorists of deaf learning cannot or will not explain (Knoors and Marschark 2014; Marschark et al. 2017).

MT is built for teaching. It is a tool that supports deaf faculty metacognition and supports detailed analysis of pedagogy (Swanwick et al. 2022). This can include the traditional focus on the efficacy of teaching and the relationships between teaching and learning. However, MT theory can also support new analyses of the overall axiology of deaf education, including ideas about what makes deaf pedagogy enjoyable, good, or beautiful. MT is flexible, extensible, and intercreative. It is the multipurpose use of the full semiotic toolkit. It is an eclectic, adaptable approach to interaction in deaf multimodal and visual pedagogies. However, let me be exactingly clear: Eclecticism is not random tools for random jobs. MT is not a relativist grab-bag or a hasty mishmash of “anything goes” methods. As prior deaf pedagogy philosophies have shown us, an unorganized “totality” can lead to more problems.

6.2. Limitations of Translanguaging

I claim that “translanguaging” as it was represented by many foundational studies and as popularly understood by my participants and by others is better defined as language transduction. If true, then translanguaging is just one subset of MT. Given the context of this special issue, my stance is not likely to be fully accepted at face value; however, my perspective could be supported or refuted by other datasets and other analyses. I invite others to examine these issues with me.

As Swanwick et al. (2022) and Wei (2018) are adamant about noting, new analyses about the transcendence of language can change how we understand even the most basic aspects of deaf pedagogy, curriculum, learning, cognition, and even language itself. MT research should be seen as a point of entry into a new problem space that aims to improve and better align the ethics and aesthetics of teaching and learning in deaf education. MT research could, for instance, deepen and clarify general translanguaging research. For example, Wei and Garcia (2022) discuss equity and power, with respect to aesthetic modes such as calligraphy and musical knowledge, which are important in many cultures, but are aspects that are implied but not explicated in their paper. Beyond the limitations of the current literature, my data demonstrate the limits of language in deaf pedagogy and curriculum and the limits of language as research data. They also suggest how MT can provide tools and concepts to transcend them.

Deaf languaging pedagogies (Bagga-Gupta 2019; Swanwick 2017a) and deaf translanguaging pedagogies (Garcia and Cole 2014; Snoddon 2017) represent educational theories originated to describe the language habits of learners. If communication modes such as color or image are used, they are afforded less status, play marginal supporting roles, and are minimally explored. Translanguaging began as a descriptive theory of what learners do with languages in social settings, but it rapidly became a prescriptive pedagogical koan (De Meulder et al. 2019). These changes and disjunctions flustered stakeholders, including those in my study, who needed the theory to do something it could not yet do. To support multimodal deaf pedagogy, they looked to deaf translanguaging research and could not find what they needed. Several times, translanguaging researchers have changed the theory in an attempt to transcend the focus on languages but problems of interpretation and application remain. As I have explored them, these problems are rooted in linguistic overdetermination. At base, I speculate that the dilemma is one of focus—where translanguaging research’s roots are in linguistic and social uses of languaging and learning, MT’s roots are in multimodal discourse theories focused on pedagogy and curriculum. Specifically, these latter domains are those that teachers can influence directly.

6.3. On Discourse and Axiology

MT is an input and an output with aesthetic and ethical affordances and constraints (Kress 2010). Developing the broadest possible pedagogical-discursive toolkit is the only way to differentiate for the heterogenous discursive toolkits of deaf learners. All deaf-perceptible modes are equipotential and necessary for teaching diverse deaf learners in an interaction framework (Skyer 2020; Skyer and Cochell 2020; Swanwick et al. 2022). My description of MT elevates non-language and quasi-language modes to be on par with language modes and suggests that communication transduction is of equal importance to language transduction. This distinction may appear pedantic, but it is not. Acknowledging that modes such as drawing, sculpture, and body movement are equally important as language modes such as speech, text, and signing, which my data show unequivocally, is an empirical reflection of the theoretical assertion that the composition of deaf students and their teachers’ semiotic toolkits differ (Kusters et al. 2017; Kurz et al. 2021).

Likewise, if the focus on MT and translanguaging studies continues to be on equity and resolving imbalances of power (as it should be), researchers and teachers should be open to the ethical potentiality of multimodality, especially of modes that are not linguistic in origin or focus (Kress 2010). Moreover, subordinating communication below language perpetuates inequity and fosters discursive injustice. The MT stance suggests that deaf education is defined not by bimodalism or bimodal languages but instead by a multimodal superfusion, in which hundreds of language and communication modes are co-present, valid and potentially equally important.

MT research has only begun. It is imperative that we continue these lines of research. The characteristics of MT offer wide utility in the classroom and in research contexts about deaf pedagogy and deaf curriculum. The MT realignment I suggest demonstrates new potentialities for the beneficial growth for all deaf agents. Deaf faculty use MT for a range of pedagogical interactions. They leverage not only visuality (Holmström and Schönström (2018) but an abundance of other intact sensory systems that coincide with multimodal semiotic networks of affect and effect. MT supports the change in form and the retention of meaning from any mode into any other mode. MT can be used for any content area, in any discipline, for cognition and metacognition. It can even be used for its own sake—just for the aesthetic pleasure afforded by change. MT is motivated (Kress 2010). Its deployment is not universalizing, but situated (Gee 2004). MT uses specific tools for specific jobs. As deaf faculty make decisions about MT based on its axiological use-value, they must consider both valences of deaf axiology, its ethics and its aesthetics.

7. Conclusions about Deaf Pedagogies of Multimodal Transduction

7.1. MT Is Literally and Metaphorically Transformative

MT occurs when knowledge changes form, when modes of knowledge transform, or when tacit knowledge is obviated. MT extends far beyond language-based modes (e.g., sign, script, speech, text, etc.). Instead, MT relates languages to communication modes such as quasi-language and non-language modes (e.g., gesture, spatiality, embodiment, image, etc.). MT includes changes from any one mode or set into any other mode or set, including when the initial, middle, or final modes circumvent language to explore modes including mathematics, color, line, layout, non-language sounds, proxemics, etc. MT differs from but supports the ways that deaf learners and teachers do language transduction, which I defined using four modal logics. These including code switching, which is a temporary language changes; chaining, which is a set of linear language representational changes; languaging, which is the active use of languages; and translanguaging, which describes broadly systemic changes to language modes. My theory of MT provides four new modal logics. The additive logic shows how new modes are included methodically. The subtractive illustrates how modes are taken away deliberately. The multiplicative logic shows how a plethora of new modes can rapidly accrete and result in complex simulacra. Finally, the divided visual attention logic supports how discrete assemblages are used in sequence or in close proximity. These communication transductions extend the modal logics already present in language transduction and flesh out what is absent in traditional translanguaging theory.

As this study’s participants demonstrated abundantly, MT entails simultaneous changes to knowledge and reality that deaf faculty use to preserve or enhance meaning throughout formal changes to knowledge. MT is an ethical approach to interactions that explore both communication and language in deaf pedagogy. Deaf faculty make real-time changes when using MT, based partly on their positive initial values about deafness and partly based on student feedback, and adjust their tactics if needed. MT is extensible and used iteratively in cycles until, as Astoria suggests, a tipping point occurs, and both faculty and students are satisfied that conceptual fulfilment has occurred. MT is valorized for aesthetic expressivity. In addition to fostering language learning, developing communication abilities, conveying curricular content, and transforming disciplinary knowledge, MT is an aesthetic change that is ethical, enjoyable, and joyful.

7.2. Situated Deaf Pedagogies

Deaf faculty enact MT in situated contexts where they interact with MT alongside content-area knowledge, general pedagogical knowledge, and knowledge specific to deaf education, including but extending beyond knowledge about Deaf Culture, ASL-based pedagogies, and the movement of the body in physical space (MOTBIPS). MT occurred in learning, teaching, pedagogy, curriculum, assessment, and feedback. It appears to be a case-independent, class-wide phenomenon that offers a common underlying structure for the praxis of deaf education. Scholars increasingly refer to deaf students as bimodal–bilinguals (De Meulder et al. 2019; Knoors and Marschark 2014; Leigh and Andrews 2017), but this focus on the ontology of language mischaracterizes the complex multimodal ontology of deaf students and deaf educators (Holcomb et al. 2021; Kurz et al. 2021). MT provides a superstructure for analyzing all of these domains.

By decentering language, deaf agents can creatively and flexibly use the widest possible range of modes and discourses. MT supports a comprehensive understanding of the complex forms of information exchanges common in deaf education’s many disciplines and sub-disciplines. Additional study is warranted, particularly with cultural, racial, or gendered subpopulations and niche areas of study, for instance: how do deaf archeologists use literal artifacts in conjunction with gestures and sign language when training new students in field studies? How do deaf human geographers use maps and embodiment to explore diaspora? How do deaf indigenous storytellers use animal bones and feathers in place-based narratives?

7.3. On Disability

Holcomb et al. (2021) suggest that deaf pedagogical translanguaging theory lacks an accounting of disability and deafness. My findings show that MT may circumvent the aporetic problems of using language-based discourses in the education of deaf pupils who are language deprived (Skyer 2021). I mentioned before the incoherence of the square peg and round hole methodology. This dilemma is exacerbated by linguistic overdetermination, where, as one example, Wei (2022) attempts to redraw a boundary by asking what is language? Perhaps instead of new definitions, new questions are warranted.

In our SR session, I asked Tessa Rose, how can a deaf student without an intact language make use of any language. She said she did not know. Neither did I. The theory of translanguaging breaks down in the face of language deprivation. In its place, MT may be useful for educators trying to make progress with deaf students diagnosed with language deprivation or who live with other language disorders, sensory disabilities, or learning disabilities, such as deafblindness, auditory-processing disorders, or autism (Gargulio and Bouck 2021; Gulati 2019). Perhaps work on MT could be useful to frame and construct an approach to teaching that entirely bypasses language deficits and to work with intact communicative and discursive assets instead. This would emphasize a pedagogy using intact abilities as a situated means to compensate for biosocial disabilities (Skyer 2020; Vygotsky 1993).

7.4. Use-Value of Multimodal Transduction

MT’s instructional utility depends on the deaf pedagogue’s judicious use of a wide, deep semiotic toolkit, which needs to be purposefully aligned to deaf learners’ toolkits to enhance their own inner sensory and discursive abilities. This toolkit cannot be limited to language alone for our reality is not limited to language. Said another way, our understanding of reality may even be limited by language. MT accounts for intra-cohort similarities and individual differences among deaf students and faculty. It subsumes translanguaging theory and supersedes the focus on language that has long defined the horizons of deaf research.

MTs support a wide range of discursive, metacognitive, and developmental processes that are educationally useful for deaf students and deaf faculty. In my study (Skyer 2021), Louis had no name for MT, but he used it consistently. He demonstrated MT to me by twisting his key lanyard. Alongside his ASL narrative, the nylon strap was transfigured into an elegant visualization of DNA’s double-helix structure. Howard explicated elements of ASL morphology using MT, meanwhile, his students constructed notations of ASL signs using visual templates, where MT drove the representational logic. Sarah Jo stated, “I cannot conceive of [deaf pedagogy] without signing, visual tools, and visual aids. [Deaf] students rely on visual [and] multimodal means…to learn in an interactive setting.” (MEMBER CHECKING, p. 26). Edward also encouraged his students to construct visual tools and videos to deeply encode their learning using MT at each stage of change. Astoria extoled, “multiple ways of teaching [that are] visual, experiential, multimodal” (OBSERVATION MEMO, p. 1). Tessa Rose encouraged her students to playfully explore MT. Under her tutelage, her students reconstructed texts as three-dimensional sculptures using Joseph Cornell’s assemblage method.

These examples show that MT is creative and expressive and social and situated. MT is an artform of deaf education. MT is also based in empirical science. MT’s characteristics are at once recognizable and ineffable. MT is metaphoric (Lakoff and Johnson 1980; Sfard 1998). Furthermore, like metaphors, MT supports the simultaneous poetic and pragmatic transference of knowledge between radically dissimilar forms. MT transcends language. MT is the purposeful augmentation of form to preserve meaning and build increasingly realistic simulacra and experiences. MT is deaf educational synesthesia3.

A final note is warranted—in multimodal transduction, deaf-centric discourses are valued for their own sake, for their inherent beauty and functionality; for the pleasure they afford the agents who create them. MT is not a base steppingstone. This warrants additional study in the context of Cherryholmes’s (1999) assertion that pragmatic educators must “bring about beautiful results in the midst of power and oppression and ignorance” (p. 5). MT is affected by axiological properties; its ethics and aesthetics suggest a plausible resolution to the teleological tension between adaptation and struggle in deaf education.

I will close by asking precisely the same question I began with: Multimodal transduction. What is it good for?

Absolutely everything.