Supervisory Feedback on Chinese Doctoral Students’ Drafts of a Research Article

Abstract

Doctoral students need guidance from both language teachers and academic supervisors for academic publication. However, previous studies have predominantly focused on corrective feedback from language teachers. The small number of studies on supervisory feedback were mainly undertaken in English-speaking countries on theses and dissertations, and mostly examined supervisors in applied linguistics, who probably have much in common with language professionals. To fill the research gaps, we investigated the foci of the feedback from a non-English-speaking supervisor on drafts of his doctoral students’ research article intended for a top conference in computer science. The results show that the supervisor commented not only on the content but also on the requirements for research writing, the logical flow of ideas, surface-level language issues, and visual elements. The findings can inform language teachers of what supervisors may value so that language professionals can provide feedback that better caters to the needs of students in research writing.

1. Introduction

The pressure to publish can be high for graduate students as well as higher education faculty. Graduate students strive to publish in refereed journals to increase competitiveness in the job market and, in many cases, to satisfy graduation requirements. Although the requirement to publish peer-reviewed journal articles has been relaxed in many universities, there may still exist “an unwritten expectation” ([1], p. 160) for many graduate students, with pressure from supervisors and students themselves. With English as the lingua franca of the research world and given the challenges of academic publication, novice writers who learn English as a foreign or second language need guidance from language teachers and academic supervisors. English for Research Publication Purposes (ERPP) can provide crucial pedagogical support for the publication success of those novice writers, and hence has been receiving increasing research attention in both the English-speaking context [2] and the non-English-speaking counterpart [3,4,5].

A large body of literature has investigated written corrective feedback from language teachers on second language (L2) writers’ texts [6,7,8]. For example, based on a comparative textual analysis of the drafts and final published versions of 15 SCI-indexed journal articles, Flowerdew and Wang [6] revealed that the second researcher, who worked as an English language teacher at a major research university in China, employed five types of revision strategies (substitution, correction, addition, deletion, and rearrangement) across four lexico-grammatical levels (morpheme, word, group, and clause/clause complex). Cheng and Zhang’s [9] investigation revealed that native English-speaking (NES) teachers of English as a foreign language tended to invest more effort in addressing global issues regarding content and organization, whereas non-native English-speaking (NNES) teachers provided more local feedback concerning linguistic errors. Hyland and Anan [10] also explored differences between NES and NNES teachers in providing feedback and found that NES teachers provided more indirect and less harsh feedback than did their NNES counterparts.

Since language teachers’ lack of content knowledge may interfere with their attempts to provide valuable feedback on students’ writing, it is worthwhile to investigate feedback from supervisors as well since their feedback can be quite different from and thus complement that provided by language professionals [11,12]. Benfield and Howard [12] compared revisions suggested by peer reviewers of the European Journal of Cardiothoracic Surgery and those suggested by a language professional (an applied linguist) for research articles. Their study revealed that feedback offered by the language professional addressed problems of form and presentation, whereas feedback from the peer reviewers focused on ideas and content. In addition, the language professional tended to focus more on the use of language for particular rhetorical effects (e.g., placing old information before new information for a better information flow), while peer reviewers were better equipped to provide suggestions regarding discipline-specific terminology and could read between the lines thanks to their subject matter knowledge. The latter finding is supported by Benfield and Feak [11].

A review of the relevant literature shows that the small number of existing studies on supervisory feedback were mainly undertaken in English-speaking countries (e.g., [13,14,15,16,17]). For instance, Basturkmen et al. [13] examined on-script feedback comments that supervisors provided for drafts of master’s and doctoral theses in three disciplinary fields across six universities in New Zealand. Morton et al.’s [16] case studies in the Australian context examined feedback comments on a master’s student’s minor thesis and a doctoral student’s assessment report (both in applied linguistics). Also set in Australia, Wang and Li [18] explored international doctoral students’ thesis writing and feedback experiences at an Australian university, based on the semi-structured, face-to-face interviews of ten non-English-speaking doctoral students from six disciplinary areas. Data analysis revealed three main themes (emotional responses, supervisory relationship, and pedagogical needs) with two tendencies (frustrated/uncertain and inspired/confident). These studies were all conducted in English-speaking countries. Even though some supervisors in English-speaking countries are non-native speakers of English, they can be more proficient in English than most of their counterparts in non-English speaking countries since they use English in their work and life on a daily basis. Research on supervisory feedback in non-English-speaking countries is lacking. One exception is Gezahegn and Gedamu [19], examining feedback received by students during doctoral studies in four Ethiopian universities. With both supervisors and supervisees working with a language other than their native one, such research is expected to provide valuable insight into supervisory feedback.

Meanwhile, previous studies on supervisory feedback mostly examined supervisors in the area of applied linguistics, who probably have more in common with language professionals than with supervisors in many other research areas [15,16,17,20]. For example, Kumar and Stracke [15] combined their linguistic backgrounds and insider perspectives as doctoral supervisors to examine the specific functions of the language used in the in-text feedback and overall feedback on the first draft of a doctoral dissertation. Similarly, a more recent study examined the power relations between two master’s students in applied linguistics and their supervisors at an English-medium university in Hong Kong through an examination of supervision meetings and a detailed textual analysis of supervision transcripts [20]. As argued by Kumar and Stracke, it is “essential to cross the boundaries” of applied linguistics since feedback is provided in disciplines other than applied linguistics as well ([15], p. 463).

Equally important, the existing studies on supervisory feedback predominantly focused on master’s theses, doctoral dissertations, and thesis and dissertation proposals, rather than writing for publication purposes [13,14,15,16,17,19,20,21,22]. For instance, Bastola and Hu [22] examined supervisory comments on drafts of master’s theses, and Xu [17] investigated a supervisor’s feedback on a draft of a doctoral dissertation proposal and the student’s responses to the feedback in the revised version. However, writing for scholarly publication purposes is distinct from writing master’s theses and doctoral dissertations in terms of writing purposes and readership [23]. Writers may employ different strategies for the two types of writing in consideration of different discourse communities. For instance, Li [24] found that a novice research writer adjusted her strategies for negotiating the knowledge contribution to the domestic discourse community (in a paper intended for publication in a journal of her country) and to the international discourse community (in a paper intended for presentation at an international conference). All these differences may lead to differences in supervisory comments, making it worthwhile to investigate supervisory comments on writing for scholarly publication purposes as well as those on theses and dissertations.

One study did examine research writing for publication purposes [25]. The researchers explored doctoral supervisors’ roles in their students’ attempts at scholarly publishing in the Chinese context and found that supervisors played a critical role, shaping students’ manuscripts and developing their “apprenticeship in scholarly publishing” ([25], p. 27). The researchers observed that one of the essential roles played by supervisors was as “manuscript correctors” and that supervisors’ corrections might encompass various aspects of writing, including the structure, word choice, grammar, and logic and flow of the manuscript ([25], p. 34). This study was conducted via semi-structured interviews. Other research methods, such as textual analysis, may also yield valuable insight.

In the literature above, there is a striking paucity of studies examining supervisory feedback provided to students writing for research publication purposes. There is also a general lack of research investigating feedback provided by supervisors from non-English-speaking countries and working in research areas other than applied linguistics. Therefore, this current study explored the feedback that a Chinese supervisor in computer science and technology provided to his doctoral supervisees on drafts of a research paper that they intended to submit for presentation at a top conference (with conference presentation generally preferred to journal publication in computer science for the timeliness of knowledge exchange, according to computer science faculty members and students that we know). The aspects of writing that the supervisor focused on when providing corrective feedback were examined. The following research questions helped guide this study:

(1)

What aspects of research writing does the doctoral supervisor focus on when providing feedback?

(2)

In what categories can the feedback be classified?

2. Materials and Methods

2.1. Participants

The supervisor whose feedback we examined was an associate professor at a renowned university in China and a prolific writer who was skilled at English writing. He received score for the English writing section on both the college entrance examination in China and the Test of English as a Foreign Language (TOEFL). In his mid-thirties, he published or presented over 100 research articles in peer-reviewed journals or at professional conferences. In addition, he published a book in English with SAGE Publishing. The supervisor was also experienced in advising students on research paper writing. A paper with one of his supervisees and another student as the first authors and with him and that student’s supervisor as the corresponding authors won the “Best Student Paper Award” at a top conference in his research area. That paper was the only winner that year, and it was also the first paper by anyone from any Asian higher education institution to have ever won that award. The other two participants were doctoral students, in their first and second years of doctoral studies, respectively. They were both novice research writers.

The supervisor and his two doctoral supervisees kindly agreed to share with the authors of this paper six versions of a paper he and two of his doctoral supervisees had completed, together with the records of their communication on two sections of the paper (“Measurement Methodology” and “Measurement Findings”). The supervisor and his supervisees used a cloud storage and file-sharing service for paper editing and a WeChat group (WeChat being a messaging and social media app) for online communication. The two supervisees would upload a subsection of the paper to the cloud service, and the supervisor would read it there. Then, in the WeChat group, he would comment on the manuscript, pointing out usages that needed improving, and the supervisees would revise accordingly. The whole writing and revising process was not affected in any way by the present study since the supervisor and his supervisees had submitted the paper to a conference by the time the authors of this paper contacted the supervisor.

2.2. Data Analysis

All supervisory feedback (i.e., the corrective comments made by the supervisor on the manuscripts) was analyzed. In addition, different versions of the paper were compared to identify usages needing improvement and the changes made in response to the corrective feedback. The records of their online communication in the WeChat group were also utilized to explore the reasons behind the changes made. All analyses were carried out from February to May 2021.

Each piece of supervisory feedback was coded for its focus. The coding process combined deductive and inductive theme analyses. For the deductive part, the process moved from categories established on the basis of previous studies to new ones detected in the data [26]. The feedback focus categories identified by Bitchener et al. [14] and Basturkmen et al. [13] were used as the preliminary coding categories (See

Table 1. Feedback focus categories identified in previous studies.

Feedback Focus	Explanation
Content	arguments, information, and claims
Requirements	genre expectations and academic conventions, formatting, referencing
Cohesion and coherence	links between and order of information and ideas
Linguistic accuracy and appropriateness	surface-level language forms and clarity of meaning

). For the inductive theme analysis, each supervisory comment was read several times by the first author of this paper to identify distinct themes that occurred repetitively; each theme was compared with those that had been identified (either in previous studies or in the present study) until no new theme was found. Then, the final forms of the themes were constructed, labeled, and defined. Credibility checks [27] were conducted by referring to the second author’s interpretation of the data to augment confidence in the results.

3. Findings

The combination of deductive and inductive coding resulted in an analytic framework of five categories (with the themes unidentified in previous studies highlighted in italics): (1) content, (2) requirements (including considerateness of reviewers and readers), (3) cohesion and coherence, (4) linguistic accuracy, appropriateness, and concision, and (5) the use of visual elements. Below, we illustrate the five categories one by one.

3.1. Content

Feedback on content addresses the completeness, accuracy, and relevance of the content presented in the research article [13]. The analysis showed that the supervisor in this study was concerned with the completeness of development, the accuracy of claims and terminology, and precise expression in anticipation of possible questions from readers.

3.1.1. Completeness of Development

Some of the supervisory comments concerning content focused on whether a section was developed completely and appropriately. For instance, the supervisor commented that an early version of the Methodology section was “simplistic” and failed to include a key element that the supervisor believed reviewers would like to see. Accordingly, he suggested that the students tell at the very beginning of this section how they had built the measurement environment to ensure fair and comprehensive measurements, and how they had figured out the working principle of a system developed by others so that they could make meticulous and elaborate measurements. The supervisor maintained that including the information would convince reviewers of the quality of their measurement and thus gain favor with reviewers.

3.1.2. Accuracy of Claims

The supervisor was also concerned with the accuracy of the claims made in the paper. For instance, one gap that the students claimed to exist in previous research was that a particular server did not respond fast enough, as indicated by the underlined parts in the original version (OV) in Example 1. However, the supervisor pointed out that this gap statement was not valid since 20 s was actually not long.

Example 1: OV: …, 20 s is too long for the server to figure out the client’s actual Internet speed, which indicates a waste on both test time and traffic costs.

The supervisor also called into question a descriptor that the students used for a speed test system. The students paraphrased the expression “the first HTML5 broadband speed test” from the official website of the system as “the leader of HTML5 broadband speed test.” The supervisor argued that, unlike “the first,” which is a factual statement, being “the leader” is a strong claim that needs to be backed by evidence. The students ended up removing the descriptor.

3.1.3. Accuracy of Terminology

One group of comments focused on the accurate use of terminology. For instance, the supervisor suggested replacing “Ali cloud” with “Alibaba Cloud” or “Aliyun Cloud.” The two suggested versions are the official English names of a cloud service provider, whereas the students’ original usage is the word-by-word translation of its Chinese name. Other comments on terminology accuracy include the suggestion of substituting “session” for “thread,” “parallel sessions” for “progressive sessions,” and “maximum speed” for “overall speed.”

3.1.4. Preciseness of Expression

Another group of comments concerning content was related to precise expression in anticipation of possible questions. For instance, the supervisor asked about the criteria for being “the quickest,” “the most reliable,” and “the nearest” (see OV in Example 2). Using “the quickest” as an example, he pointed out that it could mean either the shortest round-trip time (RTT) or the highest speed. He maintained that a clarification of which was the case would gain favor with reviewers. The students made revisions accordingly (see the revised version, or RV, in Example 2).

Example 2: OV: When the test is started, the quickest and most reliable server, not necessarily the nearest, will be chosen. RV: Before the test starts, a small pre-test is conducted where several small (under 1 KB) XHR⁴ files are sent to the client from different servers; the server which owns the shortest file transmission time will be chosen as the test server.

The supervisor expressed a similar concern about the expression “large enough” (OV, Example 3). He questioned, “How large is large enough?” and recommended reporting the exact size of the sample file, believing that reviewers would be happy to have the key information. The supervisor emphasized the importance of precision in academic writing and held that, when one used an adjective, one should ensure that the reader could understand what the adjective meant exactly. The students followed the supervisor’s suggestion and reported the size of the sample file. In addition, they added a footnote to indicate the source of this piece of information (not shown here).

Example 3: OV: The sample file is large enough to guarantee …. RV: The sample file is quite large (128 MB) to guarantee ….

In the same vein, the supervisor suggested specifying the test time needed after a claim of “saving both test time and traffic” (Example 4). According to him, the specific detail added would enable reviewers and readers to evaluate the extent to which test time was saved.

Example 4: OV: … saving both test time and traffic. RV: … saving both test time and traffic. To our experiences, under relatively stable network environments the test time is merely around 8 s.

3.2. Requirements for Research Writing

The feedback focus on requirements is related to genre expectations and academic conventions, formatting, and referencing [13]. The supervisor under study raised concerns about the appropriate borrowing and incorporation of ideas from sources, the conventional use of abbreviations, and being considerate to reviewers and readers.

3.2.1. Citation

One crucial aspect of research writing is appropriately incorporating ideas from other sources to avoid plagiarism and gain credibility. For example, the supervisor suggested including the source of their statement “speedof.me deploys 116 servers” (see OV in Example 5). Correspondingly, the students added a footnote (represented by the superscript “3”) indicating that they obtained the information from a popup window on the homepage of speedof.me. The supervisor commended the revision, holding that the extra information provided could be helpful to reviewers and readers interested in learning more about the website, and that the footnote might impress the reviewers with the meticulousness of the authors—they paid attention even to information in a popup window.

Example 5: OV: …, speedof.me deploys 116 servers located at various places across the world. RV: …, speedof.me deploys 116 servers³ located at various places across the world.

3.2.2. Abbreviation

Another issue that the supervisor raised regarding research writing was the usage of abbreviations. He noted that an abbreviation should be defined before its first use in the manuscript. As can be seen in an earlier example (RV in Example 2), a footnote was added to define the abbreviation “XHR,” as per the suggestion of the supervisor.

3.2.3. Considerateness

Being considerate to reviewers and readers is yet another requirement made by the supervisor. To him, being considerate means taking into consideration the broad readership of the target journal and describing the problem configuration adequately and clearly to establish the context for reviewers and readers. For instance, a comparison of the two versions in Example 6 reveals a difference in the URL link provided. The supervisor suggested the change because he believed that the link in the RV would be more helpful. According to him, a reader might be motivated to click on the link provided and try a speed test, only to find that the link in the OV was for the broadband news and information site, not for speed tests. The supervisor maintained that being considerate to reviewers and readers by conveying accurate and precise information should be a golden rule for academic writers.

Example 6: OV: As the UK’s largest independent broadband news and information site, thinkbroadband.com is well-known and widely used as an Internet speed test website. RV: As a major broadband news and information site in the UK, thinkbroadband.com provides its users with a web page (thinkbroadband.com/speedtest) for convenient Internet speed tests.

The supervisor also suggested providing enough information to facilitate the possible future replication of their work. He pointed out that it would be insufficient simply to state that they studied the source code “carefully” (see OV in Example 7), but rather they should elaborate on how they obtained the source code and what format the code was in. In the supervisor’s view, the information would be valuable to reviewers since researchers rarely gain access to the source code. The students made the changes accordingly (see RV in Example 7).

Example 7: OV: When exploring the inner implementation of Fast.com, we study carefully on it source code. RV: We get the client-side JavaScript source code of fast.com using the Chrome DevTools in January 2018, and discover that the code is not encrypted, thus greatly facilitating us to investigate the implementation of fast.com in a white-box manner.

3.3. Cohesion and Coherence

Cohesion and coherence refer to the links between and the order of information and ideas [13]. The supervisor paid much attention to the logical flow of ideas. For instance, he suggested moving a sentence about the features of a speed test website to the next paragraph, after describing how they examined the website, since it would make logical sense that they examined the website before discovering its features. Similarly, he suggested reversing the order of the sentences discussing a certain mechanism and the sentences presenting the research methods to reflect the fact that the mechanism was revealed by the research conducted.

3.4. Linguistic Accuracy, Appropriateness, and Concision

The fourth feedback focus category pertains to surface-level language forms and clarity of meaning [13]. The supervisor pointed out surface-level language issues with regard to accurate grammar, appropriate word choice, and concision in writing.

3.4.1. Grammatical Accuracy

The supervisor identified quite a few grammatical inaccuracies in the original version. He noticed comma splice errors, i.e., joining two independent clauses with only a comma (e.g., OV in Example 8). This type of error is common among Chinese learners of English, and many are unaware that it is problematic because of the difference in the use of commas in English and Chinese. In terms of tense, the supervisor suggested replacing the future tense and past future tense with the present simple tense. As a result, only the present simple tense is used in the revised version. Concerning conventions, a consistent issue that the supervisor raised was the use of capital letters in URL links (e.g., “Fast.com” and “SourceForge.net/speedtest”). He suggested uncapitalizing all letters in the URL links. For spelling errors involving the use of hyphens, the supervisor pointed out that a hyphen is needed for a compound word that modifies a noun but not when it functions as an object. For example, a hyphen is needed in Example 9 (since “128 MB” is used to modify “file”) but not in Example 10 (since “128 MB” is the object of the verb “reaches”). According to the supervisor, even though the inclusion or omission of a hyphen does not impede understanding, using hyphens correctly and consistently demonstrates one’s professionalism in academic writing. For other spelling errors (e.g., “afrer” instead of “after”), the supervisor suggested double-checking the manuscript with the help of text editors such as WinEdt or document preparation systems such as TeX.

Example 8: OV: It has globally distributed servers, thus the user is assigned to the nearby Netflix server during the test. RV: It has globally distributed servers, and thus the user is assigned to the nearby Netflix server during the test.

Example 9: RV: … when the 128-MB file is successfully transferred.

Example 10: RV: … the file size reaches 128 MB.

3.4.2. Word Choice Appropriateness

Another group of comments observed in the current study was related to appropriate word choice for the clear and precise expression of ideas. For example, the supervisor questioned the use of “overall” before “speed tests” (OV in Example 11), believing that an overall test is one that is considered as a whole. He suggested replacing “overall” with “comprehensive,” meaning including everything needed or relevant. On that, he further commented that their speed tests were not actually comprehensive, so they ended up removing “overall” (RV in Example 11).

Example 11: OV: …, we conduct overall black-box Internet speed tests …. RV: …, we conduct comprehensive black-box Internet speed tests ….

Revisions were also made for appropriate collocation. For instance, the supervisor suggested replacing “volatile” in Example 12 with “diverse”. He reasoned that “volatile” in science is most commonly used before a liquid or substance, indicating that it changes easily into a gas. He suggested “diverse” as a more appropriate attributive modifying “network environment”. Similarly, the supervisor pointed out that the adjective “quick” does not collocate well with the noun “speed”, and suggested replacing it with “high”. He believed that the improper collocation “quick speed” reflected negative transfer from Chinese.

Example 12: OV: … to the volatile network environments. RV: … to the diverse network environments.

3.4.3. Concision

The supervisor also made comments pertaining to concision in research article writing, an aspect that was not identified in the existing research on supervisory feedback. For instance, the supervisor pointed out the redundancy in using “meanwhile” together with “also” (Example 13). In another case, he suggested replacing “as is shown” with “as shown.” In response to the supervisors’ suggestion, one of the students checked whether “is” was optional in the phrase and found from linggle.com, a computer-assisted language search engine, that “as shown” was indeed used much more frequently than “as is shown.” The supervisor confirmed that, even though both phrases are grammatically correct, the former is more concise and more commonly used in research articles.

Example 13: OV: Meanwhile, it also publishes an open and customized speed test API ….

3.5. Visual Elements

Apart from the four categories of feedback foci presented above, the analysis also revealed a category unidentified in the existing research. It pertains to visual elements such as equations, tables, and figures.

3.5.1. Equations vs. Statement

One group of comments on the use of visual elements is related to the use of equations. The supervisor suggested removing several equations from the manuscript, as illustrated in the following example. In his view, an equation is not needed if something can be stated clearly in words; an equation should be used only when something is difficult to express in words but relatively easy to express with an equation.

Example 14: OV: It might also terminate only when the 128-MB file is successfully downloaded. The user’s speed will be calculated through the following equation:

$$Internet Speed=\frac{Last Transferred File Size}{Time Spent}$$

RV: During the test, the server first transfers a 128-KB file to the client. Every time a file is successfully transferred, the server checks whether the transmission time is longer than 8 s. If so, the test terminates and the users download speed is measured by the average download speed of this file.

3.5.2. Tables vs. Figures

The supervisor highlighted the importance of effective figures in academic writing in his field of research. According to him, reviewers may not have time to read a manuscript word by word but will definitely go over each figure. Therefore, he suggested making every figure impress reviewers favorably, if possible. He had high praise for the visual effects of figures in many papers presented at top conferences in their field and commented that those figures looked terrific and pleasing to the eye. The supervisor also expressed a clear preference for figures over tables in presenting data. He claimed that it took him quite some time to understand the data presented in a table and it was painful to stare at those numbers. He believed a much better way to present the data would be a wave chart. When one of the students expressed the concern that the chart might look very crowded with the data for ten sessions, the supervisor recommended including only the most important sessions while leaving out the less representative ones if necessary. Having said that, however, he suggested that the students first present all ten sessions in the wave chart. He would determine, on examining the chart, whether it would actually be too crowded.

4. Discussion

This study investigated the focus categories of the feedback provided by a Chinese supervisor on his doctoral students’ research article intended for submission to a top conference. Via an analysis of the supervisory feedback comments and a comparison of different versions of the research article, we found that the supervisory feedback could be classified into five categories: content, requirements, cohesion and coherence, linguistic accuracy, appropriateness and concision, and visual elements.

Four of the five categories of feedback foci revealed in this study resemble those identified by Basturkmen et al. [13], suggesting that the Chinese supervisor’s feedback foci were similar to those of supervisors in New Zealand. The supervisor placed much focus on linguistic accuracy and appropriateness as well as content. His content-focused comments were concerned with the completeness of idea development, accuracy of claims, accuracy of terminology, and precision of expressions. This is consistent with previous findings that supervisors are equipped to provide discipline-specific suggestions on matters such as terminology [12], supporting “the supervisor’s role as a provider of academic expertise” ([13], p. 441). Meanwhile, the supervisor also made several comments on grammatical accuracy and word choice appropriateness. This is different from the finding of previous studies that feedback provided by experts focuses on ideas and content whereas language teachers’ feedback addresses problems of form and presentation [11,12]. This discrepancy is likely due to the different roles played by the experts. The experts in the previous studies were peer reviewers of a journal, and it is usually not up to peer reviewers to address linguistic issues. In cases of many linguistic problems in a manuscript, they only need to recommend that the authors proofread the manuscript, perhaps with the help of a native speaker of English. The supervisor in this study, however, needed to ensure the linguistic as well as content quality of the manuscript to meet the expectations for academic writing to be highly articulate and well-proofread [13,15,16]. As revealed by Morton and Storch’s study, clumsy writing with spelling errors, typos, and poorly structured paragraphs seemed to have a strong negative impact on the reader, who would be “less inclined to believe what the writer says” and “would read much more critically” ([28], p. 20).

Different from the supervisors in Basturkmen et al.’s [13] study, who provided very few requirement- or logic-focused comments (in particular the latter), the supervisor in this study offered quite a few of both. With regard to the requirements, he kept reminding students of the need to appropriately borrow and incorporate ideas from other resources for a plagiarism-free research article. The importance of academic integrity and ethical behavior can never be overstated, especially in the context of the development of large language models (LLMs), such as ChatGPT. Measures should be taken to educate students about the proper use of the versatile tool to mitigate the risk of plagiarism [29,30]. In addition, he suggested adequate and clear descriptions for a broad readership. This is consistent with Wallwork and Southern, who maintained that the aim of academic writing is “not to force your reader to decide what certain phrases mean, but instead to make those phrases so clear that reading your paper is effortless” ([31], p. 49). The supervisor also underscored the need for a clear thread of logic. The more emphasis on logic by the supervisor in this study than by those in Basturkmen et al.’s [13] study may be due to the different natures of the text on which the supervisors provided feedback: a draft of a manuscript to be submitted to a top conference versus drafts of master’s and doctoral theses. Basturkmen, East, and Bitchener attributed the extreme rarity of logic-focused comments in their study to the “difficulty of diagnosing and commenting on problems in the coherence of writing” ([13], p. 442). Identifying an error in coherence entails deconstructing the text to understand the logic underlying it and to detect any flaw in the logic. Therefore, it is highly demanding to detect and comment on coherence errors. Nevertheless, the supervisor made quite some coherence-focused comments as a result of the high value that he attached to the logical development of the manuscript.

In addition, the current study revealed three feedback foci—concision, consideration of reviewers and readers, and the use of visual elements—which were reported neither in Basturkmen et al.’s [13] research examining supervisory comments on master’s and doctoral theses nor in Lei and Hu’s [25] study investigating supervisory comments on research writing for publication purposes. The supervisor under study suggested removing redundant words for concision, in line with advice frequently provided to novice academic writers (e.g., [32]). He also emphasized the consideration of reviewers and readers in writing, suggesting revisions intended to impress reviewers favorably. In addition, the supervisor commented on visual elements. He preferred clear and concise statements and figures to equations and tables, respectively, believing that the former could better promote understanding by the reader than the latter. One possible reason for the new feedback foci could be the ability of the supervisor in the current study to provide detailed guidance on subtle language issues and discipline-specific issues as he was a prolific research writer and skilled at English writing. In contrast, supervisors in some previous research lacked the ability to provide such guidance [21,33,34]. Another possible reason for the new finding might be that both supervisees in the current study were at their early stage of doctoral studies, and the supervisor attempted to support their development of research writing skills by providing detailed, specific guidance in the form of feedback concerning both subject knowledge and discursive competence. The supervisor provided manageable yet not overwhelming feedback [35] to the supervisees to facilitate the novice writers’ “initiation into scholarly publishing” ([25], p. 35).

Admittedly, this study has its limitations. Three major limitations relate to the fact that we focused on the feedback provided by one supervisor. For one thing, the supervisor under study was skilled at English writing, which might distinguish him from many non-native-speaking supervisors. His strong research writing ability and his confidence in it might enable him to provide rich comments on various aspects of the manuscript, perhaps more than many other non-native-speaking supervisors would. A related major limitation is that, with only one supervisor under study, it is unclear whether some of the findings are discipline-specific. The number of comments provided on the whole and in each feedback focus category could vary from discipline to discipline. It was found that math/computer science supervisors provided considerably more feedback comments on students’ theses and dissertations than did supervisors working in management/marketing and arts/humanities [13]. Future research could explore supervisors from different disciplines to obtain a more comprehensive picture of supervisory feedback. Thirdly, students’ perspectives were not taken into consideration in the current study. It would be interesting to examine students’ engagement with the feedback in future studies to see whether they responded to all the comments the supervisor provided and how they negotiated with the supervisor. Additionally, for the purpose of triangulation, semi-structured interviews could be undertaken with both the supervisees and the supervisor.

These study findings have several implications. First, they may inform language teachers, especially ERPP course instructors, of the feedback that supervisors value in research paper writing. ERPP presents challenges to language teachers since they have to go beyond teaching grammatical accuracy [36]. These findings suggest that providing feedback focused on ideas and content, such as the preference for clear and concise statements over equations and the need for a clear and logical flow of ideas, is not completely beyond the reach of language professionals. Well-informed language professionals can provide feedback that better caters to the needs of students in research writing for publication purposes. Second, supervisors’ ERPP competence plays a vital role in providing detailed and manageable guidance to supervisees. Supervisors are expected to present responsible and credible feedback since their supervisees tend to treat them as role models and learn much from them [21]. Third, efforts are needed to promote collaborative pedagogies between language teachers and supervisors [36,37]. These may facilitate the provision of detailed, comprehensive, and manageable feedback, assisting students in transitioning from “test-oriented English learning and short-composition writing to processing and producing longer research articles in English for international publications” ([38], p. 73). While LLMs can be beneficial for grammar-checking, proofreading, and editing, they are no substitute for human language teachers and supervisors. Since these models are not error-free and may lack a complete contextual understanding, they should be used in tandem with, rather than as a substitute for, human expertise and judgement [28].