The Changes in Text Cohesion of Senior High School Students Measured by Coh‐Metrix as a Function of Grade Level

: National English Curriculum Standards for General High School sets requirements for text cohesion and text cohesion is an essential component of English writing and English writing evaluation. However, previous studies explored text cohesion mainly from perspectives of college students, horizontal comparison and traditional tools. In this study, a corpus of 300 essays by grade level (senior one and senior three) are analyzed computationally by Coh‐Metrix to examine whether text cohesion improves as grade level rises. During the specific process, this paper aims to identify characteristics and discriminate differences between the text cohesion created by the two grades. The main instrument is Coh‐Metrix—a type of computational tool that offers over 200 measurements to quantify texts. This study mainly employs the measures under the category of referential cohesion, LSA and connectives of it to assess text cohesion. The results demonstrate that as the grade level increases, text cohesion gets better. Specifically, these students produce more noun overlap of adjacent sentences, argument overlap of adjacent sentences, noun overlap of all sentences, argument overlap of all sentences, stem overlap of all sentences, LSA overlap of adjacent sentences, LSA overlap of adjacent paragraphs, LSA overlap of given/new sentences all connectives and additive connectives.


INTRODUCTION
As the beginning part of this study, there are some basic information: background, purpose and significance as well as the layout of this paper.National English Curriculum Standards for General High School (2017 edition, 2020 revision) requires: senior high school students should be equipped with abilities of using discourse cohesion means to form written discourse which express meanings and embody the logical relevance of meanings.Obviously, text cohesion plays an important role in English writing.When the text cohesion is good in one text, readers can understand meanings and ideas better which the author wants to convey.At the same time, it is widely seen as an important part of writing evaluation.However, in real teaching practices, many teachers just consider students' text cohesion from an abstract angle.Fortunately, the development of computational linguistics and corpus open up the way to more comprehensive research of text cohesion.Particularly, the appearance of Coh-Metrix brings possibility for quantifying text cohesion.The purpose of this study is using Coh-Metrix to examine whether text cohesion improves with increasing grade level.Specifically, this paper identifies characteristics and differences of text cohesion on the same topic in two senior high school grades.Based on the findings, this paper is of interest to senior high school English teachers, because it can offer some new ideas for them to teach and assess English writing.

LITERATURE REVIEW
For a research, it is necessary to explain some concepts and learn about previous studies of the topic at the beginning. (

1)Text Cohesion
Text cohesion is one of the logic relationships which can effectively combine propositions and thoughts of text (Liang, 2006).Usually, it can be divided into two categories: local cohesion and global cohesion.Local cohesion refers to the use of noun phrases in place of pronouns, the definition of new terms, the addition of argument overlap between sentences, and the addition of connectives to make the relationships between ideas more clear.The term "global cohesion" refers to the practice of starting each paragraph with a topic sentence and adding topic headers to sections.The degree of difficulty and the reader's comprehension of a text are both crucially influenced by cohesion (McNamara et al., 2014).
A large proportion of previous studies on text cohesion are theoretical research.The origin, ideas, cognitive foundation, and relationship to other terms of text cohesion are all thoroughly explored (Guo, 2003;Miao, 1998).Certainly, scholars have not ignored empirical research.Liang Maochen measured students' text cohesive ability of 120 English writing divided into high grades group and low grades group through Coh-Metrix.He concluded that while there are substantial differences in students' overall writing abilities, the development of students' discourse cohesion ability and overall writing ability is essentially synchronous (Liang, 2006).Shang Yanzhi (2015) summarized cohesive errors frequently made by students in English writing and proposed teaching strategies to cultivate students' writing ability starting from textual cohesion issues.Beck, MCKeown, Omanson and Pople (1984) investigated the benefits of increasing the ease of texts for children.They revised two narrative texts aiming to solve surface, knowledge and content problems.Finally, their paper showed that all of the readers could benefit from the processing of the texts and text cohesion could improve readers' comprehension and memory of texts.To be short, text cohesion of English writing received a slightly low attention (Xie, 2020).Although, some researchers have investigated text cohesion, their research hardly employs computational tools.There is still a room to investigate text cohesion with advanced instruments.
( For some scholars, Coh-Metrix is a useful tool to research discourses' difficulty of exams or textbooks.Jiang Jinlin and Han Baocheng (2018) made use of Coh-Metrix to compare difficulty of 163 reading texts from CET 6, TOEFL and IELTS.Their findings was that texts of CET 6 were easier than others and different from the two tests in four dimensions:narrative, connectivity, lexical specificity and referential cohesion.Chen Anni and Guo Aiping (2019) researched the change in difficulty of the first four volumes of READING AND WRITING of NEW HORIZON from lexical, syntactic and cohesive dimensions through Coh-Metrix.They found that difficulty progression was not obvious both from the perspective of macroscopic view and language characteristics.These studies can provide reference basis for exam development, reading grading, and textbook compilation.The studies of automated writing evaluation based on Coh-Metrix specifically consider the reliability and validity of a certain automated essay scoring system or its application for constructing a scoring system.Zhang Guoqiang (2022) utilized Coh-Metrix and some other tools to analyze lexical complexity, syntactic complexity and text cohesion of 826 texts from CET-6.He constructed predictive models and compared the predictive variables to examine whether the reliability and validity of automated essay scoring were different between different types of writing tasks.As for S Latifi and M Gierl (2021)，they researched nine feature categories of Coh-Metrix for developing prompt-specific AES scoring models.Their findings demonstrated that Coh-Metrix features could be used to support educational goals and scoring goals in large-scale language assessments.As for prediction of writing quality, researchers always construct models to achieve their goals.McNamara et al. (2010) investigated 120 native speakers' essays and pointed out that high quality writings are with various and uncommon words as well as more complex sentences.From Du Huiying and Cai Jinheng (2013), they utilized Coh-Metrix to discover language characteristics which can affect English major writing quality of arguments.Consequently, a predictive score model was carried out.Jia Wenfeng and Zhang Peixin (2020) discussed the relationships between different text features and writing quality.Their conclusion was that text fluency, syntactic complexity, cohesion, situational models and other characteristics were relevant to essays' quality.More importantly, discourse features could effectively predict writing quality.
Coh-Metrix is also employed to identify texts' characteristics.Arthur C. Graesser, Danielle S. McNamara, and Jonna M. Kulikowich (2011) discussed five major factors that accounted for most of the variance in texts.They found out that texts could be quantified by Coh-Metrix and it was helpful for assigning the right text to the right student at the right time.Based on their findings, Andrew Elfenbein (2011) researched the usefulness of Coh-Metrix measuring textual features from a broader way.In his opinion, Coh-Metrix provided more possibilities for researchers to identify the roles of text features in comprehension for different populations at different moments in development.
Briefly speaking, although some researchers have investigated text characteristics utilizing Coh-Metrix, text cohesion is just seen as a small part in their papers.Their description or analysis of text cohesion is relatively general.In conclusion, the research on text cohesion based on Coh-Metrix is still not enough.Although some scholars have studied this topic, the participants of previous studies are almost university students while senior high school students are ignored.Meanwhile, previous studies of text cohesion have payed attention to theoretical aspects, relation with other variances or horizontal comparison between different participants or genres while longitudinal characteristics and differences are ignored.Therefore, the research, which uses Coh-Metrix to identify features and differences of text cohesion of high school students in a more specific and comprehensive way to finally examine whether text cohesion improve with increasing grade level, is still needs to be done.

METHODOLOGY (1)Research questions
This essay seeks to determine whether text cohesion varies with increasing grade level of senior high school students.Specifically, the research questions are as follows: 1.What are the characteristics of text cohesion in the two senior high school classes?Are there any differences of text cohesion in the two senior high school grades and if so, what are they?
(2)Research Materials The corpus of this paper consists of 300 texts from Pigaiwang which are on the same topic and belong to two senior high school grades in Jiangsu province (150 texts each grade).The topic is "Work Hard To Make Our Dreams Come True" and students need to write essays with a length from 300 to 500 words without limit on specific topic and genre.
(3)Research Tools and Measures Developments in computational linguistics and discourse processing provide possibilities for researchers to measure text with a wide range of indices.These indices have been gathered in Coh-Metrix 3.0 (Graesser et al., 2004).It can quantify lexical, syntactic and discourse characteristics and covers 11 parts: descriptive statistics, text easability principle component scores, referential cohesion, LSA (Latent Semantic Analysis) (Landauer et al., 2007), lexical diversity, connectives, situation model, syntactic complexity, syntactic pattern density, word information and readability.Also, Coh-Metrix overcomes the shortcomings that manual analysis are too tedious and subjective and improve quality of research results to some extent.
To just measure text cohesion, this paper generally adopts the idea of Jiang Jinlin (2016) that referential cohesion, LSA and connectives are directly relevant to text cohesion.First, referential cohesion refers to overlap in content words between local sentences or coreference which vary along local cohesion and global cohesion in Coh-Metrix (McNamara et al., 2014).Second, LSA is a advantage of Coh-Metrix, which surmounts the statistics of surface features and constructs a hidden semantic space of text.Third, connectives matters in creating cohesive connections between ideas and clauses and provide clues to the organization of texts (Cain & Nash, 2011).

DISCUSSION
This paper explored characteristics and differences of text cohesion of two grade levels.The following part will show the discussion about the previous results. (

1) Characteristics of the two grade levels' text cohesion
The results above showed the characteristics of the two grades' text cohesion compared to native speakers are both similar and different.In terms of referential cohesion, the results indicated that there were more argument overlaps in Chinese students' writings.Argument overlap occurs when there is overlap between a noun in one sentence and the same noun (in singular or plural form) in another sentence; it also occurs when there are matching personal pronouns between two sentences (McNamara et al., 2014).Then, no significant difference was showed in the category of LSA of both grades suggested that the two grades' aspects of LSA were approaching native speakers.In addition, the above results revealed that expanded temporal connectives incidence of the two grades was inferior to native speakers.

www.centuryscipub.com Volume 3 Issue 20, 2023
With reference to differences, they were displayed in category of connectives.From results above,temporal connectives incidence of senior grade one was higher than native speakers, whereas additive connectives incidence was lower.At the same time, higher value of adversative and contrastive connectives incidence was obtained by senior grade three than native speakers , while lower value of temporal connectives incidence was gained.
In sum, the results indicated that students of China overuse argument overlap and connectives especially adversative and contrastive connectives and temporal connectives while the application of expanded temporal connectives and additive connectives was not enough.The reason may be due to the materials used in this paper."Working hard to make our dreams come true" was a small issue that provided efficiency for students.For example, the word "hard" was used repeatedly in both individual and all essays.
(1) Differences of text cohesion between the two grade levels The results above revealed that significant differences exited in noun overlap of adjacent sentences, argument overlap of adjacent sentences, noun overlap of all sentences, argument overlap of all sentences, stem overlap of all sentences, LSA overlap of adjacent sentences, LSA overlap of adjacent paragraphs, LSA overlap of given/new sentences, all connectives incidence and additive connectives incidence.Meanwhile, they indicated that senior three obtained better text cohesion than senior one.Namely, text cohesion of senior high school students improved as grade level increases.
Overall, this finding lent support to the research of Crossley et al. (2011), which emphasized the importance of cohesion features in identifying grade levels.They researched and analyzed essays computationally by level (9th grade, 11th grade and college freshman).Their findings, however, indicated that these students produced fewer cohesive characteristics in text as a function of grade level, in contrast to this paper's findings that higher cohesion was associated with rising grade level.For example, in Crossley et al.'s study, ninth-grade students were more likely to produce texts that had a larger incidence of positive logical connectives and more content word overlap, but these writings were judged to be of lesser quality.A possible explanation was the the participants of the two research were with different educational levels.Namely, Crossley et al.'s study both involved college students and senior high school students, but this study only touched on the latter.Raters' criteria and objectivity of their paper was the other likely reason, because expert raters were likely high knowledge readers (McNamara, 2001).What' more, the results of this study were different from the findings of Crossley and McNamara (2012), which pointed out that high-quality L2 essays and low-quality L2 essays were distinguished from linguistic indices related to linguistic sophistication and not text cohesion.This vary may have resulted from the difference of the corpus between the two research.The essays of Crossley and McNamara were from Advanced Level Examination (HKALE) of graduating senior high school students at Hong Kong while this study' s essays were from two senior grade levels at Jiangsu province.The English proficiency of these two regions' students was considerably different from one another since the former had a deeper accumulation because of historical and cultural factors.Moreover, Freedman and Pringle (1980) noted that as grade levels rise, improvements in writing cohesion appeared to slow down.In particular, there were no distinctions between essays produced by seniors in high school and those written by third-year college students in terms of textual unity, organization, development, or coherence.This difference from this study may also result from the different educational levels of the two studies' participants.
To be short, this paper's findings differed from many others', which found out text cohesion were worse with increasing grade levels.In nature,this kind of outcome was probably the product of working memory and prior knowledge (Kellogg, 2008).More specifically, a writer's working memory capacity increases with experience.The capacity along with a writer's background knowledge allows the writer to create writings a greater depth of information.Therefore, one possible explanation for the difference may be the inadequate working memory and background knowledge of this paper's participants.

CONCLUSION
This study sought to examine whether text cohesion improves as grade level increases in senior high school.One finding revealed that argument overlap, content word, LSA overlap of adjacent paragraphs and connectives except expanded temporal connectives were overused.Also, the noun overlap of adjacent sentences, the argument overlap of adjacent sentences, the noun overlap of all sentences, the argument overlap of all sentences, the stem overlap of all sentences, the LSA overlap of adjacent sentences, and the LSA overlap of adjacent paragraphs, LSA overlap of given/new sentences, all connectives and additive connectives all showed significant differences.They suggested that senior three had more cohesive text than senior one.In particular, senior high school students' text cohesion www.centuryscipub.comVolume 3 Issue 20, 2023 gets better with increasing grade level.
Although this paper research text cohesion with computational tool Coh-Metrix and senior high school students, there are sill several limitations.First, the current study roughly considered grade level as proficiency level or quality level.Researchers can organize a pre-test to ensure participants' level in further study.Second, the sample size was small to some extent.Future study can research a larger sample size to make results more reliable and valid.Third, despite the fact that this study tries to show how text cohesion develops as grade levels rise, grade continuity was not covered.It is vital to include senior two in further research.Due to the needs of aviation, aerospace and other production fields, the large horizontal milling machine with high precision, high performance, high speed and heavy load capacity is gradually paid attention to, among which the swing Angle milling head as a key part of the large horizontal milling machine is the key object of design and optimization.Because the precision of the swing milling head determines the precision of the whole machine tool, the optimization of the swing milling head box, especially the lightweight design, plays an important role in improving the precision of the milling head and the performance of the machine tool.
2)Coh-Metrix Coh-Metrix, a web-based text analysis tool developed by McNamara et al. from the University of Memphis in the United States, provides more than 200 measures and can evaluate surface and deep characteristics of texts (McNamara et al., 2014).Domestic and foreign studies based on Coh-Metrix have focused on difficulty of reading discourse, automated writing evaluation, prediction of writing quality and texts' characteristics.