[[{“value”:”
Large Language Models (LLMs) have been widely discussed in several domains, such as global media, science, and education. Even with this focus, measuring exactly how much LLM is used or assessing the effects of created text on information ecosystems is still difficult. A significant challenge is the growing difficulty in differentiating texts produced by LLMs from human-written texts. There is a chance that unsupported AI-generated language will be misconstrued for reliable, evidence-based writing because studies have revealed that humans’ capacity to distinguish AI-generated content from human-written information is hardly better than random guessing.
In scientific research, ChatGPT-generated medical abstracts frequently avoid detection by AI systems and even by specialists. There is a chance of false information because more than 700 untrustworthy AI-generated news websites were found in the media. Individually, AI-generated text might be identical to human-written content, yet corpus-level trends show variations. When analyzing individual cases, biases can be subtly and undetectably amplified by the constant output of LLMs. Research has indicated that employing a solitary algorithm to make employment selections may result in results that are more uniform.
To overcome these issues, effective techniques for assessing LLM output at a broader scale are required. The “distributional GPT quantification” approach is one suggested technique, as it calculates the percentage of AI-generated content in a corpus without examining individual examples. This approach combines maximum likelihood estimation for texts of unclear origin with reference texts that are known to be created by humans or AI. Compared to existing AI text detection techniques, this method greatly decreases estimation errors and is far more computationally efficient.
Evidence from empirical research indicates that several adjectives are used more frequently in AI-generated texts than in texts created by humans, as seen by the abrupt increase in their usage frequency in recent ICLR reviews. This enables researchers to produce consistent and noticeable results by parameterizing their framework for probability distribution. Similar outcomes are possible when using verbs, non-technical nouns, and adverbs.
An extensive case study of writings submitted as reviews to prestigious AI conferences and publications was used to test the framework. According to the results, a tiny but noteworthy percentage of evaluations that were posted after ChatGPT’s release may have had significant AI modifications. Evaluations submitted to the Nature family publications did not show this tendency. The study also looked at how frequently and in what situations AI-generated material appears, as well as how it varies from reviews authored by experts at the corpus level.
The team from Stanford research has summarized their primary contributions as follows.
A simple and effective way has been proposed to calculate the percentage of text in a big dataset that has been significantly altered or produced by AI. This approach makes use of historical data that has been produced by AI or written by human specialists. A maximum likelihood method has been used to estimate the percentage of AI-generated text in the target corpus by utilizing this data.
A methodology has been employed to examine reviews that have been submitted to eminent scientific and ML conferences, such as EMNLP, CoRL, ICLR, NeurIPS, and EMNLP, as well as articles that have been published in Nature portfolio journals. With this case study, patterns in the application of AI can be seen since ChatGPT was released.
The team has also noted modifications at the corpus level that arise from integrating AI-generated texts into an information ecosystem. These revelations help in the comprehension of how the general landscape of scientific reviews and publications is impacted by the existence of AI-generated content.
In conclusion, the study suggests a new paradigm for effectively tracking material altered by AI in information ecosystems, highlighting the significance of assessing and analyzing LLM output overall to identify minor yet enduring effects of AI-generated language.
Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter..
Don’t Forget to join our 46k+ ML SubReddit
Find Upcoming AI Webinars here
The post Monitoring AI-Modified Content at Scale: Impact of ChatGPT on Peer Reviews in AI Conferences appeared first on MarkTechPost.
“}]] [[{“value”:”Large Language Models (LLMs) have been widely discussed in several domains, such as global media, science, and education. Even with this focus, measuring exactly how much LLM is used or assessing the effects of created text on information ecosystems is still difficult. A significant challenge is the growing difficulty in differentiating texts produced by LLMs
The post Monitoring AI-Modified Content at Scale: Impact of ChatGPT on Peer Reviews in AI Conferences appeared first on MarkTechPost.”}]] Read More AI Paper Summary, AI Shorts, Applications, Artificial Intelligence, Editors Pick, Language Model, Large Language Model, Staff, Tech News, Technology