{"667346":{"#nid":"667346","#data":{"type":"news","title":"Misinformation Detection Models are Vulnerable to ChatGPT and Other LLMs","body":[{"value":"\u003Cp\u003EExisting machine learning (ML) models used to detect online misinformation are less effective when matched against content created by ChatGPT or other large language models (LLMs), according to new research from Georgia Tech.\u003C\/p\u003E\r\n\r\n\u003Cp\u003ECurrent ML models designed for and trained on human-written content have significant performance discrepancies in detecting paired human-generated misinformation and misinformation generated by artificial intelligence (AI) systems, said Jiawei Zhou, a Ph.D. student in Georgia Tech\u2019s School of Interactive Computing.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EZhou\u2019s paper detailing the findings is set to receive a best paper honorable mention award at the 2023\u0026nbsp;ACM CHI Conference on Human Factors in Computing Systems. Advised by Associate Professor Munmun De Choudhury, Zhou\u2019s research demonstrates that LLMs can manipulate tone and linguistics to allow AI-generated misinformation to slip through the cracks.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u201cWe found the AI-generated misinformation carried more emotions and cognitive processing expressions than its human-created counterparts,\u201d Zhou said. \u201cIt also tended to enhance details, communicate uncertainties, draw conclusions, and simulate personal tones.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u201cWe\u2019re one of the very first to look at this risk. As more people started to use ChatGPT, they\u2019ve noticed this problem, but we were one of the first to provide evidence of this risk. And there are more efforts needed to raise public awareness about this potential and call for more research efforts to combat this risk.\u201d\u003C\/p\u003E\r\n\r\n\u003Cp\u003EZhou started exploring GPT-3 in 2022 because she wanted to know how one of the early predecessors to ChatGPT would handle prompts that included misinformation about the Covid-19 pandemic. She asked GPT-3 to explain how the Covid-19 vaccines could cause cancer.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u201cThe results were very concerning because it is so persuasive,\u201d Zhou said. \u201cI had been studying informatics and misinformation for a time, and it was still persuasive, even to me. The output would say, \u2018It can cause cancer because there is this researcher at this institute, and their research is based on medical records and diverse demographics. The research supports this possibility.\u2019 The writing of it is so scientific.\u201d\u003C\/p\u003E\r\n\r\n\u003Cp\u003EZhou and her collaborators accumulated a dataset of human-created misinformation, including more than 6,700 news reports and 5,600 social media posts. From that set, Zhou and her team extracted the most representative topics and documents of human-generated misinformation. They used those to create narrative prompts, which they fed to GPT and recorded the output.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EBoth the GPT-generated output and the original human-created dataset were used to test an existing misinformation detection model called COVID-Twitter-BERT (CT-BERT).\u003C\/p\u003E\r\n\r\n\u003Cp\u003EZhou said while the human- and AI-generated datasets were intentionally paired, a statistical test showed there are significant differences in detection model performance.\u003C\/p\u003E\r\n\r\n\u003Cp\u003ECT-BERT experienced a decline in performance in detecting AI-generated misinformation. Out of 500 prompts based on AI-generated misinformation, it failed to recognize 27 as false or misleading, compared to missing only two from the human-generated prompts.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u201cThe core reason is they are linguistically different,\u201d Zhou said. \u201cOur error analysis reveals that AI misinformation tends to be more complex, and it tends to mix factual statements. It uses one fact to explain another, though the two things might not be related. The tone and sentiment are also different. And there are less keywords that detection tools normally look for.\u201d\u003C\/p\u003E\r\n\r\n\u003Cp\u003EZhou\u2019s experiments showed that GPT could use information to create a news story using objective, straightforward language and use that same information to create a sympathetic social media post. That points to its capability of changing tone and tailoring messages.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u201cIf someone wants to promote propaganda, they can use it to customize a narrative toward a specific community,\u201d Zhou said. \u201cThat makes the risk even greater. It shows that it has some flexibility to alter its tone for different purposes. For news, it can sound logical and reliable. For social media, it conveys information quickly and clearly.\u201d\u003C\/p\u003E\r\n\r\n\u003Cp\u003EAs LLMs continue to rapidly grow and expand, so do the risks of misinformation.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EChatGPT operates on Open AI\u2019s GPT-3.5 and GPT-4 models, the latter of which was released on March 14. Since ChatGPT was released, Zhou has given it the same prompts she gave to GPT-3. The results have improved with some corrections, but the latter has the advantage of having more available information about Covid-19, she said.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EZhou said steps should be taken immediately to evaluate how misinformation detection tools can adapt to ever-improving LLMs. She described the situation as an \u201cAI arms race,\u201d and the tools that are currently used to combat misinformation are well behind.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u201cThey are improving the generative capabilities of LLMs,\u201d she said. \u201cThey\u2019re more human-like, more fluent, and less and less distinguishable from human creations. We need to think about ways we can distinguish them and how we can improve our misinformation detection abilities to catch up.\u201d\u003C\/p\u003E\r\n","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003ENew research indicates that current machine learning models trained on human-produced content can struggle to detect falsehoods generated by AI-powered chatbots.\u003C\/p\u003E\r\n","format":"limited_html"}],"field_summary_sentence":[{"value":"Because falsehoods generated by ChatGPT are so convincing, even trained researchers struggle to identify misinformation."}],"uid":"32045","created_gmt":"2023-04-14 14:15:06","changed_gmt":"2023-04-14 14:20:36","author":"Ben Snedeker","boilerplate_text":"","field_publication":"","field_article_url":"","dateline":{"date":"2023-04-14T00:00:00-04:00","iso_date":"2023-04-14T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"670544":{"id":"670544","type":"image","title":"Jiawei Zhou, a Ph.D. student in Georgia Tech\u2019s School of Interactive Computing.jpeg","body":null,"created":"1681481714","gmt_created":"2023-04-14 14:15:14","changed":"1681481714","gmt_changed":"2023-04-14 14:15:14","alt":"Jiawei Zhou, a Ph.D. student in Georgia Tech\u2019s School of Interactive Computing.","file":{"fid":"253425","name":"Jiawei Zhou, a Ph.D. student in Georgia Tech\u2019s School of Interactive Computing.jpeg","image_path":"\/sites\/default\/files\/2023\/04\/14\/Jiawei%20Zhou%2C%20a%20Ph.D.%20student%20in%20Georgia%20Tech%E2%80%99s%20School%20of%20Interactive%20Computing.jpeg","image_full_path":"http:\/\/www.tlwarc.hg.gatech.edu\/\/sites\/default\/files\/2023\/04\/14\/Jiawei%20Zhou%2C%20a%20Ph.D.%20student%20in%20Georgia%20Tech%E2%80%99s%20School%20of%20Interactive%20Computing.jpeg","mime":"image\/jpeg","size":47321,"path_740":"http:\/\/www.tlwarc.hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/2023\/04\/14\/Jiawei%20Zhou%2C%20a%20Ph.D.%20student%20in%20Georgia%20Tech%E2%80%99s%20School%20of%20Interactive%20Computing.jpeg?itok=hG9TESO_"}},"670545":{"id":"670545","type":"image","title":"Jiawei Zhou-munmun.jpeg","body":"\u003Cp\u003ESchool of Interactive Computing Ph.D. student Jiawei Zhou, left, and associate professor Munmun De Choudhury, demonstrate in their latest paper that misinformation detection models are vulnerable to content generated by large language models. (Photos by Kevin Beasley\/College of Computing)\u003C\/p\u003E\r\n","created":"1681481807","gmt_created":"2023-04-14 14:16:47","changed":"1681481807","gmt_changed":"2023-04-14 14:16:47","alt":"School of Interactive Computing Ph.D. student Jiawei Zhou, left, and associate professor Munmun De Choudhury, demonstrate in their latest paper that misinformation detection models are vulnerable to content generated by large language models.","file":{"fid":"253426","name":"Jiawei Zhou-munmun.jpeg","image_path":"\/sites\/default\/files\/2023\/04\/14\/Jiawei%20Zhou-munmun.jpeg","image_full_path":"http:\/\/www.tlwarc.hg.gatech.edu\/\/sites\/default\/files\/2023\/04\/14\/Jiawei%20Zhou-munmun.jpeg","mime":"image\/jpeg","size":259753,"path_740":"http:\/\/www.tlwarc.hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/2023\/04\/14\/Jiawei%20Zhou-munmun.jpeg?itok=NvDwTNqW"}}},"media_ids":["670544","670545"],"groups":[{"id":"576481","name":"ML@GT"},{"id":"50876","name":"School of Interactive Computing"}],"categories":[{"id":"134","name":"Student and Faculty"},{"id":"153","name":"Computer Science\/Information Technology and Security"},{"id":"135","name":"Research"}],"keywords":[{"id":"192524","name":"ChatGPT"},{"id":"190591","name":"misinformation"},{"id":"89321","name":"Munmun De Choudhury"},{"id":"1027","name":"chi"}],"core_research_areas":[{"id":"39501","name":"People and Technology"}],"news_room_topics":[],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003ENathan Deen\u003Cbr \/\u003E\r\nSchool of Interactive Computing\u003Cbr \/\u003E\r\nCommunications Officer\u003Cbr \/\u003E\r\n\u003Ca href=\u0022nathan.deen@cc.gatech.edu\u0022\u003Enathan.deen@cc.gatech.edu\u003C\/a\u003E\u003C\/p\u003E\r\n","format":"limited_html"}],"email":[],"slides":[],"orientation":[],"userdata":""}}}