{"668386":{"#nid":"668386","#data":{"type":"news","title":"Robustness: Making Progress by Poking Holes in Artificial Intelligence Models","body":[{"value":"\u003Cp\u003EFindings from two published studies could lead to enhancements in artificial intelligence (AI) models by focusing on their flaws.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EOne paper found that adding visual attributes to text in multimodal models could boost performance and usefulness for humans.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EAnother study determined that few-shot learning (FSL) models lack robustness against adversarial treatments and need improvements. \u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EGeorgia Tech Assistant Professor\u0026nbsp;\u003Cstrong\u003ESrijan Kumar\u003C\/strong\u003E\u0026nbsp;and Ph.D. student\u0026nbsp;\u003Cstrong\u003EGaurav Verma\u003C\/strong\u003E\u0026nbsp;lead the research being presented at the upcoming 61st Annual meeting of the Association for Computational Linguistics (ACL 2023).\u003C\/p\u003E\r\n\r\n\u003Cp\u003ECo-authors from Georgia Tech joining Kumar and Verma include\u0026nbsp;\u003Ca href=\u0022https:\/\/shivaen.org\/\u0022\u003E\u003Cstrong\u003EShivaen Ramshetty\u003C\/strong\u003E\u003C\/a\u003E\u003Cstrong\u003E\u0026nbsp;\u003C\/strong\u003Eand\u0026nbsp;\u003Ca href=\u0022https:\/\/www.linkedin.com\/in\/sarath-nookala\/\u0022\u003E\u003Cstrong\u003EVenkata Prabhakara Sarath Nookala\u003C\/strong\u003E\u003C\/a\u003E, as well as\u0026nbsp;\u003Ca href=\u0022https:\/\/www.microsoft.com\/en-us\/research\/people\/submukhe\/\u0022\u003E\u003Cstrong\u003ESubhabrata Mukherjee\u003C\/strong\u003E,\u003C\/a\u003E\u0026nbsp;a principal researcher at Microsoft Research.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EACL 2023 brings together experts from around the world to discuss topics in natural language processing (NLP) and AI research. Kumar\u2019s group offers to those discussions their work that focuses on robustness in AI models.\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u201cSecurity of AI models is paramount. Development of reliable and responsible AI models are important discussion topics at the national and international levels,\u201d Kumar said. \u201cAs Large Language Models become part of the backbone of many products and tools with which users will interact, it is important to understand when, how, and why these AI models will fail.\u201d\u003C\/p\u003E\r\n\r\n\u003Cp\u003E[\u003Ca href=\u0022https:\/\/sites.gatech.edu\/research\/acl-2023\/\u0022\u003EMICROSITE: Georgia Tech at ACL 2023\u003C\/a\u003E]\u003C\/p\u003E\r\n\r\n\u003Cp\u003ERobustness refers to the degree to which an AI model\u2019s performance changes when using new data versus training data. To ensure that a model performs reliably, it is critical to understand its robustness.\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003ETrust is of essential value within robustness, both for researchers that work in AI and consumers that use it.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EPeople lose trust in AI technology when models perform unpredictably. This issue is relevant in the ongoing societal discussion about AI security. Investigating robustness can prevent, or at least highlight, performance issues arising from unmodeled behavior and malicious attacks.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003EDeep Learning for Every Kind of Media\u003C\/strong\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003EOne aspect of AI robustness Kumar\u2019s group will present at ACL 2023 delves into multimodal deep learning. Using this method, AI models receive and apply data through modes ranging from text, images, video, and audio.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThe group\u2019s\u0026nbsp;\u003Ca href=\u0022https:\/\/faculty.cc.gatech.edu\/~srijan\/pubs\/multimodal-robustness-xmai-acl2023.pdf\u0022\u003Epaper\u003C\/a\u003E\u0026nbsp;presents a way to evaluate multimodal learning robustness called Cross-Modal Attribute Insertions (XMAI).\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EXMAI found that multimodal models perform poorly in text-to-image retrieval tasks. For example, adding more descriptive wording in search text for an image, like from \u201cgirl on a chair\u201d to \u201clittle girl on a wooden chair,\u201d caused the correct image to be retrieved at a lower rank.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EKumar\u2019s group determined this when XMAI outperformed five other benchmarks in two different task retrieval tests.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u201cBy conducting experiments in a sandbox setting to identify the plausible realistic inputs that make multimodal models fail, we can estimate various dimensions of a model\u2019s robustness,\u201d said Kumar. \u201cOnce these shortcomings are identified, these models can be updated and made more robust.\u201d\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ELabels Matter When It Comes to Adversarial Robustness\u003C\/strong\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003EPrompt-based few-shot learning (FSL) is another class of AI models that, like multimodal learning, uses text as input.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EWhile FSL is a useful framework for AI to improve task performance when labeled data is limited, Kumar\u2019s group points out in\u0026nbsp;\u003Ca href=\u0022https:\/\/faculty.cc.gatech.edu\/~srijan\/pubs\/few-shot-adversarial-robustness-acl2023.pdf\u0022\u003Etheir ACL findings paper\u003C\/a\u003E\u0026nbsp;that there is limited understanding of the methods\u2019 adversarial robustness.\u0026nbsp;\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u201cOur findings shine a light on a significant vulnerability in FSL models \u2013 a marked lack of adversarial robustness,\u201d Verma explained. \u201cThis indicates a non-trivial balancing act between accuracy and adversarial robustness of prompt-based few-shot learning for NLP.\u201d\u003C\/p\u003E\r\n\r\n\u003Cp\u003EKumar\u2019s team ran tests on six GLUE benchmark tasks, comparing FSL models with fully fine-tuned models. Here, they found a notable, greater drop in task performance of FSL models treated with adversarial perturbations than that of fully fine-tuned models.\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EIn the same study, Kumar\u2019s group found and proposed a few ways to improve FSL robustness.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThese include using unlabeled data for prompt-based FSLs and expanding to an ensemble of models trained with different prompts. The group also demonstrated that increasing the number of few-shot examples and model size led to increased adversarial robustness of FSL methods.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u201cImproved adversarial robustness of few-shot learning models is essential for their broader application and adoption,\u201d Verma said. \u201cBy securing a balance between robustness and accuracy, all from a handful of labeled instances, we can potentially implement these models in safety-critical domains.\u201d\u003C\/p\u003E\r\n","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003ESchool of Computational Science and Engineering researchers are presenting two papers the upcoming 61st Annual meeting of the Association for Computational Linguistics that explore the robustness of AI applications. The work looks to improve the reliability of these systems as a step toward creating broader public trust.\u003C\/p\u003E\r\n","format":"limited_html"}],"field_summary_sentence":[{"value":"Georgia Tech researchers are working to make AI applications more reliable and more resilient."}],"uid":"32045","created_gmt":"2023-07-07 13:09:32","changed_gmt":"2023-07-12 18:10:28","author":"Ben Snedeker","boilerplate_text":"","field_publication":"","field_article_url":"","dateline":{"date":"2023-07-07T00:00:00-04:00","iso_date":"2023-07-07T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"671126":{"id":"671126","type":"image","title":"Kumar_Verma.jpeg","body":null,"created":"1688735393","gmt_created":"2023-07-07 13:09:53","changed":"1688735393","gmt_changed":"2023-07-07 13:09:53","alt":"a composite image of Georgia Tech Assistant Professor Srijan Kumar and Ph.D. student Gaurav Verma","file":{"fid":"254133","name":"Kumar_Verma.jpeg","image_path":"\/sites\/default\/files\/2023\/07\/07\/Kumar_Verma.jpeg","image_full_path":"http:\/\/www.tlwarc.hg.gatech.edu\/\/sites\/default\/files\/2023\/07\/07\/Kumar_Verma.jpeg","mime":"image\/jpeg","size":46463,"path_740":"http:\/\/www.tlwarc.hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/2023\/07\/07\/Kumar_Verma.jpeg?itok=TUB_BEpe"}}},"media_ids":["671126"],"groups":[{"id":"47223","name":"College of Computing"},{"id":"37041","name":"Computational Science and Engineering"},{"id":"1188","name":"Research Horizons"}],"categories":[{"id":"153","name":"Computer Science\/Information Technology and Security"},{"id":"135","name":"Research"},{"id":"8862","name":"Student Research"}],"keywords":[{"id":"187915","name":"go-researchnews"}],"core_research_areas":[{"id":"39431","name":"Data Engineering and Science"},{"id":"39501","name":"People and Technology"}],"news_room_topics":[],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003EBryant Wine, Comms. Officer I\u003Cbr \/\u003E\r\nSchool of Computational Science \u0026amp; Engineering\u003Cbr \/\u003E\r\nBryant.wine@cc.gatech.edu\u003C\/p\u003E\r\n","format":"limited_html"}],"email":["Bryant.wine@cc.gatech.edu"],"slides":[],"orientation":[],"userdata":""}}}