{"653049":{"#nid":"653049","#data":{"type":"news","title":"Embracing Socratic Paradox May Lead to More Reliable Predictions from AI Models","body":[{"value":"\u003Cp\u003ENext-generation artificial intelligence (AI) models used in public health, IoT, and other critical applications will soon be able to make better decisions and more accurate predictions thanks to a bit of philosophical wisdom being instilled in them at Georgia Tech.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Ca href=\u0022https:\/\/www.cc.gatech.edu\/school-computational-science-and-engineering\u0022\u003ESchool of Computational Science and Engineering (CSE)\u003C\/a\u003E\u0026nbsp;researchers are using a new 3-year, $1.1 million National Science Foundation (NSF) Medium grant to find ways of quantifying uncertainty in current AI models that use time-series data to make predictions.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EFrom here, they hope to essentially teach the models what\u0026rsquo;s known as the Socratic paradox: \u0026ldquo;I know that I know nothing.\u0026rdquo;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;The key hurdle is that current deep learning neural models are poor at quantifying their uncertainty and are often overconfident in their predictions. Quantifying uncertainty will allow a model to say, \u0026lsquo;I don\u0026#39;t know,\u0026rsquo; when facing unknown or unexpected situations,\u0026rdquo; said\u0026nbsp;\u003Cstrong\u003EB.\u003C\/strong\u003E\u0026nbsp;\u003Cstrong\u003EAditya\u003C\/strong\u003E\u0026nbsp;\u003Cstrong\u003EPrakash\u003C\/strong\u003E, CSE associate professor and co-principal investigator for the project.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EBecause it doesn\u0026rsquo;t recognize what it doesn\u0026rsquo;t know, a current model may guess at answers and move forward as if it has guessed correctly. This is particularly problematic with time-series data \u0026ndash; like public health monitoring and forecasting \u0026ndash;\u0026nbsp;where guesses and wrong answers can lead to lower levels of confidence in predictions generated by current-generation models.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EPrakash (left) and\u0026nbsp;\u003Cstrong\u003EChao\u003C\/strong\u003E\u0026nbsp;\u003Cstrong\u003EZhang\u0026nbsp;\u003C\/strong\u003E(right), CSE assistant professor and lead PI, are working with co-PI\u0026nbsp;\u003Cstrong\u003EShuochao\u003C\/strong\u003E\u0026nbsp;\u003Cstrong\u003EYao\u003C\/strong\u003E\u0026nbsp;from George Mason University to address these limitations. Along with quantifying the amount of uncertainty, the team is working to better understand the types and sources of predictive uncertainty.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;We need principled models which are flexible enough to model uncertainties from multiple sources in the datasets and also produce accurate predictions,\u0026rdquo; Zhang said. \u0026ldquo;Qualifying uncertainty will allow us to dynamically select a subset of models that are more reliable, which can improve the efficiency of the system and decisions at run time.\u0026rdquo;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EBecause so much depends on specific tasks and data, the researchers say it\u0026rsquo;s difficult to provide accurate estimates of efficiency improvements based on their novel approach. Preliminary results in disease forecasting, however, indicate the team\u0026rsquo;s new models, algorithms, and techniques can outperform previous state-of-the-art models up to 2.5x in accuracy and 2.4x in reliability.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThe techniques and tools emerging from this project, formally titled\u0026nbsp;\u003Ca href=\u0022https:\/\/www.nsf.gov\/awardsearch\/showAward?AWD_ID=2106961\u0022\u003E\u003Cem\u003ECollaborative Research: Principled Uncertainty Quantification in Deep Learning Models for Time Series Analysis (NSF Award #\u003C\/em\u003E\u0026nbsp;\u003Cem\u003E2106961)\u003C\/em\u003E\u003C\/a\u003E, will be open source, and the research findings will be integrated into existing courses, tutorials, and workshops.\u003C\/p\u003E\r\n\r\n\u003Ch3\u003EFacebook Research Award\u003C\/h3\u003E\r\n\r\n\u003Cp\u003EPrakash and Zhang are taking another tack as well to improve the current state of predictive modeling.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThe pair recently earned a\u0026nbsp;\u003Ca href=\u0022https:\/\/research.fb.com\/programs\/research-awards\/proposals\/2021-statistics-for-improving-insights-models-and-decisions-request-for-proposals\/#award-recipients\u0022\u003EFacebook 2021 Statistics for Improving Insights, Models, and Decisions\u003C\/a\u003E\u0026nbsp;research award. Their proposal,\u0026nbsp;\u003Cem\u003ENon-Parametric Methods for Calibrated Hierarchical Time-series Forecasting\u003C\/em\u003E, was one of 10 winners recently announced by Facebook Research.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EAccording to the team, nonparametric statistical methods can be very flexible and effective for modeling time-series data when there are many unknowns in a dataset. Generally speaking, this is because nonparametric tools analyze group medians rather than group means. As a result, scientists can better understand outliers and can use them to strengthen.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;Our goal is to develop principled end-to-end models that incorporate hierarchical constraints and behaviors. We will also incorporate signals from different views such as demographics signals, time-series signals, and mechanistic models, to let them mutually reinforce each other to make the models more accurate, reliable, and robust,\u0026rdquo; said Zhang.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EOnce the project is complete, the results of the research will be used to improve predictive modeling in healthcare, public health, and a variety of industrial applications.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;For example,\u0026rdquo; said Prakash, \u0026ldquo;Facebook can use this technique to forecast demands in their data centers in different geographic regions. This will give them lead time to make their infrastructure more robust and efficient.\u0026rdquo;\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":"","field_summary_sentence":[{"value":"Researchers from the School of CSE are using new NSF and Facebook Research grants to better understand uncertainty in that use time-series data to make predictions."}],"uid":"32045","created_gmt":"2021-11-19 17:56:42","changed_gmt":"2021-11-24 04:08:09","author":"Ben Snedeker","boilerplate_text":"","field_publication":"","field_article_url":"","dateline":{"date":"2021-11-19T00:00:00-05:00","iso_date":"2021-11-19T00:00:00-05:00","tz":"America\/New_York"},"extras":[],"hg_media":{"653050":{"id":"653050","type":"image","title":"Socrates Statue Sky","body":null,"created":"1637344839","gmt_created":"2021-11-19 18:00:39","changed":"1637344839","gmt_changed":"2021-11-19 18:00:39","alt":"A statue of Socrates with blue sky ","file":{"fid":"247714","name":"shutterstock_socrates-18040215[1] copy.jpeg","image_path":"\/sites\/default\/files\/images\/shutterstock_socrates-18040215%5B1%5D%20copy.jpeg","image_full_path":"http:\/\/www.tlwarc.hg.gatech.edu\/\/sites\/default\/files\/images\/shutterstock_socrates-18040215%5B1%5D%20copy.jpeg","mime":"image\/jpeg","size":402163,"path_740":"http:\/\/www.tlwarc.hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/images\/shutterstock_socrates-18040215%5B1%5D%20copy.jpeg?itok=vDZmxUKG"}}},"media_ids":["653050"],"groups":[{"id":"1188","name":"Research Horizons"}],"categories":[{"id":"135","name":"Research"}],"keywords":[{"id":"187915","name":"go-researchnews"}],"core_research_areas":[],"news_room_topics":[{"id":"71881","name":"Science and Technology"}],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003EAlbert Snedeker, Communications Manager II\u003Cbr \/\u003E\r\n\u003Ca href=\u0022mailto:albert.snedeker@cc.gatech.edu?subject=Socratic%20Paradox\u0022\u003Ealbert.snedeker@cc.gatech.edu\u003C\/a\u003E\u003C\/p\u003E\r\n","format":"limited_html"}],"email":["albert.snedeker@cc.gatech.edu"],"slides":[],"orientation":[],"userdata":""}}}