{"662098":{"#nid":"662098","#data":{"type":"event","title":"PhD Defense by Nirbhay Modhe","body":[{"value":"\u003Cp\u003E\u003Cstrong\u003ETitle\u003C\/strong\u003E: Leveraging Value-awareness for Online and Offline Model-based Reinforcement Learning\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDate: Thursday, October 27th, 2022\u003C\/p\u003E\r\n\r\n\u003Cp\u003ETime: 9:00 AM - 11:00 AM Eastern Time\u003C\/p\u003E\r\n\r\n\u003Cp\u003ELocation (virtual): \u003Ca href=\u0022https:\/\/bluejeans.com\/264974579\/4014\u0022\u003Ehttps:\/\/bluejeans.com\/264974579\/4014\u003C\/a\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ENirbhay Modhe\u003C\/strong\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003EPh.D. Candidate\u003C\/p\u003E\r\n\r\n\u003Cp\u003ESchool of Interactive Computing\u003C\/p\u003E\r\n\r\n\u003Cp\u003ECollege of Computing\u003C\/p\u003E\r\n\r\n\u003Cp\u003EGeorgia Institute of Technology\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ECommittee\u003C\/strong\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Dhruv Batra (advisor), School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Zsolt Kira, School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Mark Riedl, School of Interactive Computing, Georgia Institute of Technology\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Gaurav Sukhatme, University of Southern California\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDr. Ashwin Kalyan, Allen Institute for AI (AI2)\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ESummary\u003C\/strong\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003EModel-based Reinforcement Learning (RL) lies at the intersection of planning and learning for sequential decision making. Value-awareness in model learning has recently emerged as a means to imbue task or reward information into the objective of model learning, in order for the model to leverage specificity of a task. While finding success in theory as being superior to maximum likelihood estimation in the context of (online) model-based RL, value-awareness has remained impractical for most non-trivial tasks.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThis thesis aims to bridge the gap in theory and practice by applying the principle of value-awareness to two settings -- the online RL setting and offline RL setting. First, within online RL, this thesis revisits value-aware model learning from the perspective of minimizing performance difference, obtaining a novel value-aware model learning objective as a direct upper bound of it. Then, this thesis investigates and remedies the issue of stale value estimates that has so far been holding back the practicality of value-aware model learning. Using the proposed remedy, performance improvements are presented over maximum-likelihood based baselines and existing value-aware objectives, in several continuous control tasks, while also enabling existing value-aware objectives to become performant.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EIn the offline RL setting, this thesis takes a step back from model learning and applies value-awareness towards better data augmentation. Such data augmentation, when applied to model-based offline RL algorithms, allows for leveraging unseen states with low epistemic uncertainty that have previously not been reachable within the assumptions and limitations of model-based offline RL. Value-aware state augmentations are found to enable better performance on offline RL benchmarks compared to existing baselines and non-value-aware alternatives.\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":"","field_summary_sentence":[{"value":"Leveraging Value-awareness for Online and Offline Model-based Reinforcement Learning"}],"uid":"27707","created_gmt":"2022-10-13 16:13:53","changed_gmt":"2022-10-13 16:13:53","author":"Tatianna Richardson","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2022-10-27T10:00:00-04:00","event_time_end":"2022-10-27T12:00:00-04:00","event_time_end_last":"2022-10-27T12:00:00-04:00","gmt_time_start":"2022-10-27 14:00:00","gmt_time_end":"2022-10-27 16:00:00","gmt_time_end_last":"2022-10-27 16:00:00","rrule":null,"timezone":"America\/New_York"},"extras":[],"groups":[{"id":"221981","name":"Graduate Studies"}],"categories":[],"keywords":[{"id":"100811","name":"Phd Defense"}],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1788","name":"Other\/Miscellaneous"}],"invited_audience":[{"id":"78761","name":"Faculty\/Staff"},{"id":"78771","name":"Public"},{"id":"78751","name":"Undergraduate students"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}