{"633268":{"#nid":"633268","#data":{"type":"event","title":"*POSTPONED* ML@GT and ISyE Joint Seminar: Warren B. Powell, Princeton University","body":[{"value":"\u003Cp\u003E\u003Cstrong\u003E**This event has been postponed**\u003C\/strong\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003EML@GT and ISyE invite you to a seminar by Warren B. Powell, professor of operations research and financial engineering at Princeton University.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EFor scheduling information, please contact\u0026nbsp;Anton Kleywegt at\u0026nbsp;\u003Ca href=\u0022mailto:anton.kleywegt@isye.gatech.edu\u0022\u003Eanton.kleywegt@isye.gatech.edu\u003C\/a\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Ch4\u003ETitle\u003C\/h4\u003E\r\n\r\n\u003Cp\u003EFrom Reinforcement Learning to Stochastic Optimization: A Universal Framework for Sequential Decision Analytics\u003C\/p\u003E\r\n\r\n\u003Ch4\u003EAbstract\u003C\/h4\u003E\r\n\r\n\u003Cp\u003ESequential decisions are an almost universal problem class, spanning dynamic resource allocation problems, control problems, discrete graph problems, active learning problems, as well as two-agent games and multiagent problems.\u0026nbsp; Application settings span engineering, the sciences, transportation, health services, medical decision making, energy, e-commerce and finance.\u0026nbsp; A rich problem class involves systems that must actively learn about the environment, possibly via drones or robots.\u0026nbsp; In multi-agent systems, we may need to learn about the behavior of other agents.\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThese problems have been addressed in the academic literature using a variety of modeling and algorithmic frameworks, including dynamic programming, stochastic programming, stochastic control, simulation optimization, approximate dynamic programming\/reinforcement learning, and even multiarmed bandit problems.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EI will describe a universal modeling framework that can be used for \u003Cem\u003Eany\u003C\/em\u003E sequential decision problem in the presence of different sources of uncertainty.\u0026nbsp; The framework is centered on an optimization problem that optimizes over policies (rules for making decisions), where we show that there are two fundamental strategies for designing policies (policy search and policies based on lookahead approximations), each of which further divide into two classes, creating four (meta)classes of policies that are the foundation of \u003Cem\u003Eany\u003C\/em\u003E solution approach that has ever been proposed for a sequential problem.\u0026nbsp; I will demonstrate these policies in two broad contexts: pure learning problems (\u0026ldquo;bandit problems\u0026rdquo;) and dynamic resource allocation problems, where I will use a simple energy storage problem to show that each of the four classes (and a hybrid) can be made to work best.\u003C\/p\u003E\r\n\r\n\u003Ch4\u003EBio\u003C\/h4\u003E\r\n\r\n\u003Cp\u003EWarren Powell is a faculty member in the Department of Operations Research and Financial Engineering at Princeton University where he has taught since 1981. In 1990, he founded\u0026nbsp;\u003Ca href=\u0022http:\/\/www.castlelab.princeton.edu\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003ECASTLE Laboratory\u003C\/a\u003E\u0026nbsp;which spans research in computational stochastic optimization with applications initially in transportation and logistics. In 2011, he founded the\u0026nbsp;\u003Ca href=\u0022http:\/\/energysystems.princeton.edu\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003EPrinceton laboratory for ENergy Systems Analysis (PENSA)\u003C\/a\u003E\u0026nbsp;to tackle the rich array of problems in energy systems analysis. In 2013, this morphed into \u0026ldquo;CASTLE Labs,\u0026rdquo; focusing on computational stochastic optimization and learning.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EIn the 1980\u0026rsquo;s, he designed and wrote\u0026nbsp;\u003Ca href=\u0022http:\/\/castlelab.princeton.edu\/Papers\/Braklow%20Graham%20et%20al%20%20Interactive%20Optimization%20Improves%20Service%20at%20Yellow.pdf\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003ESYSNET\u003C\/a\u003E, an interactive optimization model for load planning at Yellow Freight System, where it is still in use after 25 years. In 1988, he founded the Princeton Transportation Consulting Group which marketed the model as\u0026nbsp;\u003Ca href=\u0022https:\/\/castlelab.princeton.edu\/superspin\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003ESuperSPIN\u003C\/a\u003E, which was adopted by the entire less-than-truckload industry, stabilizing an industry where 80 percent of the companies went bankupt in the first post-deregulation decade.\u0026nbsp;\u003Ca href=\u0022https:\/\/castlelab.princeton.edu\/superspin\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003ESuperSPIN\u003C\/a\u003E\u0026nbsp;was used in the planning of American Freightways (which became FedEx Freight), Roadway Package System (which became FedEx Ground), and Overnight Transportation (which became UPS Freight). SuperSPIN stabilized the LTL trucking industry in the 1990\u0026rsquo;s, following its deregulation in 1980.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EAlso in the 1980\u0026rsquo;s he developed a series of models for truckload trucking, starting with\u0026nbsp;\u003Ca href=\u0022http:\/\/castlelab.princeton.edu\/Papers\/Powell-OperationalPlanningModelforDVA.pdf\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003ELoadMAP\u003C\/a\u003E\u0026nbsp;(written by Ken Nickerson \u0026rsquo;84), which then evolved to an integrated stochastic model for driver assignment called\u0026nbsp;\u003Ca href=\u0022https:\/\/castlelab.princeton.edu\/micromap\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003EMicroMAP\u003C\/a\u003E\u0026nbsp;(the senior thesis of David Cape \u0026rsquo;87). As of 2011, MicroMAP was being used to dispatch over 66,000 drivers for 20 of the largest truckload carriers in the U.S.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EHe has started three consulting firms: Princeton Transportation Consulting Group (1988),\u0026nbsp; Transport Dynamics (1995), and Optimal Dynamics (2016) (CEO is his son Daniel Powell), but he has continued to do his developmental work through CASTLE Laboratory at Princeton University, where he has worked with the leading companies in less-than-truckload trucking (Yellow Freight System\/YRC), parcel shipping (United Parcel Service), truckload trucking (Schneider National), rail (primarily Norfolk Southern Railway), air (Netjets and Embraer), as well as the Air Mobility Command. As he moved into energy, he has worked with PJM Interconnections (the grid operator for the mid-Atlantic states), and PSE\u0026amp;G (the utility that serves 75 percent of New Jersey).\u0026nbsp;\u003Ca href=\u0022https:\/\/castlelab.princeton.edu\/impact-on-industry\/\u0022\u003EClick here for a complete list.\u003C\/a\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003EMotivated by these applications, he developed a method for bridging dynamic programming with math programming to solve very high-dimensional stochastic, dynamic programs using the modeling and algorithmic framework of\u0026nbsp;\u003Ca href=\u0022http:\/\/adp.princeton.edu\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003Eapproximate dynamic programming\u003C\/a\u003E. This work has been used in a variety of applications including\u0026nbsp;\u003Ca href=\u0022https:\/\/castlelab.princeton.edu\/wagner\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003Efleet management at Schneider National\u003C\/a\u003E\u0026nbsp;(50,000 variables per time period, and a state variable with 10^{20}\u0026nbsp;\u003Cem\u003Edimensions\u003C\/em\u003E), the\u0026nbsp;\u003Ca href=\u0022http:\/\/castlelab.princeton.edu\/Papers\/Powell-SMART_JOC_2011.pdf\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003ESMART energy resource planning model\u003C\/a\u003E\u0026nbsp;(175,000 time periods), and\u0026nbsp;\u003Ca href=\u0022https:\/\/castlelab.princeton.edu\/plasma\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003Elocomotive optimization at Norfolk Southern\u003C\/a\u003E.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EHe identified four fundamental classes of policies for solving sequential decision problems, integrating fields such as stochastic programming, dynamic programming (including approximate dynamic programming\/reinforcement learning), robust optimization, optimal control and stochastic search (to name a few). This work identified a new class of policy called a\u0026nbsp;\u003Cem\u003Eparametric cost function approximation\u0026nbsp;\u003C\/em\u003E(\u003Ca href=\u0022https:\/\/castlelab.princeton.edu\/jungle\/\u0022\u003Eclick here for more information\u003C\/a\u003E).\u003C\/p\u003E\r\n\r\n\u003Cp\u003EHis work in industry is balanced by contributions to the\u0026nbsp;\u003Ca href=\u0022https:\/\/castlelab.princeton.edu\/castle-lab-theory\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003Etheory of stochastic optimization, and machine learning.\u003C\/a\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003EPrizes and awards \u0026ndash; Recipient\u0026nbsp;\u003Cem\u003EDocteur Honoris Causa\u003C\/em\u003E\u0026nbsp;from the University of Quebec in Montreal in 2013. Winner,\u0026nbsp;\u003Ca href=\u0022https:\/\/castlelab.princeton.edu\/wagner\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003EDaniel Wagner Prize\u003C\/a\u003E\u0026nbsp;for extending approximate dynamic programming to very high-dimensional problems for Schneider National. Best Paper Prize from the Society for Transportation Science and Logistics (once for this problem, and once for our ADP model for locomotive management at Norfolk Southern). His students have won many awards (Dantzig Prize for best dissertation in Operations Research, several winners of the Transportation Science dissertation prize, Doing Good with Good OR Competition honorable mention, Nicholson Prize finalist). Finalist in the prestigious Edelman competition in 1987 and 1991. Informs Fellows Award, Presidential Young Investigator Award.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EBooks: He is the author of\u0026nbsp;\u003Ca href=\u0022http:\/\/adp.princeton.edu\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003EApproximate Dynamic Programming: Solving the curses of dimensionality\u003C\/a\u003E\u0026nbsp;and co-author (with Ilya Ryzhov) of\u0026nbsp;\u003Ca href=\u0022http:\/\/optimallearning.princeton.edu\/\u0022 rel=\u0022noopener noreferrer\u0022 target=\u0022_blank\u0022\u003EOptimal Learning\u003C\/a\u003E\u0026nbsp;(both published by Wiley). Co-editor (with J. Si, A. Barto, and D. Wunsch)\u0026nbsp;\u003Cem\u003ELearning and Approximate Dynamic Programming: Scaling up to the Real World.\u003C\/em\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003EJust the numbers: $50+ million in research funding (in 2020 dollars), 250+ refereed papers, two books (plus an edited volume), ~60 Ph.D. students and post-docs (~30 in academia and research laboratories), 10 Masters, 200+ undergraduate senior theses,\u0026nbsp;\u003Ca href=\u0022http:\/\/scholar.google.com\/citations?user=vDw80QEAAAAJ\u0026amp;hl=en\u0022\u003Eh-number (on Google) of 65, 18,000+ citations,\u003C\/a\u003E\u0026nbsp;36,000+ visitors per year to my websites, 7,000+ connections on LinkedIn (some miniscule number on Facebook)\u0026hellip; (let me know if you can think of any more).\u003C\/p\u003E\r\n\r\n\u003Cp\u003EHe has served in numerous leadership and service roles, including President of the Transportation Science Section, Informs board of directors, director of several NSF workshops, Area Editor for transportation at Operations Research (8 years), and numerous prize, review and service committees. In 1991 he co-founded the triennial conference TRISTAN, now the leading international conference for transportation systems analysis. In 2003 he designed the Informs Impact Prize and served as the first chair in 2004.\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":"","field_summary_sentence":[{"value":"ML@GT and ISyE invite you to a seminar by Warren B. Powell, professor of operations research and financial engineering at Princeton University."}],"uid":"34773","created_gmt":"2020-03-04 14:15:22","changed_gmt":"2020-03-13 13:06:04","author":"ablinder6","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2020-04-02T14:00:00-04:00","event_time_end":"2020-04-02T15:30:00-04:00","event_time_end_last":"2020-04-02T15:30:00-04:00","gmt_time_start":"2020-04-02 18:00:00","gmt_time_end":"2020-04-02 19:30:00","gmt_time_end_last":"2020-04-02 19:30:00","rrule":null,"timezone":"America\/New_York"},"extras":[],"groups":[{"id":"47223","name":"College of Computing"},{"id":"1299","name":"GVU Center"},{"id":"576481","name":"ML@GT"},{"id":"50877","name":"School of Computational Science and Engineering"},{"id":"50875","name":"School of Computer Science"},{"id":"50876","name":"School of Interactive Computing"}],"categories":[],"keywords":[],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1795","name":"Seminar\/Lecture\/Colloquium"}],"invited_audience":[{"id":"78761","name":"Faculty\/Staff"},{"id":"177814","name":"Postdoc"},{"id":"78771","name":"Public"},{"id":"174045","name":"Graduate students"},{"id":"78751","name":"Undergraduate students"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003EKyla Hanson\u003C\/p\u003E\r\n\r\n\u003Cp\u003Ekhanson@cc.gatech.edu\u003C\/p\u003E\r\n","format":"limited_html"}],"email":[],"slides":[],"orientation":[],"userdata":""}}}