{"633197":{"#nid":"633197","#data":{"type":"event","title":"(POSTPONED) Warren Powell workshop on Stochastic Optimization and Machine Learning ","body":[{"value":"\u003Cp\u003EWorkshop:\u003C\/p\u003E\r\n\r\n\u003Cp\u003EA Unified Framework on Sequential Decisions under Uncertainty\u003C\/p\u003E\r\n\r\n\u003Cp\u003EGeorgia Tech, ISyE\u003C\/p\u003E\r\n\r\n\u003Cp\u003EWarren B Powell\u003Cbr \/\u003E\r\nPrinceton University\u003Cbr \/\u003E\r\nApril 3, 2020\u003C\/p\u003E\r\n\r\n\u003Cp\u003ESequential decision problems arise in application areas that include engineering, the sciences, transportation, logistics, health services, medical decision making, energy, e-commerce and finance, along multiagent problems that arise with drones and robotics.\u0026nbsp; These problems have been addressed in the academic literature using a variety of modeling and algorithmic frameworks, including dynamic programming, stochastic programming, optimal control, simulation optimization, approximate dynamic programming\/reinforcement learning, and even multiarmed bandit problems.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EIn sharp contrast with the field of deterministic math programming which enjoys a canonical framework that is used around the world, sequential decision problems are described in the literature using at least eight fundamental modeling languages plus at least six more derivative dialects.\u0026nbsp; Further, application communities have developed a wide range of solution methods that reflect the characteristics of specific problem classes.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EIn this workshop, I will introduce a single, canonical modeling framework that covers every sequential decision problem.\u0026nbsp; The framework consists of five dimensions (drawing heavily on the approach used by stochastic control) that naturally represent real-world problems and map directly into software (and vice versa).\u0026nbsp; The framework clearly lays out the elements of any sequential decision problem that have to be modeled, which helps to clarify the understanding of complex systems.\u003C\/p\u003E\r\n\r\n\u003Cp\u003ESpecial attention will be given to the modeling of state variables, which are a surprising source of confusion in the research literature.\u0026nbsp; I will illustrate the different classes of state variables, including belief states, using a series of energy storage problems.\u0026nbsp; Through proper handling of state variables, I show how we can model pure learning problems, pure resource allocation problems, hybrid learning\/resource allocation problems, and contextual problems.\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EOur modeling strategy is centered on the challenge of optimizing over policies (functions for making decisions).\u0026nbsp; I will describe four (meta) classes of policies, that can be divided into two broad groups: the policy search class (finding the best function that works well over time), and lookahead policies that approximate the downstream impact of making a decision now.\u0026nbsp; I will claim that these four classes are universal: any method proposed in the literature (or used in practice) falls into one of these four classes, or a hybrid of two or more.\u0026nbsp; I will illustrate parametric cost function approximations that are widely used in practice, but almost completely ignored by the academic community.\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EI will illustrate the four classes of policies in the context of pure learning problems, and the much richer class of state-dependent problems.\u0026nbsp; In the process, I will highlight the strengths of policies in the policy search class (simplicity, ability to incorporate structure) and the weaknesses (tunable parameters).\u0026nbsp; I will use an energy storage problem to show that we can make each of the four classes of policies (and possibly a hybrid) work best depending on the characteristics of the data.\u0026nbsp;\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003ESequential decision problems arise in application areas that include engineering, the sciences, transportation, logistics, health services, medical decision making, energy, e-commerce and finance, along multiagent problems that arise with drones and robotics.\u0026nbsp; These problems have been addressed in the academic literature using a variety of modeling and algorithmic frameworks, including dynamic programming, stochastic programming, optimal control, simulation optimization, approximate dynamic programming\/reinforcement learning, and even multiarmed bandit problems.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EIn sharp contrast with the field of deterministic math programming which enjoys a canonical framework that is used around the world, sequential decision problems are described in the literature using at least eight fundamental modeling languages plus at least six more derivative dialects.\u0026nbsp; Further, application communities have developed a wide range of solution methods that reflect the characteristics of specific problem classes.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EIn this workshop, I will introduce a single, canonical modeling framework that covers every sequential decision problem.\u0026nbsp; The framework consists of five dimensions (drawing heavily on the approach used by stochastic control) that naturally represent real-world problems and map directly into software (and vice versa).\u0026nbsp; The framework clearly lays out the elements of any sequential decision problem that have to be modeled, which helps to clarify the understanding of complex systems.\u003C\/p\u003E\r\n\r\n\u003Cp\u003ESpecial attention will be given to the modeling of state variables, which are a surprising source of confusion in the research literature.\u0026nbsp; I will illustrate the different classes of state variables, including belief states, using a series of energy storage problems.\u0026nbsp; Through proper handling of state variables, I show how we can model pure learning problems, pure resource allocation problems, hybrid learning\/resource allocation problems, and contextual problems.\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EOur modeling strategy is centered on the challenge of optimizing over policies (functions for making decisions).\u0026nbsp; I will describe four (meta) classes of policies, that can be divided into two broad groups: the policy search class (finding the best function that works well over time), and lookahead policies that approximate the downstream impact of making a decision now.\u0026nbsp; I will claim that these four classes are universal: any method proposed in the literature (or used in practice) falls into one of these four classes, or a hybrid of two or more.\u0026nbsp; I will illustrate parametric cost function approximations that are widely used in practice, but almost completely ignored by the academic community.\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EI will illustrate the four classes of policies in the context of pure learning problems, and the much richer class of state-dependent problems.\u0026nbsp; In the process, I will highlight the strengths of policies in the policy search class (simplicity, ability to incorporate structure) and the weaknesses (tunable parameters).\u0026nbsp; I will use an energy storage problem to show that we can make each of the four classes of policies (and possibly a hybrid) work best depending on the characteristics of the data.\u0026nbsp;\u003C\/p\u003E\r\n","format":"limited_html"}],"field_summary_sentence":[{"value":"A Unified Framework on Sequential Decisions under Uncertainty"}],"uid":"34868","created_gmt":"2020-03-02 18:10:38","changed_gmt":"2020-03-13 01:36:04","author":"sbryantturner3","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2020-04-03T10:00:00-04:00","event_time_end":"2020-04-03T18:00:00-04:00","event_time_end_last":"2020-04-03T18:00:00-04:00","gmt_time_start":"2020-04-03 14:00:00","gmt_time_end":"2020-04-03 22:00:00","gmt_time_end_last":"2020-04-03 22:00:00","rrule":null,"timezone":"America\/New_York"},"extras":[],"groups":[{"id":"1242","name":"School of Industrial and Systems Engineering (ISYE)"}],"categories":[],"keywords":[],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1795","name":"Seminar\/Lecture\/Colloquium"}],"invited_audience":[{"id":"78761","name":"Faculty\/Staff"},{"id":"177814","name":"Postdoc"},{"id":"78771","name":"Public"},{"id":"174045","name":"Graduate students"},{"id":"78751","name":"Undergraduate students"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}