{"650228":{"#nid":"650228","#data":{"type":"event","title":"ISyE Seminar- Robert Nowak","body":[{"value":"\u003Cp\u003E\u003Cstrong\u003ETitle:\u003C\/strong\u003E What Kinds of Functions Do Neural Networks Learn?\u003Cbr \/\u003E\r\n\u003Cbr \/\u003E\r\n\u003Cstrong\u003EAbstract: \u003C\/strong\u003ENeural nets have made an amazing comeback during the past decade. Their empirical success has been truly phenomenal, but neural nets are poorly understood in a mathematical sense compared to classical methods like splines, kernels, and wavelets.\u0026nbsp; This talk describes recent steps towards a mathematical theory of neural networks comparable to the foundations we have for classical nonparametric methods. Surprisingly, neural nets are minimax optimal in a wide variety of classical univariate function spaces, including those handled by splines and wavelets. In multivariate settings, neural nets are\u0026nbsp; solutions to data-fitting problems cast in entirely new types of multivariate function spaces characterized through total variation (TV) measured in the Radon transform domain.\u0026nbsp; And deep (multilayer) neural nets naturally represent compositions of functions in these Radon-BV (bounded variation) spaces.\u0026nbsp; Remarkably, this theory provides novel explanations for many notable empirical discoveries in deep learning, including the benefits of \u0026ldquo;skip connections\u0026rdquo; and sparse and low-rank \u0026ldquo;weight\u0026rdquo; matrices. Radon-BV spaces set the stage for the nonparametric theory of neural nets.\u003Cbr \/\u003E\r\n\u003Cbr \/\u003E\r\n\u003Cstrong\u003EBio:\u003C\/strong\u003E Rob holds the Nosbusch Professorship in Engineering at the University of Wisconsin-Madison. His research focuses on signal processing, machine learning, optimization, and statistics.\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003E\u003Cstrong\u003EAbstract:\u003C\/strong\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003ENeural nets have made an amazing comeback during the past decade. Their empirical success has been truly phenomenal, but neural nets are poorly understood in a mathematical sense compared to classical methods like splines, kernels, and wavelets.\u0026nbsp; This talk describes recent steps towards a mathematical theory of neural networks comparable to the foundations we have for classical nonparametric methods. Surprisingly, neural nets are minimax optimal in a wide variety of classical univariate function spaces, including those handled by splines and wavelets. In multivariate settings, neural nets are\u0026nbsp; solutions to data-fitting problems cast in entirely new types of multivariate function spaces characterized through total variation (TV) measured in the Radon transform domain.\u0026nbsp; And deep (multilayer) neural nets naturally represent compositions of functions in these Radon-BV (bounded variation) spaces.\u0026nbsp; Remarkably, this theory provides novel explanations for many notable empirical discoveries in deep learning, including the benefits of \u0026ldquo;skip connections\u0026rdquo; and sparse and low-rank \u0026ldquo;weight\u0026rdquo; matrices. Radon-BV spaces set the stage for the nonparametric theory of neural nets.\u003Cbr \/\u003E\r\n\u0026nbsp;\u003C\/p\u003E\r\n","format":"limited_html"}],"field_summary_sentence":[{"value":"What Kinds of Functions Do Neural Networks Learn?"}],"uid":"34868","created_gmt":"2021-08-30 19:42:50","changed_gmt":"2021-09-07 19:11:04","author":"sbryantturner3","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2021-09-10T12:00:00-04:00","event_time_end":"2021-09-10T13:00:00-04:00","event_time_end_last":"2021-09-10T13:00:00-04:00","gmt_time_start":"2021-09-10 16:00:00","gmt_time_end":"2021-09-10 17:00:00","gmt_time_end_last":"2021-09-10 17:00:00","rrule":null,"timezone":"America\/New_York"},"extras":[],"groups":[{"id":"1242","name":"School of Industrial and Systems Engineering (ISYE)"}],"categories":[],"keywords":[],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1795","name":"Seminar\/Lecture\/Colloquium"}],"invited_audience":[{"id":"78761","name":"Faculty\/Staff"},{"id":"177814","name":"Postdoc"},{"id":"78771","name":"Public"},{"id":"174045","name":"Graduate students"},{"id":"78751","name":"Undergraduate students"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}