{"589505":{"#nid":"589505","#data":{"type":"news","title":"Bo Knows Neural Network Training \u2013 Xie in the Spotlight at Machine Learning Workshop","body":[{"value":"\u003Cp\u003ETraining neural networks is no easy feat. Just ask Georgia Tech School of Computational Science and Engineering Ph.D. student \u003Ca href=\u0022https:\/\/www.linkedin.com\/in\/boxie-gatech\/\u0022\u003E\u003Cstrong\u003EBo Xie\u003C\/strong\u003E\u003C\/a\u003E.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EHowever, in a presentation this week at the \u003Ca href=\u0022https:\/\/simons.berkeley.edu\/\u0022\u003ESimons Institute for the Theory of Computing\u003C\/a\u003E at the University of California, Berkeley, Xie suggested a simple approach might be the most effective.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EXie was a guest speaker for a machine learning workshop and presented the event\u0026rsquo;s Spotlight Talk, titled \u003Ca href=\u0022https:\/\/youtu.be\/VZSTCkX2R84\u0022\u003E\u003Cem\u003ESemi-Random Units for Learning Neural Networks with Guarantees\u003C\/em\u003E\u003C\/a\u003E\u003Cem\u003E.\u003C\/em\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDuring the talk, Xie described that despite the challenges of solving non-convex optimization problems, there is evidence that simple gradient-based algorithms may be effective in working toward minimizing\u0026nbsp;neural network training errors.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EAlthough these algorithms are widely used in practice, Xie \u0026ndash; who expects to graduate this summer \u0026ndash; said the jury is still out as to why they work so well in neural network training.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;It is a mystery, in theory, why it would work so well because training a neural network is a difficult, non-convex problem,\u0026rdquo; said Xie. \u0026ldquo;This means that gradient descent can easily get stuck in a bad local optimum.\u0026rdquo;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EBad local optima usually equate to failures of learning\u0026nbsp;for a neural network. With this failure, the only option is to start over again \u0026ndash; possibly an exponential number of times \u0026ndash; to achieve a global optimum, which is best described as a definitive best solution.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EHowever, Xie\u0026rsquo;s research demonstrates that these solutions can be guaranteed with high probability using gradient-based algorithms. In turn, these positive outcomes represent successful learning for a neural network.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;In the short term, this work provides more understanding of the optimization learning landscape for a deep neural network,\u0026rdquo; said Xie. \u0026ldquo;We know more about why simple gradient descents will not be stuck in local optimal.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;Beyond this, my hope is that this work will inspire people to design more efficient algorithms for learning neural networks. It will allow us to train a better model with less time.\u0026rdquo;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EXie first became interested in machine learning as an undergraduate student at Beijing University of Posts and Telecommunications. He was intrigued by some early machine learning related technologies like face detection and spam email detection.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;I was fascinated about how to design algorithms that can learn from data instead of being manually programmed to do intelligent tasks,\u0026rdquo; said Xie.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EFollowing his graduation, Xie plans to be a machine learning researcher in industry.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;I want to work on real-world large-scale problems. It is a really exciting time to do research in machine learning and artificial intelligence since they are transforming our lives in every aspect.\u0026rdquo;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EXie\u0026#39;s primary academic adviser is Assistant Professor \u003Ca href=\u0022http:\/\/www.cc.gatech.edu\/~lsong\/\u0022\u003E\u003Cstrong\u003ELe Song\u003C\/strong\u003E\u003C\/a\u003E.\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":"","field_summary_sentence":[{"value":"Ph.D. student Bo Xie discussed his research on neural network training at Berkeley\u0027s Simon\u0027s Institute."}],"uid":"32045","created_gmt":"2017-03-30 12:49:33","changed_gmt":"2017-03-30 14:34:07","author":"Ben Snedeker","boilerplate_text":"","field_publication":"","field_article_url":"","dateline":{"date":"2017-03-30T00:00:00-04:00","iso_date":"2017-03-30T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"589507":{"id":"589507","type":"image","title":"Bo Xie","body":null,"created":"1490878599","gmt_created":"2017-03-30 12:56:39","changed":"1490878599","gmt_changed":"2017-03-30 12:56:39","alt":"","file":{"fid":"224608","name":"Bo Xie_CSE PhD_linkedin pic.jpg","image_path":"\/sites\/default\/files\/images\/Bo%20Xie_CSE%20PhD_linkedin%20pic.jpg","image_full_path":"http:\/\/www.tlwarc.hg.gatech.edu\/\/sites\/default\/files\/images\/Bo%20Xie_CSE%20PhD_linkedin%20pic.jpg","mime":"image\/jpeg","size":51140,"path_740":"http:\/\/www.tlwarc.hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/images\/Bo%20Xie_CSE%20PhD_linkedin%20pic.jpg?itok=hL3A2VnD"}}},"media_ids":["589507"],"groups":[{"id":"47223","name":"College of Computing"},{"id":"50877","name":"School of Computational Science and Engineering"},{"id":"50875","name":"School of Computer Science"}],"categories":[],"keywords":[],"core_research_areas":[{"id":"39431","name":"Data Engineering and Science"}],"news_room_topics":[],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003EAlbert \u0026quot;Ben\u0026quot; Snedeker, Communications Manager\u003C\/p\u003E\r\n\r\n\u003Cp\u003E404-894-7153\u003C\/p\u003E\r\n","format":"limited_html"}],"email":["albert.snedeker@cc.gatech.edu"],"slides":[],"orientation":[],"userdata":""}}}