{"622215":{"#nid":"622215","#data":{"type":"news","title":"Artificial Intelligence Agents Begin to Learn New Skills from Watching Videos","body":[{"value":"\u003Cp\u003EData is a hot word in 2019 and according to \u003Cstrong\u003EAshley Edwards\u003C\/strong\u003E, there is a lot of data out there that can be used more efficiently for teaching robots and artificial agents how to do a variety of tasks.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EEdwards, a recent computer science Ph.D. graduate from Georgia Tech, details her research in a new paper, \u003Ca href=\u0022https:\/\/arxiv.org\/pdf\/1805.07914.pdf\u0022\u003E\u003Cem\u003EImitating Latent Policies from Observation\u003C\/em\u003E\u003C\/a\u003E.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThe new approach uses imitation learning from observation and video data. This new way of thinking could eventually\u0026nbsp;teach agents how to do tasks like make a sandwich, play a videogame, or even drive a car, all from watching videos.\u0026nbsp;In most experiments, Edwards and her fellow researchers algorithm was able to complete a task in 200 to 300 steps while previous methods have gone into the thousands.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;This approach is exciting because it unpeels another layer for how we can train artificial agents to work with humans. We have hardly skimmed the surface of this problem space, but this is a great next step,\u0026rdquo;\u0026nbsp;said\u0026nbsp;\u003Cstrong\u003ECharles Isbell,\u0026nbsp;\u003C\/strong\u003Edean designate of the College of Computing and paper co-author.\u003C\/p\u003E\r\n\r\n\u003Cp\u003ETo accomplish this, researchers have an agent watch a video and guess what actions are being taken. In the paper, this is referred to as a latent policy. Given that guess, the agent tries to predict movements and learn what to do. When the agent is then placed into an actual environment, it can take what it has learned from the videos and apply its knowledge to real-world actions.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EIn previous research using \u0026ldquo;imitation from observation,\u0026rdquo; humans must physically show agents how to do an action or train a computer to use a dynamic model to learn how to do a new task, both of which are time-consuming, expensive, and potentially dangerous.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;There are thousands of videos out there documenting people doing things, but it can be hard to know what they are doing in a way that can be applied to artificial systems,\u0026rdquo; said Edwards.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EFor example, there are countless hours of dashcam footage from autonomous cars driving on streets, but there isn\u0026rsquo;t much information about why self-driving cars make the decisions that they do. The videos rarely have detailed telemetry information about the vehicle, like what angle the steering wheel was pointed when the car moved a certain way. Edwards and her team hope that their algorithm will be able to analyze video footage and piece together not only how to do an action, but why.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDuring their research, Edwards and her co-authors performed four experiments to prove their idea. Using a platform game called Coinrun, they trained an agent to jump over platforms and avoid traps to solve a task. They also used classic control environments in their experiments to get a cart to balance a pole and teach a mountain car to drive itself up a mountain.\u003C\/p\u003E\r\n\r\n\u003Cp\u003ETheir approach was able to beat the expert in two of the experiments and was considered \u0026ldquo;state-of-the-art\u0026rdquo; in all four. \u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDespite its achievements, the current model is only created for discrete actions like moving right, left, forward or backward one step at a time. So, Edwards and her team are continuing to push their work forward toward being able to achieve smoother and more continuous actions for their models.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThis research is one of 18 accepted papers from \u003Ca href=\u0022http:\/\/www.ml.gatech.edu\/\u0022\u003Ethe Machine Learning Center at Georgia Tech\u0026rsquo;s (ML@GT)\u003C\/a\u003E and will be presented at the \u003Ca href=\u0022https:\/\/icml.cc\/Conferences\/2019\u0022\u003E36\u003Csup\u003Eth\u003C\/sup\u003E Annual International Conference on Machine Learning (ICML)\u003C\/a\u003E held June 9 through 15 in Long Beach, Calif.\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":"","field_summary_sentence":[{"value":"Using video and existing data, Georgia Tech researchers are teaching artificial agents how to do a variety of tasks more efficiently."}],"uid":"34773","created_gmt":"2019-06-04 15:00:34","changed_gmt":"2019-06-05 21:59:07","author":"ablinder6","boilerplate_text":"","field_publication":"","field_article_url":"","dateline":{"date":"2019-06-04T00:00:00-04:00","iso_date":"2019-06-04T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"622214":{"id":"622214","type":"image","title":"Georgia Tech researchers are looking at how to more efficiently teach robots and artificial agents how to do tasks using video. ","body":null,"created":"1559660261","gmt_created":"2019-06-04 14:57:41","changed":"1559660261","gmt_changed":"2019-06-04 14:57:41","alt":"Screen capture of YouTube ","file":{"fid":"237001","name":"con-karampelas-1178812-unsplash.jpg","image_path":"\/sites\/default\/files\/images\/con-karampelas-1178812-unsplash.jpg","image_full_path":"http:\/\/www.tlwarc.hg.gatech.edu\/\/sites\/default\/files\/images\/con-karampelas-1178812-unsplash.jpg","mime":"image\/jpeg","size":287447,"path_740":"http:\/\/www.tlwarc.hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/images\/con-karampelas-1178812-unsplash.jpg?itok=VL6Fo7oN"}}},"media_ids":["622214"],"groups":[{"id":"47223","name":"College of Computing"},{"id":"576481","name":"ML@GT"},{"id":"50876","name":"School of Interactive Computing"}],"categories":[],"keywords":[],"core_research_areas":[{"id":"39501","name":"People and Technology"},{"id":"39521","name":"Robotics"}],"news_room_topics":[],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003EAllie McFadden\u003C\/p\u003E\r\n\r\n\u003Cp\u003ECommunications Officer\u003C\/p\u003E\r\n\r\n\u003Cp\u003Eallie.mcfadden@cc.gatech.edu\u003C\/p\u003E\r\n","format":"limited_html"}],"email":["allie.mcfadden@cc.gatech.edu"],"slides":[],"orientation":[],"userdata":""}}}