{"156851":{"#nid":"156851","#data":{"type":"news","title":"Easy Guider: Intuitive Visual Control Provides Faster Remote Operation of Robots","body":[{"value":"\u003Cp\u003EUsing a novel method of integrating video technology and familiar control devices, a research team from the Georgia Institute of Technology is developing a technique to simplify remote control of robotic devices.\u003C\/p\u003E\u003Cp\u003EThe researchers\u0027 aim is to enhance a human operator\u0027s ability to perform precise tasks using a multi-jointed robotic device such as an articulated mechanical arm. The new approach has been shown to be easier and faster than older methods, especially when the robot is controlled by an operator who is watching it in a video monitor.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EKnown as Uncalibrated Visual Servoing for Intuitive Human Guidance of Robots, the new method uses a special implementation of an existing vision-guided control method called visual servoing (VS). By applying visual-servoing technology in innovative ways, the researchers have constructed a robotic system that responds to human commands more directly and intuitively than older techniques.\u003C\/p\u003E\u003Cp\u003E\u0022Our approach exploits 3-D video technology to let an operator guide a robotic device in ways that are more natural and time-saving, yet are still very precise,\u0022 said Ai-Ping Hu, a senior research engineer with the Georgia Tech Research Institute (GTRI). \u0022This capability could have numerous applications \u2013 especially in situations where directly observing the robot\u0027s operation is hazardous or not possible \u2013 including bomb disposal, handling of hazardous materials and search-and-rescue missions.\u0022\u003C\/p\u003E\u003Cp\u003EA paper on this technology was presented at the 2012 IEEE International Conference on Robotics and Automation held in St. Paul, Minn.\u003C\/p\u003E\u003Cp\u003EFor decades articulated robots have been used by industry to perform precision tasks such as welding vehicle seams or assembling electronics, Hu explained. The user develops a software program that enables the device to cycle through the required series of motions, using feedback from sensors built into the robot.\u003C\/p\u003E\u003Cp\u003EBut such programming can be complex and time-consuming. The robot must typically be maneuvered joint by joint through the numerous actions required to complete a task. Moreover, such technology works only in a structured and unchanging environment, such as a factory assembly line, where spatial relationships are constant.\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EThe Human Operator\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003EIn recent years, new techniques have enabled human operators to freely guide remote robots through unstructured and unfamiliar environments, to perform such challenging tasks as bomb disposal, Hu said. Operators have controlled the device in one of two ways: by \u0022line of sight\u0022 \u2013 direct user observation \u2013 or by means of conventional, two-dimensional camera that is mounted on the robot to send back an image of both the robot and its target.\u0026nbsp;\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EBut humans guiding robots via either method face some of the same complexities that challenge those who program industrial robots, he added. Manipulating a remote robot into place is generally slow and laborious.\u003C\/p\u003E\u003Cp\u003EThat\u0027s especially true when the operator must depend on the imprecise images provided by 2-D video feedback. Manipulating separate controls for each of the robot\u0027s multiple joint axes, users have only limited visual information to help them and must maneuver to the target by trial and error.\u003C\/p\u003E\u003Cp\u003E\u0022Essentially, the user is trying to visualize and reconstruct a 3-D scenario from flat 2-D camera images,\u0022 Hu said. \u0022The process can become particularly confusing when operators are facing in a different direction from the robot and must mentally reorient themselves to try to distinguish right from left. It\u0027s somewhat similar to backing up a vehicle with an attached trailer \u2013 you have to turn the steering wheel to the left to get the trailer to move right, which is decidedly non-intuitive.\u0022\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EThe Visual Servoing Advantage\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003ETo simplify user control, the Georgia Tech team turned to visual servoing (a term synonymous with visual activation).\u0026nbsp; Visual servoing has been studied for years as a way to use video cameras to help robots re-orient themselves within a structured environment such as an assembly line.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003ETraditional visual servoing is calibrated, meaning that position information generated by a video camera can be transformed into data meaningful to the robot. Using these data, the robot can adjust itself to stay in a correct spatial relationship with target objects.\u0026nbsp;\u0026nbsp;\u0026nbsp;\u003C\/p\u003E\u003Cp\u003E\u0022Say a conveyor line is accidently moved a few millimeters,\u0022 Hu said. \u0022A robot with a calibrated visual servoing capability can automatically detect the movement using the video image and a fixed reference point, and then readjust to compensate.\u0022\u003C\/p\u003E\u003Cp\u003EBut visual servoing offers additional possibilities. The research team \u2013 which includes Hu, associate professor Harvey Lipkin of the School of Mechanical Engineering, graduate student Matthew Marshall, GTRI research engineer Michael Matthews and GTRI principal research engineer Gary McMurray -- has adapted visual-servoing technology in ways that facilitate human control of remote robots.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EThe new technique takes advantage of both calibrated and uncalibrated techniques.\u0026nbsp; A calibrated 3-D \u0022time of flight\u0022 camera is mounted on the robot \u2013 typically at the end of a robotic arm, in a gripping device called an end-effector. This approach is sometimes called an eye-in-hand system, because of the camera\u0027s location in the robot\u0027s \u0022hand.\u0022\u003C\/p\u003E\u003Cp\u003EThe camera utilizes an active sensor that detects depth data, allowing it to send back 3-D coordinates that pinpoint the end-effector\u0027s spatial location.\u0026nbsp; At the same time, the eye-in-hand camera also supplies a standard, uncalibrated 2-D grayscale video image to the operator\u0027s monitor.\u003C\/p\u003E\u003Cp\u003EThe result is that the operator, without seeing the robot, now has a robot\u0027s-eye view of the target. Watching this image in a monitor, an operator can visually guide the robot using a gamepad, in a manner somewhat reminiscent of a first-person 3-D video game.\u0026nbsp;\u003C\/p\u003E\u003Cp\u003EIn addition, visual-servoing technology now automatically actuates all the joints needed to complete whatever action the user indicates on the gamepad \u2013 rather than the user having to manipulate those joints one by one. In the background, the Georgia Tech system performs the complex computation needed to coordinate the monitor image, the 3-D camera information, the robot\u0027s spatial position and the user\u0027s gamepad commands.\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003ETesting System Usability\u003C\/strong\u003E\u003C\/p\u003E\u003Cp\u003E\u0022The guidance process is now very intuitive \u2013 pressing \u0027left\u0027 on the gamepad will actuate all the requisite robot joints to effect a leftward displacement,\u0022 Hu said. \u0022What\u0027s more, the robot could be upside down and the controls will still respond in the same intuitive way \u2013 left is still left and right is still right.\u0022\u003C\/p\u003E\u003Cp\u003ETo judge system usability, the Georgia Tech research team recently conducted trials to test whether the visual-servoing approach enabled faster task-completion times. Using a gamepad that controls an articulated-arm robot with six degrees of freedom, subjects performed four tests: they used visual-servoing guidance as well as conventional joint-based guidance, in both line-of-sight and camera-view modes.\u003C\/p\u003E\u003Cp\u003EIn the line-of-sight test, volunteer participants using visual-servoing guidance averaged task-completion times that were 15 percent faster than when they used joint-based guidance. However, in camera-view mode, participants using visual-servoing guidance averaged 227 percent faster results than with the joint-based technique.\u003C\/p\u003E\u003Cp\u003EHu noted that the visual-servoing system used in this test scenario was only one of numerous possible applications of the technology.\u0026nbsp; The research team\u0027s plans include testing a mobile platform with a VS-guided robotic arm mounted on it. Also underway is a proof-of-concept effort that incorporates visual-servoing control into a low-cost, consumer-level robot.\u003C\/p\u003E\u003Cp\u003E\u0022Our ultimate goal is to develop a generic, uncalibrated control framework that is able to use image data to guide many different kinds of robots,\u0022 he said.\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; \u003Cbr \/\u003E\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; \u0026nbsp;\u003Cbr \/\u003E\u003Cstrong\u003EResearch News \u0026amp; Publications Office\u003C\/strong\u003E\u003Cbr \/\u003E\u003Cstrong\u003EGeorgia Institute of Technology\u003C\/strong\u003E\u003Cbr \/\u003E\u003Cstrong\u003E75 Fifth Street, N.W., Suite 309\u003C\/strong\u003E\u003Cbr \/\u003E\u003Cstrong\u003EAtlanta, Georgia\u0026nbsp; 30308\u0026nbsp; USA\u003C\/strong\u003E\u003Cbr \/\u003E\u003Cbr \/\u003E\u003Cstrong\u003EMedia Relations Contact\u003C\/strong\u003E: John Toon (404-894-6986)(\u003Ca href=\u0022mailto:jtoon@gatech.edu\u0022\u003Ejtoon@gatech.edu\u003C\/a\u003E).\u003C\/p\u003E\u003Cp\u003E\u003Cstrong\u003EWriter\u003C\/strong\u003E: Rick Robinson\u003Cbr \/\u003E\u003Cbr \/\u003E\u003C\/p\u003E","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003EUsing a novel method of integrating video technology and familiar control devices, a research team from the Georgia Institute of Technology is developing a technique to simplify remote control of robotic devices.\u0026nbsp;\u0026nbsp;\u0026nbsp;\u003C\/p\u003E","format":"limited_html"}],"field_summary_sentence":[{"value":"A technique known as uncalibrated visual servoing could make the remote control of robots more intuitive."}],"uid":"27303","created_gmt":"2012-09-25 13:13:30","changed_gmt":"2016-10-08 03:12:50","author":"John Toon","boilerplate_text":"","field_publication":"","field_article_url":"","dateline":{"date":"2012-09-25T00:00:00-04:00","iso_date":"2012-09-25T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"156811":{"id":"156811","type":"image","title":"Uncalibrated Visual Servoing","body":null,"created":"1449178872","gmt_created":"2015-12-03 21:41:12","changed":"1475894792","gmt_changed":"2016-10-08 02:46:32","alt":"Uncalibrated Visual Servoing","file":{"fid":"195305","name":"visual-servoing64.jpg","image_path":"\/sites\/default\/files\/images\/visual-servoing64_0.jpg","image_full_path":"http:\/\/www.tlwarc.hg.gatech.edu\/\/sites\/default\/files\/images\/visual-servoing64_0.jpg","mime":"image\/jpeg","size":1222654,"path_740":"http:\/\/www.tlwarc.hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/images\/visual-servoing64_0.jpg?itok=V1vmTawp"}},"156821":{"id":"156821","type":"image","title":"Uncalibrated Visual Servoing 2","body":null,"created":"1449178872","gmt_created":"2015-12-03 21:41:12","changed":"1475894792","gmt_changed":"2016-10-08 02:46:32","alt":"Uncalibrated Visual Servoing 2","file":{"fid":"195306","name":"visual-servoing46.jpg","image_path":"\/sites\/default\/files\/images\/visual-servoing46_0.jpg","image_full_path":"http:\/\/www.tlwarc.hg.gatech.edu\/\/sites\/default\/files\/images\/visual-servoing46_0.jpg","mime":"image\/jpeg","size":1392359,"path_740":"http:\/\/www.tlwarc.hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/images\/visual-servoing46_0.jpg?itok=FGrla3b4"}},"156831":{"id":"156831","type":"image","title":"Uncalibrated Visual Servoing 3","body":null,"created":"1449178872","gmt_created":"2015-12-03 21:41:12","changed":"1475894792","gmt_changed":"2016-10-08 02:46:32","alt":"Uncalibrated Visual Servoing 3","file":{"fid":"195307","name":"visual-servoing126.jpg","image_path":"\/sites\/default\/files\/images\/visual-servoing126_1.jpg","image_full_path":"http:\/\/www.tlwarc.hg.gatech.edu\/\/sites\/default\/files\/images\/visual-servoing126_1.jpg","mime":"image\/jpeg","size":1026900,"path_740":"http:\/\/www.tlwarc.hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/images\/visual-servoing126_1.jpg?itok=dr4WbmK6"}}},"media_ids":["156811","156821","156831"],"groups":[{"id":"1188","name":"Research Horizons"}],"categories":[{"id":"145","name":"Engineering"},{"id":"152","name":"Robotics"}],"keywords":[{"id":"44451","name":"Ai-Ping Hu"},{"id":"416","name":"GTRI"},{"id":"1356","name":"robot"},{"id":"44461","name":"robot arm"},{"id":"667","name":"robotics"},{"id":"44441","name":"visual servoing"}],"core_research_areas":[{"id":"39501","name":"People and Technology"},{"id":"39521","name":"Robotics"},{"id":"39541","name":"Systems"}],"news_room_topics":[],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003EJohn Toon\u003C\/p\u003E\u003Cp\u003EResearch News \u0026amp; Publications Office\u003C\/p\u003E\u003Cp\u003E(404) 894-6986\u003C\/p\u003E\u003Cp\u003E\u003Ca href=\u0022mailto:jtoon@gatech.edu\u0022\u003Ejtoon@gatech.edu\u003C\/a\u003E\u003C\/p\u003E","format":"limited_html"}],"email":["jtoon@gatech.edu"],"slides":[],"orientation":[],"userdata":""}}}