Abstract

In this talk, I will present two specific methods for computational video and some thoughts about trends in this area in general.

First, I will describe a novel algorithm for video stabilization that generates stabilized videos by employing L1-optimal camera paths to remove undesirable motions. Our method allows for video stabilization beyond conventional filtering, that only suppresses high frequency jitter. An additional challenge in videos shot from mobile phones are rolling shutter distortions. We propose a solution based on a novel mixture model of homographies parametrized by scanline blocks to correct these rolling shutter distortions. Our method does not rely on a-priori knowledge of the readout time nor requires prior camera calibration. Our novel video stabilization and calibration free rolling shutter removal has been deployed on YouTube, resulting in successfully stabilizing millions of videos. We also discuss several extensions to the stabilization algorithm and present technical details behind the widely used YouTube Video Stabilizer, running live on youtube.com

Second, I will describe an efficient and scalable technique for spatio-temporal segmentation of long video sequences using a hierarchical graph-based algorithm. We begin by over-segmenting a volumetric video graph into space-time regions grouped by appearance. We then construct a region graph over the obtained segmentation and iteratively repeat this process over multiple levels to create a tree of spatio-temporal segmentations. This hierarchical approach generates high quality segmentations, and allows subsequent applications to choose from varying levels of granularity. We demonstrate the use of spatio-temporal segmentation as users interact with the video, enabling efficient annotation of objects within the video. This system is now available for use via the videosegmentation.com site. I will describe some applications of how this system is used for dynamic scene understanding.

This talk is based on efforts of research by Matthias Grundmann, Daniel Castro, and S. Hussain Raza, as part of their research efforts as students at Georgia Tech. Some parts of the work described were also completed with Matthias Grundmann, Vivek Kwatra, and Mei Han at Google Research.

Bio

Irfan Essa is the Associate Dean for Off-Campus and Special Initiatives in the College of Computing. He is also a professor in the School of Interactive Computing (IC) and an adjunct professor in the School of Electrical and Computer Engineering. 

Essa works in the areas of Computer Vision, Computer Graphics, Computational Perception, Robotics and Computer Animation, Machine Learning, and Social Computing, with potential impact on Video Analysis and Production (e.g., Computational Photography & Video, Image-based Modeling and Rendering, etc.) Human Computer Interaction, Artificial Intelligence, Computational Behavioral/Social Sciences, and Computational Journalism research.

The author of more than 150 scholarly articles in leading journals and conference venues, Essa has received several best paper awards. He has been awarded the NSF CAREER Award and was elected to the grade of IEEE Fellow. Also, he has held extended research consulting positions with Disney Research and Google Research and was an adjunct faculty member at Carnegie Mellon’s Robotics Institute. 

He joined the Georgia Tech faculty in 1996 after holding a research faculty position at the Massachusetts Institute of Technology in the Media Lab for eight years.

Abstract

Using the latest technology, we can safely hijack your most trusted senses, thereby fooling your brain into believing you are in another world. Virtual reality (VR) has been around for a long time, but due to the recent convergence of sensing, display, and computation technologies, there is an unprecedented opportunity to explore this form of human augmentation with lightweight, low-cost materials and simple software platforms. This is an intense form of human-computer interaction (HCI) that requires re-examining core engineering principles with a direct infusion of perceptual psychology research.

Developing systems that optimize classical criteria might lead to overcomplicated solutions that are too slow or costly in practice, and yet could make no perceptible difference to users. Simple adaptation of techniques that were developed for on-screen viewing, such as cinematography and first-person shooter game play, often lead to unpleasant VR experiences due the presentation of unusual stimuli or due to mismatches between the human vestibular system and other senses. With the rapid rise in consumer VR, fundamental research questions are popping up everywhere, slicing across numerous disciplines from engineering to sociology, to film, to medicine. This talk will provide some perspective on where we have been and where we might be going next.

Bio

Steven LaValle, a professor of computer science at the University of Illinois at Urbana-Champaign, is a specialist in the interdisciplinary field of robotics and is recognized as a world leader in motion planning, a fundamental research area not only in robotics, but also in other research and commercial applications, ranging from computational biology to virtual prototyping, architectural planning, and video-game design. His most recent work has centered on determining the minimal sensing requirements needed to solve tasks using machines that combine sensing, actuation, and computation. 

Abstract

In this presentation we will discuss how hard automation in many cases is getting replaced by flexible automation. The ideal factory is no longer a sequence of fixed processing stations, but a swarm of heterogenous stationary and mobile manipulation systems that can be re-configured to produce one-off products in a system that is fully integrated from design to manufacturing. The integration of new sensors, mixed-human-robot interaction, and cloud services offer an opportunity to rethink modern manufacturing.

Bio

Henrik I. Christensen is the KUKA Chair of Robotics and a Distinguished Professor of Computing. He is also the director of the Institute for Robotics and Intelligent Machines at Georgia Tech.

Christensen received his first degree in mechanical engineering and subsequently received M.Sc. and Ph.D. degrees from Aalborg University in Denmark. His research focuses on human centered robotics, specifically perception, human-robot interaction, and systems modeling. A large number of companies have commercialized his research and he has maintained active collaborations with researchers and companies across three continents. Christensen also serves as an advisor to numerous companies and agencies across the world.

The author of more than 300 contributions in the areas of computer vision, artificial intelligence, and computer vision, Christensen held positions at Aalborg University, Royal Institute of Technology, and the University of Pennsylvania before joining the Georgia Tech faculty.

Additionally, Christensen was the founder of the European Network of Excellence in Robotics (1999-2006), and in 2011, he founded the U.S. Robotics Virtual Organization. He was the coordinator of the effort to formulate the National Robotics Roadmap, which was presented to congress in 2009 and 2013.

Christensen serves on seven editorial boards across the areas of computer vision and robotics. He is the editor in chief of Robotics and Trends in Robotics and serves as the senior technology lead for the Robotics Technology Consortium (RTC). He is also a member on the boards of the Robot Industry Association (RIA) and the College Industry Council for Material Handling Industry Education (CICHME).

Abstract

The last few years have seen significant progress in our understanding of how one should structure multi-robot systems. New control, coordination, and communication strategies have emerged, and in this talk, we discuss some of these developments. In particular, we will show how one can go from global, geometric, team-level specifications to local coordination rules for achieving and maintaining formations, area coverage, and swarming behaviors. One aspect of this process concerns how users can interact with networks of mobile robots in order to inject new, global information and objectives. We will also investigate what global objectives are fundamentally implementable in a distributed manner on a collection of spatially distributed and locally interacting agents.

Bio

Magnus Egerstedt is the Schlumberger Professor in the School of Electrical and Computer Engineering, where he serves as associate chair for Research and External Affairs. After completing his Ph.D., he was a postdoctoral scholar at Harvard University.

As the director of the Georgia Robotics and Intelligent Systems Laboratory (GRITS Lab), Egerstedt conducts research in the areas of control theory and robotics, focusing on control and coordination of complex networks, such as multi-robot systems, mobile sensor networks and cyber-physical systems.

Egerstedt is the deputy editor-in-chief for IEEE Transactions on Network Control Systems and the past editor for electronic publications for the IEEE Control Systems Society. Additionally, he is a Fellow of the IEEE and a recipient of the National Science Foundation CAREER Award. He also received the HKN Outstanding Teacher Award, the Alumni of the Year Award from the Royal Institute of Technology and the Outstanding Junior Faculty Member Award from Georgia Tech's School of Electrical and Computer Engineering. 

Abstract

We seek to develop methods to capture, quantify, and characterize human manipulatory skill (and its acquisition). Such an understanding is critical in the context of skill training for human performance of manipulation tasks for diverse audiences (ranging from astronauts to assembly line operators, pilots to surgeons, musicians to handicraft makers). However, in lieu of an abstract treatment, we concretize our efforts in the context of surgical procedural assessment/training.

Surgical procedural performance involves interplay of a highly dynamic system of intercoupled perceptual, sensory, and cognitive components—at the current stage, however, our focus is on sensorimotor procedural performance. Our operant hypothesis is that while human manipulation behavior may be based on the dynamic interaction between the neuromuscular system and its dynamic environment (human-machine interface + task dynamics), it becomes manifest in the ensuing spatiotemporal patterns. Hence, we take a sensing/systems perspective and propose to track, measure and record, under carefully controlled conditions, low-level dynamic behaviors of users (novitiates to experts) as they perform skilled surgical tasks. Our immediate goal is to then determine the underlying structure (“skill-level” or “signature”) of a proceduralist, with its clear ramifications to accreditation and certification, despite the significant spatiotemporal variability of populations (human), coupling characteristics (device), and the interactions (environment).

We will present early results from our skill assessment implementation efforts in two contexts: (A) The da Vinci Robotic Minimally Invasive Surgery (RMIS) case that involves multi-degree-of-freedom dexterous motion components, features better instrumentation and sensing, but is currently performed without haptic feedback; and (B) Percutaneous Kidney Biopsy case that initially is more unstructured and open-ended but ultimately has more-constrained (1 DOF) motions, yet depends critically on the sense of touch.

Time permitting, we will also provide a brief overview of the many other related research initiatives, including uneven terrain locomotion systems; human-computer interfaces for mediated teleoperation; haptic user-interface design; and video-understanding methods for human activity monitoring. 

Bio

Venkat N. Krovi is an associate professor in the Mechanical and Aerospace Engineering Department at the State University of New York (SUNY) at Buffalo, with adjunct appointments in Electrical Engineering, Pathology and Anatomical Sciences, and Gynecology-Obstetrics.

Krovi obtained his Ph.D. from the Department of Mechanical Engineering and Applied Mechanics at the University of Pennsylvania in December 1998 under the supervision of Prof. Vijay Kumar and Prof. G.K. Ananthasuresh. From January 1999 to September 2001, he was a tenure-track assistant professor in the Mechanical Engineering Department at McGill University in Montreal.

In September 2001, Krovi joined the Mechanical and Aerospace Engineering Department at the State University of New York at Buffalo as a tenure-track assistant professor and received tenure in June 2007. At Buffalo, Krovi directs the Automation, Robotics and Mechatronics (ARM) Lab and the Computer Integrated Surgery (CIS) Lab. His research program focuses on lifecycle treatment (design, modeling, analysis, control, implementation, and verification) of novel mechanical and mechatronic systems with emphasis on both the theoretical formulation and experimental validation. The rich theory of kinematics, dynamics, and control of constrained articulated mechanical systems forms the intellectual basis of his research.

Increasingly, Krovi’s research has become more focused on the biological domain where inherent multi-scale irregularities, inhomogeneities, and nonlinearities offer considerable challenges to complete characterization and understanding. His approach has been to investigate the working principles of biological systems from a mechatronic viewpoint with the goal of developing improved design and implementation methodologies for bio-inspired machines and systems. His group’s research activities have won several conference and journal best paper and best poster awards. 

Krovi has served on technical program committees and conference editorial boards of several IEEE and ASME conferences. Currently, he is serving as an associate editor of the IEEEE Transactions on Robotics and as the general conference chair for the ASME IDETC 2014. 

Abstract

In this talk I present recent work from the Socially Intelligent Machines Lab at Georgia Tech. One of the focuses of our lab is on socially guided machine learning, building robot systems that can learn from everyday human teachers. We look at standard machine learning interactions and redesign interfaces and algorithms to support the collection of learning input from naive humans. This talk covers results on building computational models of reciprocal social interactions, high-level task goal learning, low-level skill learning, and active learning interactions using several humanoid robot platforms.  

Bio

Andrea L. Thomaz is an associate professor in the School of Interactive Computing at Georgia Tech. She joined the faculty in 2007. Thomaz earned a B.S. in Electrical and Computer Engineering from the University of Texas at Austin in 1999, and Sc.M. and Ph.D. degrees from MIT in 2002 and 2006. She has published in the areas of Artificial Intelligence, Robotics, and Human-Robot Interaction. Currently, she directs the Socially Intelligent Machines lab, which is affiliated with the Institute for Robotics and Intelligent Machines (IRIM) and the GVU Center. Her research aims to computationally model mechanisms of human social learning in order to build social robots and other machines that are intuitive for everyday people to teach.

Abstract

With the rapidly growing popularity of the Intuitive Surgical da Vinci system, robotic minimally invasive surgery (RMIS) has crossed the threshold from the laboratory to the real world. There are now hundreds of thousands of human-guided robotic surgeries performed every year, and the number is growing rapidly. This is a unique example of the growing paradigm of human-machine collaborative systems—namely systems of machines and computers collaborating with people to perform tasks that neither can perform as effectively alone.

While designed to be an amplifier of human ability, collaborative systems like the da Vinci pose a number of new challenges. In particular, the learning curve is steep and involves mastery of a new and rapidly evolving set of skills and techniques. Yet, at the same time, the availability of easily acquired data also creates new opportunities to create new engineering paradigms for modeling, evaluating, and improving human skill.

In this talk, I will describe our work toward developing effective human-machine collaborative teams. I will spend most of the talk discussing approaches to modeling complex manipulation tasks within in the context of surgery. By creating models for the “Language of Surgery,” I will show that we are able to evaluate the style and efficiency of surgical motion. These models also lead naturally to methods for supporting, or even automating, component actions in surgery as well as in other domains. At the end of the talk, I will briefly touch on our recent work, building on these ideas, to develop collaborative systems for manufacturing.

This talk includes joint work with Sanjeev Khudanpur and Rene Vidal.

Bio

Gregory D. Hager is a professor and chair of Computer Science at Johns Hopkins University and the deputy director of the NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology. His research interests include collaborative robotics, time-series analysis of image data, image-guided robotics, and medical applications of image analysis and robotics. He is also chair of the Computing Community Consortium and is currently a member of the governing board of the International Federation of Robotics Research. He is a fellow of the IEEE for his contributions to vision-based robotics. 

Abstract

Termites build huge, complex structures through the collective actions of millions of independent agents. These natural systems inspire the research area of collective construction, whose goal is to develop autonomous multi-robot systems that build large-scale structures according to user specifications. I will discuss the design and realization of such a system in which climbing robots flexibly build structures using specialized building blocks. Robots act independently under decentralized control, using local information, onboard sensing, and implicit coordination through manipulation of a shared environment. A user can specify a target structure using a high-level representation, and robots follow simple rules that guarantee the correct completion of that structure. I will also briefly discuss ongoing experimental studies of African termites, aimed at better understanding their individual behaviors and how these give rise to the structures they produce.

Bio

Justin Werfel is a research scientist at the Wyss Institute for Biologically Inspired Engineering at Harvard University. His research includes work on swarm robotics, evolutionary theory, DNA self-assembly, social insect behavior, and other topics in complex systems and self-organization. 

Werfel received an A.B. in physics from Princeton and a S.M. (EECS) from MIT. He completed his Ph.D. in computer science at MIT in 2006, developing algorithms to allow swarms of simple robots to autonomously build user-specified structures. His postdoctoral research at Harvard included further exploration of collective construction, work on the evolution of cooperative and altruistic behaviors at the New England Complex Systems Institute, and cancer modeling at Harvard Medical School/Children’s Hospital Boston. 

Abstract

The emergence of robotic products that serve as automated tools, assistants, and collaborators promises tremendous benefits across a range of everyday settings from the home to manufacturing facilities. While these products promise interactions that can be far more complex than those with conventional products, their successful integration into the human environment requires these interactions to be also natural and intuitive. To achieve complex but intuitive interactions, designers and developers must simultaneously understand and address computational and human challenges. In this talk, I will present my group’s work on building human-centered guidelines, methods, and tools to address these challenges in order to facilitate the design of robotic technologies that are more effective, intuitive, acceptable, and even enjoyable. In particular, I will present a series of projects that demonstrates how a marrying of knowledge about people and computational methods can enable effective user interactions with social, assistive, and telepresence robots, as well as the development of novel tools and methods that support complex tasks across the key stages of analysis, synthesis, and evaluation in the design process. I will additionally present ongoing work that applies these guidelines to the development of real-world applications of robotic technology. 

Bio

Bilge Mutlu is an assistant professor of computer science, psychology, and industrial engineering at the University of Wisconsin–Madison. He received his Ph.D. degree from Carnegie Mellon University’s Human-Computer Interaction Institute in 2009. 

Mutlu’s background combines training in interaction design, human-computer interaction, and robotics with industry experience in product design and development. He is a former Fulbright Scholar and the recipient of the NSF CAREER award and several paper awards and nominations, including HRI 2008, HRI 2009, HRI 2011, UbiComp 2013, IVA 2013, RSS 2013, and HRI 2014. His research has been featured in national and international press, including the NewScientist, MIT Technology Review, Discovery News, Science Nation, and Voice of America.

Additionally, Mutlu has served on the steering committee of the HRI Conference and the editorial board of IEEE Transactions on Affective Computing, co-chairing the program committees for HRI 2015 and ICSR 2011 and the program sub-committees on design for CHI 2013 and CHI 2014.

Bio

Karen Liu is an associate professor in the School of Interactive Computing at Georgia Tech. She received her B.E. degree from the National Taiwan University in 1999, and her M.S. and Ph.D. degrees in Computer Science from the University of Washington in 2001 and 2005 respectively. Before joining Georgia Tech, Liu was an assistant professor at the University of Southern California since 2006.

Liu's research interests are in computer graphics and animation, including physics-based animation, character animation, numerical methods, robotics and computational biomechanics. The goal of her research is to expand computer-generated character animation from a visualization tool to an interdisciplinary research areas centered at the studies of human motion and autonomous control. Specifically, she focuses on three interrelated research directions: (1) Integrating insights of biomechanics to enhance physics-based computational models for synthesis of human motion; (2) Designing efficient algorithms for controlling and simulating optimal motion; and (3) Synthesizing autonomous behaviors with synthetic sensory and physiology.

Liu received a National Science Foundation CAREER Award and was named one of the Young Innovators Under 35 in 2007 by Technology Review.

Bio

Luis Sentis joined the Department of Mechanical Engineering at The University of Texas at Austin as an assistant professor in 2010. Between 2007 and 2009 he was a postdoctoral fellow at Stanford University. He received his Ph.D. in 2007 from Stanford University and his B.S. (Honors Thesis) from the Polytechnic University of Catalonia (UPC) in 1996. He worked as a control engineer in Silicon Valley between 1996 and 1998.

Sentis directs the Human Centered Robotics Laboratory, where his research focuses on characterizing, controlling, and building humanoid robots. His control methods have been used in some of the most advanced humanoid robots such as the Meka Humanoid Robot and simulations of the Honda Asimo Humanoid robot. He conducts research on algorithms for the compliant skills of humanoid robots, design of mechanical and embedded hardware for humanoid systems, and biomechatronics.

Additionally, Sentis has created a new class called Topics in Human Centered Robotics to explore foundations and applications of robots for the assistance and augmentation of humans in their daily lives. He is also the co-designer of the open-source software architecture, the Compliant Whole-Body Control Framework, which endows complex compliant skills to humanoid robots. His research has been funded by the Office of Naval Research, the NSF National Robotics Initiative, and the DARPA Robotics Challenge.