{"620309":{"#nid":"620309","#data":{"type":"news","title":"Research Reveals Possibly Fatal Consequences of Algorithmic Bias","body":[{"value":"\u003Cp\u003ESelf-driving cars are supposed to make driving safer, but they may endanger the lives of certain groups. New Georgia Tech research suggests that pedestrians with darker skin may be more likely to get hit by self-driving cars than those with lighter skin.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThe researchers tested machine learning (ML) object detection models\u0026shy; to see how well they could see people with different skin tones. Their results revealed models were nearly 5 percent less likely to detect darker-skinned pedestrians.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThis predictive imbalance remained regardless of how researchers accounted for variables in the training data set, such as time of day, partially blocked views of pedestrians, and pixel size of the person.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;Companies don\u0026rsquo;t want the public to know about any issues of inaccuracy, so consumers need to learn to ask a lot of questions,\u0026rdquo; said \u003Cstrong\u003E\u003Ca href=\u0022http:\/\/jamiemorgenstern.com\/\u0022\u003EJamie Morgenstern\u003C\/a\u003E\u003C\/strong\u003E, School of Computer Science (SCS) assistant professor and the study\u0026rsquo;s lead author.\u003C\/p\u003E\r\n\r\n\u003Cdiv\u003E\u003Cstrong\u003E\u003Ca href=\u0022https:\/\/www.scs.gatech.edu\/news\/616279\/human-rights-may-help-shape-artificial-intelligence-2019\u0022\u003E[RELATED: \u0026#39;Human Rights\u0026#39; May Help Shape Artificial Intelligence in 2019]\u003C\/a\u003E\u003C\/strong\u003E\u003C\/div\u003E\r\n\r\n\u003Ch2\u003E\u003Cstrong\u003EBiased data\u003C\/strong\u003E\u003C\/h2\u003E\r\n\r\n\u003Cdiv\u003EThe prediction system is only one possible source of the inequity. The training data is another. The researchers used one of the most comprehensive publicly available self-driving car training datasets and wanted to determine if it represented all skin tones evenly. They classified the images using Fitzpatrick skin typing, a scale to predict UV sensitivity, and found the dataset has roughly 3.5 times as many examples of people with lighter skin.\u003C\/div\u003E\r\n\r\n\u003Cp\u003EThis discrepancy might introduce problems because of the ML method known as loss function, which determines how well an algorithm models a dataset. A model learns by measuring loss function between predicted values and actual values. The goal is to get as small of a loss function output as possible, indicating the model fits the data well. This approach is more accurate with larger subsets in the data, but can minimize the value of smaller groups. In effect, this 3.5 difference made the results even more accurate for lighter-skinned pedestrians.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EDespite the bias, Morgenstern remains optimistic. The team was able to correct for the inequity by reweighing the model to better analyze smaller groups.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThe findings, \u003Ca href=\u0022https:\/\/arxiv.org\/pdf\/1902.11097.pdf\u0022\u003Epublished earlier this month\u003C\/a\u003E, have attracted media coverage and some criticism. Much of this stems from the fact that Morgenstern and her fellow researchers \u0026shy;\u0026ndash; \u003Ca href=\u0022https:\/\/www.ic.gatech.edu\/\u0022\u003ESchool of Interactive Computing\u003C\/a\u003E Assistant Professor \u003Cstrong\u003E\u003Ca href=\u0022https:\/\/www.cc.gatech.edu\/~judy\/\u0022\u003EJudy Hoffman\u003C\/a\u003E\u003C\/strong\u003E and machine learning Ph.D. student \u003Cstrong\u003E\u003Ca href=\u0022https:\/\/www.linkedin.com\/in\/benjamin-r-wilson\u0022\u003EBenjamin Wilson\u003C\/a\u003E\u003C\/strong\u003E \u0026mdash; were not able to investigate ML models and training data actually used by the self-driving car industry because they are not publicly available.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003E\u003Ca href=\u0022https:\/\/www.scs.gatech.edu\/news\/615576\/georgia-tech-researchers-improve-fairness-machine-learning-pipeline\u0022\u003E[RELATED: Georgia Tech Researchers Improve Fairness in the Machine Learning Pipeline]\u003C\/a\u003E\u003C\/strong\u003E\u003C\/p\u003E\r\n\r\n\u003Ch2\u003E\u003Cstrong\u003EA bigger problem\u003C\/strong\u003E\u003C\/h2\u003E\r\n\r\n\u003Cp\u003EThis is not the first study of ML systems having varying predictive accuracy on different demographics. Other researchers have found examples in the financial sector. Yet in many of these scenarios, developers won\u0026rsquo;t take responsibility, according to Morgenstern.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;Developers blame any biased outcomes of their system on biased historical trends, such as the fact that more loans were applied for and issued in whiter neighborhoods, or biased training data,\u0026rdquo; she said. \u0026ldquo;For example, if the training labels used for creditworthiness instead reflect only the decisions of lenders who are now known to have had higher predictive accuracy on white applicants.\u0026rdquo;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EWith self-driving cars, however, a system developer would have a harder time blaming object detection system bias on historical trends or behavior of certain demographic groups. This was what appealed to Morgenstern about this research.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u0026ldquo;There is no capacity for arguing that historical behavior of some group should affect the trade-offs made by self-driving cars,\u0026rdquo; Morgenstern said. \u0026ldquo;No one deserves to be hit by a car.\u0026rdquo;\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":"","field_summary_sentence":[{"value":"Georgia Tech researchers found both the models and data for self-driving cars have bias."}],"uid":"34541","created_gmt":"2019-04-10 21:34:29","changed_gmt":"2019-04-10 21:46:47","author":"Tess Malone","boilerplate_text":"","field_publication":"","field_article_url":"","dateline":{"date":"2019-04-10T00:00:00-04:00","iso_date":"2019-04-10T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"620310":{"id":"620310","type":"image","title":"Crosswalk","body":null,"created":"1554932751","gmt_created":"2019-04-10 21:45:51","changed":"1554932751","gmt_changed":"2019-04-10 21:45:51","alt":"Crosswalk","file":{"fid":"236194","name":"architecture_buildings_bus_business_cars_city_cityscape_clouds-1495895.jpg!d.jpeg","image_path":"\/sites\/default\/files\/images\/architecture_buildings_bus_business_cars_city_cityscape_clouds-1495895.jpg%21d.jpeg","image_full_path":"http:\/\/www.tlwarc.hg.gatech.edu\/\/sites\/default\/files\/images\/architecture_buildings_bus_business_cars_city_cityscape_clouds-1495895.jpg%21d.jpeg","mime":"image\/jpeg","size":319882,"path_740":"http:\/\/www.tlwarc.hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/images\/architecture_buildings_bus_business_cars_city_cityscape_clouds-1495895.jpg%21d.jpeg?itok=hNeMUxDW"}}},"media_ids":["620310"],"groups":[{"id":"47223","name":"College of Computing"},{"id":"50875","name":"School of Computer Science"}],"categories":[],"keywords":[],"core_research_areas":[{"id":"39431","name":"Data Engineering and Science"},{"id":"39501","name":"People and Technology"}],"news_room_topics":[],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003ETess Malone, Communications Officer\u003C\/p\u003E\r\n\r\n\u003Cp\u003Etess.malone@cc.gatech.edu\u003Ca href=\u0022mailto:tess.malone@cc.gatech.edu\u0022\u003Etess.malone@cc.gatech.edu\u003C\/a\u003E\u003C\/p\u003E\r\n","format":"limited_html"}],"email":["tess.malone@cc.gatech.edu"],"slides":[],"orientation":[],"userdata":""}}}