Discussing Trust, Ethics, and Responsibility in ML at ICML, VLDB, and ICLR
With the question of ethics and responsibility in ML research beginning to resonate within the global AI community, we take a look at some of the workshops this year dedicated to this cause.
AI research and related discussions are in full swing — this comes as no surprise to anyone. The rhetoric of responsible AI, on the other hand, still has some catching up to do. According to a survey from last year, currently, only 25% of companies consider unbiased AI a worthy goal. Big Tech has taken steps to change that, but by and large, the private sector has yet to make responsible AI its priority.
Meanwhile, when you apply for a bank loan, it’s often the machine that decides whether you get approved or rejected based on an algorithm that cross-references such factors as age, gender, postal code, and language. Many job applications are now assessed this way, too. When you get into a modern car, speech recognition is used, and how well you’re understood (or not at all) comes down to your accent. Naturally, these factors raise sensible concerns over how ML models are built and implemented.
Some world leaders are well aware of the importance of controlled and well-maintained AI practices. For instance, last year, the European Parliament published a comprehensive, 100-page report on the potential issues associated with unchecked AI development. WEF and OECD have also been vocal in opposing black box algorithms and unethical data usage.
Clearly, the research that ML scientists pursue and oversee today has far-reaching implications. This development has to make space for transparent pipelines built to accommodate fair and ethical decision-making, free of discrimination and injustice. Or free of corrupted data at the very least — that would be the first red flag, with significantly more troubles ahead.
Many ML workshops and conferences the world over have begun to regularly focus their attention on these crucial issues in hopes of making a difference. Let’s look at 3 of the most interesting examples from this year.
→ Responsible AI: An Interdisciplinary Perspective on Practical Limitations and Tradeoffs among Fairness, Explainability, Safety, Robustness, and Beyond
As part of the ICLR conference this year, an interdisciplinary workshop was held on May 7. The workshop brought together experts in various AI sub-fields, from academia and business to government — both researchers and practitioners. The aim was to evaluate ML pipelines from the perspective of fairness, safety, and interpretability. Through a series of talks, the event raised questions of transparency and bias (black box algorithms) as reflected in ethical and legal guidelines and practices, among them in criminal justice, medicine, and education.
Speakers
All of the talks were recorded and are available here.
📣 This talk titled “Towards Interpreting Responsibly: Good Intentions are Not Enough” is by a staff research scientist at Google Brain, Dr. Been Kim. Kim offered her perspective on the future of ML through the prism of social responsibility. She presented a family of work and reexamined the goals and methods in interpretable ML.
📣 The next talk titled “Challenges and Opportunities of Working with Computer Scientists as a Social Scientist” is by an associate professor at the University of Southern California, Dr. Eric Rice. Eric discussed contemporary research in AI, shared his insights into AI-driven intervention programs, and presented 3 main findings of his work on HIV prevention interventions with homeless youth.
📣 Another talk titled “Towards Creating Models People Can Use: Experiences from Health Applications” is by a John L. Loeb associate professor at Harvard University, Dr. Finale Doshi-Velez. Finale discussed human-subject experiments that had tested interfaces designed to facilitate interaction between AI and clinicians in the context of antidepressant treatment. She revealed the most common requests clinicians had made for improvement, as well as their reaction to correct and incorrect AI recommendations.
About ICLR
The International Conference on Learning Representations (ICLR) is an international ML conference held every spring. The first ICLR conference took place in 2013 and has since risen to become a major gathering of computer scientists. With almost 3000 paper submissions this year and a 30% acceptance rate, today the conference is considered to be in the top 3 global events dedicated to AI and ML.
→ Trust, Ethics, and Excellence in Crowdsourced Data Management at Scale
As part of the VLDB’s 47th installment this year, a crowd science workshop will be held on August 20, the last day of the conference. Named “Trust, Ethics, and Excellence in Crowdsourced Data Management at Scale”, the workshop will cover three primary concerns of the crowdsourcing approach to data labeling: large-scale data excellence, crowd-AI interplay, and trust and ethics.
During the course of the workshop, three keynote speakers will examine the role of crowdsourcing in large-scale research with a focus on the welfare of crowd workers and trustworthiness of annotated data.
Speakers
📣 The first workshop talk titled “Computation and Organizations” is by an award-winning author and Associate Professor at Stanford University, Dr. Michael Bernstein. Michael will examine the future of work and online platforms. He will discuss how work may be organized in the future, and how the current infrastructure leads to disenfranchisement and alienation. He will then describe possible counters to these negative outcomes, both from the perspective of engineering and design, and from the perspective of policy.
📣 The second talk titled “HealthBytes: Fostering A Sustainable Future for Crowd Work” is by Dr. Ujwal Gadiraju, an Assistant Professor at Delft University of Technology in the Netherlands. While most concerns and solutions associated with crowdsourcing marketplaces have been focused primarily on data quality, Dr. Gadiraju will examine the domain’s less-explored problem — the wellbeing of crowd workers. By using MTurk and Prolific as examples, the researcher will dissect microtask crowdsourcing from the perspective of those who complete the tasks, including their pay and their physical and mental health. The need for contractual laws governing crowdsourcing will be discussed.
📣 The last talk of the series titled “Digital Community Driven Crowdsourcing” is by Wuraola Oyewusi, a Research and Innovation Lead at Data Science Nigeria. The speaker will offer her insight into the “humanizing” aspect of crowdsourcing, namely how crowd workers can interact not only with crowdsourcing platforms, but also within global and regional communities. Human-to-human learning and feedback within the context of crowdsourcing will be discussed and new worker-training models offered.
About VLDB
Launched in 1975, International Conference on Very Large Data Bases (VLDB) is one of the foremost annual gatherings of researchers and developers dedicated to database management. Organized by the US-based VLDB Endowment and held in a different country each year, the much-anticipated VLDB 2021 is set to take place in Copenhagen, Denmark, between August 16 and 20.
→ Socially Responsible ML
As part of the ICML’s 38th installment this year, a virtual workshop will be held on July 24. The workshop will bring together theoretical and applied researchers with the goal of discussing how to build socially responsible ML pipelines. Among other topics, the questions of vulnerability to security and privacy attacks, data leakage, as well as racial and gender biases will be raised. The workshop will address a lack of transparency, and how it can lead to deliberate or accidental data perturbations with real-life consequences.
About ICML
The International Conference on Machine Learning (ICML) is the leading global conference in the field of AI and ML. First held in Pittsburgh in 1980, this annual event is dedicated to statistics and data science with a focus on machine vision, computational biology, speech recognition, and robotics. It’s currently one of the fastest growing AI gatherings in the world. The conference dates this year are July 18–24.
The main takeaway
As AI research moves forward, the questions of morality and fairness are becoming more relevant by the day. The need for transparent and well built ML pipelines is at the epicenter of this growing concern. The objective is to make sure that the obvious benefits of emerging technologies are not overshadowed by the problems stemming from irresponsible AI development. The workshops mentioned in this article examine these ethical issues at length and offer some solutions.
