Designing and Evaluating Human-AI Collaboration

Abstract

How can artificial intelligence systems most effectively assist humans in problem solving? This talk will offer a framework for addressing this question in high-stakes, real-world computer vision applications in medicine and misinformation. Specifically, this talk will discuss empirical results from two large-scale digital experiments on the dynamics of physician-machine partnerships in store-and-forward teledermatology diagnosis and AI assistance in deepfake detection. The findings in both experiments reveal that overall increases in task accuracy by humans with AI assistance come with unexpected trade-offs on other performance metrics.

Biography

Matt Groh is assistant professor at Northwestern University Kellogg School of Management in the Management and Organizations department and a core faculty member of Northwestern Institute on Complex Systems (NICO). His research examines human-AI collaboration with a focus on misinformation, medical diagnosis, and empathy. Before joining Northwestern, Matt worked as a data scientist at multiple startups, a non-profit, the World Bank, and DARPA and cofounded a startup. Matt has a BA from Middlebury College where he majored in economics and minored in Arabic and mathematics and an MA and PhD from MIT in Media Arts and Sciences.

Zoom Link: https://northwestern.zoom.us/j/91563101700?pwd=aVdXbWRMdTJSd3hEcnlhYnhsQnJ6QT09

Panopto:  https://northwestern.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=51a093fb-285b-4846-bf07-b0ee012bd2dd

Research Interests/Area

Human-AI Collaboration, Computational Social Science, Affective Computing, Synthetic Media

White-Box Computational Imaging: Measurements to Images to Insights 

Abstract

Computation and machine learning hold tremendous potential to improve the quality and capabilities of imaging methods used across science, medicine, engineering, and art. Despite their impressive performance on benchmark datasets, however, deep learning methods are known to behave unpredictably on some real-world data, which limits their trusted adoption in safety-critical domains. Accordingly, in this talk I will describe white-box, interpretable methods for photorealistic volumetric reconstruction that match or exceed the performance of black-box neural alternatives. I will also present recent theoretical results that guarantee correct and efficient reconstruction using our white-box approach in nonlinear computed tomography. 

Biography

Sara Fridovich-Keil is a postdoctoral fellow at Stanford University, where she works with Mert Pilanci and Gordon Wetzstein on foundations and applications of machine learning and signal processing in computational imaging. She is currently supported by an NSF Mathematical Sciences Postdoctoral Research Fellowship. Sara received her PhD in electrical engineering and computer sciences in May 2023 from UC Berkeley, where she was advised by Ben Recht and supported by an NSF GRFP fellowship. Sara received her BSE in electrical engineering from Princeton University in 2018, where she was advised by Peter Ramadge and supported, in part, by a Barry Goldwater Scholarship.

Zoom Link: https://northwestern.zoom.us/j/95206250523?pwd=T05wK0k0c0ROcklDTlVMektxV2Rndz09

Panopto:  https://northwestern.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=f3ac4dd3-f183-4b50-8163-b0ee012c8e6d

Research Interests/Area

Computational imaging; signal processing; machine learning; computer vision

On Data Ecology, Data Markets, the Value of Data, and Dataflow Governance

Abstract

Data shapes our social, economic, cultural, and technological environments. Data is valuable, so people seek it, inducing data to flow. The resulting dataflows distribute data and thus value. For example, large Internet companies profit from accessing data from their users, and engineers of large language models seek large and diverse data sources to train powerful models. It is possible to judge the impact of data in an environment by analyzing how the dataflows in that environment impact the participating agents. My research hypothesizes that it is also possible to design (better) data environments by controlling what dataflows materialize; not only can we analyze environments but also synthesize them. In this talk, I present the research agenda on “data ecology,” which seeks to build the principles, theory, algorithms, and systems to design beneficial data environments. I will also present examples of data environments my group has designed, including data markets for machine learning, data-sharing, and data integration. I will conclude by discussing the impact of dataflows in data governance and how the ideas are interwoven with the concepts of trust, privacy, and the elusive notion of “data value.” As part of the technical discussion, I will complement the data market designs with the design of a data escrow system that permits controlling dataflows.

Biography

In my research, I ask what is the value of data and explore the potential of data markets to unlock that value. My group collaborates with economists, legal scholars, statisticians, and domain scientists. We build systems to share, discover, prepare, integrate, and process data. I have traditionally worked on distributed query processing systems and continue to do so. I have received a SIGMOD'23 Test-of-time-Award. I am an assistant professor in the Department of Computer Science and on the Committee of Data Science at The University of Chicago. Before UChicago, I did a postdoc at MIT with Sam Madden and Mike Stonebraker. And before that, I completed a PhD at Imperial College London with Peter Pietzuch.

Zoom Link:  https://northwestern.zoom.us/j/96564331079?pwd=Ukl2MFNzM2sxc2FSZi9Oa0hHUllYUT09

Panopto: https://northwestern.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=569d51db-14d0-4ae3-80ea-b0f5011ffc7c

Research Interests/Area

Data Science, Data Management, Systems

CS Department Events

Bagel Thursday

Other Events & Opportunities

Entrepreneurship Open House

Startup Matchmaking at The Garage

CS News

NU News