Bulletin #8 Friday 23rd, February, 2024
|
Important Dates & Reminders
Saturday, March 9, 2024 Winter Classes End
Monday, March 11, 2024 Winter Examinations Begin
Saturday, March 16, 2024 Spring Break Begins
|
|
We want to hear from you! Please send any upcoming news and events to news@cs.northwestern.edu to be included in future bulletins &/featured on our socials/website.
Events must be emailed at least two (2) business days in advance.
|
|
In this Issue
Upcoming Seminars:
Monday 26th February
"Accessible Foundation Models: Systems, Algorithms, and Science" (Tim Dettmers)
Wednesday 28th February
"Paths to AI Accountability" (Sarah Cen)
Monday 4th March
"Understanding Language Models through Discovery and by Design" (John Hewitt)
CS Events:
Bagel Thursday | Feb 29
CSPAC Workshop Series | Various Dates
Northwestern Events
|
|
Missed a seminar? No worries!
View past seminars via the Northwestern CS Website
(northwestern login required).
|
|
February
26th - Tim Dettmers
28th - Sarah Cen
March
4th - John Hewitt
|
Monday/ CS Seminar
February 26th / 12:00 PM
In Person / Mudd 3514
|
|
|
" Accessible Foundation Models: Systems, Algorithms, and Science"
|
Abstract
The ever-increasing scale of foundation models, such as ChatGPT and AlphaFold, has revolutionized AI and science more generally. However, increasing scale also steadily raises computational barriers, blocking almost everyone from studying, adapting, or otherwise using these models for anything beyond static API queries. In this talk, I will present research that significantly lowers these barriers for a wide range of use cases, including inference algorithms that are used to make predictions after training, finetuning approaches that adapt a trained model to new data, and finally, full training of foundation models from scratch. For inference, I will describe our LLM.int8() algorithm, which showed how to enable high-precision 8-bit matrix multiplication that is both fast and memory efficient. LLM.int8() is based on the discovery and characterization of sparse outlier sub-networks that only emerge at large model scales but are crucial for effective Int8 quantization. For finetuning, I will introduce the QLoRA algorithm, which pushes such quantization much further to unlock finetuning of very large models on a single GPU by only updating a small set of the parameters while keeping most of the network in a new information-theoretically optimal 4-bit representation. For full training, I will present SWARM parallelism, which allows collaborative training of foundation models across continents on standard internet infrastructure while still being 80% as effective as the prohibitively expensive supercomputers that are currently used. Finally, I will close by outlining my plans to make foundation models 100x more accessible, which will be needed to maintain truly open AI-based scientific innovation as models continue to scale.
Biography
Tim Dettmers’ research focuses on making foundation models, such as ChatGPT, accessible to researchers and practitioners by reducing their resource requirements. This involves developing novel compression and networking algorithms and building systems that allow for memory-efficient, fast, and cheap deep learning. These methods enable many more people to use, adapt, or train foundation models without affecting the quality of AI predictions or generations. He is a PhD candidate at the University of Washington and has won oral, spotlight, and best paper awards at conferences such as ICLR and NeurIPS. He created the bitsandbytes library for efficient deep learning, which is growing at 1.4 million installations per month and received Google Open Source and PyTorch Foundation awards.
|
|
Wednesday/ CS Seminar
February 28th / 12:00 PM
In Person / Mudd 3514
|
|
|
"Paths to AI Accountability"
|
Abstract
We have begun grappling with difficult questions related to the rise of AI, including: What rights do individuals have in the age of AI? When should we regulate AI and when should we abstain? What degree of transparency is needed to monitor AI systems? These questions are all concerned with AI accountability: determining who owes responsibility and to whom in the age of AI. In this talk, I will discuss the two main components of AI accountability, then illustrate them through a case study on social media. Within the context of social media, I will focus on how social media platforms filter (or curate) the content that users see. I will review several methods for auditing social media, drawing from concepts and tools in hypothesis testing, causal inference, and LLMs.
Biography
Sarah is a final-year PhD student at MIT in the Electrical Engineering and Computer Science Department advised by Professor Aleksander Mądry and Professor Devavrat Shah. Sarah utilizes methods from machine learning, statistical inference, causal inference, and game theory to study responsible computing and AI policy. Previously, she has written about social media, trustworthy algorithms, algorithmic fairness, and more. She is currently interested in AI auditing, AI supply chains, and IP Law x Gen AI.
Research Interests/Area
Responsible AI, ethics in AI, AI policy + statistical inference, causal inference
|
|
Monday/ CS Seminar
March 4th / 12:00 PM
In Person / Mudd 3514
|
|
|
"Understanding Language Models through Discovery and by Design"
|
Abstract
Whereas we understand technologies like airplanes or microprocessors well enough to fix them when they break, our tools for fixing modern language models are coarse. This is because, despite language models' increasing ubiquity and utility, we understand little about how they work. In this talk, I will present two lines of research for developing a deep, actionable understanding of language models that allows us to discover how they work, and fix them when they fail. In the first line, I will present structural probing methods for discovering the learned structure of language models, finding evidence that models learn structure like linguistic syntax. In the second line, I will show how we can understand complex models by design: through the new Backpack neural architecture, which gives us precise tools for fixing models.
Biography
John is a PhD student in Computer Science at Stanford University, working with Percy Liang and Christopher Manning on discovering the learned structure of neural language models, and designing them to be more understandable, diagnosable, and fixable. He was an NSF Graduate Fellow, and received a B.S.E in Computer and Information Science from the University of Pennsylvania. John has received an Outstanding Paper Award at ACL 2023, a Best Paper Runner Up at EMNLP 2019, an Honorable Mention for Best Paper at the Robustness of Few-Shot Learning in Foundation Models Workshop (R0-FoMo)@NeurIPS 2023, and an Outstanding Paper Award at the Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackBoxNLP)@EMNLP 2020.
Research Interests/Area
Natural Language Processing
|
|
|
CS PhD Advisory Council is a PhD student-led organization. Our mandate is to interface between PhD students and faculty on academic issues. Reach us at cspac@u.northwestern.edu
|
|
|
|
|
|
Leap into delicious bagels and coffee with your fellow CS students and faculty at Bagel Thursday!
|
|
|
|
|
|
|
|
|
Thursday, February 29th 2024; 9AM-11AM
|
|
|
Mudd 3514
2233 Tech Drive
|
|
|
|
|
Northwestern Medicine Healthcare AI Forum
|
The Northwestern Medicine Healthcare AI Forum dives into cutting-edge developments in the field of AI for healthcare. Presenters share the latest published research and technology innovation, and facilitate discussion among attendees.
Open to the entire Northwestern Medicine community, the forum is presented by the Center for Collaborative AI in Healthcare, Institute for Artificial Intelligence in Medicine (I.AIM).
|
|
|
|
|
|
|
|
|
Fridays Bi-Weekly 10:00 AM CT
|
|
|
3rd Annual Traditional Spring Pow Wow- Hosted by NAISA
|
Hosted by Northwestern's Native American and Indigenous Student Alliance
Contact: NAISAPOWWOW@gmail.com
|
|
|
|
|
|
|
|
|
Saturday, April 27, 2024
11:00 AM - 5:00 PM
|
|
|
Welsh Ryan Arena
2705 Ashland Ave, Evanston, IL 60208
|
|
|
|
Advancing Compiler Technology
|
Six papers reflecting multidisciplinary Northwestern Computer Science collaborations in compilers have been accepted into prestigious conferences this year.
Read More
|
|
|
|
Northwestern CS, YWCA Advance ‘Tech Lab’ Initiative
|
Supported by a Racial Equity and Community Partnership grant from Northwestern, Northwestern Computer Science and YWCA Evanston/North Shore are helping remove racial barriers in the technology field through the YW Tech Lab economic empowerment training program.
Read More
|
|
|
|
Prioritizing Ethics in the Computer Science Curriculum
|
Sara Owsley Sood will receive The Alumnae of Northwestern University’s Award for Curriculum Innovation, which recognizes and supports faculty who have innovative ideas for new courses, methods of instruction, and components of existing classes.
Read More
|
|
|
|
|
One Northwestern Researcher’s Mission to Elevate Healthcare
|
After his own near-death event, Northwestern Engineering Professor Sanjay Mehrotra has devoted the last half of his research career to improving healthcare decision-making through data science and predictive analytics.
Read More
|
|
|
|
AI2: Breaking ground on artificial intelligence
|
Northwestern Qatar has launched a new initiative to contribute to research, teaching and professional development in artificial intelligence.
Read More
|
|
|
|
© Robert R. McCormick School of Engineering and Applied Science, Northwestern University
|
|
Northwestern Department of Computer Science Mudd Hall, 2233 Tech Drive, Third Floor, Evanston, Illinois, 60208 Unsubscribe
|
|
|
|
|
|