ML with New Compute Paradigms (MLNCP) at NeurIPS 2024

Welcome to the MLNCP Workshop at NeurIPS 2024!

The workshop will take place on Sunday, the 15th December at NeurIPS 2024 in Vancouver, Canada. The currently assigned meeting rooms are 114, 115 in the Vancouver Conference Center.

Digital computing is approaching fundamental limits and faces serious challenges in terms of scalability, performance, and sustainability. At the same time, generative AI is fuelling an explosion in compute demand. There is, thus, a growing need to explore non-traditional computing paradigms, such as (opto-)analog, neuromorphic hardware, and physical systems. Expanding on last year's successful NeurIPS workshop, which was the first of its kind in this community, we aim to bring together researchers from machine learning and alternative computation fields to establish new synergies between ML models and non-traditional hardware. Co-designing models with specialized hardware, a feature that has also been key to the synergy of digital chips like GPUs and deep learning, has the potential to offer a step change in the efficiency and sustainability of machine learning at scale. Beyond speeding up standard deep learning, new hardware may open the door for efficient inference and training of model classes that have been limited by compute resources, such as energy-based models and deep equilibrium models. So far, however, these hardware technologies have fallen short due to inherent noise, device mismatch, a limited set of compute operations, and reduced bit-depth. As a community, we need to develop new models and algorithms that can embrace and, in fact, exploit these characteristics. This workshop aims to encourage cross-disciplinary collaboration to exploit the opportunities offered by emerging AI accelerators both at training and at inference.

For Authors of Submissions

Please submit the camera-ready version (deanonymized, reviewer feedback taken into account) by the 6th of December.
For Posters, please consider NeurIPS guidelines for posters. Authors of oral presentations are invited to also bring a poster to the workshop.

Tentative Workshop Schedule

Time Event Duration
Morning Session (ML focus)
09:00 - 09:20 Opening remarks & Awards 20 min
09:20 - 09:40 Zico Kolter (15 min) + 5 min Q&A 20 min
09:40 - 09:50 Contributed Talk I (8 min) + 2 min Q&A 10 min
09:50 - 10:00 Contributed Talk II (8 min) + 2 min Q&A 10 min
10:00 - 10:20 Dimitry Krotov (15 min) + 5 min Q&A 20 min
10:20 - 10:30 Break 10 min
10:30 - 10:40 Contributed Talk III (8 min) + 2 min Q&A 10 min
10:40 - 11:00 Azalia Mirsoheini (15 min) + 5 min Q&A 20 min
11:00 - 11:10 Contributed Talk IV (8 min) + 2 min Q&A 10 min
11:10 - 11:30 Break 20 min
11:30 - 12:00 ML Panel discussion 30 min
Poster Session and Lunch Break
12:00 - 13:00 Poster session 1 hour
13:00 - 13:40 Lunch break 40 min
Afternoon Session (Hardware focus)
13:40 - 14:00 Mike Davies (15 min) + 5 min Q&A 20 min
14:00 - 14:10 Contributed Talk V (8 min) + 2 min Q&A 10 min
14:10 - 14:30 Clara Wanjura (15 min) + 5 min Q&A 20 min
14:30 - 14:40 Contributed Talk VI (8 min) + 2 min Q&A 10 min
14:40 - 14:50 Break 10 min
14:50 - 15:10 Jannes Gladrow (15 min) + 5 min Q&A 20 min
15:10 - 15:30 Phillip Stanley-Marbell (15 min) + 5 min Q&A 20 min
15:30 - 15:40 Contributed Talk VII (8 min) + 2 min Q&A 10 min
15:40 - 16:00 Patrick Coles (15 min) + 5 min Q&A 20 min
16:00 - 16:30 Break & Poster session 30 min
16:30 - 17:00 Hardware Panel discussion 30 min

Speakers and Panellists

Zico Kolter

Zico Kolter

Prof. Zico Kolter is a professor at Carnegie Mellon University and Director of the Machine Learning Department as well as Chief Scientist of AI research at Bosch. His research spans several areas within machine learning and he is well-known for innovation in deep learning architectures as well as AI robustness and safety. He is a recipient of the DARPA Young Faculty Award, a Sloan Fellowship, and best paper awards at NeurIPS, ICML (honorable mention), AISTATS (test of time), IJCAI, KDD, and PESGM. Prof. Kolter recently joined the board of directors of OpenAI.

Dimitry Krotov

Dimitry Krotov

Dmitry Krotov is a physicist working on neurobiologically inspired machine learning. He is a member of the research staff at the MIT-IBM Watson AI Lab and IBM Research in Cambridge, MA. Prior to this, he was a member of the Institute for Advanced Study in Princeton. His work mainly focuses on the theory of associative memory and energy-based neural architectures.

Azalia Mirsoheini

Azalia Mirsoheini

Prof. Azalia Mirhoseini is an assistant professor in the computer science department at Stanford University. Her research focuses on developing capable, reliable, and efficient AI systems. She has made significant contributions to decision-making problems in chip design, self-improving AI models, and scalable deep learning optimization. Before joining Stanford, Azalia worked at industry AI labs, including Anthropic and Google Brain. At Google Brain, she co-founded the ML for Systems team, which focused on automating and optimizing computer systems and chip design. Azalia Mirhoseini's work has been recognized through several prestigious awards, including the MIT Technology Review's 35 Under 35 Award and the Best ECE Thesis Award at Rice University.

Mike Davies

Mike Davies

Mike Davies is the Director of Intel’s Neuromorphic Computing Lab, a position he has held since 2017. His work focuses on developing neuromorphic computing systems, that is, chips that are inspired by the principles of the human brain. These systems aim to create more efficient and powerful computing architectures by integrating memory and computation into a web of artificial neurons that exchange simple messages. Mike Davies joined Intel in 2011 following the acquisition of Fulcrum Microsystems, where he had been involved in IC development for 11 years.

Clara Wanjura

Clara Wanjura

Clara Wanjura is leading a Minerva Fast Track Group at the Max Planck Institute for the Science of Light since 2024. After her undergraduate studies at Ulm University, she moved to the University of Cambridge where she received her PhD in 2022. She became a postdoctoral researcher at the Max Planck Institute for the Science of Light in the group of Florian Marquardt in 2022 and received a Minerva Fast Track Fellowship in 2024.

Phillip Stanley-Marbell

Phillip Stanley-Marbell

Phillip Stanley-Marbell is Professor of Physical Computation at the University of Cambridge and founder of Signaloid. He was a Faculty Fellow at the Alan Turing Institute in London (2017 to 2021) and a Royal Academy of Engineering Enterprise Fellow (2022). Prior to moving to the UK in 2017, he was a researcher in the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT. From 2012 to 2014, he was with the Core OS organization at Apple. Prior to Apple, he spent several years (2008–2012) as a permanent research staff member at IBM Research in Zürich, Switzerland. He completed his Ph.D. at Carnegie Mellon University in 2007. Before his Ph.D., he spent several periods at Bell Labs: With the Lucent Data Networking division (1999), in a project spun out of the group that created UNIX, where he contributed to commercial products based on the Inferno operating system, and in the Lucent Microelectronics division, with a group that designed ASICs for telephony applications (1995, 1996).

Patrick Coles

Patrick Coles

Patrick Coles is the Chief Scientist at Normal Computing, a deep tech AI startup known for pioneering thermodynamic computing. His work focuses on developing energy-efficient AI hardware systems that can unlock new capabilities, as well as decision-making under uncertainty. Before joining Normal Computing, Patrick Coles was the head of Quantum Computing at Los Alamos National Laboratory.

Ting Cao

Ting Cao

Ting Cao is a Principal Research Manager in the Heterogeneous Extreme Computing (HEX) group within the Systems and Networking research area at Microsoft Research. Ting received her Ph.D. from the Research School of Computer Science at the Australian National University. Her work has received a range of awards, such as 2012 ACM Research highlights, 2012 IEEE Micro Top Picks, 2021 ACM SIGMOBILE Research highlights, Best paper awards at various conferences as well as Huawei's Future star award.

Organisers

The workshop is organised by the following people:

Sponsors

Contact: MLwithNewCompute _at_ googlegroups.com