ML with New Compute Paradigms (MLNCP) at NeurIPS 2023

Welcome to the MLNCP Workshop at NeurIPS 2023!

This workshop aims to bring together ML researchers with academic and industrial researchers building novel AI accelerators. The goal is to enable interaction between the two groups and kick-start a new feedback cycle between models and accelerators and to enable hardware-model co-design. We welcome relevant algorithmic or model-innovations as well as results demonstrated on accelerators in the following categories:

• photonic or optical compute
• neuromorphic compute
• in-memory compute
• low-precision and edge-compute
• analog compute
• biologically-plausible machine-learning

The workshop will be held on December 16th, 2023 as part of the NeurIPS conference in New Orleans, Louisiana.

Abstract

As GPU computing comes closer to a plateau in terms of efficiency and cost due to Moore' s law reaching its limit, there is a growing need to explore alternative computing paradigms, such as (opto-)analog, neuromorphic, and low-power computing. This NeurIPS workshop aims to unite researchers from machine learning and alternative computation fields to establish a new hardware-ML feedback loop. By co-designing models with specialized accelerators, we can leverage the benefits of increased throughput or lower per-flop power consumption. Novel devices hold the potential to further accelerate standard deep learning or even enable efficient inference and training of hitherto compute-constrained model classes. However, new compute paradigms typically present challenges such as intrinsic noise, restricted sets of compute operations, or limited bit-depth, and thus require model-hardware co-design. This workshop's goal is to foster cross-disciplinary collaboration to capitalize on the opportunities offered by emerging AI accelerators.

Speakers

• 

  Prof Peter McMahon

  Cornell University

  Expert in quantum and classical unconventional computers, including photonic AI accelerators.

• 

  Prof Jesús del Alamo

  MIT

  Expert in electronic, magnetic, optical and quantum materials and devices, especially nanoscale materials, devices, and systems.

• 

  Prof Julie Grollier

  Thales/CNRS

  Expert in building and designing neuromorphic devices, memristors, and spintronics.

• 

  Dr Kailash Gopalakrishnan

  Encharge.ai

  Expert in AI accelerator architecture, system design, low-bit precision machine-learning, and hardware-software codesign.

• 

  Prof Mihaela van der Schaar

  University of Cambridge

  Expert in ML theory.

• 

  Prof Ruqi Zhang

  Purdue University

  Expert in low-precision machine-learning and approximate inference.

Schedule

Morning Session:

• 09:00 - 09:10: Opening remarks (10 min)
• 09:10 - 09:30: Prof Peter McMahon: Computing with physical systems: reimagining special-purpose computers from the bottom up (15 min) + 5 min Q&A
• 09:30 - 09:40: Contributed Talk I: SpiNNaker2: A Large-Scale Neuromorphic System for Event-Based and Asynchronous Machine Learning (8 min) + 2 min Q&A
• 09:40 - 09:50: Contributed Talk II: Scaling-up Memristor Monte Carlo with magnetic domain-wall physics (8 min) + 2 min Q&A
• 09:50 - 10:10: Prof Jesús del Alamo: Analog AI Accelerators (15 min) + 5 min Q&A
• 10:10 - 10:25: Break (15 min)
• 10:25 - 10:35: Jannes Gladrow: Analog Iterative Machine (AIM) - analog computation for optimization and machine-learning inference (8 min) + 2 min Q&A
• 10:35 - 10:45: Contributed Talk III: Bayesian Metaplasticity from Synaptic Uncertainty (8 min) + 2 min Q&A
• 10:45 - 11:05: Prof Julie Grollier: Training physical systems with Equilibrium Propagation (15 min) + 5 min Q&A
• 11:05 - 11:25: Prof Ruqi Zhang: Low-precision Sampling for Probabilistic Deep Learning (15 min) + 5 min Q&A

Poster Session and Lunch Break:

• 11:25 - 12:30: Poster session I (1 hour)
• 12:30 - 13:30: Lunch break

Afternoon Session:

• 01:30 - 01:50: Prof Mihaela van der Schaar: The Optimization Conundrum: Balancing Data, Speed, and Accuracy in Real-Time Machine Learning (15 min) + 5 min Q&A
• 01:50 - 02:00: Contributed Talk IV: A Lagrangian Perspective on Dual Propagation (8 min) + 2 min Q&A
• 02:00 - 02:10: Contributed Talk V: Scaling of Optical Transformers (8 min) + 2 min Q&A
• 02:10 - 02:30: Dr Kailash Gopalakrishnan: Low-Precision & Analog Computational Techniques for Sustainable & Accurate AI Inference & Training (15 min) + 5 min Q&A
• 02:30 - 04:00: Poster session II and Coffee Break (1.5 hours)
• 04:00 - 05:00: Panel discussion (1 hour)

Organisers

The workshop is organised by the following people:

• Jannes Gladrow (Microsoft Research)
• Benjamin Scellier (Rain.ai)
• Eric Xing (Carneggie Mellon, Mohamed bin Zayed University of Artificial Intelligence)
• Babak Rahmani (Microsoft Research)
• Francesca Parmigiani (Microsoft Research)
• Paul R. Prucnal (Princeton University)
• Cheng Zhang (Microsoft Research)

Sponsors