TiC-LM: A Web-Scale Benchmark for Time-Continual LLM Pretraining
This paper was accepted at the Scalable Continual Learning for Lifelong Foundation Models (SCLLFM) Workshop at NeurIPS 2024.
Large Language Models (LLMs) trained on...
Revisit Large-Scale Image–Caption Data in Pre-training Multimodal Foundation Models
Recent advancements in multimodal models highlight the value of rewritten captions for improving performance, yet key challenges remain. Notably, the role of synthetic...
Apple Workshop on Natural Language Understanding 2024
Progress in natural language processing enables more intuitive ways of interacting with technology. For example, many of Apple’s products and services, including Siri...
SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
Large Language Models (LLMs) have transformed natural language processing, but face significant challenges in widespread deployment due to their high runtime cost. In...
Interpreting and Improving Optimal Control Problems With Directional Corrections
Many robotics tasks, such as path planning or trajectory optimization, are formulated as optimal control problems (OCPs). The key to obtaining high performance...
Mutual Reinforcement of LLM Dialogue Synthesis and Summarization Capabilities for Few-Shot Dialogue Summarization
In this work, we propose Mutual Reinforcing Data Synthesis (MRDS) within LLMs to improve few-shot dialogue summarization task. Unlike prior methods that require...
Modeling Speech Emotion With Label Variance and Analyzing Performance Across Speakers and Unseen Acoustic...
Spontaneous speech emotion data usually contain perceptual grades where graders assign emotion score after listening to the speech files. Such perceptual grades introduce...
Universally Instance-Optimal Mechanisms for Private Statistical Estimation
We consider the problem of instance-optimal statistical estimation under the constraint of differential privacy where mechanisms must adapt to the difficulty of the...
The Role of Prosody in Spoken Question Answering
Spoken language understanding research to date has generally carried a heavy text perspective. Most datasets are derived from text, which is then subsequently...
Fundamental Challenges in Evaluating Text2SQL Solutions and Detecting Their Limitations
In this work, we dive into the fundamental challenges of evaluating Text2SQL solutions and highlight potential failure causes and the potential risks of...
