Tobias Falke

Researcher in Machine Learning and Foundation Models

I’m a Principal Applied Scientist at Amazon AGI in Berlin, working on foundation model pretraining. My work spans the full pretraining stack: model architecture (MoEs in particular), training recipes, scaling, experimental design and evaluation. Most recently, I also worked on automating research, agentic systems and self-improving systems.

I joined Amazon in 2019 after completing my PhD at UKP Lab, Technische Universität Darmstadt, where I worked on NLP for document exploration: text summarization, information extraction and interactive learning. Before moving into LLM pretraining, I spent several years working on language understanding for Amazon Alexa.

Writing

All posts →

Publications

2026

MoE Routing Testbed: Studying Expert Specialization and Routing Behavior at Small Scale

Tobias Falke, Nicolas Anastassacos, Samson Tan, Chankrisna Richy Meas, Chandana Satya Prakash, Nitesh Sekhar, M Saiful Bari, Krishna Kompella and Gamaleldin F. Elsayed

arXiv:2604.07030

pdf bib

2023

Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models

Saleh Soltan, Andy Rosenbaum, Tobias Falke, Qin Lu, Anna Rumshisky and Wael Hamza

Findings of ACL 2023. Toronto, Canada.

pdf bib

Regression-Free Model Updates for Spoken Language Understanding

Andrea Caciolai, Verena Weber, Tobias Falke, Alessandro Pedrani and Davide Bernardi

ACL 2023 (Industry Track). Toronto, Canada.

pdf bib

Sharing Encoder Representations across Languages, Domains and Tasks in Large-Scale Spoken Language Understanding

Jonathan Hueser, Judith Gaspers, Thomas Gueudre, Chandana Prakash, Jin Cao, Daniil Sorokin, Quynh Do, Nicolas Anastassacos, Tobias Falke, Turan Gojayev, Mariusz Momotko, Denis Romasanta Rodriguez, Austin Doolittle, Kartik Balasubramaniam, Wael Hamza, Fabian Triefenbach and Patrick Lehnen

ACL 2023 (Industry Track). Toronto, Canada.

pdf bib

2021

Feedback Attribution for Counterfactual Bandit Learning in Multi-Domain Spoken Language Understanding

Tobias Falke and Patrick Lehnen

EMNLP 2021. Virtual.

pdf bib

Multilingual Paraphrase Generation For Bootstrapping New Features in Task-Oriented Dialog Systems

Subhadarshi Panda, Caglar Tirkaz, Tobias Falke and Patrick Lehnen

NLP4ConvAI @ EMNLP 2021. Virtual.

pdf bib

2020

Leveraging User Paraphrasing Behavior In Dialog Systems To Automatically Collect Annotations For Long-Tail Utterances

Tobias Falke, Markus Boese, Daniil Sorokin, Caglar Tirkaz and Patrick Lehnen

COLING 2020 (Industry Track). Virtual.

★ Best Paper Award — Industry Track

pdf bib

Data-Efficient Paraphrase Generation to Bootstrap Intent Classification and Slot Labeling for New Features in Task-Oriented Dialog Systems

Shailza Jolly, Tobias Falke, Caglar Tirkaz and Daniil Sorokin

COLING 2020 (Industry Track). Virtual.

pdf bib

2019

Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference

Tobias Falke, Leonardo F. R. Ribeiro, Prasetya Ajie Utama, Ido Dagan and Iryna Gurevych

ACL 2019. Florence, Italy.

pdf bib data

Fast Concept Mention Grouping for Concept-Map-Based Multi-Document Summarization

Tobias Falke and Iryna Gurevych

NAACL 2019. Minneapolis, MN, USA.

pdf bib code

Automatic Structured Text Summarization with Concept Maps

Tobias Falke

PhD Thesis. Technische Universität Darmstadt, Germany.

pdf bib

2018

What's Important in a Text? An Extensive Evaluation of Linguistic Annotations for Summarization

Markus Zopf, Teresa Botschen, Tobias Falke, Benjamin Heinzerling, Ana Marasovic, Todor Mihaylov, P.V.S. Avinesh, Eneldo Loza Mencia, Johannes Fürnkranz and Anette Frank

SNAMS 2018. Valencia, Spain.

pdf bib

2017

System Z^FO: Default Reasoning with System Z-like Ranking Functions for Unary First-Order Conditional Knowledge Bases

Christoph Beierle, Tobias Falke, Steven Kutsch and Gabriele Kern-Isberner

International Journal of Approximate Reasoning, Volume 90, 2017.

pdf bib code

Concept-Map-Based Multi-Document Summarization using Concept Coreference Resolution and Global Importance Optimization

Tobias Falke, Christian M. Meyer and Iryna Gurevych

IJCNLP 2017. Taipei, Taiwan.

pdf bib code

Bringing Structure into Summaries: Crowdsourcing a Benchmark Corpus of Concept Maps