PhD in Computer Science, University of Trento (Nov 2023 – Present)
Ciao, I’m Leonardo! I’m a PhD student in Information Engineering and Computer Science at the University of Trento, supervised by Prof. Raffaella Bernardi and co-supervised by Prof. Barbara Plank.
My research lies at the intersection of artificial intelligence and cognitive science, focusing on the reasoning capabilities of large language models (LLMs). More broadly, I am interested in understanding how these models solve reasoning problems, examining the mechanisms that drive their decisions, and comparing them to formal systems and human reasoning.
If you have questions about my research or are interested in collaborating, please reach out!
News
- Mechanistic Interpretability Lab at AILC Lectures 2026! I will lead the Mechanistic Interpretability lab at the AILC Lectures on Computational Linguistics 2026 in Bolzano. The lab materials are available here.
- New preprint! Our new preprint is now available: FALSIFYBENCH: Evaluating Inductive Reasoning in LLMs with Rule Discovery Games.
- New paper accepted at ACL ‘26 Findings! Our paper How Language Models Conflate Logical Validity with Plausibility: A Representational Analysis of Content Effects has been accepted at ACL ‘26 Findings.
- Internship at Booking.com! I will join Booking.com as a PhD Machine Learning Research Intern for a three-month internship beginning in August 2026 at the Amsterdam office.
- New paper accepted at EACL ‘26! Our paper Teaching Small Language Models to Learn Logic through Meta-Learning has been accepted at EACL ‘26.
- New preprint! Our preprint How Language Models Conflate Logical Validity with Plausibility: A Representational Analysis of Content Effects became available.
- Visiting UvA! I joined the ILLC at the University of Amsterdam as a visiting researcher to work with Prof. Sandro Pezzelle.
- New paper accepted at EMNLP ‘25! Our work The Validation Gap: A Mechanistic Analysis of How Language Models Compute Arithmetic but Fail to Validate It was accepted at EMNLP ‘25.
- New preprint! Our preprint Teaching Small Language Models to Learn Logic through Meta-Learning became available.
- New paper accepted at ACL ‘25! Our paper LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks was accepted at ACL ‘25.
- New preprint! Our preprint The Validation Gap: A Mechanistic Analysis of How Language Models Compute Arithmetic but Fail to Validate It became available.
- New paper accepted at EMNLP ‘24! Our paper A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferences was accepted at EMNLP ‘24.
- Starting my PhD! I started my PhD in Information Engineering and Computer Science at the University of Trento.
- Master’s degree in Cognitive Science! I received my master’s degree in Cognitive Science from CIMeC, University of Trento.
- Best Paper Award: Evaluation! Our paper ChatGPT’s Information Seeking Strategy: Insights from the 20-Questions Game received the Best Paper Award in the Evaluation category at INLG 2023.
- New paper accepted at INLG 2023! Our work ChatGPT’s Information Seeking Strategy: Insights from the 20-Questions Game was accepted at INLG 2023.
Education
MSc in Cognitive Science (Computational Linguistics), University of Trento (Sep 2021 – Oct 2023)
BA in Philosophy (Logic and Philosophy of Language), University of Trento (Sep 2018 – Jul 2021)
Papers
All publications · Google Scholar
FALSIFYBENCH: Evaluating Inductive Reasoning in LLMs with Rule Discovery Games,
2026,
ArXiv preprint
Leonardo Bertolazzi
,
Katya Tentori
,
Raffaella Bernardi
How Language Models Conflate Logical Validity with Plausibility: A Representational Analysis of Content Effects,
2026,
Findings of the Association for Computational Linguistics: ACL 2026
Leonardo Bertolazzi
,
Sandro Pezzelle
,
Raffaella Bernardi
Teaching Small Language Models to Learn Logic through Meta-Learning,
2026,
Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics
Leonardo Bertolazzi
,
Manuel Vargas Guzmán
,
Raffaella Bernardi
,
Maciej Malicki
,
Jakub Szymanik
The Validation Gap: A Mechanistic Analysis of How Language Models Compute Arithmetic but Fail to Validate It,
2025,
The 2025 Conference on Empirical Methods in Natural Language Processing
Leonardo Bertolazzi
,
Philipp Mondorf
,
Barbara Plank
,
Raffaella Bernardi
LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks,
2025,
The 63rd Annual Meeting of the Association for Computational Linguistics
Anna Bavaresco
,
Raffaella Bernardi
,
Leonardo Bertolazzi
,
Desmond Elliott
,
Raquel Fernández
,
Albert Gatt
,
Esam Ghaleb
,
Mario Giulianelli
,
Michael Hanna
,
Alexander Koller
,
André F. T. Martins
,
Philipp Mondorf
,
Vera Neplenbroek
,
Sandro Pezzelle
,
Barbara Plank
,
David Schlangen
,
Alessandro Suglia
,
Aditya K Surikuchi
,
Ece Takmaz
,
Alberto Testoni
A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferences,
2024,
The 2024 Conference on Empirical Methods in Natural Language Processing
Leonardo Bertolazzi
,
Albert Gatt
,
Raffaella Bernardi