I work on making reasoning systems precise, verifiable, and usable in law and policy.

I am a Postdoctoral Scholar affiliated with Stanford Law School and Stanford NLP. I work with Dan Ho, Chris Manning and Jacob Goldin. I completed my Ph.D. in Computer Science at Yale (2021.9 - 2025.9). My dissertation is on advancing LLM reasoning from logic to law (advised by Drago Radev).

Previously, I interned at Google Deepmind, Meta Superintelligence Labs and AWS, working on advancing reasoning capabilities in large language models. I completed my B.Eng Computer Science degree in Nanyang Technological University, Singapore.

I was fortunate to have been supported by Emily Hau, Scott Shapiro and Ruzica Piskac during my PhD studies.

Selected Publications

• Creativity or Brute Force? Using Brainteasers as a Window into the Problem-Solving Abilities of Large Language Models
  Sophia S. Han, Stephen Xia, Grant Zhang, Howard Dai, Chen Liu, Lichang Chen, Hoang Huy Nguyen, Hongyuan Mei, Jiayuan Mao, R. Thomas McCoy
  NeurIPS 2025. Video Presentation. Featured in MIT Technology Review. Interview on “Do Models Think Like Us”.
• CourtReasoner: Can LLM Agents Reason Like Judges?
  Sophia S. Han, Shannon Zejiang Shen, Chen Liu, Yixin Liu, Roque K. Thuo, Sonia Knowlton, Ruzica Piskac, Scott J Shapiro, Arman Cohan
  EMNLP 2025.
• GraphIC: A Graph-Based In-Context Example Retrieval Model for Multi-Step Reasoning
  Jiale Fu, Yaqing Wang, Simeng Han, Jiaming Fan, Xu Yang
  AAAI 2026
• Measuring what Matters: Construct Validity in Large Language Model Benchmarks
  Andrew M. Bean, Ryan Othniel Kearns, Angelika Romanou, Franziska Sofia Hafner, Harry Mayne +11…, Sophia S. Han, +24…
  NeurIPS 2025 Datasets and Benchmarks. Featured in NBC News. Featured in The Guardian.
• Scheherazade: Evaluating Chain-of-Thought Math Reasoning in LLMs with Chain-of-Problems
  Stephen Miner, Yoshiki Takashima, Sophia S. Han, Ferhat Erata, Timos Antonopoulos, Ruzica Piskac, Scott J Shapiro
  NeurIPS 2025 Workshop MATH-AI: The 5th Workshop on Mathematical Reasoning and AI
• ATEB: Evaluating and Improving Advanced NLP Tasks for Text Embedding Models
  Sophia S. Han, Frank Palma Gomez, Tu Vu, Zefei Li, Daniel Cer, Hansi Zeng, Chris Tar, Arman Cohan, Gustavo Hernandez Abrego
  ACL 2025 Workshop: Towards Knowledgeable Foundation Models
• FOLIO: Natural Language Reasoning with First-Order Logic
  Sophia S. Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alex Wardle-Solano, Hannah Szabo, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander R. Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Victoria Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev
  EMNLP 2024 (Video presentation).
• P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains
  Sophia S. Han, Aaron Yu, Rui Shen, Zhenting Qi, Martin Riddell, Wenfei Zhou, Yujie Qiao, Yilun Zhao, Semih Yavuz, Ye Liu, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Dragomir Radev, Rex Ying, Arman Cohan
  EMNLP 2024 (Video presentation,) Top-10 most-cited paper in EMNLP 2024
• Learning to Reason via Mixture-of-Thoughts for Logical Reasoning
  Tong Zheng*, Lichang Chen*, Sophia S. Han, R. Thomas McCoy, and Heng Huang
• Towards Artificial Intelligence Research Assistant for Expert-Involved Learning
  Tianyu Liu*, Sophia S. Han*, Xiao Luo, Hanchen Wang, Pan Lu, Biqing Zhu, Yuge Wang, Keyi Li, Jiapeng Chen, Rihao Qu, Yufeng Liu, Xinyue Cui, Aviv Yaish, Yuhang Chen, Minsheng Hao, Chuhan Li, Kexing Li, Arman Cohan, Hua Xu, Mark Gerstein, James Zou, Hongyu Zhao
• Straight to the Gradient: Learning to Use Novel Tokens for Neural Text Generation
  Xiang Lin, Sophia S. Han, Shafiq Joty
  ICML’21 (as long talk ~3%)
• Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation
  Alexander Fabbri, Sophia S. Han, Haoyuan Li, Haoran Li, Marjan Ghazvininejad, Shafiq Joty, Dragomir Radev, Yashar Mehdad
  NAACL’21

Awards

• Meta Research Grant 2024 on Complex Reasoning.
• Best Final Year Thesis Gold Medal (2021.6).
• SM2 Scholarship, a full scholarship issued by Ministry of Education and Nanyang Technological University, Singapore
• National Physics Olympiad Second Prize, China.
• NUS Hackathon Top-8, Yale-NUS hack4climate Datathon 2nd Place in the Data Science Category, NTU Hackathon 2nd Prize.

Recent talks

Invited talk at AAAI 2026 Workshop: Bridge Program on Advancing LLM-Based Multi-Agent Collaboration, January 2026.
From First Principles to Real-World Applications: Advancing LLM Reasoning from Logic to Law. Invited talk at Zhejiang University, November 2025.
From First Principles to Real-World Applications: Advancing LLM Reasoning from Logic to Law. Invited talk at UCSB, October 2025.
From First Principles to Real-World Applications: Advancing LLM Reasoning from Logic to Law. Thesis Defense talk at Yale, September 2025. Slides. Video.
From First Principles to Real-World Applications: Advancing LLM Reasoning from Logic to Law. Invited talk at Rice University, September 2025.
From First Principles to Real-World Applications: Advancing LLM Reasoning from Logic to Law. Invited talk at Google Deepmind, August 2025.
Advancing Reasoning in Large Language Models: from Fundamentals to Real-World Applications. Invited talk at CMU, April 2025.
Advancing Reasoning in Large Language Models: from Fundamentals to Real-World Applications. Invited talk at UCSD, Nov 2024.
Advancing Reasoning in Large Language Models: from Fundamentals to Real-World Applications. Thesis prospectus talk at Yale, May 2024.

Teaching

• Teaching fellow: Topics in Natural Language Processing at Yale.
• Teaching fellow: AI Foundation Models at Yale.

Services and Organization

• Organizing Committee: DataMFM: Emerging Directions in Data for Multimodal Foundation Models at CVPR 2026.
• Organizing Committee: MATH-AI: The 5th Workshop on Mathematical Reasoning and AI at NeurIPS 2025.
• Organizing Committee: Knowledge-Intensive Multimodal Reasoning at ICCV 2025.
• Chair: Widening NLP.
• General Chair: New England NLP (NENLP) 2025.
• Session Chair: BoF session on Complex Reasoning with LLMs at NAACL 2025.
• Organizing Committee: Yale AI4Research Meeting Seminar.
• Ogannizing Committee: Automatic Summarization for Creative Writing Workshop at COLING 22
• Reviewer: ACL, EMNLP, NAACL, NeurIPS, ICLR, ICML, TACL.
• Member: Yale Women in School of Engineering & Applied Science.

Miscellaneous

• As a jack-of-all-trades in art, I had the privilege of performing in an ensemble at Yale’s historic Woolsey Hall. I also learned to sing from Marissa Katz and Cynthia Eggers.
• I have lived in Guangzhou, Shenzhen, Los Angeles, Singapore, New Haven, New York, Mountain View, San Diego, Palo Alto, Redwood City and Jersey City.
• Open Source Society Technical Director, Hackers for Charity Subcommittee.