Contact
San Francisco, CA
Email: noahrshinn[at]gmail[dot]com
Github
Google scholar
About
I am a programmer and researcher. Previously, I was a researcher at Sierra, where I joined as one of the first employees.
Before that, I worked on type prediction and code generation in the Programming Language Group at Northeastern University, and excited-state molecular dynamics in the Computational Photochemistry Group.
I was also a member of the avionics team in the NU Aerospace group.
Papers
-
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Shunyu Yao, Noah Shinn, Pedram Razavi, Karthik Narasimhan
International Conference on Learning Representations (ICLR) 2025
paper |
code
-
Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions
Federico Cassano, Luisa Li, Akul Sethi, Noah Shinn, Abby Brennan-Jones, Edward Berman, George Chakhnashvili, Anton Lozhkov, Carolyn Anderson, Arjun Guha
Conference on Language Modeling (COLM) 2024
paper |
code
-
Type Prediction With Program Decomposition and Fill-in-the-Type Training
Federico Cassano, Ming-Ho Yee, Noah Shinn, Arjun Guha, Steven Holtzen
paper |
code
-
Reflexion: Language Agents with Verbal Reinforcement Learning
Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, Shunyu Yao
Neural Information Processing Systems (NeurIPS) 2023
paper |
code
Other
- web-browser: A tool that automates web search, traversal, and extraction of information on unstructured web pages
- public-web: A prototype of a public network of APIs
- gitm: A search tool for git and Github
- The Collaborative Browser: a web browser for all users
- llm-chat: Question answering in your terminal
- gcl: Tool for git commits and GitHub issues
- bashllm: Tool for auto cli suggestions
- A blog post written with Ashwin Gopinath
- kstateagent: Classification + q-learning written in OCaml and Rust
- erica: A project from a hackathon written in Rust and Python
- Leetcode Hard Gym: A hard benchmark for programming tests
τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains
Shunyu Yao, Noah Shinn, Pedram Razavi, Karthik Narasimhan
International Conference on Learning Representations (ICLR) 2025
paper |
code
Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions
Federico Cassano, Luisa Li, Akul Sethi, Noah Shinn, Abby Brennan-Jones, Edward Berman, George Chakhnashvili, Anton Lozhkov, Carolyn Anderson, Arjun Guha
Conference on Language Modeling (COLM) 2024
paper |
code
Type Prediction With Program Decomposition and Fill-in-the-Type Training
Federico Cassano, Ming-Ho Yee, Noah Shinn, Arjun Guha, Steven Holtzen
paper |
code
Reflexion: Language Agents with Verbal Reinforcement Learning
Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, Shunyu Yao
Neural Information Processing Systems (NeurIPS) 2023
paper |
code