Ziqiao "Martin" Ma

Here is a virtual but warm greeting from Ziqiao, as you just came across this page! As many of my friends found it hard to pronounce my name (马子乔, pronounced as Tsz-Chaw in Mandarin), it's absolutely fine to just call me Martin alternatively.

I am a 3rd year Ph.D. candidate in Computer Science and Engineering at the University of Michigan. I work with Professor Joyce Chai as a member of the Situated Language and Embodied Dialogue (SLED) Lab. I intern(ed) with Adobe Research and Amazon Alexa AI. Prior to that, I obtained my Dual-Bachelor Degree in Computer Science at the University of Michigan and Electrical and Computer Engineering at Shanghai Jiao Tong University.

The three constant themes of my research are Language, Interaction, and Embodiment. My current work focuses on connecting large language models to the world and to humans, as well as computational inquiry of human cogntiion. More generally, I am interested in aspects of Natural Language Processing (Language Grounding to Vision and Robotics, Theory of Mind, Language Acquisition), Embodied AI (World Model, Embodied Instruction Following, Human-Robot Communication), and Machine Learning (Cognitively Motivated Learning, Graph Representation Learning, Data-Efficient Learning). I am honored to have received an Outstanding Paper Award at ACL 2023, an Amazon Alexa Prize Award, and an Outstanding GSI Award for teaching. I am a co-organizer of SpLU-RoboNLP, and serve as a regular PC member/reviewer for NLP venues (e.g., ACL, EMNLP, NAACL, EACL, COLING) and ML venues (e.g., ICLR, NeurIPS, TMLR).

Updates

⚠ ⚠ ⚠

• SLED members Run Peng (RL for NLP and Robotics), Xuweiyi Chen (3D Grounding), Jacob Sansom (Cognitive Modeling), Zekun Wang (Cognitive Modeling), Xijia Zhang (Human Robot Interaction) and Sikai Li (Human Robot Interaction) are looking for grad school opportunities. They are extremely talented, self-motivated, and pleasant to work with. Please consider them if you have openings!

News

• [Oct. 2023] The 4th SpLU-RoboNLP Workshop will be with ACL 2024. Submit your best work and join us in Bangkok :)
• [Oct. 2023] One paper to appear in EMNLP 2023, see you in Singapore :)
• [Sep. 2023] One paper to appear in NeurIPS 2023, see you in New Orleans :)
• [Jul. 2023] Our paper "World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models" was selected for the Outstanding Paper Award at ACL 2023!!
• [Jun. 2023] We won the 1st place in the Alexa Prize SimBot Challenge!!
• [May. 2023] Two papers to appear in ACL 2023, and I will serve as an on-site volunteer in Toronto :)
• [Apr. 2023] One paper to appear in IJCAI 2023, see you in Macau :)

Experiences

Education

• University of Michigan, Ann Arbor | Sept 2021 - May 2026 (Expected)
  Ph.D. in Computer Science and Engineering, Certificate in Cognitive Science
  Advisor: Joyce Chai
• University of Michigan, Ann Arbor | Sept 2019 - May 2021
  B.S.E in Computer Science, Minor in Mathematics (Summa Cum Laude)
  Advisors: Joyce Chai, Qiaozhu Mei, Jie Liu
• Shanghai Jiao Tong University, Shanghai | Sept 2017 - Aug 2021
  B.S.E in Electrical and Computer Engineering
  
• Technische Universität Berlin, Berlin | Jan 2019 - Feb 2019
  Visiting Student
  
• Chengdu Foreign Languages School, Chengdu | Sept 2011 - June 2017
  High School

Industry

• Adobe, San Jose | Jan 2024 - Apr 2024
  Research Scientist Intern
  
• Amazon, Sunnyvale | May 2023 - Aug 2023
  Applied Scientist Intern
  

Teaching

• EECS 595 Natural Language Processing, University of Michigan.
• EECS 492 Introduction to Artificial Intelligence, University of Michigan.
• VE 492 Introduction to Artificial Intelligence, Shanghai Jiao Tong University.
• VE 280 Programming and Data Structure, Shanghai Jiao Tong University.

Selected Awards

• Outstanding Paper Award, ACL 2023.
• First Place, Amazon Alexa Prize, 1st SimBot Challenge, 2023.
• Outstanding GSI Award, University of Michigan, 2022.
• James B. Angell Scholar, University of Michigan, 2021.

Selected Fellowships

• Rackham Doctoral Intern Fellowship, University of Michigan, 2024.
• China National Scholarship, 2018.
• John Wu & Jane Sun Excellence Scholarship, Shanghai Jiao Tong University, 2017.

Academic Services

• Co-organizer, the 4th SpLU-RoboNLP Workshop, ACL 2024.
• Poster/Demo Session Chair, Michigan AI Symposium, 2022.
• Program Committee: ACL 2023, EMNLP 2023.
• Reviewer: ARR (since 2023), ACL (since 2023), EMNLP (since 2022), COLING (since 2022), TMLR (since 2023).

• Personal

Publications

* indicates equal contributions; § indicates mentoring.

Language Grounding to Human Interactions (esp. Theory of Mind)

Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models

Ziqiao Ma, Jacob Sansom, Run Peng, Joyce Chai

EMNLP Findings (Theme Track), 2023

[web] [paper] [data]

Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialogue

Cristian-Paul Bara, Ziqiao Ma, Yingzhuo Yu, Julie Shah, Joyce Chai

IJCAI, 2023

[paper] [code]

Language Grounding to Vision (esp. Multimodal Learning)

Inversion-Free Image Editing with Natural Language

Sihan Xu*, Yidong Huang*, Jiayi Pan, Ziqiao Ma§, Joyce Chai

Preprint, 2023

[web] [paper] [code]

CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulation

Sihan Xu*, Ziqiao Ma*, Yidong Huang, Honglak Lee, Joyce Chai

NeurIPS, 2023

[web] [paper] [code]

World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models

Ziqiao Ma*, Jiayi Pan*, Joyce Chai

ACL (Outstanding Paper Award), 2023

[paper] [code] [model] [video] [slides] [poster]

Language Grounding to Robotics (esp. Embodied AI)

SEAGULL: An Embodied Agent for Instruction Following through Situated Dialog

Yichi Zhang, Jianing Yang, Keunwoo Peter Yu, Yinpei Dai, Shane Storks Yuwei Bao, Jiayi Pan, Nikhil Devraj, Ziqiao Ma, Joyce Chai

Alexa Prize SimBot Challenge (1st Place), 2023

[paper]

DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents

Ziqiao Ma, Benjamin VanDerPloeg*, Cristian-Paul Bara*, Huang Yidong*, Eui-In Kim, Felix Gervits, Matthew Marge, Joyce Chai

EMNLP Findings, 2022
Presented at SereTOD Workshop, 2022

[paper] [code] [data] [video] [slides] [poster]

DANLI: Deliberative Agent for Following Natural Language Instructions

Yichi Zhang, Jianing Yang, Shane Storks, Jiayi Pan, Nikhil Devraj, Ziqiao Ma, Keunwoo Peter Yu, Yuwei Bao, Joyce Chai

EMNLP (Oral), 2022

[paper] [code] [video] [slides]

Data-Efficient Learning on Semi-Structured Data

Partition-Based Active Learning for Graph Neural Networks

Jiaqi Ma*, Ziqiao Ma*, Joyce Chai, Qiaozhu Mei

TMLR (Survey Certificate), 2023
Presented at DLG-KDD Workshop (Oral), 2022

[paper] [code] [poster]

AI for Social Good and Community Services

NLP Reproducibility For All: Understanding Experiences of Beginners

Shane Storks, Keunwoo Peter Yu, Ziqiao Ma, Joyce Chai

ACL (Theme Track), 2023

[paper] [code] [video] [slides] [poster]

Non-Academic: Game Design

Contracts

[trailer][itch.io][indidb][tigsource]

• A turn-based tactic video game.

Floating

[play online][download]

• A physics-based puzzle video game.

• More

Misc

Mentoring

• Jiayi Pan, B.S. @ UMICH. Now Ph.D. @ UCB.

• Connecting language to the world: Understanding and generating situated language that is grounded to sensorimotor experiences and the physical world.
• Connecting language to humans: Cognitively motivated human-agent interaction and computational inquiry into human language acquisition/processing.
• Human-like Learning: Machine learning that is (inter)active, data-efficient and continual on (semi-)structured data.

Fun Facts

• I was born and raised up in Chengdu, the home of pandas.
• I am proud of Chengdu Foreign Languages School, my high school, and identify myself as a CFLSer. 成外人永远不会成外人.
• I am INFJ according to the Myers-Briggs, and my friends said that I exhibit stereotypical traits of this personality type...lol
• I love literature and drama. I am particularly interested in Shakespeare's plays, ancient Chinese drama, Latin American literature, and modern Asian literature. Here is one of my essays about Cross-dressed Heroines in Comedies.
• I am a super fan of Nightwish. My favourite album is Imaginaerum.

Friends

• Jimmy Zhang SDE @ Microsoft, Suzhou, China.
• Sway Ye Ph.D. @ SJTU, Shanghai, China.
• Joy Dong Ph.D. @ UMICH, Ann Arbor, US.
• Haoyang Zhang Ph.D. @ UIUC, Champaign, US.
• ...(if you want to get linked just message me 😏)

Mental Support

• Counseling and Support Services @ SJTU
• Resources for Stress and Mental Health @ Michigan.

Schedule

If you would like to have a random chat with me, please visit my calendly page.

Get In Touch

You are welcome to drop me a message :)