About Me

Here is a virtual but warm greeting from Ziqiao, as you just came across this page! As many of my friends found it hard to pronounce my name (马子乔, pronounced as Tsz-Chaw in Mandarin), it's absolutely fine to just call me Martin alternatively.

Bio

Ziqiao (Martin) Ma is a 4th year Ph.D. candidate in Computer Science and Engineering advised by Professor Joyce Chai. His work has been supported in part by the Weinberg Cognitive Science Fellowship. He is also a part-time researcher at Adobe Research. Previously, he worked with Amazon AGI. Martin obtained dual-bachelor degrees from the University of Michigan and Shanghai Jiao Tong University. He received an Outstanding Paper Award at ACL 2023, an Amazon Alexa Prize Award. He teaches Natural Language Processing and won an Outstanding Guaduate Student Instructor Award. He co-organized SpLU-RoboNLP @ ACL 2024, and serve as a regular PC member/reviewer for various venues.

My Research

The three constant themes of my research are Language, Interaction, and Embodiment from a scalable and cognitive angle. My current research focus is two-fold:

Grounding and alignment: connecting language to everything non-linguistics.
  • I (mostly) agree with Freda Shi's view on the definition of grounding, and grounding != alignment. We wrote something on alignment, and I will find some time to put down my thoughts on the difference between alignment and grounding (TODO list +1)..
  • Grounding language to the physical world: Understanding and generating language that is grounded to sensorimotor experiences and physical situations. There are more to look at beyond 2D grounding, e.g., video, 3D, generative world models.
  • Grounding language to human interactions: (Co-)situated Human-AI interaction in shared environment with disparate mental states, and collaborations towards a common ground.
  • Alignment in post-training and at inference time: Human-like learning that is (inter)active, data-efficient, and continual on (semi-)structured data upon pre-trained systems.
  • Applications of frontier models in situated/embodied agents as well as content generation.
Scalable representations as computational abstractions of cognition.
  • I (mostly) agree with Alex Warstadt and Samuel Bowman's view on What Artificial Neural Networks Can Tell Us About Human Language Acquisition.
  • Scaling law and developmental psychology: Exploring the developmental trajectories of large-scale frontier models and the emergent cognitive capabilities over the course of development.
  • Benchmarking and computational inquiry of human-like cognition: Computational abstractions to inquire about potential mechanisms of how humans acquire cognitive skills, process language, as well as collaborate.
  • Cross-cultural and cross-lingual conventions of cognition. (Languages are dying! Under-represented languages are dear to my heart but I plan (try hard) not to do (too much) research on this topic before I finish my PhD lol)

    • I include a more dynamic and spontaneous sharing of my thoughts here.

    Quick Pointers

    CSE595 NLP / CV (Aug 2024) / Google Scholar / Semantic Scholar

    Updates

    News

    Archived news...
    • [May. 2023] I started my intern with Amazon Alexa AI (Amazon AGI)!!
    • [Sep. 2022] I will serve as the Poster/Demo Session Chair for Michigan AI Symposium 2022.
    • [Aug. 2022] I will be the Graduate Student Instructor for EECS 595 (NLP) in Fall 2022 at Michigan.
    • [Aug. 2021] I graduated from SJTU!
    • [May. 2021] I graduated from Michigan!
    • [Mar. 2021] I will join the family of the Michigan AI as a Ph.D. student this fall. Go Blue!
    • [Dec. 2020] I will be the Instructional Aide for EECS 492 (Intro. AI) in Winter 2021 at Michigan.
    • [May. 2020] I will be the Teaching Assistant for VE 492 (Intro. AI) and VE 280 (Program. & Intro. Data Struct.) Summer 2020 at SJTU.

    Paper Alerts

    Archived news...
    • [Apr. 2023] One paper to appear in IJCAI 2023, see you in Macau :)
    • [Oct. 2022] Two papers to appear in EMNLP 2022, and I will serve as an on-site volunteer in Abu Dhabi :)

    Seminar Talks

    Previous talks...
    • [20240712] Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations @ Deep Learning: Classics and Trends (DLCT). Host: Rosanne Liu
    • [20240705] Language Grounding to the Visual World and Human Interactions: How Far Are We from Embodied Dialogue Agents @ Data Science Group, KAIST.
    • [20240627] Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations @ CoCoDev Seminar. Host: Abdellah Fourtassi
    • [20240529] Language Grounding to the Visual World and Human Interactions: How Far Are We from Embodied Dialogue Agents @ University of Maryland. Host: Furong Huang

    Experiences

    Education

    Industry

    Teaching @ UMich

    Previous Teaching @ SJTU...
    • VE 492 Introduction to Artificial Intelligence, Shanghai Jiao Tong University. (Summer 2020)
    • VE 280 Programming and Data Structure, Shanghai Jiao Tong University. (Summer 2020)
    • VP 141 Physics Lab I, Shanghai Jiao Tong University. (Summer 2019)
    • VY 200 Academic Writing II, Shanghai Jiao Tong University. (Spring 2019)
    • VY 100 Academic Writing I, Shanghai Jiao Tong University. (Fall 2018)

    Guest Lectures

    Selected Awards/Recognitions

    Selected Fellowships/Scholarships

    Academic Services

    The 4th International Combined Workshop on Spatial Language Understanding and Grounded Communication for Robotics (SpLU-RoboNLP @ ACL 2024)

    Co-organizer

    [Homepage/CFP][Proceedings]

    The 5th Michigan AI Symposium: AI & Accessibility (2022)

    Poster/Demo Session Chair

    [Homepage/CFP]

    Publications

    Show by... ( Recent Selection / Cognitively Motivated Collection / Publication Year / All Research Topics )

    Research Topics: Language Grounding to Visual Perception / Language Grounding to Human Interaction / Language-Guided Multimodal Generation / Embodied/Situated Language Intelligence / Alignment and (Inter)active Learning / Teaching & Community Services

    * indicates equal contributions; § indicates correspondence and mentoring.

    Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference under Ambiguities
    Zheyuan Zhang*, Fengyuan Hu*, Jayjun Lee*, Freda Shi, Parisa Kordjamshidi, Joyce Chai, Ziqiao Ma§

    Preprint 2024

    Paper

    Towards Bidirectional Human-AI Alignment: A Systematic Review for Clarifications, Framework, and Future Directions
    Hua Shen, Tiffany Knearem, Reshmi Ghosh, Kenan Alkiek, Kundan Krishna, Yachuan Liu, Ziqiao Ma, Savvas Petridis, Yi-Hao Peng, Li Qiwei, Sushrita Rakshit, Chenglei Si, Yutong Xie, Jeffrey P. Bigham, Frank Bentley, Joyce Chai, Zachary Lipton, Qiaozhu Mei, Rada Mihalcea, Michael Terry, Diyi Yang, Meredith Ringel Morris, Paul Resnick, David Jurgens

    Preprint 2024

    Paper

    Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models
    Yue Zhang*, Ziqiao Ma*, Jialu Li*, Yanyuan Qiao*, Zun Wang*, Joyce Chai, Qi Wu, Mohit Bansal, Parisa Kordjamshidi

    Preprint 2024

    Paper

    Multi-Object Hallucination in Vision-Language Models
    Xuweiyi Chen*, Ziqiao Ma*, Xuejun Zhang*, Sihan Xu, Shengyi Qian, Jianing Yang, David Fouhey, Joyce Chai

    NeurIPS 2024 / The 3rd Workshop on Advances in Language and Vision Research (ALVR) @ ACL 2024

    Web / Paper

    Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations
    Ziqiao Ma*, Zekun Wang*, Joyce Chai

    The 1st Workshop on LLMs and Cognition (LLMCog) @ ICML 2024 (Oral)

    Paper / Code

    DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiences
    Yidong Huang, Jacob Sansom, Ziqiao Ma§, Felix Gervits, Joyce Chai

    IROS 2024 / The 1st Vision and Language for Autonomous Driving and Robotics Workshop (VLADR) @ CVPR 2024

    Web / Paper

    GroundHog: Grounding Large Language Models to Holistic Segmentation
    Yichi Zhang, Ziqiao Ma, Xiaofeng Gao, Suhaila Shakiah, Qiaozi Gao, Joyce Chai

    CVPR 2024

    Web / Paper

    Inversion-Free Image Editing with Language-Guided Diffusion Models
    Sihan Xu*, Yidong Huang*, Jiayi Pan, Ziqiao Ma§, Joyce Chai

    CVPR 2024

    Web / Paper / GitHub /

    CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulation
    Sihan Xu*, Ziqiao Ma*, Yidong Huang, Honglak Lee, Joyce Chai

    NeurIPS 2023

    Web / Paper / GitHub /

    Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models
    Ziqiao Ma, Jacob Sansom, Run Peng, Joyce Chai

    EMNLP 2023 (Findings)

    Paper / GitHub

    Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialogue
    Cristian-Paul Bara, Ziqiao Ma, Yingzhuo Yu, Julie Shah, Joyce Chai

    IJCAI 2023

    Paper / GitHub

    World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models
    Ziqiao Ma*, Jiayi Pan*, Joyce Chai

    ACL 2023 (🏆 Outstanding Paper Award)

    Paper / GitHub / Poster

    NLP Reproducibility For All: Understanding Experiences of Beginners
    Shane Storks, Keunwoo Peter Yu, Ziqiao Ma, Joyce Chai

    ACL 2023 (Theme Track)

    Paper / GitHub / Poster

    SEAGULL: An Embodied Agent for Instruction Following through Situated Dialog
    Yichi Zhang, Jianing Yang, Keunwoo Peter Yu, Yinpei Dai, Shane Storks Yuwei Bao, Jiayi Pan, Nikhil Devraj, Ziqiao Ma, Joyce Chai

    Alexa Prize SimBot Challenge 2023 (🏆 1st Place)

    Paper

    DOROTHIE: Spoken Dialogue for Handling Unexpected Situations in Interactive Autonomous Driving Agents
    Ziqiao Ma, Benjamin VanDerPloeg*, Cristian-Paul Bara*, Huang Yidong*, Eui-In Kim, Felix Gervits, Matthew Marge, Joyce Chai

    EMNLP 2022 (Findings)

    Paper / GitHub / Data

    DANLI: Deliberative Agent for Following Natural Language Instructions
    Yichi Zhang, Jianing Yang, Shane Storks, Jiayi Pan, Nikhil Devraj, Ziqiao Ma, Keunwoo Peter Yu, Yuwei Bao, Joyce Chai

    EMNLP 2022 (Oral)

    Paper / GitHub

    Partition-Based Active Learning for Graph Neural Networks
    Jiaqi Ma*, Ziqiao Ma*, Joyce Chai, Qiaozhu Mei

    TMLR 2023 (Survey Certificate) / The 8th International Workshop on Deep Learning on Graphs (DLG) @ KDD 2022 (Oral)

    Paper / GitHub / Poster

    Misc

    Game Design (More)

    Mentoring

    I understand that access to research oppotunities can be hard, particularly for beginners and the underrepresented. If there is a match in research interests, I am happy to collaborate with undergrads and masters when I have the bandwidth. Please check my Research and Mentoring Statements and fill out our application form to indicate that you want to collaborate with me.
    I've been fortunate to have (co-)mentered and collaborated with these amazingly talented young researchers:

    Fun Facts

    Random Tours

    Mental Support

    Chat?

    If you would like to have a random (virtual) coffee chat with me, please visit my calendly page.

    Get In Touch

    You are welcome to drop me a message :)

    • Phone

      xxx-xxx-xxxx

    • marstin0607

    • ziqiao_ma

    • Address

      Bob and Betty Beyster Building 4909,
      2260 Hayward Street,
      Ann Arbor, MI 48109.