I am Jing Gu, a Ph.D student at the University of California, Santa Cruz working with Prof. Xin (Eric) Wang. Previously I was a master student at the University of California, Davis working with Prof. Zhou Yu. I was in Google Research as an intern working on Embodied AI in 2022 summer. This summer, I am a research intern at Adobe.

News

[2023.11] I received a Scholar Award from NeurIPS 2023. Thanks for the NeurIPS for the travel support!
[2023.10] Our workshop AVLR (Advances in Language and Vision Research) 2024 is accepted to be held at ACL 2024!
[2023.10] Our paper R2H has been accepted by EMNLP 2023!
[2023.09] PHOTOSWAP is accepted to NeurIPS 2023!
[2023.06] Our SlugJARVIS team wins the Third Place ($50,000) in the inaugural Alexa Prize SimBot Challenge! [Media Coverage]
[2022.11] One paper accepted to AACL Finding!
[2022.10] Our Athena team received an Amazon Alexa Prize Award to work on Alexa Prize SocialBot Grand Challenge 5.
[2022.08] Our paper JARVIS: A Neuro-Symbolic Commonsense Reasoning Framework for Conversational Embodied Agents is on arXiv!
[2022.05] Our SlugJARVIS team won the Alexa Prize SimBot Public Benchmark Challenge! [Media Coverage]
[2022.04] Invited Talk in Chinese about Vision-and-Language Navigation in AI Drive! [Video in Chinese]
[2022.03] One paper about Vision-and-Language Navigation Survey accepted to ACL 2022!
[2022.03] Our SlugJARVIS team received an Amazon Alexa Prize Award to work on Alexa Prize SimBot Challenge. [link]
[2021.09] Started Ph.D. journey at UCSC!
[2021.05] One short paper accepted to ACL 2021!
[2021.04] One paper accepted to EACL 2021!
[2020.11] One paper accepted to AAAI 2021!
[2020.09] One paper accepted to EMNLP Finding!
[2020.10] One paper accepted to EMNLP WNUT workshop!

Research

My research interests are mainly Computer Vision, Natural Langauge Processing, and Embodied AI.

Publication & Manuscript

• PHOTOSWAP: Personalized Subject Swapping in Images
  Jing Gu, Yilin Wang, Nanxuan Zhao, Tsu-Jui Fu, Wei Xiong, Qing Liu, Zhifei Zhang, He Zhang, Jianming Zhang, HyunJoon Jung, Xin Eric Wang
  NeurIPS 2023
  [paper] [code] [website]

• R2H: Building Multimodal Navigation Helpers that Respond to Help
  Yue Fan, Jing Gu, Kaizhi Zheng, Xin Eric Wang
  EMNLP 2023
  [paper]

• SlugJARVIS: Multimodal Commonsense Knowledge-based Embodied AI for SimBot Challenge
  Jing Gu*, Kaizhi Zheng*, Kaiwen Zhou Yue Fan, Xuehai He Jialu Wang Zonglin Di, Xin Eric Wang
  Alexa Prize SimBot Challenge Proceedings
  [paper]

• JARVIS: A Neuro-Symbolic Commonsense Reasoning Framework for Conversational Embodied Agents
  Kaizhi Zheng*, Jing Gu*, Kaiwen Zhou*, Yue Fan*, Jialu Wang*, Zonglin Di, Xuehai He, Xin Eric Wang
  Preprint
  SoCal NLP 2022
  Winner Model of the Alexa Prize SimBot Public Benchmark Challenge [link]
  [paper]

• Memformer: Memory-Augmented Transformer
  Qingyang Wu, Zhenzhong Lan, Jing Gu, Zhou Yu
  in Proc. of AACL 2022 Findings
  [paper]

• Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
  Jing Gu*, Eliana Stefani, Qi Wu, Jesse Thomason, Xin Eric Wang
  in Proc. of ACL 2022
  [paper] [code]

• A Tailored Pre-Training Model for Task-Oriented Dialog Generation
  Jing Gu*, Qingyang Wu*, Chongruo Wu, Weiyan Shi, Zhou Yu
  in Proc. of ACL 2021
  [paper] [code]

• Perception Score, a Learned Metric for Open-ended Text Generation Evaluation
  Jing Gu, Qingyang Wu, Zhou Yu
  in Proc. of AAAI 2021
  [paper]

• Data Annealing for Informal Language Understanding Tasks
  Jing Gu, Zhou Yu
  in Proc. of EMNLP 2020 Findings
  [paper]

• Flow-Aware Structual Model for Conversational Question Generation
  Jing Gu, Mostafa Mirshekari, Zhou Yu, Aaron Sisto
  in Proc. of EACL 2021 [paper] [code]

• ConQuest: Contextual Question Paraphrasing through Answer-Aware Synthetic Question Generation
  Mostafa Mirshekari, Jing Gu, Aaron Sisto
  EMNLP 2021 WNUT Workshop [paper]

Service

Program Committe Member (Reviewer)

• ACL 2021, Sigdial 2021, EMNLP 2021, NeurIPS 2021, ICLR 2022, ACL Rolling Review 2021&2022, NeurIPS 2022, EMNLP 2022, ACL 2023, ICML 2023, IEEE RA-L 23, EMNLP 2023, NeurIPS 2023

Honors & Awards

• Scholar Award from NeurIPS 2023
• Third place ($50,000) in the inaugural Alexa Prize SimBot Challenge! [Media Coverage]
• Winner of the Alexa Prize SimBot Public Benchmark Challenge, May 2022
• Team Leader of SlugJarvis, one of the ten teams selected for Alexa SimBot challenge, ongoing competition, 2021-2023
• Outstanding Graduates Award of Shanghai, June. 2018
• Excellent Graduation Thesis, June. 2018
• Excellent Student Cadre of Shanghai, Dec. 2017

Misc

• Teaching Assitant for Machine Learning, 2021
• Teaching Assitant for Natural Language Processing, 2019
• Teaching Assitant for Machine Dependent Programming, 2020
• Research Scientist in Searchable AI Corp., 2020

One last word

Keep Hungry!