I am Jing Gu, a Ph.D student at the University of California, Santa Cruz working with Prof. Xin (Eric) Wang. Previously I was a master student at the University of California, Davis working with Prof. Zhou Yu. I was in Google Research as an intern working on Embodied AI in 2022 summer. This summer, I am a research intern at Adobe.
News
[2023.11] I received a Scholar Award from NeurIPS 2023. Thanks for the NeurIPS for the travel support!
[2023.10] Our workshop AVLR (Advances in Language and Vision Research) 2024 is accepted to be held at ACL 2024!
[2023.10] Our paper R2H has been accepted by EMNLP 2023!
[2023.09] PHOTOSWAP is accepted to NeurIPS 2023!
[2023.06] Our SlugJARVIS team wins the Third Place ($50,000) in the inaugural Alexa Prize SimBot Challenge! [Media Coverage]
[2022.11] One paper accepted to AACL Finding!
[2022.10] Our Athena team received an Amazon Alexa Prize Award to work on Alexa Prize SocialBot Grand Challenge 5.
[2022.08] Our paper JARVIS: A Neuro-Symbolic Commonsense Reasoning Framework for Conversational Embodied Agents is on arXiv!
[2022.05] Our SlugJARVIS team won the Alexa Prize SimBot Public Benchmark Challenge! [Media Coverage]
[2022.04] Invited Talk in Chinese about Vision-and-Language Navigation in AI Drive! [Video in Chinese]
[2022.03] One paper about Vision-and-Language Navigation Survey accepted to ACL 2022!
[2022.03] Our SlugJARVIS team received an Amazon Alexa Prize Award to work on Alexa Prize SimBot Challenge. [link]
[2021.09] Started Ph.D. journey at UCSC!
[2021.05] One short paper accepted to ACL 2021!
[2021.04] One paper accepted to EACL 2021!
[2020.11] One paper accepted to AAAI 2021!
[2020.09] One paper accepted to EMNLP Finding!
[2020.10] One paper accepted to EMNLP WNUT workshop!
Research
My research interests are mainly Computer Vision, Natural Langauge Processing, and Embodied AI.
Publication & Manuscript
PHOTOSWAP: Personalized Subject Swapping in Images
Jing Gu, Yilin Wang, Nanxuan Zhao, Tsu-Jui Fu, Wei Xiong, Qing Liu, Zhifei Zhang, He Zhang, Jianming Zhang, HyunJoon Jung, Xin Eric Wang
NeurIPS 2023
[paper] [code] [website]R2H: Building Multimodal Navigation Helpers that Respond to Help
Yue Fan, Jing Gu, Kaizhi Zheng, Xin Eric Wang
EMNLP 2023
[paper]SlugJARVIS: Multimodal Commonsense Knowledge-based Embodied AI for SimBot Challenge
Jing Gu*, Kaizhi Zheng*, Kaiwen Zhou Yue Fan, Xuehai He Jialu Wang Zonglin Di, Xin Eric Wang
Alexa Prize SimBot Challenge Proceedings
[paper]JARVIS: A Neuro-Symbolic Commonsense Reasoning Framework for Conversational Embodied Agents
Kaizhi Zheng*, Jing Gu*, Kaiwen Zhou*, Yue Fan*, Jialu Wang*, Zonglin Di, Xuehai He, Xin Eric Wang
Preprint
SoCal NLP 2022
Winner Model of the Alexa Prize SimBot Public Benchmark Challenge [link]
[paper]Memformer: Memory-Augmented Transformer
Qingyang Wu, Zhenzhong Lan, Jing Gu, Zhou Yu
in Proc. of AACL 2022 Findings
[paper]Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions
Jing Gu*, Eliana Stefani, Qi Wu, Jesse Thomason, Xin Eric Wang
in Proc. of ACL 2022
[paper] [code]A Tailored Pre-Training Model for Task-Oriented Dialog Generation
Jing Gu*, Qingyang Wu*, Chongruo Wu, Weiyan Shi, Zhou Yu
in Proc. of ACL 2021
[paper] [code]Perception Score, a Learned Metric for Open-ended Text Generation Evaluation
Jing Gu, Qingyang Wu, Zhou Yu
in Proc. of AAAI 2021
[paper]Data Annealing for Informal Language Understanding Tasks
Jing Gu, Zhou Yu
in Proc. of EMNLP 2020 Findings
[paper]Flow-Aware Structual Model for Conversational Question Generation
Jing Gu, Mostafa Mirshekari, Zhou Yu, Aaron Sisto
in Proc. of EACL 2021 [paper] [code]ConQuest: Contextual Question Paraphrasing through Answer-Aware Synthetic Question Generation
Mostafa Mirshekari, Jing Gu, Aaron Sisto
EMNLP 2021 WNUT Workshop [paper]
Service
Program Committe Member (Reviewer)
- ACL 2021, Sigdial 2021, EMNLP 2021, NeurIPS 2021, ICLR 2022, ACL Rolling Review 2021&2022, NeurIPS 2022, EMNLP 2022, ACL 2023, ICML 2023, IEEE RA-L 23, EMNLP 2023, NeurIPS 2023
Honors & Awards
- Scholar Award from NeurIPS 2023
- Third place ($50,000) in the inaugural Alexa Prize SimBot Challenge! [Media Coverage]
- Winner of the Alexa Prize SimBot Public Benchmark Challenge, May 2022
- Team Leader of SlugJarvis, one of the ten teams selected for Alexa SimBot challenge, ongoing competition, 2021-2023
- Outstanding Graduates Award of Shanghai, June. 2018
- Excellent Graduation Thesis, June. 2018
- Excellent Student Cadre of Shanghai, Dec. 2017
Misc
- Teaching Assitant for Machine Learning, 2021
- Teaching Assitant for Natural Language Processing, 2019
- Teaching Assitant for Machine Dependent Programming, 2020
- Research Scientist in Searchable AI Corp., 2020
One last word
Keep Hungry!