A video–text retrieval model for videos containing multiple events.
Gengyuan Zhang
AI researcher,
Based in Europe (Germany/UK)
Vision, Intelligence, Human & Machine
Building, Evaluating, Leveragingvisual intelligence
Three horizontally overlapping ink circles represent research directions. Blue: Benchmarking visual intelligence; orange: Domain applications; yellow: Machines can see. Follow a direction to explore its projects.
Tap a label to highlight; tap again to clear. Open highlighted link ↗
News
- Oct 2026
- Sep 2026
- Sep 2026
- Sep 2026
ReEXplore at the Embodied Multimodal Reasoning Workshop, ECCV 2026.
- Mar 2026
I visited the Torr Vision Group at the University of Oxford from 16 Mar to 31 Jul 2026.
- Dec 2025
Visual Intelligence
Multimodal understanding, evaluation, and systems
Machines
can see
Models, Agents, Systems
A streaming vision-language agent that handles queries and visual evidence arriving at different times.
Paper & authors
AViLA: Asynchronous Vision-Language Agent for Streaming Multimodal Data Interaction
G. Zhang*, T. Hannan*, H. Kleiner, B. Aydemir, X. Xie, J. Lan, T. Seidl, V. Tresp, and J. Gu
Benchmarking
visual intelligence
Multi-image re-representation: visual tools, a grounding-focused benchmark, and a trained agent.
Paper & authors
Rethinking Multi-Image Re-Representation in Multi-Image Understanding
G. Zhang, X. Han, X. Xie, T. Liu, and V. Tresp
Localizing video events with a reference image and a refinement text.
Paper & authors
Localizing Events in Videos with Multimodal Queries
G. Zhang*, M. L. A. Fok*, J. Ma, Y. Xia, D. Cremers, P. Torr, V. Tresp, and J. Gu
Domain
applications
Retrospective experience replay and hierarchical frontier selection for embodied exploration.
Paper & authors
G. Zhang, M. Ding, J. Wu, R. Liao, and V. Tresp
A surgical understanding benchmark and an environment for evaluating tool-using generalist agents.
Paper & authors
How to Train a Surgeon? Benchmarking Generalist Agents in Surgical Scene Understanding
G. Zhang, X. Han, S. Zhou, A. Koukash, G. Ghazaei, J. Gu, J. Wu, Y. Zhu, N. Navab, and V. Tresp
Publications & preprints, 19
* Equal contribution.
How to Train a Surgeon? Benchmarking Generalist Agents in Surgical Scene Understanding
G. Zhang, X. Han, S. Zhou, A. Koukash, G. Ghazaei, J. Gu, J. Wu, Y. Zhu, N. Navab, and V. Tresp
Rethinking Multi-Image Re-Representation in Multi-Image Understanding
G. Zhang, X. Han, X. Xie, T. Liu, and V. Tresp
ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replay
G. Zhang, M. Ding, J. Wu, R. Liao, and V. Tresp
Conformalized Large Language Models under Configuration Shift
Y. Zhu, J. Yu, L. Li, G. Zhang, Z. Yang, S. Staab, P. Dokania, P. Torr, J. Tang, and E. Kharlamov
Approximate or Perish: Spectral MLP-KAN Diffusion with Attentive Function Learning for Unsupervised Hyperspectral Image Restoration
H. Jiang, J. Ma, G. Du, J. Sun, G. Zhang, Z. Zhang, and K. Luo
AViLA: Asynchronous Vision-Language Agent for Streaming Multimodal Data Interaction
G. Zhang*, T. Hannan*, H. Kleiner, B. Aydemir, X. Xie, J. Lan, T. Seidl, V. Tresp, and J. Gu
Localizing Events in Videos with Multimodal Queries
G. Zhang*, M. L. A. Fok*, J. Ma, Y. Xia, D. Cremers, P. Torr, V. Tresp, and J. Gu
Perceive, Query & Reason: Enhancing Video QA with Question-Guided Temporal Queries
R. Amoroso*, G. Zhang*, R. Koner, L. Baraldi, R. Cucchiara, and V. Tresp
Memory Helps, but Confabulation Misleads: Understanding Streaming Events in Videos with MLLMs
G. Zhang*, M. Ding*, T. Liu, Y. Zhang, and V. Tresp
Multimodal Pragmatic Jailbreak on Text-to-image Models
T. Liu, Z. Lai, J. Wang, G. Zhang, S. Chen, P. Torr, V. Demberg, V. Tresp, and J. Gu
Unveiling the "Fairness Seesaw": Discovering and Mitigating Gender and Race Bias in Vision-Language Models
J. Lan, U. Schlegel, T. Hannan, G. Zhang, H. Chen, and T. Seidl
FedBiP: Heterogeneous One-Shot Federated Learning with Personalized Latent Diffusion Models
H. Chen, H. Li, Y. Zhang, J. Bi, G. Zhang, Y. Zhang, P. Torr, J. Gu, D. Krompass, and V. Tresp
CL-Cross VQA: A Continual Learning Benchmark for Cross-Domain Visual Question Answering
Y. Zhang, H. Chen, A. Frikha, D. Krompass, G. Zhang, J. Gu, and V. Tresp
Can Vision-Language Models Be a Good Guesser? Exploring VLMs for Times and Location Reasoning
G. Zhang, Y. Zhang, K. Zhang, and V. Tresp
VideoINSTA: Zero-shot Long Video Understanding via Informative Spatial-Temporal Reasoning with LLMs
R. Liao, M. Erler, H. Wang, G. Zhai, G. Zhang, Y. Ma, and V. Tresp
SPOT! Revisiting Video-Language Models for Event Understanding
G. Zhang*, J. Bi*, J. Gu, Y. Chen, and V. Tresp
Multi-Event Video-Text Retrieval
G. Zhang, J. Ren, J. Gu, and V. Tresp
A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models
J. Gu, Z. Han, S. Chen, A. Beirami, B. He, G. Zhang, R. Liao, Y. Qin, V. Tresp, and P. Torr
Time-dependent Entity Embedding is not All You Need: A Re-evaluation of Temporal Knowledge Graph Completion Models under a Unified Framework
Z. Han*, G. Zhang*, Y. Ma, and V. Tresp
Artifacts
Interactive experiments, research tools, datasets, and papers.
Print Labs
Four interactive labs for print texture, layout, paper objects, and element composition.
Mosaic
Rethinking Multi-Image Re-Representation in Multi-Image Understanding.
ICQ-Highlight
A benchmark, dataset, and code for localizing events with multimodal queries.
WikiTiLo
An image dataset for probing time and location reasoning in vision-language models.
MeVTR
Code and evaluation resources for Multi-Event Video-Text Retrieval.
About me
I am a PhD researcher at LMU Munich, supervised by Prof. Volker Tresp. My thesis is titled “Promises and Limits of Language-centered Visual Understanding”.
Experience
Research Assistant
LMU Munich
Multimodal learning, visual/video understanding, and multimodal agents.
Visiting PhD Student
University of Oxford, Torr Vision Group
Agentic environments and memory for decision-making under uncertainty.
Applied Scientist Intern
Amazon, London
Omni-modal audio–video retrieval and retrieval robustness.
Intern
Agile Robots AG, Munich
Robotic object localization, grasping, and real-time perception.
Student Assistant
Technical University of Munich
Perception for the RobMoSys robotic platform.
Education
PhD in Computer Science
LMU Munich
Expected completion: Dec 2026.
MSc in Electrical Engineering and Information Technology
Technical University of Munich
BEng in Information Science and Engineering
Zhejiang University
Funding
Towards an Intelligent Agent for Streaming Surgical Video Understanding
Reviewing
CVPR 2024–2026; ECCV 2024–2026; NeurIPS 2024–2026; COLING 2024; ARR 2025–2026.
Outside research
Outside research: plants, travelling, cooking, Crusader Kings III, and my dachshund.