Zijian Chen

Ph.D Student @sjtu1896. Working on #Visual quality assessment, #AI evaluation, and #AI4Humanity (Oracle bone characters). Opinions are my own.

[Zijian Chen @_ZijianChen]

Understanding a scientific figure is one thing. Writing code to reproduce it is another.

We introduce FigCodeBench to evaluate multimodal LLMs on figure-to-code reproduction.

Project: https://t.co/Izj8Vid5qD

#mllm #benchmark #ai #figcodebench #coding

A thread (1/4)

[Zijian Chen @_ZijianChen]

@ZipMind_AI Thanks for the insightful comment! We haven't tried adding grids or tick marks to the non-grid images yet, but it's a great idea. We plan to investigate this in future work to quantify how much of the gap such visual cues alone can bridge.

[Zijian Chen @_ZijianChen]

Understanding a scientific figure is one thing. Writing code to reproduce it is another.

We introduce FigCodeBench to evaluate multimodal LLMs on figure-to-code reproduction.

Project: https://t.co/Izj8Vid5qD

#mllm #benchmark #ai #figcodebench #coding

A thread (1/4)

[Zijian Chen @_ZijianChen]

@ZipMind_AI 4. Further case study reveals that although current MLLMs have largely mastered basic data-to-geometry mapping, they still exhibit substantial weaknesses in fine-grained aesthetic rendering.

[Zijian Chen @_ZijianChen]

@ZipMind_AI 3. Meanwhile, we find a significant divergence in the coding strategies adopted by different MLLMs, highlighting that few models can strike a perfect balance between precise visual comprehension and reliable code generation.

[Zijian Chen @_ZijianChen]

@ZipMind_AI 2. We observe a significant performance gap across different programming languages, where rendering via descriptive LaTeX constitutes a formidable bottleneck for all evaluated models.

[Zijian Chen @_ZijianChen]

FigCodeBench examines the perception and reasoning bottlenecks between seeing a figure and reproducing it through code.

Explore the benchmark and results:
https://t.co/2CqXZVz2xn

Dataset:
https://t.co/r8xdxEhHDp

[Zijian Chen @_ZijianChen]

Does the output look right? Does the code faithfully represent the figure?

We evaluate 24 multimodal LLMs using image- and code-oriented metrics, alongside the Mean Machine Opinion Score (MMOS) and Figure-Code Fidelity (FCF).

A thread (3/4)

[Zijian Chen @_ZijianChen]

@GoogleDeepMind cool

[Zijian Chen @_ZijianChen]

Our research, "Oracle Bone Inscriptions Information Processing: A Comprehensive Survey", is available on @ResearchGate: https://t.co/RyNqFjIKS0

[Zijian Chen @_ZijianChen]

Watch the performance of GPT-5.

[Zijian Chen @_ZijianChen]

#GPT5 #Benchmark #biological_motion #LLMs
We propose BioMotion Arena, a biological motion-based visual preference evaluation framework for large models.

Project: https://t.co/jiCCjDwEg7
Paper: https://t.co/Vs93cFjCsI

[Zijian Chen @_ZijianChen]

Current benchmarks adopt either ground-truth-based score-form evaluation on static datasets or indistinct textual chatbot-style human preferences collection, which may not provide users with immediate, intuitive, and perceptible feedback on performance differences.

[Zijian Chen @_ZijianChen]

@sama We propose BioMotion Arena, a biological motion-based visual preference evaluation framework for large models.
GPT-5 (newly added) leads the board, with reasoning-focused closed-source models close behind.
Project: https://t.co/jiCCjDwEg7
Paper: https://t.co/Vs93cFjCsI

[Zijian Chen @_ZijianChen]

• Across the board, today’s models show little grasp of biological motion priors, falling far short of smooth, plausible limb dynamics

[Zijian Chen @_ZijianChen]

Key results
• GPT-5 (newly added) leads the board, with reasoning-focused closed-source models close behind.
• Open-source generalists lag dramatically; code-specialized models often emit random, discontinuous dot clouds that viewers unanimously reject as “non-human.”

[Zijian Chen @_ZijianChen]

1st JND versus the parameter ratio of the language backbone to the visual backbone.

We observe that LMMs with stronger detail perception capabilities generally have a lower Param_L/Param_V.

Dataset and code are coming soon!

A thread (6/n)

[Zijian Chen @_ZijianChen]

#JND #MLLM #perception What is the minimal magnitude of changes that LMMs can perceive?

We propose the LMM-JND to explore the perception granularity of large multimodal models (LMMs).

Arxiv: https://t.co/tanZAKgTk2
Project: https://t.co/FEXPIB9fI2

A thread (1/n)

[Zijian Chen @_ZijianChen]

What do LMM-JND curves look like?

The width of the trough between two high-level peaks represents the perceptual redundancy interval of LMM.

1. Evident perceptual redundancy intervals in Qwen-VLs & GPT-4o.
2. InternVL2.5 displays more frequent fluctuations.

A thread (5/n)