GG-EZ: regional adaptation framework for vision-language models in SEA

General-purpose models such as LLaVA, InternVL, Phi-Vision, and SDXL often misidentify Southeast Asian food, scripts, and traditions. Full retraining on regional data is expensive and usually requires proprietary data that local teams do not have.

We asked whether targeted adaptation could improve cultural accuracy without full retraining, and how small that intervention could be. The answer depends on where cultural knowledge sits in model weights and how efficiently we can change it. Automated scores cannot judge cultural accuracy alone, so the evaluation also required expert regional raters.

We proposed Anthropogenic Regional Adaptation, based on the claim that structured regional knowledge can teach cultural context without raw scale. We tested curated retrieval, targeted regional fine-tuning, and systematic evaluation of cultural accuracy.

I led the image-generation work. I fine-tuned the model on curated Southeast Asian images, then merged its weights with the original model at several ratios. This let us control regional accuracy against global image quality. Regional raters scored cultural accuracy at 1.569 versus a 1.491 baseline. Automated tests retained more than 98% of global quality. Regional adaptation did not require a comparable loss in general ability.

Regional annotators scored cultural accuracy, while DPG-Bench measured global image quality. We tracked both for every variant to map regional gains against global losses.