GG-EZ: regional adaptation framework for vision-language models in SEA

General-purpose models such as LLaVA, InternVL, Phi-Vision, and SDXL often misidentify Southeast Asian food, scripts, and traditions. However, full retraining on regional data is expensive. It usually needs proprietary data that local teams lack.

We asked whether targeted adaptation could improve cultural accuracy without full retraining. Moreover, we asked how small that intervention could be. The answer depends on where model weights hold cultural knowledge and how efficiently we can change them. However, automated scores alone cannot judge cultural accuracy. Therefore, the evaluation also needed expert regional raters.

We proposed Anthropogenic Regional Adaptation to teach models cultural context through structured regional knowledge. The approach rests on the claim that this knowledge can reduce reliance on large data volumes. Therefore, we tested retrieval from curated sources, targeted regional fine-tuning, and systematic checks of cultural accuracy.

I led the image-generation work. First, I fine-tuned the model on curated Southeast Asian images. Then, I merged its weights with the original model at several ratios. This controlled the balance between regional accuracy and global image quality. Regional raters scored cultural accuracy at 1.569 versus a 1.491 baseline. Moreover, automated tests retained more than 98% of global quality.

Regional annotators scored cultural accuracy, while DPG-Bench measured global image quality. We tracked both for every variant. Therefore, we could map regional gains against global losses.