Skip to main navigation Skip to search Skip to main content

AI 图像生成的精细化可控:结构化与非结构化 提示词及其轻量微调的对比研究

Translated title of the contribution: Fine-tuned control of AI image generation: a comparative study of structured and unstructured prompts and lightweight model fine-tuning
  • Wei Zhang*
  • , Yayuan Luo
  • *Corresponding author for this work
  • Karolinska Institutet
  • Liaoning Technical University

Research output: Contribution to journalArticlepeer-review

Abstract

To mitigate instability and poor controllability in text-to-image(T2I) generation, this study integrates prompt engineering and lightweight fine-tuning in artificial-intelligence-generated content(AIGC) systems. We examine how prompt formats and fine-tuned models jointly affect controllability of image generation and propose a reproducible pipeline built on “structured prompts + LoRA fine-tuning.” The method addresses the challenge of achieving precise control in LLM-assisted image generation for packaging design workflows. Building on FLUX-dev, we create a four-quadrant, 100-theme dataset that systematically varies prompt structure and LoRA configurations. The proposed method is evaluated using CLIPScore and automated aesthetic ratings. Statistical significance is examined through linear mixed-effects models and paired tests, while visual quality is assessed through isomorphic renderings and dual global–detail inspections. To assess the generalizability of the structured prompt strategy, we also evaluate the dataset with the cross-modal DALL·E3 model. Results show that structured prompts mainly stabilize overall layout and hier-archy, whereas LoRA chiefly sharpens edges and material details. For base T2I models, structured prompts consistently improve image–text alignment, while LoRA substantially elevates aesthetic scores. Different large models exhibit distinct strengths, and newer versions do not inherently surpass older ones; the key is pairing high-quality prompts with LoRA settings suited to each version. These findings provide both theoretical insights and practical guidance for prompt engineering and cost-effective model adaptation.

Translated title of the contributionFine-tuned control of AI image generation: a comparative study of structured and unstructured prompts and lightweight model fine-tuning
Original languageChinese (Traditional)
Pages (from-to)908-918
Number of pages11
JournalCAAI Transactions on Intelligent Systems
Volume21
Issue number4
DOIs
StatePublished - Jul 2026
Externally publishedYes

Fingerprint

Dive into the research topics of 'Fine-tuned control of AI image generation: a comparative study of structured and unstructured prompts and lightweight model fine-tuning'. Together they form a unique fingerprint.

Cite this