title: Artistic Portrait Generation
emoji: 🎨
colorFrom: yellow
colorTo: gray
sdk: gradio
sdk_version: 5.22.0
app_file: app.py
pinned: true
license: apache-2.0
models:

AisingioroHao0/IP-Adapter-Art
guozinan/PuLID
stabilityai/stable-diffusion-xl-refiner-1.0
xinsir/controlnet-openpose-sdxl-1.0

IP Adapter Art：

Introduction

IP Adapter Art is a specialized version that uses a professional style encoder. Its goal is to achieve style control through reference images in the text-to-image diffusion model and solve the problems of instability and incomplete stylization of existing methods. This is a preprint version, and more models and training data coming soon.

How to use

can be used to conduct experiments directly.

For local experiments, please refer to a demo.

Local experiments require a basic torch environment and dependencies:

conda create -n artadapter python=3.10
conda activate artadapter
pip install -r requirements.txt
pip install git+https://github.com/openai/CLIP.git
pip install -e .

Comparison with Existing Style Control Methods in Diffusion Models

Evaluation using StyleBench style images. Image quality is evaluated using improved aesthetic predictor

	CLIP Style Similarity	CSD Style Similarity	CLIP Text Alignment	Image Quality	Average
DEADiff	61.99	43.54	20.82	60.76	46.78
StyleShot	63.01	52.40	18.93	55.54	47.47
Instant Style	65.39	58.39	21.09	60.62	51.37
Art-Adapter(ours)	67.03	65.02	20.25	62.23	53.63

Examples of Text-guided Stylized Generation

Artistic Portrait Generation

Pipeline

We built an artistic portrait generation pipeline using Art-Adapter, PuLID, and ControlNet. The structure is shown in the figure below.

Examples

Stylize ControlNet Parameter Visualization

Citation

@misc{ipadapterart,
  author = {Hao Ai, Xiaosai Zhang},
  title = {IP Adapter Art},
  year = {2024},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/aihao2000/IP-Adapter-Art}}
}