Flux IPA

In the previous chapters, we shared the Flux Redux model, which can generate an image similar to the input image. However, it also has a drawback: the weight of the input image is too high, making it impossible to fine-tune through Prompt. Although the weight of the Redux model can be fine-tuned through a plugin, the effect is not always ideal.

This chapter will introduce how to use the Flux IPA model, which can also generate an image similar to the input image, similar to the Redux model. However, the biggest difference is that it can fine-tune the image through Prompt. The effect is similar to the image below. I uploaded a Mona Lisa image and added the words “wearing sunglasses” or “wearing earphone” in the prompt, then used the Flux IPA model to generate two images. You can see that the two images on the right are relatively similar to the Mona Lisa, with the addition of sunglasses or earphones in the scene.

​

1. Install Flux IPA Extension

Currently, there are two Flux IPA models on the market: one from Xlab and another from InstantX. I recommend using the InstantX Flux IPA because, based on my tests, it has lower memory requirements and produces good results.

To use the InstantX Flux IPA model, you need to install the Flux IPA extension. You can install this extension through ComfyUI’s ComfyUI-Manager. For detailed installation methods, refer to the Install ComfyUI Extension article.

​

2. Download Flux IPA Model

After installing the extension, you need to download the Flux IPA model. You can download the Flux IPA model here, downloading only the ip-adapter.bin file and placing it in the /models/ipadapter-flux directory.

​

3. Flux IPA Workflow

After downloading the model, you can build a Flux IPA workflow as shown in the image below.

If you look closely at this workflow, it actually adds an Apply IPAdapter Flux Model node (Figure ①) to the basic Flux workflow. Then continue connecting along ①, connecting the ②, ③, and ④ nodes in the image to the Apply IPAdapter Flux Model node.

It’s important to note that the IPAdapter model differs from Redux in that it does not pass data and Prompt together to the Flux model, but instead passes data and Prompt separately to the Flux model. Therefore, the output node of Apply IPAdapter Flux Model is not Conditioning, but model.

In addition to the workflow shown above, you can also try a simpler version. I found that the simpler version performed better in my tests. I mainly used the Flux ComfyUI workflow recommended in the Flux ComfyUI workflow, then modified it based on this workflow.