How to Use ComfyUI: Workflows, Models, and Setup

ComfyUI is a free, locally run application designed for AI image and video generation, offering users superior control compared to standard prompt-based tools. It provides full visibility and adjustment capabilities for every stage of the generation process. While this flexibility introduces a learning curve, this guide will walk you through the setup, from installation to executing and customizing your first workflow.

Prerequisites

To use ComfyUI, your computer must have sufficient GPU resources to support the models and workflows you intend to run. More complex models and resource-intensive workflows generally demand higher VRAM capacity.

You will also need the specific model files required by your workflow. Depending on the architecture, these may include checkpoints, diffusion models, VAEs, text encoders, LoRAs, or other components. These files are typically stored in the ComfyUI/models directory.

If your local hardware lacks the necessary GPU power, consider running ComfyUI on a remote GPU-equipped desktop. This approach offloads the generation workload to the remote GPU while allowing you to access the interface from your standard computer.

Installing ComfyUI

For Windows and macOS users, the official desktop application is the recommended entry point for beginners. Alternative methods, such as manual installation or using the ComfyUI command-line tool, are also available. The best approach depends on your specific operating system and configuration.

Once installed, launch the application to access the interface. You will find the workflow canvas along with the tools needed to create and manage your workflows.

The Importance of ComfyUI Workflows

A ComfyUI workflow defines the exact process by which an image or video is generated, dictating the models, settings, and processing steps involved in the final output.

This offers significantly more control than a simple prompt box. You can swap models, integrate LoRAs, use input images, fine-tune generation parameters, upscale results, or insert additional processing steps.

Workflows are also designed for reuse. Instead of reconstructing setups from scratch, you can save configurations that yield desired results and modify specific settings as needed. Additionally, you can import and adapt workflows created by the community to fit your own requirements.

Anatomy of a ComfyUI Workflow

Workflows consist of interconnected nodes, each handling a specific aspect of the generation pipeline. The connections between these nodes dictate how data flows through the system.

A standard text-to-image workflow typically includes nodes for loading models, inputting prompts, initializing image data, generating the image, decoding the output, and saving the final file.

• Model Loader: Loads the base model used for generation.
• Text Encoder: Translates your prompt into data interpretable by the model.
• Sampler: Executes the generation process based on selected parameters.
• VAE: Facilitates conversion between latent data and visual images.
• Save Image: Writes the final generated image to local storage.

You do not need to construct every workflow from zero. ComfyUI offers various templates, and a vast library of community-created workflows can be downloaded and opened directly.

Using Existing Workflows

The most efficient way to start is by utilizing pre-existing workflows. ComfyUI includes sample workflows for various tasks, and community platforms host numerous additional options.

Workflow images often embed the workflow data in their metadata. You can simply drag the image into the ComfyUI interface or use the Workflows → Open menu to load it. The canvas will display the nodes and settings already configured.

After loading, verify which models are required. If files are missing, ComfyUI can identify gaps in supported templates. For other workflows, you may need to locate and install the missing models manually.

Sourcing Models for ComfyUI

Models are commonly available on repositories like Hugging Face and Civitai, as well as on the original project pages. The key is ensuring the model is compatible with your chosen workflow.

Do not assume universal compatibility; different model architectures often require specific loaders and supporting files.

Before downloading, verify the following:

• Model architecture and version
• Required ComfyUI workflow compatibility
• Model file format
• Recommended VRAM and hardware requirements
• Any necessary VAE, text encoder, LoRA, or supporting files
• Licensing terms and usage restrictions

ComfyUI supports various file types, each stored in specific directories. For instance, checkpoints go in models/checkpoints, LoRAs in models/loras, and VAEs in models/vae. Newer architectures may use directories like models/diffusion_models and models/text_encoders.

Installing a Model

Once you have downloaded a model, place it in the directory expected by the workflow and select it in the corresponding model loader node.

For example, a checkpoint should be stored in:

ComfyUI/models/checkpoints/

A LoRA would typically be placed in:

ComfyUI/models/loras/

If the new model does not appear in the dropdown list, refresh the interface or restart ComfyUI.

Installing Custom Nodes

Advanced workflows often rely on custom nodes not included in the base installation. Missing dependencies will appear as missing nodes in the canvas.

The built-in Manager facilitates the installation of these custom nodes. Alternatively, you can install them manually by placing their repositories in the custom_nodes directory and resolving their dependencies.

Exercise caution and only install custom nodes from trusted sources, as they contain executable code with potential security and dependency implications.

Executing and Customizing Workflows

With all models and nodes installed, review critical settings such as the model, prompt, image dimensions, and sampling parameters.

When ready, click the Queue button to initiate the workflow. ComfyUI will process each step and generate the output defined by your node graph.

You can easily modify individual components without rebuilding the workflow. Add a LoRA, connect an input image, switch samplers, or insert upscalers to refine the results.

Preserving Your Workflows

Save workflows you plan to reuse. Note that a workflow file contains the node structure and settings but not the model files themselves. Maintain a record of the specific models and custom nodes required.

This is crucial when transferring workflows to a new computer or cloud desktop, as you will need to reinstall the corresponding models and nodes to ensure compatibility.

Experience ComfyUI on DaDesktop

There is no need to purchase a new GPU solely for ComfyUI. If your local hardware is insufficient, you can run the application on a cloud desktop on demand.

DaDesktop offers cloud desktops equipped with dedicated GPU resources, ideal for AI image and video generation workloads. You can install ComfyUI, download necessary models, and develop workflows without investing in local GPU hardware.

Learn more about AI image and video generation on DaDesktop. You can also view the available GPUs to select a configuration that matches your model and workflow requirements.