generative-visual-prompt | NeurIPS 2022 ] Code for `` Generative Visual Prompt

generative-visual-prompt is a Python library. generative-visual-prompt has no bugs, it has no vulnerabilities and it has low support. However generative-visual-prompt build file is not available and it has a Non-SPDX License. You can download it from GitHub.

Generative models (e.g., GANs and diffusion models) learn the underlying data distribution in an unsupervised manner. However, many applications of interest require sampling from a specific region of the generative model's output space or evenly over a range of characteristics. To allow efficient sampling in these scenarios, we propose Generative Visual Prompt (PromptGen), a framework to achieve distributional control over pre-trained generative models by incorporating knowledge of arbitrary off-the-shelf models. PromptGen defines control as an energy-based model (EBM) and samples images in a feed-forward manner by approximating the EBM with invertible neural networks, avoiding optimization at inference. We demonstrate how PromptGen can control several generative models (e.g., StyleGAN2, diffusion autoencoder, StyleNeRF, NVAE) using various off-the-shelf models:. PromptGen requires no training data, and the only supervision comes from off-the-shelf models that help define the control. It samples images in a feed-forward manner, which is highly efficient, and it also stands alone at inference, meaning that we can discard the off-the-shelf models after training. PromptGen not only offers generality for algorithmic design and modularity for control composition, but also enables iterative controls when some controls are contingent on others.