Introducing Generative Fill
This Tuesday, Adobe added a fascinating new tool to its Photoshop beta. The tool, called "Generative Fill," uses cloud-based image synthesis to generate new content in selected image areas based on a text description. This AI-powered functionality operates thanks to Adobe Firefly, Adobe's in-house image synthesis model.
The Magic of Generative Fill and Adobe Firefly
Generative Fill employs an AI technique known as "inpainting," a strategy previously featured in AI tools like DALL-E and Stable Diffusion. Firefly, the deep learning AI model driving this process, has been trained using millions of images from Adobe's stock library. The purpose? To generate realistic imagery based on textual descriptions. For example, if a user types "a clown on a computer monitor," Firefly will generate multiple options to choose from. The final result is a seamlessly integrated, AI-synthesized image within an existing image.
How Does It Work?
To utilize Generative Fill, users first select an area in their image they'd like to modify. A "Contextual Task Bar" appears, where users can type in a description of the desired content. This data is sent to Adobe's servers for processing and then the results return to the app. Users can select from several options or ask for more to browse.
When the Generative Fill tool is applied, it creates a new "Generative Layer" to enable non-destructive modifications of image content. This allows for additions, extensions, or removals driven by text prompts, which automatically adjust to the existing image's perspective, lighting, and style.
What Else is New?
In addition to Generative Fill, Adobe Firefly has introduced other AI-based features to the Photoshop beta. It can now remove parts of an image, erase objects, or extend an image's dimensions by generating new surrounding content - a technique known as "outpainting."
Although these features have been available in OpenAI's DALL-E 2 and various Stable Diffusion releases, Adobe's introduction of these to its flagship image editor signifies a significant stride. Nevertheless, there are potential pitfalls related to issues like harmful or socially stigmatized content generation, misuse of artists' images, and disinformation.
Adobe has implemented measures to mitigate these concerns. It blocks certain keywords associated with copyrighted, violent, and sexual content, and relies on usage terms to prohibit generating abusive, illegal, or confidential content. Adobe is also enhancing its Content Authenticity Initiative, using Content Credentials to trace the provenance of generated files.
Availability and Access
Currently, Generative Fill is available to all Creative Cloud members with a Photoshop subscription or trial through the "Beta apps" section. It is not available for commercial use, those under 18, in China, and supports only English text prompts. However, Adobe aims to make Generative Fill available to all Photoshop customers by year-end.
For those without a Creative Cloud subscription, Generative Fill can be tried for free on the Adobe Firefly website using an Adobe login. The waitlist for its Firefly beta has also been removed.
Stay tuned for more updates as Adobe continues to advance its tools, giving us more freedom to create our imagined visuals with AI's help!
