AI model allows for near-instant image creation on consumer-grade hardware

An AI model that creates images as the user types, using only modest and affordable hardware, has been announced by the Surrey Institute for People-Centred Artificial Intelligence (PAI) at the University of Surrey.

The model, NitroFusion, represents a world first and has been made open source by its developers—SketchX, a lab within PAI—a move that fundamentally transforms access to AI-enabled image creation models for creative professionals. The research is published on the arXiv preprint server.

Professor Yi-Zhe Song, Director of SketchX and Co-Director of PAI, said, “NitroFusion represents a paradigm shift in making AI accessible to everyone, eliminating the need for large computing resources and the long waiting times between prompt and result that are common with most image generation platforms.”

Typically, similar technology is available only to corporate giants with vast computing resources. However, NitroFusion runs on a single consumer-grade graphics card—marking a decisive step forward in bringing advanced AI capabilities to individual creators, small studios, and educational institutions. The almost instant creation of images allows rapid artistic iterations and greater control over the generated imagery.

Dar-Yen Chen, the Ph.D. researcher who helped to develop the project at PAI, said, “NitroFusion leverages a novel dynamic adversarial framework that works like a panel of specialized art critics, each evaluating different aspects of the generated image to ensure high quality in a single step. The system’s flexible architecture allows users to optionally use between one to four refinement steps, providing direct control over the balance between generation speed and image quality.”

Professor Song added, “With NitroFusion, we’re not just releasing another image generation model—we’re pioneering an entirely new approach which democratizes AI interaction.

“Following our DemoFusion release last year, which provided a new way to upscale AI-generated images, this innovation further establishes our position at the forefront of making powerful AI technology accessible to all.”

This breakthrough delivers multiple leaps for the users and industry:

Instant image generation that responds as users type—a first in the field—enabling rapid iteration, greater control and better experimentation
Improved sustainability through greatly reduced energy consumption
Consumer-grade affordable hardware requirements (e.g. a single high-performance GPU) that mean individuals and small studios can create imagery affordably
Open-source availability enables global innovation, adaptation and variations
No cloud dependencies or subscription fees.

Professor Adrian Hilton, Director of the Institute for People-Centred AI at the University of Surrey, said, “We believe we’re the first in the world to achieve interactive image generation at this scale and efficiency. This opens up access to state-of-the-art AI for image generation and is just the beginning of our commitment to democratizing creative AI tools. Our Institute will continue to develop open-source, groundbreaking technologies that put professional-grade AI capabilities into the hands of creators everywhere.

“We’re particularly proud of the great work that our SketchX Lab, creating new concepts and advancing the science of generative AI. Our research is focused on ensuring that the future of creative AI technology is inclusive, responsible and accessible to all, and we’re keen to continue to work with organizations that share this ethos.”

More information:
Dar-Yen Chen et al, NitroFusion: High-Fidelity Single-Step Diffusion through Dynamic Adversarial Training, arXiv (2024). DOI: 10.48550/arxiv.2412.02030