On Monday, researchers at Nvidia unveiled a new artificial intelligence (AI) model that has the ability to move objects within an image. The program, called DiffUHaul, can transport an object from one location to another without changing the image’s shape or backdrop since it can spatially grasp the context of the image. This technique’s distinctive feature is that it is training-free, which means that no pre-training data was used in its development. The business demonstrated the new technique at the Asia 2024 meeting of the Special Interest Group on Computer Graphics and Interactive Techniques (SIGGRAPH).
Researchers from Nvidia outlined the new AI tool in a study. The technology was created in partnership with Reichman University, Tel Aviv University, and The Hebrew University of Jerusalem. The researchers’ goal with the new tool was to address a major difficulty with AI picture generating models: the inability to spatially aware move elements within an image.
The study emphasizes how AI models’ lack of spatial reasoning has kept this specific editing task a barrier for AI researchers. Because they don’t comprehend how a movement in a 2D world would be interpreted spatially, existing visual models are only able to comprehend the context of a picture.
Nvidia says this problem can be resolved with DiffUHaul. The tool’s denoising step makes use of attention masking, which is based on picture diffusion architecture. To maintain the appearance of the high-level object, this is done. BlobGEN is a novel method that incorporates spatial awareness into the AI tool. Furthermore, using the localized model in the appropriate location, new methods were employed to reconstruct actual images.
The AI will be able to physically reposition the object and change the background in response to a text prompt that users input on the front end that highlights the object they wish to modify. The AI editing tool’s ability to comprehend the shape changes that accompany spatial movement was not evident in the company’s demonstrations. For example, the shape of an airborne balloon is altered when it is transported to the ground. However, due to a lack of training, the AI might not be able to recognize that.
DiffUHaul uses BlobGEN for spatial understanding, which enables robust object dragging without fine-tuning.
To learn more on this, visit to learn and read more about this.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behaviour or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
