Nvidia’s text-to-video tech could take your GIF game to the next level
The only limit to your GIFs is your imagination
When you purchase through links on our site, we may earn an affiliate commission.Here’s how it works.
Now thatChatGPTand Midjourney are pretty much mainstream, the next big AI race is text-to-video generators – andNvidiahas just shown off some impressive demos of the tech that could soon take your GIFs to a new level.
Anew research paper and micro-sitefrom Nvidia’s Toronto AI Lab, called “High-Resolution Video Synthesis with Latent Diffusion Models”, gives us a taste of the incredible video creation tools that are about to join the ever-growing list of thebest AI art generators.
Latent Diffusion Models (or LDMs) are a type of AI that can generate videos without needing massive computing power. Nvidia says its tech does this by building on the work of text-to-image generators, in this case Stable Diffusion, and adding a “temporal dimension to the latent space diffusion model”.
In other words, its generative AI can make still images move in a realistic way and upscale them to using super-resolution techniques. This means it can produce short, 4.7-second long videos with a resolution of 1280x2048, or longer ones at the lower resolution of 512x1024 for driving videos.
Our immediate thought on seeing the early demos (like the ones above and below) is how much this could boost our GIF game. Okay, there are bigger ramifications, like the democratization of video creation and the prospect of automated film adaptations, but at this stage text-to-GIF seems to be the most exciting use case.
Simple prompts like ‘a storm trooper vacuuming on the beach’ and a ‘teddy bear is playing the electric guitar, high definition,4K’ produce some pretty usable results, even if there are naturally artifacts and morphing with some of the creations.
Right now, that makes text-to-video tech like Nvidia’s new demos most suitable for thumbnails and GIFs. But, given the rapid improvements seen inNvidia’s AI generation for longer scenes, we probably won’t have to wait for longer text-to-video clips in stock libraries and beyond.
Get the best Black Friday deals direct to your inbox, plus news, reviews, and more.
Sign up to be the first to know about unmissable Black Friday deals on top tech, plus get all your favorite TechRadar content.
Analysis: The next frontier for generative AI
Nvidia isn’t the first company to show off an AI text-to-video generator. We recently sawGoogle Phenakimake its debut, revealing its potential for 20-second clips based on longer prompts. Its demos also show an albeit more ropey clip that’s over two minutes long.
The startup Runway, which helped created the text-to-image generator Stable Diffusion, also revealed itsGen-2 AI video modellast month. Alongside responding to prompts like ‘the late afternoon sun peeking though the window of a New York City loft’ (the result of which is above), it lets you provide an still image to base the generated video on and lets you request styles to be applied to its videos, too.
The latter was also a theme of the recent demos forAdobeFirefly, which showedhow much easier AI is going to make video editing. In programs like Adobe Premiere Rush, you’ll soon be able to type in the time of day or season you want to see in your video and Adobe’s AI will do the rest.
The recent demos from Nvidia,Google, and Runway show that full text-to-video generation is in a slightly more nebulous state, often creating weird, dreamy or warped results. But, for now, that’ll do nicely for our GIF game – and rapid improvements that’ll make the tech suitable for longer videos (asdemos from the more recent OpenAI Sora show) are now just around the corner.
Mark is TechRadar’s Senior news editor. Having worked in tech journalism for a ludicrous 17 years, Mark is now attempting to break the world record for the number of camera bags hoarded by one person. He was previously Cameras Editor at both TechRadar and Trusted Reviews, Acting editor on Stuff.tv, as well as Features editor and Reviews editor on Stuff magazine. As a freelancer, he’s contributed to titles including The Sunday Times, FourFourTwo and Arena. And in a former life, he also won The Daily Telegraph’s Young Sportswriter of the Year. But that was before he discovered the strange joys of getting up at 4am for a photo shoot in London’s Square Mile.
How to delete a character from Character AI
How to turn off Meta AI
Phishing attacks surge in 2024 as cybercriminals adopt AI tools and multi-channel tactics
