In April, a New York-based company called Runway AI unveiled technology that allows people to create videos, like a cow at a birthday party or a dog chatting on a smartphone, simply by typing a sentence into a box on a computer screen.
The four-second videos were blurry, choppy, distorted and annoying. But it was a clear sign that AI technologies would create increasingly compelling videos in the coming months and years.
Just 10 months later, San Francisco start-up OpenAI unveiled a similar system that creates videos that look like they’ve come straight out of a Hollywood movie. One demonstration included short videos — created in minutes — of woolly mammoths walking through a snowy meadow, a monster staring at a melting candle, and a Tokyo street scene that appears to have been captured by a camera running around the city .
OpenAI, the company behind chatbot ChatGPT and still image generator DALL-E, is among the many companies struggling to improve this kind of instant video creation, including startups like Runway and tech giants like Google and Meta, the owner of Facebook and Instagram. The technology could speed up the work of experienced cinematographers while completely replacing less experienced digital artists.
It could also become a quick and cheap way to create online misinformation, making it even harder to tell what’s real online.
“I’m absolutely horrified that this would affect a closely contested election,” said Oren Etzioni, a University of Washington professor who specializes in artificial intelligence. He is also the founder of True Media, a non-profit organization that works to identify online misinformation in political campaigns.
OpenAI calls its new system Sora, after the Japanese word for sky. The team behind the technology, including researchers Tim Brooks and Bill Peebles, chose the name because it “evokes the idea of limitless creative potential.”
In an interview, they also said the company had not yet released Sora to the public because it was still working to understand the risks of the system. Instead, OpenAI shares the technology with a small group of academics and other outside researchers who will “red team,” a term for looking for ways of abuse.
“The intent here is to give a preview of what’s on the horizon so people can see the potential of this technology — and we can get feedback,” said Dr. Brooks.
OpenAI already tags system-generated videos with watermarks that identify them as AI-generated, but the company acknowledges that these can be removed. They can also be difficult to detect. (The New York Times added “Generated by AI” watermarks to videos with this story.)
The system is an example of artificial intelligence being created, which can instantly generate text, images and sounds. Like other artificial intelligence generation technologies, OpenAI’s system learns by analyzing digital data — in this case, videos and captions that describe what those videos contain.
OpenAI declined to say how many videos the system learned or where they came from, except that the training included both publicly available videos and videos licensed by copyright holders. The company says little about the data used to train its technologies, likely because it wants to maintain an edge over competitors — and has been sued multiple times for using copyrighted material.
(The New York Times sued OpenAI and its partner Microsoft in December, alleging copyright infringement in news content related to AI systems.)
Sora creates videos in response to brief descriptions such as “a beautifully rendered papercraft world of a coral reef, filled with colorful fish and sea creatures.” While videos can be impressive, they are not always perfect and can include strange and absurd images. The system, for example, recently produced a video of someone eating a cookie — but the cookie never got any smaller.
DALL-E, Midjourney and other still image producing devices have improved so rapidly in recent years that they now produce images that are almost indistinguishable from photographs. This has made it harder to spot misinformation online, and many digital artists complain that it has made it harder for them to find work.
“We all laughed in 2022 when Midjourney first came out and said, ‘Oh, that’s cute,'” said Reid Southen, a film artist in Michigan. “Now people are losing their jobs at Midjourney.”
