Runway’s Gen-1 allows users to transform existing videos using text, image, or video input.
Runway, the AI startup known for its video-to-video generative AI model called Gen-1, has launched its first mobile app on iOS. The app allows users to transform existing videos based on text, image, or video input, similar to a style transfer tool. However, unlike style transfer, the app generates entirely new videos as an output instead of applying filters.
Gen-1 Functionality
Runway’s Gen-1 allows users to upload a video and apply an aesthetic or theme to it. For example, a video of someone cycling in the park can be transformed to look like a watercolor painting or charcoal sketch. The app allows users to apply these transformations directly from their phones, making the process of creating this type of video much more fluid.
Output of Generative AI
The generative AI-powered app’s output can often be peculiar owing to its inherent nature. When a user incorporates a claymation effect, the resulting models deviate from genuine claymation, with limbs fluctuating in size, features melting and smudging, and warping between frames. Nevertheless, despite these constraints, the app’s entertainment value remains unaltered.
Limitations of the App
The app is currently limited to Runway’s Gen-1 model, and only allows footage shorter than five seconds to be used. Certain prompts such as generating nudity are banned, and it seems copyright-protected work is off-limits as well. The app only supports Runway’s Gen-1 model at the moment, but the purely generative Gen-2 will be added soon, according to Runway’s CEO Cristóbal Valenzuela.
Valenzuela’s Comments
Valenzuela drew a comparison between the present generation of generative AI and the 19th-century “optical toys” phase in which scientists and inventors produced a variety of devices that were primitive in their capabilities but served as precursors to modern cameras. Runway’s mobile app is reminiscent of these toys, but Valenzuela envisions the significant impact that such tools will have in the future.
Conclusion
While the app may not be suitable for professional production work, it offers a huge sense of possibility for future tools like this. The app is not seamless and has certain limitations, but it makes creating videos a more fluid process. The processing is done in the cloud and takes around two to three minutes to create each video. Runway’s mobile app feels like an optical toy, but it offers a glimpse into the future of generative AI.
