Review of the most interesting neural networks in autumn 2023 - winter 2024: we create videos, teach leading foreign languages

  The boom in the neural network, despite the lull, has not disappeared anywhere. As before, new, interesting II-services appear that facilitate work and creativity in various fields. In the article, we talk about novelties this fall, which will come in handy when creating various creatives — not only in arbitration, but also in creative life. 

    HeyGen - high-quality deep fakes in foreign languages ​​HeyGen - a generative neural network from the OpenAI company for recognizing speech in video and translating it into another language. This is not the main function of the service, but it is precisely thanks to it that HeyGen has become popular.

 

    Founded several years ago by Joshua Xu and Wayne Liang, a platform called Movio was designed to create videos with AI avatars and voiceovers. This concept was preserved, only later added tools for dubbing texts in different languages, voice recognition and creating deepfakes while preserving the character's facial expressions. In September 2023, already under the name HeyGen, the developers presented a functionally expanded beta version. 

Features of HeyGen 

• Creating videos for various tasks - advertising, presentations, training, videos for social networks; 
• Choose an avatar for the video from the proposed options of 100+ races, ages and images. Unique emotions, facial expressions and gestures are developed for each avatar; 
• A selection of more than 50 voices with translations into popular languages ​​and dialects. Available languages ​​include English, French, German, Spanish, Italian, Dutch, Portuguese, Chinese, Japanese, and Hindi. 

There is no possibility of translation and dubbing in Russian yet, but it is possible to translate from Russian into other languages. 

Users also get the following options:

• Adjustment of speed, tonality and voice intonation with additional selection of pauses and accents in speech; 
• Selecting a background from the HeyGen collection or using your own downloaded from your computer;

 

• Adding music — your own or selected in the service library;
• Adding logos, titles, watermarks and other creative details that make the video unique; Saving videos, including in HD quality with a resolution of 1080 pixels.
• Getting a link for the site or sharing in social networks.

How to work with HeyGen 

    You can start working with videos for free by clicking the "Try HeyGen for Free" button, after which you need to create an email account or log in with Google, Facebook* or Single Sign-On.

The HeyGen dashboard interface is simple and uncomplicated, all menu names are intuitive even for a person who does not speak English well. In the upper part, there are four active menus with which you can create an avatar or generate a video creative - Instant Avatar, Photo Avatar, Template, AI Script.

 

    The main work with the video is done in the Template menu. When you click on the command, a form for creating a horizontal/vertical video appears on the screen. At the same time, you can choose the video format - advertising, presentation, news, commercial, and so on. After selecting the basic settings, the service redirects to the built-in video editor for working with a specific video. Here you can choose translation options, upload your own audio scripts, add pauses and use other editing methods, including AI.

    The advantage of HeyGen is that there are many additional tools for working with voice and visuals. For example, the Clone voice command allows you to duplicate your own voice in all the main languages ​​of the AI ​​service. TalkingPhoto functionality helps to "animate" static photos, which is convenient when creating animated avatars. HeyGen also makes it possible to integrate the created creatives with hundreds of applications, systems, social networks, CRM, etc. using the Zapier cloud platform. 

HeyGen tariffs 

    The free version of the service allows you to work with two 30-second videos or one 59-second video. There are two conditional credits for this, each credit is a minute of generated video. The Creator and Business plans offer different amounts of monthly credits – from 15 to 300 at different prices. The cheapest option of using 15 credits of the Creator plan will cost $24 per month, the most expensive - 300 credits of the Business plan - $720 per month. A 20% discount applies when choosing an annual tariff. HeyGen also has a customized Enterprise offering with a customized set of tools.

 

    When paying for the service, you will need a foreign bank card, Russian cards are not supported. 

Advantages and disadvantages of HeyGen The advantages of the service include: 

• Ease of installation. Any person who knows a minimum of English can understand the settings when creating a video. 
• High quality creatives. At the output, users receive videos with clear sound and good diction of the presenter; 
• A decent variety of settings. A large number of customized announcer templates with different characteristics of voices and backgrounds allow you to make professional videos. 

Among the disadvantages, users note: 

• The relative mechanistic nature of voices. With the use of templates, slight voice artificiality is noticeable, especially in long videos; 
• Lack of detailed drawing of emotions on faces. When carefully examining the faces, users note the static facial expressions, which leads to emotionlessness in some episodes; 
• Long wait for results. The process of generating videos, even on paid tariffs, can take up to several hours. This is most likely caused by peak server loads. 

Conclusion 

    HeyGen quickly became a popular neural network. It was especially appreciated by bloggers and lovers of unusual creativity. However, many people are put off by the price of use: $24 per month to generate 15 minutes of video is a cost for ordinary users.

 

Masterpiece - we draw and compose texts together with Yandex

    Masterpiece from Yandex in the form of a prototype appeared back in April 2023. In the first months after the presentation, there was a lot of criticism of the service. However, the neural network has been constantly updated, and the fall release introduces Masterpiece as a full-fledged tool for creating images, text and videos. Free AI from Yandex works on the basis of cascade diffusion technology: it creates a basic image and then gradually adds details to it according to a given prompt. The output shows four different images to choose from. 

    Since its launch, Masterpiece has been conceived as a large community of users who can create content and publish it in a shared feed. At the same time, people were invited to rate and comment on creatives, subscribe to each other. Thus, Yandex's neural network actually builds a social network around its AI. 

    In the summer of 2023, YandexGPT — a language model that generates text on request — was connected to Masterpiece. In August, the neural network "learned" to glue together several images in videos. Features of Masterroom from Yandex To create your own creatives, the Masterroom program must be downloaded from Google Play or AppStore. In the browser, you can only view other people's images and like them.

What the program looks like when requested 

 

    The application allows you to generate images with a resolution of 1024x1024, independently set the desired resolution and aspect ratio will not work. However, the input prompt box has modifying prompts - "professional photo", "realistic", "bright lighting", "Dali style" and so on. The language model of YandexGPT when generating text works in the same way as in the Alice service, only the header must also appear in the Masterwork neural network. Several images generated by text request are immediately offered to choose from.

    When creating a video, you can generate a short clip of 4 seconds with a frequency of 24 fps. During the creation process, Masterpiece asks to describe the main idea and offers 4 shots to choose from. At the same time, the user must decide on the type of animation that will "animate" the frame - timelapse, zoom, panorama, flight, rise, rotation or gradual change of the picture. Video creation tools are now, however, available to a small number of active neural network users. All created content appears in the general feed, any picture, in particular someone else's, can be saved by clicking on the ellipsis and selecting the appropriate item. As with any other neural network, Masterpiece has limitations. 

    Prohibited during generation are prompts indicating specific personalities, mentioning politics or religion, as well as eroticism, violence and cruelty. It takes a little more than 5-10 seconds to generate the text of the neural network, images are formed within a minute. How to launch Masterroom After downloading the program to the device, you need to log in to it under the Yandex account. A general feed of posts with published creatives of other users will appear on the main page. 

    All entries can be sorted by date or popularity, in the subscription mode, only the posts of authors who have a subscription are displayed. To create a creative, you need to click the "+" button at the bottom of the program, and then select the "image" or "text" item. In the window, the neural network offers to write a request for an image or text. When you click on a question, hints appear, which you can swipe to the left.

 

    If you do not like the received creative, you can click on the "Regenerate" button and clarify the request in the field of the prompt editor. Failed images are deleted when you click the Trash icon. 

Advantages and disadvantages Masterpiece 

The advantages of the popular neural network from Yandex include: 

• Availability and ease of use. The service interface is maximally simplified and understandable even to those who have never used AI technologies. An additional advantage is the prompts that appear on the screen when creating an illustration or text; 
• A community has been created around the neural network. The attempt to turn Masterpiece into a social network can be considered successful — people enjoy creating, liking and commenting on each other's work; 
• Free service. All Masterspace tools are available for free without any restrictions. 

Disadvantages of Masterpiece: 

• Errors during generation. Like any other neural network, technology from Yandex produces images that are incorrect in detail; 
• Weak drawing of details. The risk of errors increases when the prompts are highly detailed. This applies not only to images with limbs, but also, for example, inscriptions;
• Skew towards abstraction. Masterpiece generates abstract-surrealistic or painted images quite well, but there are often problems with realism. 

 

    Conclusions The social and creative scope of Masterpiece is not bad: the service offers people a social network with the possibility of generating creatives in three formats. But objectively: so far AI from Yandex lags behind the received images from the Russian-language analogue Kandinsky from Sber and even more so from the most popular Midjourney. However, this is the only multi-disciplinary AI service in Runet for the fall of 2023 with a fairly simple toolkit.

 

We create short films together with Runway Gen-2 Runway Gen-2 - 

is a neural network for video generation from the developers of the popular Stable Diffusion, a service that allows you to generate high-quality images of various formats, including realistic ones. 

 

The first version of the AI-video platform allowed you to convert original images and downloaded videos, in the updated version you can create a video from a text request from scratch. 

Features of Runway Gen-2 

The service makes it possible to create professional, sometimes realistic videos. From the additional functions, you can choose to apply a mask, add stylization to the specified template. The tools also include storyboarding, rendering, and customization. 

The peculiarity of the service is that the generated videos on the free tariff do not exceed 4 seconds in duration, and the requests themselves cannot be longer than 320 characters. On paid tariffs, the video length increases to 15 seconds. 

You can use Runway Gen-2 both in the browser version and in the program for iOS. How to use Runway Gen-2 In the browser version, you need to click Try runway for free to register, then create an account or log in via Google, Apple account or Single Sign-On.

How to use Runway Gen-2 

    In the browser version, to register, click Try runway for free, then create an account or log in via Google, Apple account or Single Sign-On. 

 

    The dashboard interface is intuitively simple and clear. On the left is the toolbar, which includes menus for creating videos, images, and 3D objects. The central part contains several buttons with the main tools – generating a video from a downloaded reference picture, video or text prompt. Popular tools are also placed on a separate panel with icons. 

    At the bottom of the screen, in the Discover and Remix section, there are tiled examples of inspirational videos.

 

    To create a video, you need to select the "Start with Image" or "Start with Text" menu, after which you need to enter a request in the "Text" field. An image for reference is selected in the "Image" field, and a text description is added to the reference illustration via the "Image+Description" menu. 

 

    When you click on the icon of additional settings, commands are opened: Seed - A number that determines the randomness of generation in one style. Numbers of successfully generated videos can be saved for future replay; Interpolate — a tool for smoothing video frames to increase smoothness; Upscale — function to improve video resolution for greater clarity and detail; Remove watermark - removing the watermark on the finished video. The Runway Gen-2 settings also have detailed editing tools: Inpainting – removes unnecessary objects from the video; Remove Background – works with the background of the video. 

Tariffs Runway Gen-2 

    In  the free version, the service offers 125 credits for three projects with a total size of up to 5 GB without the possibility of removing watermarks. One second of generation equals 5 credits. The "Standard" tariff, worth $12 per month or $144 per year, opens up additional possibilities: 625 credits for an unlimited number of projects with a total video size of up to 100 GB.

 

The "Pro" tariff, worth $28 per month or $336 annually, opens up even more possibilities - 2,250 credits when uploading 500 GB of video, as well as a number of additional options for editing and exporting videos. The "Unlimited" tariff at a price of $76 per month and $912 per year allows you to remove all restrictions on editing, with a bonus of 2,250 monthly renewable credits. 

Advantages and disadvantages of Runway Gen-2 

The advantages of the neural network include: 

• Simplicity and convenience of work. The interface of II-service is intuitive to anyone who has a minimum command of the English language, and access to tools is open both through the side menu and through the interactive buttons of the dashboard; 
• Lots of editing tools. Runway Gen-2 offers a decent set of useful options for generating videos both from a text request and from downloaded reference illustrations or your own videos; 
• Good quality of the finished material. Skillful use of prompts, references and subtle settings allows you to create full-fledged videos - not only entertaining, but also advertising, presentation and others. 

Among the disadvantages of Runway Gen-2, users note: 

• Problems with detailing. Despite the overall high quality of the generated video, there are difficulties with drawing individual interior details and complex, multi-faceted actions; 
• Problems in drawing limbs. There are also difficulties with the detailing of human limbs. Also, sometimes extra objects or double characters are generated in the videos, not specified in the prompt. 

Conclusion 

    To use Runway Gen-2 fully, you need to switch to at least a standard tariff. This will allow you to experiment more or less freely with the video, resulting in a version of the video closer to the idea. In general, currently, the maintenance of the already implemented functionality looks solid.