Students of the Faculty of Automation and Computer Engineering of Novosibirsk State Technical University (NSTU) have created a visual product search system that allows you to find the right product in the marketplace catalog based on photos. The development is based on artificial intelligence, which understands the essence of search, and not just compares pixels.
The development is intended primarily for integration into mobile applications of Russian online stores and marketplaces. This will simplify the search for customers, make it intuitive and fast, increase conversion and sales levels for sites, reducing the time between interest in a product and its search, and create a new convenient tool for stores working with visual search for goods from clothing to furniture.
According to Egor Antonyants, the head of the project, a lecturer at the Department of Automated Control Systems at NSTU-NETI, the system is based on the Siamese neural network (SNN), which is trained to understand the essence of an image, and not just look for a thing based on the elements of a digital image. SNN converts a photo into a unique digital "fingerprint" and quickly finds products with the most similar vectors in the database, and recognizes the same object, even if the photo has different lighting, angle, or complex background. This is necessary so that the customer sees the product they like in the store, takes a picture of it and can find the same model in online stores.
"The task was to make the system not only fast, but also accurate, so we simplified and optimized the architecture of the neural network, which allowed us to maintain high recognition accuracy comparable to large commercial analogues, but with much lower requirements for computing resources," said Egor Antonyants.
The advantage of the NSTU-NETI development is the recognition accuracy. Similar visual search technologies are already used in some major domestic and foreign marketplaces and applications, but almost all of them are either less resistant to imperfect shooting conditions (for example, poor light), or require powerful servers to process requests.
"The recognition accuracy of our technology was more than 95%, even on user photos with poor quality. At the same time, the system can be integrated to work on mobile devices, which makes it ready for implementation in real applications without the need for expensive equipment," said Dmitry Shipunov, one of the main developers of the project, a third—year student at AVTF, adding that in the future the technology can be adapted for other tasks where it is needed quickly and efficiently. accurately compare images, for example, in warehouse accounting systems.
Earlier, Mikhail Kireenko and Danil Matveev, undergraduates at the Faculty of Automation and Computer Engineering of NSTU-NETI, created an application for the automated creation of songs in which an AI model recreates a certain voice within the framework of the Project Activity discipline under the guidance of Egor Antonyants.