Novosibirsk State Technical University has developed a software tool that analyzes the movements of visitors in stores in real time and builds heat maps of their activity. The project is based on computer vision technologies and neural networks.
Designed for retail, the development allows you to instantly receive data on customer behavior, which will help make trading floors more convenient and efficient. According to the project manager, Egor Antonyants, an assistant at the Department of Automated Control Systems at NSTU-NETI, the system analyzes the video stream from a surveillance camera in real time.
"It is based on a convolutional neural network model that detects people in a frame. The program counts the number of visitors, remembers the trajectories of their movement and the places of stops. All this information is instantly superimposed on the video in the form of a color gradient — the so-called heat map. The warmer the color the more people have visited this point," said Egor Antonyants.
Unlike the common system of counting visitors to retail outlets, when each customer is simply fixed at the door, the new algorithm allows you to see the full picture of human behavior in the hall. Technically, the work is built in several stages: first, the neural network identifies people in each frame, then the program accumulates data about their location, and then, using mathematical algorithms, transforms them into an understandable visual map in real time.
According to Dmitry Gordienko, the main developer of the project, a fourth-year student of the Faculty of Automation and Computer Engineering, there are analogues of such a software tool, but they are either too expensive for small shops, or they give only general statistics, but they do not show at the moment what exactly attracted a person's attention.
"Our development allows us to assess right now how well the goods are laid out on the new showcase or in which area of the hall visitors spend the most time. We have clearly tested this: in one of the test videos, you can clearly see how customers stop at the ice cream display case, and the seller is always in the checkout area. Such information can be used to instantly adjust the layout of goods," Dmitry Gordienko said.
The student also added that the tool can be useful not only in retail, but also in any public spaces where it is important to quickly understand the flow of people. We are talking about museums, exhibition centers and airports. In the future, the system can be adapted for more complex analytics and integration with existing video surveillance systems.
Earlier, students of the Faculty of Automation and Computer Engineering of NSTU-NETI created a visual product search system that allows you to find the right product in the marketplace catalog based on photos.