An E-Commerce Product Recommendation Method Based on Visual Search and Customer Satisfaction

Introduction

The ubiquity of the Internet has provided the foundation for explosive growth in online information fueling an increase in consumer purchase behaviors via e-commerce sites. According to a survey by Statista using global statistical data from e-commerce retail purchases, sales have continued to increase at an annual growth rate of approximately 20% since 2014, reaching 4,206 billion dollars in 2020, with a continued growth forecast (Sabanoglu, 2020a).

Consumers often scrutinize the product information provided by e-commerce sites when purchasing products. Due to the excessive amount of information available, people will often spend a lot of time choosing the appropriate products. According to a 2020 worldwide survey of digital shopper preferences by the global statistical data firm Statista, 61% of digital shoppers preferred searching and navigation (for easy-to-find products) as their source of inspiration for online shopping. (Sabanoglu, 2020b). Introducing a product recommendation method would be an effective way to narrow shoppers’ choices and allow them to be more selective.

On existing e-commerce sites, consumers can input keywords to search for their desired products. In this case, consumers need to grasp the product information in advance, which is difficult for indescribable products or consumers who are unfamiliar with the e-commerce site. For example, when a shopper sees a shirt or a backpack he/she likes, the user should specify its shape, color, and other features. It is a challenge to describe the style of the product, and the results recommended by the existing e-commerce search mechanism may differ from what is expected by the shopper. Also, when a shopper wants to buy a specified product, he/she may not know the product name, which will make the product search more difficult.

Recently, a visual (image-based) product search has started being used by shoppers. There are already many relatively mature algorithms that have been devised for image recognition in real-world systems. At present, image-based product search recommendation systems make selections based on similarities between user-inputted images and shapes, colors, and other features of products. Some of this information may be combined with sales and rating data of the products for comprehensive analysis, but the recommendation effect is still not very satisfying for shoppers. For example, when the same product is sold in multiple colors, but the product image is only displayed in one of the colors. If the user-inputted image does not match the color of the product display image, the user may miss the product. Furthermore, the quality of recommended products cannot be guaranteed when searching for products based on the similarities between user-inputted images and product images that do not necessarily match the real ones. According to the survey, the second most important source of inspiration for online shoppers is product ratings and reviews, which accounted for 53% of shoppers’ preferences for making online buying choices (Sabanoglu, 2020b).

Therefore, in this work, the authors aim to generate a product recommendation method to support visual product searches based on the similarities of both visual and textual information. For this, the authors not only measure the similarities between user-inputted images and product images but also measure the degree of customer satisfaction for each product by analyzing product reviews and product repeat purchase rates.

To sum up, the authors’ contributions are as follows:

• •

  The authors conduct image analysis on the user-inputted image and product images to measure the similarity between the user-inputted image and each product image based on the SIFT features and the surrounding text of the images.

• •

  The authors conduct sentiment analysis on product reviews and combine user repeat purchase behavior to measure the degrees of product satisfaction.

• •

  The authors evaluate the ranking accuracy and the usefulness of the proposed product recommendation method by analyzing the product data and the review data in e-commerce.