Area 6: Product Search and Brand Tracking in Social Media

Brand tracking over social media is very important for generating business intelligence to companies to understand the online sentiments, interests and concerns of customers. Increasingly, more and more social images and videos either have none or no meaningful text annotations. In order to track brand information more completely for various products in the domains of fashion, automobile and electronics, visual (and in turn logo) recognition is an important component of this research. To develop and validate the research, we built a large dataset covering more than 100 products with logos, and including about 500 training images for each logo. The dataset is setup by harvesting about one week of micro-blogs (many with images) from Twitter and Sina Weibo, as well as mining images from Google Image Search, Flickr, and videos from YouTube.

 

 

 Figure 6.1. Logo search on user generated images


 Figure 6.2. Logo recognition pipeline

 

In this research, we first classify the logos into image type and text type, depending on whether the logo has a specific image pattern like “Puma” or composed mostly of letters like “Nikon”, as shown in Figure 6.1. Figure 6.2 presents the logo recognition pipeline. We are addressing classes of logos in which most of state-of-the-art techniques tend to fail or perform badly. These classes include: (a) when Image logos are very small (fewer than 30x30 pixels in size) or have simple design which provides very few features for matching; and (b) when logos come with poor imaging conditions such as blurring, bad illumination and extreme viewpoint. To tackle these problems, we investigate both matching-based and classification-based techniques as well as incorporate context to help in logo recognition. So far, we have built the first prototype on image logo recognition, and are working towards text logo recognition. At the same time, we are working on visual brand tracking in microblogs and social video sharing sites in which many images and video have no or unreliable text annotations