Reverse image search


Reverse image search is a content-based image retrieval query technique that involves providing the CBIR system with a sample image that it will then base its search upon; in terms of information retrieval, the sample image is very useful. In particular, reverse image search is characterized by a lack of search terms. This effectively removes the need for a user to guess at keywords or terms that may or may not return a correct result. Reverse image search also allows users to discover content that is related to a specific sample image or the popularity of an image, and to discover manipulated versions and derivative works.
A visual search engine is a search engine designed to search for information on the World Wide Web through a reverse image search. Information may consist of web pages, locations, other images and other types of documents. This type of search engines is mostly used to search on the mobile Internet through an image of an unknown object. Examples are buildings in a foreign city. These search engines often use techniques for content-based image retrieval.
A visual search engine searches images, patterns based on an algorithm which it could recognize and gives relative information based on the selective or apply pattern match technique.

Uses

Reverse image search may be used to:
  • Locate the source of an image.
  • Find higher resolution versions.
  • Discover webpages where the image appears.
  • Find the content creator.
  • Get information about an image.

    Algorithms

Commonly used reverse image search algorithms include:

Image search

An image search engine is a search engine that is designed to find an image. The search can be based on keywords, a picture, or a web link to a picture. The results depend on the search criterion, such as metadata, distribution of color, shape, etc., and the search technique which the browser uses.

Image search techniques

Two techniques currently used in image search:
Search by metadata: Image search is based on comparison of metadata associated with the image as keywords, text, etc. and it is obtained by employing a set of images sorted by relevance. The metadata associated with each image can reference the title of the image, format, color, etc. and can be generated manually or automatically. This metadata generation process is called audiovisual indexing.
Search by example: In this technique, also called reverse image search, the search results are obtained through the comparison between images using content-based image retrieval computer vision techniques. During the search the content of the image is examined, such as color, shape, texture or any visual information that can be extracted from the image. This system requires a higher computational complexity, but is more efficient and reliable than search by metadata.
There are image searchers that combine both search techniques. For example, the first search is done by entering a text. The images obtained are then used to refine the search.

Video search

A video search engine is a search engine designed to search video on the net. Some video searchers process the search directly in the Internet, while others shelter the videos from which the search is done. Some searchers also enable to use as search parameters the format or the length of the video. Usually the results come with a miniature capture of the video.

Video search techniques

Currently, almost all video searchers are based on keywords to perform searches. These keywords can be found in the title of the video, text accompanying the video or can be defined by the author. An example of this type of search is YouTube.

3D Models searcher

A searcher of 3D models aims to find the file of a 3D modeling object from a database or network. At first glance the implementation of this type of searchers may seem unnecessary, but due to the continuous documentary inflation of the Internet, every day it becomes more necessary indexing information.

3D Models search techniques

These have been used with traditional text-based searchers, where the authors of the indexed material, or Internet users, have contributed these tags or keywords. Because it is not always effective, it has recently been investigated in the implementation of search engines that combine the search using text with the search compared to 2D drawings, 3D drawings and 3D models.
Princeton University has developed a search engine that combines all these parameters to perform the search, thus increasing the efficiency of search.

Mobile visual search

A mobile image searcher is a type of search engine designed exclusively for mobile phones, through which you can find any information on Internet, through an image made with the own mobile phone or using certain words. Mobile Visual Search solutions enable you to integrate image recognition software capabilities into your own branded mobile applications. Mobile Visual Search bridges the gap between online and offline media, enabling you to link your customers to digital content.

Introduction

Mobile phones have evolved into powerful image and video processing devices equipped with high-resolution cameras, color displays, and hardware-accelerated graphics. They are also increasingly equipped with a global positioning system and connected to broadband wireless networks. All this enables a new class of applications that use the camera phone to initiate search queries about objects in visual proximity to the user. Such applications can be used, e.g., for identifying products, comparison shopping, finding information about movies, compact disks, real estate, print media, or artworks.

Process

Typically, this type of search engine uses techniques of query by example or Image query by example, which use the content, shape, texture and color of the image to compare them in a database and then deliver the approximate results from the query.
The process used in these searches in the mobile phones is as follows:
First, the image is sent to the server application. Already on the server, the image will be analyzed by different analytical teams, as each one is specialized in different fields that make up an image. Then, each team will decide if the submitted image contains the fields of their speciality or not.
Once this whole procedure is done, a central computer will analyze the data and create a page of the results sorted with the efficiency of each team, to eventually be sent to the mobile phone.

Application in popular search systems

Yandex

Images offers a global reverse image and photo search. The site uses standard Content Based Image Retrieval technology used by many other sites, but additionally uses artificial intelligence-based technology to locate further results based on query. Users can drag and drop images to the toolbar for the site to complete a search on the internet for similar looking images. The Yandex images searches some obscure social media sites in addition to more common ones offering content owners means of tracking plagiarism of image or photo intellectual property.

Google Images

Google's Search by Image is a feature that uses reverse image search and allows users to search for related images by uploading an image or copying the image URL. Google accomplishes this by analyzing the submitted picture and constructing a mathematical model of it. It is then compared with other images in Google's databases before returning matching and similar results. When available, Google also uses metadata about the image such as description. In 2022 the feature was replaced by Google Lens as the default visual search method on Google, and the old Search by Image function remains available within Google Lens.

TinEye

is a search engine specialized for reverse image search. Upon submitting an image, TinEye creates a "unique and compact digital signature or fingerprint" of said image and matches it with other indexed images. This procedure is able to match even very edited versions of the submitted image, but will not usually return similar images in the results.

Lenso.ai

Lenso.ai is a web-based reverse image search engine that uses artificial intelligence to identify image sources and detect visually similar images, and perform facial recognition searches.

Pixsy

Pixsy reverse image search technology detects image matches on the public internet for images uploaded to the Pixsy platform. New matches are automatically detected and alerts sent to the user. For unauthorized use, Pixsy offers a compensation recovery service for commercial use of the image owners work. Pixsy partners with over 25 law firms and attorneys around the world to bring resolution for copyright infringement. Pixsy is the strategic image monitoring service for the Flickr platform and users.

eBay

ShopBot uses reverse image search to find products by a user uploaded photo. eBay uses a ResNet-50 network for category recognition, image hashes are stored in Google Bigtable; Apache Spark jobs are operated by Google Cloud Dataproc for image hash extraction; and the image ranking service is deployed by Kubernetes.

SK Planet

uses reverse image search to find related fashion items on its e-commerce website. It developed the vision encoder network based on the TensorFlow inception-v3, with speed of convergence and generalization for production usage. A recurrent neural network is used for multi-class classification, and fashion-product region-of interest detection is based on Faster R-CNN. SK Planet's reverse image search system is built in less than 100 man-months.