This is how Facebook’s AI improves photo descriptions for the visually impaired

Facebook has been introducing functions in Spain for some time to make the use of the social network easier for people with a disability, such as visual impairment. The social network has made public the operation of your image recognition system, which will allow visually impaired people to hear a description of a photo. Even if the user who has uploaded the photo has not put a text.

Since 2016 Facebook has an automatic alternative text (AAT) technology which is capable of generating a description for a photo. A new version of this technology has been released, where there are various improvements. Among them is the best ability to recognize faces or the number of people in a photo.

The latest Google accessibility experiment you can try: so you can speak just by moving your eyes

Google has created an application so that people with reduced mobility can speak using only the movement of their eyes, you can now try it.

Facebook knows what there is a photo

Facebook’s AAT technology allows you to recognize objects in a photo, so that you know if there are people and how many, the objects present, animals or the place where the photo was taken (if a monument or place of interest is recognized in it) or say if it has been taken indoors or exterior, for example. To make this possible, a neural network is used that has been trained with millions of examples and parameters.

This technology can even detect relative size of objects

. You can tell if there are three people in a photo, the position of those people in the photo and analyze the position of each of them with respect to the space in which said image was taken. You can also determine which are the most prominent or main and minor elements in a photo. In addition, it is increasingly accurate in this type of analysis.

Initially, Facebook’s AAT was able to recognize about 100 simple concepts (tree, mountain …), while currently there are more than 1,200 concepts that you can recognize. To train this technology, the social network uses photos both uploaded in its app and on Instagram, using their hashtags. This large database allows them to recognize a multitude of elements, such as types of food, events, the activity they carry out, the skin color or gender of people.

This Facebook technology mainly seeks help visually impaired people, issuing an accurate description of what is seen in said image. Although at the same time the enormous amount of data that the social network collects to be able to know what is in each photo precisely is surprising. According to the social network, these descriptions are available in a total of 45 different languages.

The entry How Facebook AI improves the descriptions of photos for the visually impaired appears first in The Free Android.