In March 2016, at the annual Build conference, Microsoft showed off a technology that was impressive in many ways even more than anything that technology showed that day. The touching video told the story of a blind programmer of the company who created unusual glasses with built-in artificial intelligence that can scan the surrounding space and describe it in words.
This technology is called Seeing AI and is now available to iPhone owners as a separate application of the same name.
The principle remains the same: by opening the program on your smartphone, you can point the camera at an object - the program will recognize it and tell you what is in front of you. You can even catch a person in the frame, and Seeing AI will tell if he is sitting or standing, smiling or frowning, young or old. At home, it will help a blind person understand what kind of product he took from the shelf - for this you just need to scan the barcode from the box. But one of the most useful features of the app is currency recognition. Since US dollars are the same size and it is simply impossible to tactilely distinguish banknotes of different values - as well as to understand how much you got out of your pocket - Seeing AI solves this problem.
The application is built on the basis of neural networks, this technology is used, for example, in self-driving cars and image recognition programs. Most of the data is processed by the processor of the smartphone, so almost all functions work without a permanent connection to the Internet. However, for some, access to the network may still be needed - for example, to recognize handwritten text or scan a complex scene.
In a short interview with The Verge, Sakib Shaikh, the head of Seeing AI development and the same blind programmer from the video, said that he most often uses the application to identify road signs and read menus. In the latter case, the program not only voices what is written on paper, but also tells you when and in which direction a person needs to move the camera in order to “read” further. And Shaikh considers the speed of work to be the main advantage of his program: for example, Seeing AI recognizes people's faces in a split second.
The application is still available only to iPhone owners, and you can download it only in the American App Store. The program does not speak Russian either, however, given the usefulness and ambitiousness of the project, the appearance of Seeing AI on Android and in app stores in other countries can be called a matter of time.>