Amazon searches and the future of AI in libraries

 Hi All--

There is an interesting report by two of Amazon's researchers about how it ranks its searches. This study was conducted by Daria Sorokina and Erick Cantu-Paz. Daria went to Lomonosov Moscow State University for her master's degree and then to Cornell University for her PhD. Erick went to Instituto Tecnológico Autónomo de México for his bachelor's degree and then to the University of Illinois Urbana-Champaign for his PhD. They both worked for Amazon--A9, which developed Amazon's search engines for all of its sites--when the report was written and published the report for an international conference where it was reviewed.

Amazon's report is specifically about how Amazon ranks its products in search results to be relevant to customers. Amazon trains its searches. This means that it focuses on how consumers react to its results and modifies them to be popular with them. The researchers' methodology was to record consumer reactions--purchases--and then, through several models, produce more relevant results. An additional method was to count searches to see whether the relevant search, the purchase, happened with relatively few searches.

Search engines can be quite complicated because they must be the store clerk and the store all at once. That means that they must guess what the customer wants and so must be able to interpret language and cues from the customer. Thus, search engines must use natural language processing (NLP) to judge whether, for instance, a customer wanted a "casual dress" as in something a girl might wear and not dress shoes. Search engine calibration was done partly beforehand and also live through customer responses and data analysis. There were, apparently, discrepancies between what consumers wanted at the top of the page and what consumers bought.

This is applicable to public library technology use because we use search engines all the time and, increasingly, AI. By training searches and correctly interpreting prompts, we can redirect to the right book, for instance. Thus, NLP is very important for automated understanding of when people do not know which book they want. AI is a major improvement on more traditional Google-like keyword searches but can often misunderstand major changes in prompts--it's an inductive machine, primarily--and hallucinates. Time will tell whether further improvements will fix this.

--Nicholas Bullen

References

Cantu-Paz, E. & Daria, S. (2016). Amazon search: the joy of ranking products. SIGIR.  http://doi.org/10.1145/2911451.2926725