Hey Siri, find me a Chinese restaurant..Ahh, no Italian. And make a reservation for two for 8 PM this Sunday. No, wait. Saturday evening.
This command can cause confusion in every reader, but the computer understands it. And acts on it. What kind of technology makes this possible? This is what Prof. Dragomir Radev, Professor of Computer Science at Yale University and Director of the Natural Language Processing (NLP) lab at Yale, went into detail to explain.
Being the only Bulgarian professor at the prestigious Yale University, Radev is also the co-founder of the North American Computational Linguistics Open Competition and has coached the US national team in the International Linguistics Olympiad to several gold medals. He was back in Sofia, Bulgaria last month for the INSAIT conference Emerging Trends in AI and Computing Research where he delivered a keynote on AI, componential models, and NLP.
Dragomir Radev graduated high school in Sofia, back in 1986. His passion for linguistics was ignited back then when he first tried competing in national contests, but his true immersion into computer linguistics and NLP research began when he left to study in the US in 1991. For years, he has worked on text summarization, semantic parsing, natural language interfaces to databases, sentiment analysis, and question-answering. His long-term vision is to build an infrastructure for computers and humans to interact in a fluent and natural way, as well as to use NLP to develop educational and medical applications.
Words into numbers? The science behind NLP
“The computer is created in such a way that it can only think in numbers,” Radev begins explaining the concept from the basics. “Their memory is made of bits and bytes and every bit corresponds to either 0s or 1s. Let’s dig a step further. Think about a simple picture that you see on your computer. This picture represents a specific matrix that combines millions of pixels. And every pixel corresponds to a specific color, thus a specific set of bytes,” he goes on.
The same is true for words, as he clarifies. Computers are not tailored to accept and process information in a text or semantic format. Thus, it has to translate everything into numbers, or as computer scientists call them – tensors. A tensor is a concept from linear algebra that corresponds to the notion of a vector.
“When I say, or type something in Chinese or English, the computer has to automatically translate this into numbers. Every single word is represented in the form of a specific tensor and the computer is then able to train itself and draw historical analogs to build on its semantic knowledge,” Radev exemplifies, adding that by using compositional semantics the computer makes bigger vectors of words that form word combinations. “Thus, it knows that the verb after the word “dog” would be “bark”, not “purr”. All of these complex mathematical and computational processes are part of the standard method for training and validating any type of AI, including NLP,” he says.
How often do we interact with NLP?
Every day. And Dragomir Radev explains what trends lead to the widespread use of this technology.
“Natural language processing is gaining so much interest these days for a couple of reasons. First, there is a huge set of data that can be used to train computer systems. Companies such as Google and Microsoft have access to virtually the whole internet which means lots of text. Second, computers have become super powerful, and third, globalization has created a need for cross-cultural conversations. Russian, Chinese, and Spanish are popular languages but think about having to communicate with someone from Latvia or Bulgaria,” he shares.
Basically, all the search engines we use are based on NLP. Every time we go on Google Maps, use personal assistant apps, or voice commander and messaging, we interact with NLP. What has changed in the past decade though is the maturity of the technology and its intelligence. We all remember making fun of the automatic translations of Google, but today, besides understanding the grammatical, modern search engines understand the semantics of our search intent.
The unexplored opportunities for NLP
As Radev breaks down, NLP is beginning to find use cases in the financial sector. Various banks, and investment funds extract relevant financial information from news outlets and use that to translate it into numbers, feed data into mathematical equations, and retrieve predictions and advice for investment decisions.
Another vertical is Healthcare. Additionally, there are verticals where NLP is already widely used but there is still substantial room for improvement. “In e-commerce, for example, the dialogue regime can be done a lot better. Dialogue technology means that computers can answer questions based on previous communication without having the full context of every specific question. Currently, Microsoft, SalesForce, and Facebook are all working on such dialogue technologies,” Radev shares.
How is university research commercialized?
University labs write publications, and make prototypes, but do not go into integrating these solutions into businesses. However, pretty much every piece of research, code, and findings done in university labs is put into a GitHub repository and becomes available for public use. Individuals and companies can then use the code to integrate into their products or tweak and improve it in a way that will fit their technological needs. Nowadays, more and more companies such as Google and Facebook have started to open their own internal laboratories.
The deep tech potential of Bulgaria and the role of INSAIT
“When Martin Vechev, the Architect of INSAIT, reached out to me around 2 years ago with the idea that Bulgaria can become a place that gathers world-class computer and AI scientists, I immediately resonated with his vision. It’s worth noting that the main goal of INSAIT as a research center is not to produce applications, but publications,” Radev says.
The quality of every research center depends on the people that work there and its prestige is judged by the number of publications and research citations. Unfortunately, now in the computer science research field, there is no presence of Bulgarian research labs. However, there are many Bulgarian researchers who work abroad and have been recognized worldwide.
“Bulgaria sits at the center of SEE and there is great potential to attract top scientists from Greece, Romania, and even Poland and the Czech Republic. We have amazing talent that is forced to leave and work in the UK or Switzerland because they cannot find the same opportunity here. With INSAIT, the future deep tech and computer science researchers will be able to develop as world-class researchers in Sofia,” Dragomir Radev concludes.
