Born and raised in Serbia, with a heritage which is part Montenegrin and part Bosnian, computer scientist Petar Veličković’s academic journey has taken him all the way to being an Affiliated Lecturer at Cambridge and a Staff Research Scientist at Alphabet’s DeepMind.
Veličković’s research during the past years has been focused on teaching machines how to assist humans with proving difficult theorems and creating new approaches that would tackle different open problems and issues. One of these examples includes implementing graph machine learning mechanisms to estimate travel times in Google Maps, a project that Veličković has been working on during the past period.
AI has the potential to revolutionize many different fields, such as healthcare, transportation, finance, and agriculture, and one of the areas where Veličković is most known for is graph machine learning and the industrial and scientific application of such methods. He also presented parts of his research during the AI2FUTURE conference held in Zagreb, Croatia earlier this year.
In an interview for The Recursive, Veličković reflects on his career, the practical use of AI and ML in different industries and sectors, and the potential of the Balkan region to develop and implement such technologies.
The Recursive: How did your academic journey begin and what sparked your interest in graph machine learning?
Petar Veličković: I finished high school in Belgrade, and after that, I got a fully funded undergraduate position in Computer Science at the Trinity College of the University of Cambridge. Then, after my bachelor’s in Computer Science, I went straight for a doctoral degree at the same university to study Machine Learning and Artificial Intelligence.
And after those three years, I joined DeepMind as a full-time research scientist. I’m also an Affiliated Lecturer at the University of Cambridge, where I teach a Master’s course on graph machine learning.
My general interests are in geometric deep learning. So, basically trying to build machine learning techniques that work on nontrivially structured data or data with some interesting geometric structure with it, specifically focusing on the ways that such models can be used to imitate or augment reasoning systems.
Additionally, I am quite active in scientific and industrial applications of these methods. Perhaps the ones that I’m most well known for include using these graph machine learning mechanisms to estimate travel times in Google Maps. I was part of a research team at DeepMind that built a graph neural network model that is now deployed in production in Google Maps.
So, wherever you are, whenever you ask Google Maps for the travel time from point A to point B, the travel time estimate that you get is served by the graph neural network model that we have developed.
We have also worked for the past few years on using these graph machine-learning models to uncover hidden structures inside mathematical objects. In collaboration with several top-tier mathematicians from the Universities of Oxford and Sydney, they use the tools that we built to come up with brand new top-tier mathematical results and conjectures in very diverse areas of mathematics.
These results were just by themselves very significant. And they were both published in top-tier mathematical journals. But the overall significance of the idea of using AI to find the hidden structure of mathematical objects, and help humans know where to look when they want to derive new theories about them, led to our work being published on the cover of Nature last December.
How would you describe the geometric deep learning architectures and how are they used in practice?
Geometric, in this sense, primarily refers to the existence of some interesting structure in your data that you might want to exploit. One very common example is if you have data, which looks like a graph, but every single node of that graph also has some spatial coordinates in a 3D space.
And in those kinds of settings, very often, there are certain things you can do to these coordinates such as rotate them, translate them, and things like that. And you should still expect the same results when you apply such transformations. One very common area where these methods are being applied is in bioinformatics and chemistry, where the basic unit that you’re studying is the molecule, and molecules are obviously graphs of atoms connected together by chemical bonds, and each atom has rough coordinates in 3D space.
So, you can rotate the entire molecule and it is still the same molecule, right? And geometric deep learning allows you to build models of these molecules such that you can predict interesting stuff on them. For example, in biotech and drug discovery, you can use these models to predict could this molecule be a possible cure for a particular disease, or will certain pairs of drugs interact well together or cause a negative side effect, or something like that.
Geometric deep learning allows us a nice way to process these kinds of data in the machine learning system while being mindful of the fact that there are these rotational or geometric symmetries that we might want to take into account.
What are your thoughts on the potential that AI has and its implementation in different industries and sectors?
AI can have a pretty transformative impact on a lot of industries and there’s a variety of ways in which this can be accomplished. I think one side where we’re already seeing a great deal of success is building specialized systems that are optimized for a particular downstream application.
So, build a machine learning model that is very mindful of the specific scientific or industrial tasks that you’re solving. And then making sure to optimize it for the specifics of that problem, doing certain things—such as scaled-up search—that humans would not be a very good fit for. And therefore automating away some parts of the process that might be really hard, really expensive, or really tedious.
I think the best example of that is the AlphaFold model, which was a geometric deep-learning model applied to protein folding by DeepMind. This basically caused a huge step change in how accurately we’re able to predict what proteins will look like, based on just their one-dimensional sequence alone without having to do expensive and slow crystallography experiments. Now, suddenly we have a much faster way of inferring structure for these very important biomolecules, which govern basically all of the processes inside the cell. And therefore enable all sorts of industrial applications when you have a potent model like this.
These are specialized solutions, and I think we’re already seeing a lot of great success stories like that. Of course, there’s also been a lot of chat around building models that are just generally powerful and therefore can help in a variety of ways.
Specifically, these are models that have been trained on the entire contents of the Internet, for example, and can therefore serve as a sort of resource that captures all of that knowledge in some human-digestible way.
I think we still have a long way to go before these models are reasonably robust to be very industrially viable. However, I think they already can be quite transformative, for example, as brainstorming companions. Because they have access to so much more context than we, as humans will be able to read in our lifetimes, they can be a pretty good candidate since you just prompt them by asking them certain queries, and they might give you some inspiration for what to look at or what to try.
While the answers may not be very correct, or very grounded, they might still inspire you to know where to look or to know what to explore further. And I think that can be quite invaluable.
What is the potential for developing these technologies in the Balkans and how can we use AI and ML to improve the quality of life?
Our countries generally have a lot of raw talent. So we have a lot of people who are really strong in these areas, especially their mathematical foundations. I think the main limitation behind why maybe we’re not growing as much as we should, as a function of that talent that we have, is that we also have a bit of a mentality that is not very incentivized to seek opportunities that students in the West might be more incentivized to seek.
For example, when I came to study at Cambridge, I had access to a lot of offers for internships, to do research placements. And in such an environment, you generally start to get the feeling that you’re entitled to those things.
Whereas I think in Eastern Europe, we don’t get as many opportunities as this on a regular basis. And naturally, people might feel like they’re less worthy of a particular offer, even though they might even be technologically stronger than some applicant that comes from the West.
I think we need to become more encouraged and break the ice more often. And this might allow us to grow much faster as a community, because we have a lot of talent, we’re just not necessarily always encouraged to deploy it in the right way in specific industries where it can be quite beneficial.
There are a lot of aspects where we could build specialized applications of artificial intelligence for our own, say, language area. So a lot of the existing state-of-the-art language modeling solutions are for the English language or other more commonly spoken languages, we could try to call upon our specialist skills to improve these models in the context of our own languages in our region.
We can build models that are predictive of the kinds of industries that are currently most actively present in the region. In the example of Serbia, we have quite a few strategic aims to have good machine learning models that could be deployed in agriculture or to forecast electricity usage.
Additionally, biotechnology is another area where we can be quite transformative since it requires a lot of people with good domain expertise, which we certainly have. Further, biotech research is often not really a function of who has the best resources, or computers to train these systems. It’s still more of a function of how smart your solution is, and I feel like this is the kind of problem space where we can more easily have a transformative impact.
What would be the emerging AI and ML trends that we can expect to see in 2023?
The main trends in AI and ML are going to be connected to the powerful general-purpose models out there. I think there are going to be a lot of exciting products being built over the coming years that just make advantage of the amazing tech we already have.
And that will start to convince more and more people of the utility of AI in industry. Also, I think we’re also going to see some pretty big scientific advances just generally with the use of AI because there are so many great specialized examples that some people in the past were much more skeptical of, but now are starting to buy more and more into those technologies.
Obviously, the more people looking at these applications, the more likely it is that we’ll have some breakthrough results.
The part that I am personally most excited about, and it relates well to my area of research, is that I would really like to build the next generation of really powerful models that, in the long run, are not only able to give kind of a nice, plausible-sounding summary for various questions that they get asked but also to try to really carefully ground their answers in facts and to be stable under certain ways in which these questions can be asked to them.
