Digital communication platform to support community life

AI algorithms are playing an increasingly important role in a wide range of scientific projects around the world. At the Institute of Science and Technology Austria (ISTA), machine learning is also being used in several projects to analyse large data sets and gain valuable insights. By using AI, researchers can automate tasks that would otherwise be extremely time-consuming, enabling them to ask advanced questions in their respective fields. 

The applications

‘AI/ML is a complementary strategy that allows us to identify structures in our dataset that we would otherwise overlook,’ says Professor Sandra Siegert. Although she is generally positive about the use of AI in her work, she is aware of its limitations, particularly its tendency to reinforce biases.

"We are aware that data input is crucial. If certain data parameters are linked, this can distort the results. It is also essential to provide as much detail as possible about the experimental data. One challenge, for example, is that studies often do not describe the “gender” of the animal or only provide an approximate indication of the area of the brain from which the sample was taken. However, these are biologically critical parameters that will have a major impact on the data readout," Siegert notes.

In a completely different discipline, Assistant Professor Caroline Muller and her group Dynamic Processes in the Atmosphere and Ocean are less concerned about introducing bias. The group studies global high-resolution climate simulations.

They use AI/ML tools to examine these huge data sets and determine which aspects of a storm's environment cause it to become more elongated or more circular. ‘I think AI/ML has great potential in the sciences, as we often deal with large amounts of data. For example, some of the research in the geosciences relies on large datasets from satellite observations. These approaches allow us to process large amounts of data very efficiently,’ says Muller.

‘We use AI/ML to understand physical processes, such as the development of clouds and storms. We don't use AI/ML for predictions, so we're not too concerned about bias and error. Our main focus is on interpreting the AI/ML results and ensuring that they can be understood from physical principles,’ she adds.

The hype

As with all aspects of AI, it is important to understand the hype surrounding it. It is commonly assumed that AI is a one-stop shop that can bring about a paradigm shift in everything we do. This is far from the truth. At best, current AI models can read a large amount of data, create a probability distribution, and suggest something as the most likely explanation for what that data suggests. At worst, the results are flawed.

In addition to the moderate use of deep learning in several projects, mathematicians and computer scientists at the ISTA campus are also working on improving AI itself.

Professor Christoph Lampert, whose work focuses on machine learning, says that ‘AI has the potential to support and take over many menial tasks and serve as a tool for increasing motivation and creativity.’ At the same time, he warns that ‘AI is not about solving problems.’ Rather, its role is to automate tasks, ideally in such a way that the result is no worse or even better than if a human were to do them.

He cites two famous examples: AlphaGo, which plays the game of Go better than any human, and AlphaFold, which predicts the three-dimensional structure of a protein as soon as it receives information about the amino acid sequence that makes up that protein.

AlphaFold is a vivid example of how AI can support science, but also one that shows that you can't get very far with a single tool in your shed. AlphaFold only offers a hypothesis for the structure of protein folding; the proposed structure still needs to be verified experimentally.

According to Lampert, one area where more research is needed is AI's tendency to act as a multiplier of existing biases. ‘We see this particularly in the recently developed large language models (LLMs) such as ChatGPT, which are predominantly trained on (often biased and inaccurate) internet data,’ he says.

This is also one of the areas targeted by ISTA's newest assistant professor in this field, Francesco Locatello. His research focuses on advancing AI and machine learning to understand cause-and-effect relationships. This is the next big step in the development of artificial intelligence: causal AI.

Until now, AI technologies have struggled to process causal relationships, cannot distinguish between coincidence and correlation, and are not yet very reliable. Locatello and his research group want to change that.

Marco Mondelli, assistant professor and head of the Data Science, Machine Learning and Information Theory group, also noted that trustworthy AI and model robustness are important research topics. Recent work from Mondelli's group predicts the robustness of high-dimensional models (with over millions of parameters).

This work has the potential to help users predict which model is theoretically best suited. "The next generation of questions will involve high-dimensional problems: both models and datasets are getting bigger and bigger, and this size creates enormous practical problems. Training large models (e.g., LLMs) is now something that only (very) few technology companies can afford to do. I believe that a theory can be helpful in this regard by providing precise guarantees for problems in high dimensions," says Mondelli.

Efforts are currently underway at ISTA to solve the problems associated with the size of the models. The Alistarh group recently presented their work describing their SparseGPT pruning method, which reduces the size of large models without loss of accuracy. While the model behind ChatGPT remains proprietary, other large language models like this one have been made openly available – and the public is eager to experiment with them.

With a recently awarded ERC Proof of Concept Grant, Alistarh and his team now want to bring their approaches even closer to potential users.

"Our techniques reduce the overhead of distributed training of ML models, which can be very high for very large and accurate models. We are now bringing these methods closer to practitioners. We are doing this by building a software library that enables them to train large AI models efficiently on standard computers.‘ In this context, Alistarh emphasises the impact of his research on the ’democratisation" of AI.

The missing theory

Professor Herbert Edelsbrunner is a mathematician whose work in topological data analysis aims to improve AI tools. It is a niche area, as the mathematics that Edelsbrunner and his group use is not typical for the AI research community.

Edelsbrunner believes that ‘the gaps in theory are enormous. In short, we don't know why deep learning works as well as it does.’ ‘The current wave of AI is based on very successful experimental work that builds on a lot of theory,’ says Edelsbrunner. ‘But the theory is lagging behind, and there is an urgent need to develop it further.’ According to Edelsbrunner, there is currently a lot of experimental work that finds applications in people's everyday lives, but this work is poorly understood and unpredictable.

Here he agrees with Lampert, who also believes that the pressing questions in AI research currently are to understand how and why AI systems actually work, rather than just building them; how to develop more natural AI systems; how to develop more efficient AI systems; and how to use AI systems for good.

ISTA researchers greatly appreciate the potential of AI, but point out that AI has not yet been precisely defined. What is considered AI has changed dramatically over the past few decades. The question of definition is not usually asked, either in the popular conception of AI or in AI research. Although the available AI tools seem to be becoming richer by the day, a cautious and reserved approach to their reliability is the best recipe given the current scenario.