AI is firmly back in the news
The recent surge of interest in AI is evidenced by OpenAI’s ChatGPT and image-generating systems such as MidJourney and Stable Diffusion. This is a positive development, since the revolutionary changes that will occur in the coming decades should not take us by surprise.
One of the leading companies in the AI field is IBM. One of its senior executives, Alessandro Curioni, recently spoke on the London Futurists Podcast to discuss IBM’s current AI, quantum computing, and related projects. Curioni has been with IBM for 25 years, and is an IBM Fellow, Director of IBM Research, and Vice President for Europe and Africa.
IBM’s grand challenges
IBM has been inventing the future of computing for 70 years. It has achieved a series of impressive grand challenges, including Deep Blue’s victory over Gary Kasparov in chess in 1996, Watson’s win against Ken Jennings in the TV quiz Jeopardy in 2011, and a machine capable of debating a world debating champion in 2018.
The first of these machines were rules-based, and the later ones used deep learning, based on training models on large data sets. The new paradigm is large language models (LLMs) or foundation models, which use self-supervision to train. This involves randomly masking one word from each sentence in a huge number of sentences from the web, and trying to guess what the word is. This approach can be applied to any kind of structured data, such as images, videos, computer code, industrial process data streams, and the language of science.
Narrower focus
IBM is building large language models, but for specific applications as opposed to general use, like ChatGPT. For instance, it is developing systems specialising in organic chemistry and business. Specialised machines are more capable of going deeper and are less brittle, as they can use better quality data and remove bias more easily.
One of the reasons why ChatGPT performs better than GPT-3 is the use of Reinforcement Learning with Human Feedback (RLHF). OpenAI, which created these systems, hired many people to comment on the system’s output and label any biased or offensive passages. This does lead to the joke that AI stands for Affordable Indians, but the humans are only used during the training, not in operation.
IBM hopes to create a large model in a particular domain which can be trained on the proprietary data of client organisations in that domain. This would be a major improvement over the traditional approach of developing a new model for each application.
More efficient chips designs
IBM is also looking to improve the efficiency and sustainability of AI and computing with regard to chip design. Large language models approach the scale of computation in the human brain, yet they use the same energy as a small town, while the brain uses the same energy as a lightbulb.
Curioni states that IBM is taking three steps to reduce the power demand of advanced AI systems. The first is neuromorphic chips, such as IBM’s True North and Loihi, which are modelled more closely on human neurons than traditional chips. The second is memristors, where process and memory storage take place on the same chip, reducing energy used for retrieving and re-storing data. The third is spiking neural networks, which only transmit information when the particular function is required, unlike traditional chips where each neuron transmits information all the time.
These three steps can bring about two to four orders of magnitude improvements in energy efficiency.
Breakthrough in quantum computing
IBM is also a leading player in quantum computing, alongside Google and Microsoft. It recently announced a breakthrough in quantum cryptography which will enable data being transmitted today to remain secure even when quantum computers are built that can break today’s encryptions.
Together with a number of academic partners, IBM developed a new type of encryption called quantum safe crypto, which is based on high-dimension lattice cryptography and is believed to be unbreakable by quantum computers. Over the last decade, a large research programme was conducted to assess potential types of quantum safe crypto, and last July, four algorithms emerged as the strongest. Three of those four were developed in Curioni’s laboratory in Zurich, and the winner has just been selected.
The next step is migrating data from old encryption forms to this new form. This task is becoming urgent due to the ‘quantum apocalypse’ paper published in December 2022 by a team of Chinese researchers, which suggested they had worked out how to breach today’s encryption technologies. It was quickly realised that they were not all the way there, but it may not be long before someone does achieve it – perhaps in the next two or three years. The US government has directed that all its agencies must be quantum safe by 2025, and other governments and companies are doing the same. IBM’s breakthrough may have come just in time.
