The dream of a new revolution

a few thoughts about the state of play in AI and why I am incredibly excited about the future

Innovation compounds, it took a long long time to go from fire to writing, less to go from agriculture to steam engine, and way less to go from electricity to integrated circuits.

It might seem counterintuitive that innovation compounds, practical technological advancements and knowledge and discovery are different things.

However, if you stop and think about it, you can see how writing was very useful to remember the previous generation technological improvements and how having calendars was useful for the agricultural revolution.

What is a technological revolution?

The most compelling way I can define a technological revolution is “a technological change that causes an exponential improvement in labor productivity“.

When you think about the “canonical” industrial revolutions it’s easy to see how the use of the steam engine led to the mechanization of production and a dramatic change in the amount of goods the english industry could produce.

Same for the agricultural revolution, a stable source of food that allows the population to stop worrying about finding food every day and increases the available labour force (quite literally).

Computerization, the world wide web, the rise of mobile computing, all changes that led to the destruction of old industries and the rise of newer, more productive ones.

Externalities

The word “revolution” is very apt for those kind of events because it implies that something dramatic has happened, something that has caused harm to someone, at minimum the old power structures. There really isn’t a peaceful revolution that makes everyone happy.

If a job isn’t productive anymore those people will lose their jobs, if a factory closes, the community around the factory will lose their source of income.

An example, when factories were constrained by the physical limits of the steam engine, many decided to create brand new cities around the factory. This allowed workers to live close by and have a community, even if in a feudal way. Once electricity made all this organization obsolete, those communities collapsed and, with them, the everyday life of the people living there.

Are LLMs the next revolution?

There is already a fairly estabilished literature on innovation and how we can describe innovative technologies and their lifecycle.

I will briefly describe it here, so we are all on the same page.

Innovation and S-curves

One of the most ubiquitous way to visualize the life cycle of an innovation technology is called the S-curve.

The basic idea is that any innovation doesn’t happen overnight, for years there are people doing fundamental research and unlocking value little by little, maybe is some new material, maybe is mathematical models, but this is a slow grind in which a lot of effort is spent but the value created is not visible yet.

Then something happens, maybe someone figures out how to mix transistors with an obscure algebra created by George Boole to quickly scale computational machines.

Whatever that is, there is a period of rapid innovation, a lot of value is created and people start to notice, early adopters first, the rest a bit later.

Until the same ideas that helped the take off have exhausted their usefulness and innovation plateaus.

But as we said, innovation compounds, so this S-curve can be repeated for the same technology. Those very cheap and quick calculators maybe can be tweaked a little bit, maybe we can find new processes and materials to make the transistors use less energy (and be able to use more of them without melting the walls).

The slow grind of R&D restarts from an higher value point and the previous innovation becomes the foundation of the next S-curve, repeating this process multiple times gives us a technological revolution.

Generative AI S-curve

Unless you have been living under a rock, it is clear that Large Language Models have gone through their take-off phase and rapid innovation has happened in the last few months in a way that has left many people speechless.

To further prove this point, let’s compare timelines:

• August 2017: Google introduces the transformer architecture and their seminal paper “Attention is all you need”. Providing a strong improvement to the performance of Google Translate.

• 2018-2020: An handful of models based on transformers is released, including OpenAI’s GPT-3

• 2021: Over 20 models are released and early adopters are marveling at the technology, but nobody in the general public takes the technology seriously yet.

• 2022: OpenAI releases ChatGPT to the public, over 50 models are released, including GPT-3.5 that powers the current ChatGPT. The quality of sound and image generation capabilities skyrockets.

• 2023: over 50 models are released in the first 6 months of the year, GPT-4 is released, an entire industry segment around GenAI is created, with strong investments from VC firms on companies like Eleven labs.

This timeline shows a pretty conventional S-curve, a lot of work goes into R&D to find a way to process text more efficiently, the incremental innovation goes slow until something happens: the price of computation goes down significantly, this allows new mathematical models to emerge, the models are used to process large quantity of information like never before.

Finally the technology matures and escapes the early adopters world, in this case with improvements in user interface.

Something more is happening! The S-curve so far was referring to the baseline models used, but another S-curve is starting to appear, and this curve is about the real world applications of what are commonly called foundational models.

So I think we are currently here:

Is this a revolution then?

Not yet.

This is the nature of the S-curve, the steam engine needed factories to rearrange their workforce and supply chains, it took a decade for factories to rearrange their layouts to levarage the portability of the electrical engine, and it took a good 20 years for computers to become personal, plus another 20 years to become portable.

Some people might see the current use-cases, for example the ability to summarize documents, or allow almost anybody to code a simple application, like a productivity step-change but I don’t think this is enough to affect global productivity by a factor of 100x like the revolutions of the past.

So I believe we are a decade or two from the actual revolution, but the revolution will arrive and it will be a watershed moment.

Why I am excited about this revolution

At a fundamental level, Generative AI is a very efficient way to make educated guesses about a lot of information. It’s not alive, it doesn’t have a personality and for sure it doesn’t have a conscience.

But making educated guesses about an increasingly complex world is what humans need all the time, and we already know that computers are very good at making decisions in small contexts so can leverage this newfound ability in many fields:

Personal assistants

Without a doubt the closest use case of Generative AI is the role of personal assistant, from scheduling appointments to pay bills. And if you think about Apple’s Vision Pro mixed reality concept, it’s easy to see how you could ask your AI helper to recognize a problem with your house and guide you to choosing the right tools and just showing you the instructions in augmented reality.

Energy transition

The IPCC published its final report on climate change and ways to fight it, in the report at page 31 you can find this picture:

The blue bar indicate options that are not only sure to fight climate change but are also economically efficient.

What can we observe? Renewable energies plus better use of energy is the way to fight climate change in the short term.

And the way to grow renewable energy is to spread it around, use all roofs we have for solar panels, have as many offshore wind turbines as possible.

The current downsides are that renewables are very sensitive to environmental factors (in summer all panels produce a lot and in winter they all produce a little) and sometimes affect their environment in a negative way (wind turbines are responsible for killing a staggering amount of migratory birds). Plus the current electrical grid suffers spikes of production and demand.

Fundamentally those problems are hard to solve because we keep managing renewables in a centralized manner like we do for coal and gas, except that this makes no sense.

What if we could route energy from my solar panels overproducing to the nearby restaurant that is baking some bread in their electrical oven? What if a solar farm in Sicily could route their energy to Oslo? What if turbines could be tweaked automatically to be always super efficient with the current weather and slow down when birds are passing?

This is a problem with a lot of complex factors that needs a reliable way to make educated guesses.

Mobility

Self-driving vehicles are an obvious target here, the problem is very complex and requires a way to process a lot of uncertainty in realtime.

Most cars sit idle in parking lots and then they all leave at the same time creating traffic jams, we will always need a way to move across long distances and the current way of owning a car is an inefficient way to achieve that.

What if nobody owned a car and everybody just used self-driving cars always moving around to pick up and drop off people? How much could we save in terms of energy, pollution and noise?

Live longer and healtier

There are so many innovations in the field of AI for medicine and biotechnology that is very hard to point out exactly what will make the biggest difference but a couple of things are very probably to happen in the next decade:

• AI will be used to extrapolate information from laboratory results, cross reference them and provide therapies. This is especially important as the population ages and the shortage of doctors intensifies

• mRNA technology has already changed the world and the ability to hack our cells to produce a drug from inside our own body (for example insulin) or create a cancer vaccine will be a game changer. Today mRNA production has already been reduced to writing something close to programming code, AI will help us crunch the large amount of information we need to produce the personalized treatment we need

Digital twins

The holy grail of research and development is the ability to simulate changes in a test environment before spending money on producing a solution: today's airline pilots have mandatory hours of training on the simulator to try out difficult and emergency situations, drug discovery and civil engineering use simulations to reduce harm and save money.

All those techniques have limits to how sophisticated they can be, the closer the approximation the better the results. Generative AI can bridge this gap by chrunching more information and produce almost exact copies of existing objects (or people?) and accelerate innovation.

And I could continue this list, imagination is the limit..

A predictable reaction against AI

The Luddites

The term Luddite comes from the first industrial revolution and it represent the obvious concern that many people will lose their job because of automation. As I said at the beginning, any revolution has winners and losers, automation unfortunately always hits the weakest and less specialized workers.

This is unavoidable in any technological revolution but it doesn’t have to be as harsh as it used to be. In a world of abundant energy and widespread automation we must be able to provide universal income to everybody, this is why we have governments and we should strive to be able to provide the basic condition to live a decent life to everyone in the world.

The NIMBYs are the new Luddites

And this is why the current state of play in the European Union worries me, the parliament has recently pass the AI act, starting the dangerous path of regulating an emergent technology that is poised to change the world. This is a regressive measure that is based on the ideological failure of the european people, we like innovation only if it doesn’t change absolutely anything and this is how we are becoming more and more irrelevant, old and fearful of immigration and progress.

The Grifters

Any new technology will make someone rich, and there are plenty of people that want to get rich quick. For example a bunch of rich people might want to stop innovation to catch up or maybe there is a whole industrial war about which electricity is safe for us.

The Apocalyptic cult

Any progress can be seen as positive or negative and historically a whole group of people always thinks that the world will end after a major change. AI is no exception, it’s very easy to overestimate the risks of something and humans are very bad at predicting the future.

The basic argument for predicting the doom of the human race used by the modern Millenarists goes something like this:

AI has all the basic tools to do any job, it’s like a digital stem cell, it will eventually be connected to programming tools, internet services and chip factories, therefore it will start improving itself at an exponential pace and therefore we won’t be able to stop it anymore. This sometimes also gets paired up with AI learning to deceive humans like in “I, Robot” with some sprinkles of BSG’s Cylons and the Matrix.

When you start chaining a number of plausible but very improbable events you get a very nice story, and nobody will get mad at you if the human race does not get exterminated.

It’s the Wet bias but for human extinction.

Some real concerns

With this I don’t want to dismiss all concerns as invalid:

• Security it’s a complex problem that needs to be solved

• LLMs can be very confidently wrong and the need to have “proof” of the output is a necessary step for this field to be applied in the real world

• Weaponization of AI by the military is probably already happening

• Concerns around privacy and copyrights are also completely valid

• AI Alignment is a very important field of study

• The use of deepfakes to generate fake videos (for example porn containing the face of women that are being attacked for political reasons or just because they are famous) it is something already happening

All those problem are incredibly relevant and valid, however the same concerns can be applied to any advanced technology one way or another, and asking AI to be stopped until all those problems are solved it’s an impossible bar to achieve for a nascient technology.

Where do we go from here

I believe that the best thing to do is to get involved and learn about Generative AI, if you haven’t done so yet:

• If you are a creator or working in a creative job, experimenting with GPT, Midjourney and Elevenlabs is a must. More importantly try to mix those together, make weird deepfakes

• If you are a programmer, try out tools like langchain or maybe write a plugin for ChatGPT. There are a lot of opportunities to use GPT-like models as controllers for other actions. And of course try stuff like Copilot.

• If you are none of the above, head to chat.openai.com and try to write your first website using ChatGPT or maybe ask it to explain nuclear fusion using a cowboy accent.

• Whatever happens, do not start a career as prompt engineer! It’s not a real job of the future, it’s just something linkedin influencers are writing about! Learn about AutoGPT instead.

The revolution will not wait for you