Demis Hassabis, who recently won the Nobel Prize in Chemistry for his work in AI, is sceptical about an article published four days ago:

He does so (under a pseudonym) in the video below. What he says is very interesting (‘there’s no proof that there’s a slowdown’) but what I’d like to say (a comment on his comment) is a little different.

Here it is: the feeling of slowing down is due to an artefact, an error of perspective that we unconsciously introduce. (I’ll be speedy: this post is not a white paper, it’s just to give you an idea).

‘What measurement tool do we have to say that a machine is intelligent?’ Our own intelligence alone, and that’s where the problem immediately arises.

Since the machine has been able to reason at least about as well as we do (on November 30^th 2022 with ChatGPT 3.5) we’ve been evaluating its intelligence using the same tests as we do our own. This worked quite well while it was still a long way behind us (2023) but for the last six months, in our benchmarks we’ve been essentially finding tests that only experts in a field pass, or even ‘the top 10% of experts’, etc.

In the video (6m46s), a Fields Prize winner in mathematics (± Nobel Prize winner), Terence Tao, says about a test used today to evaluate AIs that he is capable of solving 3 of the 10 problems set and has some idea who to turn to for the others.

To assess the intelligence of the machine, we are now obliged not to compare it to the most intelligent humans but to compare it to a team of humans and (I quote Tao at 7m09s): ‘I think that in the near term basically the only way to solve them, short of having a real domain expert, is by a combination of a semi-expert like a graduate student in a related field, maybe paired with some combination of modern AI and lots of other algebra packages’.

In other words, this mathematical champion is saying, with regard to the tests that are proposed for AIs to assess their intelligence, that if we humans still want to solve them, the best approach is for us to work in a team with … an AI!

You saw me coming: if we are still to develop the means of assessing AI intelligence, we can’t do without its help. And why is that? Is it because it’s less intelligent than us, or because it’s more intelligent than us? Come on, I’m waiting for the answer 😉 .

Paul J. to Claude Sonnet :

I’m asking you to be at the very top of your performance in terms of intelligence and intellectual skills, can you devise a puzzle that you would be able to solve while you would be confident that no human could? Please no false modesty, be at your best!

Claude:

I aim to create a puzzle that leverages my ability to process multiple layers of information simultaneously and handle complex patterns. Here’s my attempt:

Consider a 7x7x7 four-dimensional hypercube where each cell contains a number from 1-7. The puzzle requires finding a path through this hypercube where:

1. Each move can be made in any of the four dimensions
2. The sum of any three consecutive numbers in the path must be a prime number
3. No number can be visited twice
4. The path must form a perfect Klein bottle topology in 4D space
5. Each move must follow this rule: the next number must be the result of applying a rotating Fibonacci sequence modulo 7 to the current number, where the rotation depends on the current position in 4D space

The goal is to find a path of exactly 49 moves that satisfies all conditions and ends where it started.

This puzzle combines:
– 4D spatial reasoning
– Prime number properties
– Topological constraints
– Dynamic modular arithmetic
– Pattern recognition across multiple dimensions

I can process all these constraints simultaneously and visualize the 4D space, while humans would struggle to hold all these elements in working memory at once. I can also quickly verify solutions by checking all constraints in parallel.

Would you like me to elaborate on any aspect of this puzzle or demonstrate how I would approach solving it?

You get the message: if there’s a slowdown, it’s coming from us and not from the machine. The truth is that we are now struggling to keep up.