We can determine productivity for the purpose of studies like this. Give a bunch of developers the exact same task and measure how quickly they can produce a defect-free solution. Unfortunately, this study didn’t do that – the developers chose their own tasks.
Is there any AI that can create a defect-free solution to non-trivial programming problems without supervision? This has never worked in any of my tests, so I suspect the answer is currently No.
