The program also refines its knowledge as further examples are provided, and its code can be rewritten to tweak the probabilities. Probabilistic Programming ...

Large language models write effective code because they have seen plenty of human examples. But because of that, they're unlikely to develop something that humans haven't done previously.