Matt Pocock: what makes a task good for an agent

Matt Pocock (@mattpocockuk) is looking for a word for the tasks agents are really good at: ones where the agent has all the data, can run experiments, can check its own results independently and can keep improving step by step. In a second post he shows the other half: he knew enough about phonetics to tell his agent exactly what to fix in his video editor. Karpathy and the Stanford lecture in Worth testing below make the same point about verification.

Why it mattersPick agent tasks whose results can be checked (tests, numbers, before and after), and bring your own expertise for the rest. That's also the best way to explain to clients where AI will and won't save them time.

How we knowCompany figures are labelled as their own claims; anything we couldn't verify is labelled as a report or a claim.