Skip to main content

Looks like we're around 1000 lying monkeys

IEEE Spectrum: Will the "AI Scientist" Bring Anything to Science?

A tool to take over the scientific process continues a controversial trend

(note: the AI *ONLY* completely makes up numbers 10 percent of the time which the authors thing is "probably" unacceptable.)
spectrum.ieee.org/ai-for-scien…

#ai #monkeys #llms

But then there’s the problem of hallucinations—the technical term for an AI making stuff up. Lu says that although they instructed the AI scientist to only use numbers from its experimental journal, “sometimes it still will disobey.” Lu says the model disobeyed less than 10 percent of the time, but “we think 10 percent is probably unacceptable.” He says they’re investigating a solution, such as instructing the system to link each number in its paper to the place it appeared in the experimental log. But the system also made less obvious errors of reasoning and comprehension, which seem harder to fix.

Will the "AI Scientist" Bring Anything to Science?

Eliza Strickland (IEEE Spectrum)
#ai #LLMs #monkeys
⇧