AI and the Future of Lies

On May 12, author/filmmaker Steven Rosenbaum released his latest book, The Future of Truth: How AI Shapes Reality.

Between Rosenbaum’s reputation as a journalist in the technology space and the book’s topic, it had received a fair amount of buzz prior to its release. 

Though the book did initially receive positive reviews, its day in the sun would be short-lived. On May 19, Benjamin Mullin at the New York Times published an article claiming that the book contained “more than a half-dozen misattributed or fake quotes.”

One of those quoted, journalist Kara Swisher, said that she had never said the quote in question and that it made her, “sound like I have a stick up my butt.”

While that was bad enough, things got worse for Rosenbaum after he revealed that the quotes were because he had “used AI tools ChatGPT and Claude during the research, writing, and editing process.”

In short, in a book about how AI is reshaping truth, he used AI systems that ended up fabricating quotes and attributing them to real people who never said them. The irony of the story has not been lost on other publications. 

Rosenbaum, for his part, has acknowledged the errors and said that he is working with his editors to address them and that they will be fixed in future editions. He also said that he disclosed the use of AI in the book’s acknowledgments. 

Unfortunately, that will probably not be enough to rescue Rosenbaum’s reputation, which has taken a massive hit over the scandal. 

That, in turn, should serve as a warning to other authors who are considering dabbling with AI.

The Big Problem

The irony of the situation is obvious. Writing a book about the dangers of AI while using AI is bad enough. However, taking it that next step and not fact-checking the AI’s output is beyond the pale.

Rosenbaum seems to understand this. In an interview with Ars Technica, he said, “The fact that someone writing critically about AI and verification could still encounter these failures tells you how pervasive and persuasive these systems have become.”

While that’s true, it misses the much bigger point. The book isn’t being sold as being authored by Claude and ChatGPT. It is Rosenbaum’s name on the cover. While we can argue about whether his AI disclosure was adequate, there is not much doubt that he is listed as the author of the book.

Being the author of a work means you reap the credit and other rewards for it. However, it also means that you carry the risk of that work. I’ve talked about this before in the context of ghostwriters. Though there are situations where using a ghostwriter is acceptable, that doesn’t mean that the person whose name is on the cover doesn’t ultimately bear the responsibility. 

For Rosenbaum, the situation is no different than if he had used a human ghostwriter or had an acknowledged second author do most of the heavy lifting (similar to Jane Goodall). Simply put, authors can’t claim the credit for work done by AI and then try to pass the blame to AI when it goes awry. The author (and the editors) are ultimately responsible for the book’s content, regardless of how it got there.

To be clear, I don’t think Rosenbaum would disagree with this. But, while he’s right that this story is illustrative of the dangers of AI, it is the wrong lesson to learn.

Instead, the lesson to learn is about the responsibility that comes with authorship. An author has an obligation to ensure that what they publish is accurate, original, and ethical. This means, at the very least, ensuring the accuracy of information contained in the book, regardless of where it came from.

Rosenbaum did not do that, and if one of the individuals “quoted” decided to sue, it would be him and his publisher, not Anthropic or OpenAI, that would be on the hook.

That alone should be a major wake-up call.

The Non-Shortcut

This begs a simple question: why didn’t Rosenbaum, or his publisher, fact-check the work before publication?

Rosenbaum is an established author and journalist who has been working for over 30 years. He was actively working on a book about how AI distorts truth. Why was he, or others involved, not more careful?

While it’s impossible to know for certain, I suspect that it’s because, if they had, there wouldn’t have been much reason to use AI at all.

The reason is simple. Anyone who has done any form of research writing knows that the time spent putting words on paper (or screen) is only a tiny fraction of the time spent on a piece. Most of it is spent finding and reviewing sources, learning about the topic at hand, and determining how it will all fit together.

If Rosenbaum had gone through and fact-checked every claim the AI made, it would have likely taken about the same amount of time as doing the research from scratch. The whole point of using AI is to save time and other resources. 

In that regard, AI serves much like a ghostwriter or researcher. By hiring a human assistant, you save time at the expense of money. However, every minute you spend validating or editing their work, that deal becomes less valuable. Yet, if they do something illegal or unethical, you are the one who will face the repercussions.

With humans, you can strike a balance. You can study their history, previous work, and their competency. You can make decisions about how much you trust them and make arrangements accordingly. 

With AI, you don’t have that option. Even as AI improves, its propensity for “hallucinations” remains and is a massive problem, especially over larger works, like a book.

Though humans make mistakes as well, it’s easier to predict the number and kind of mistakes. It’s also easier to check their work as you can examine all the notes. AI often doesn’t offer that. It’s also impossible to predict where or how often AI will make mistakes because it moves forward with equal confidence regardless of how little information it has.

In short, AI needs to be treated as what it is: the work of an overly-confident novice. If you wouldn’t hire such a person to research your book, maybe don’t use AI to do it either.

Bottom Line

Back in March, I wrote about how, when using AI in certain environments, you’re trading time saved for understanding of the material. That is very much true in this case. While I don’t dispute Rosenbaum’s expertise, he would have had a much greater understanding of this particular topic if he had done all of his own research.

However, as this case illustrates, there is another trade-off. Especially when you rely on AI research, you’re trading time for accuracy. Even though AI continues to improve, it will always make mistakes and hallucinate. If you want a work to be as correct as possible, there is simply no substitute for human writing, editing, and fact-checking.

It reminds me of the old adage, if you want something done right, you have to do it yourself. That includes not having AI perform research, make editorial/creative decisions, or otherwise run unchecked on a project.

To be clear, there might be times where this tradeoff is acceptable. But I don’t think a book about how AI is impacting our ability to tell truth from fiction is one of them.

Rosenbaum’s choice to use AI, even with a disclosure, was not appropriate for the topic. As illustrative as this story is in some ways, it’s done real damage to Rosenbaum’s reputation and has undermined the book.

In short, Claude and ChatGPT will not be held to account for these errors; it’s Rosenbaum and his publisher that will. 

That’s something others need to think about as they weigh the pros and cons of AI-assisted writing.