Grading Google’s AI on Copyright Questions
Last month, Google began rolling out its generative AI search summary system to more and more users. To call the system controversial would be an understatement.
First, many webmasters complain that the feature users their content in a way that discourages searchers from clicking through. Many web admins have already had their traffic significantly reduced due to changes in Google’s algorithms that have catapulted lower-quality sites to the top of the results.
The AI has become a laughingstock for its inaccuracies. It famously recommended using glue to help cheese stick to a pizza and that users eat one small rock per day. The viral answers were so laughable that Google scaled back its AI search answers.
However, despite the issues Google’s AI continues to face, it doesn’t appear that it is going away. Instead, many people will likely use the system to get information about various topics.
That raises the question: How accurate is the system? I’ve decided to test it. I’ve asked it five basic questions about copyright law, and I will grade its answers.
With that in mind, let’s see how Google’s AI did.
Note: I am not a lawyer; nothing in this is legal advice. Though I am knowledgeable about copyright, these questions are deliberately basic and generic. If you want specific advice for your situation, seek out an attorney. Also, I am not including the AI answers in the text to avoid potential issues with having AI text on the site.
Q1: How do I Register a Copyright?
The AI starts well here. It lists four steps to register a copyright: visiting the Copyright Office site, filling out the application, paying the fee and making the required deposits.
However, when it begins discussing fees, it makes its first blunder. The AI says the fees range “from $25 to $140”. However, neither number is correct. According to the US Copyright Office’s Fees Page, the cheapest registration is $45. Conversely, registrations can cost up to $500 if you’re registering a vessel design.
The rest of the AI’s answer is largely correct but also unneeded. The information would given to the user as they went through the process. However, it does indicate that it is difficult to register multiple works at once, even though there are many ways to do so.
Surprisingly, there is no actual link to the copyright office registration system. Even though the sources include the registration portal, the answer isn’t clear on where to go. I suspect many would have to perform a second search to get that link.
My Grade: D
Setting aside the errors regarding fees, the answer it gave is still somewhat accurate. I found the tone and implications misleading, but I don’t think it actively misleads the reader. It just doesn’t help them either. However, that is still a major factual error that needs to be called out.
Q2: What is Fair Use?
The first paragraph of this answer is excellent. However, that’s likely because the system pulled much of it from the US Copyright Office’s website.
However, the rest of the answer is where things go off the rails. Though the AI lists some areas where fair use is more likely to be found and acknowledges that different countries have different rules, it does not mention the four factors used to decide fair use.
That’s a significant omission, even in a basic answer. Someone could easily walk away from this explanation with a skewed idea of what is fair use. However, it’s not inaccurate, just incomplete. The AI skips over fundamentals to talk about minutia.
My Grade: C
Ultimately, this answer doesn’t have anything completely inaccurate. It’s just incomplete. However, it doesn’t indicate more complexity beyond the answer or encourage people to learn more.
Q3: What Does Copyright Protect?
The answer to this question is pretty much spot on. The AI explains that copyright is a type of intellectual property that protects original works of authorship as soon as they are fixed into a tangible medium of expression.
The answer then provides a reasonably complete list of protected works and the rights a copyright holder has. Finally, it distinguishes between copyright and trademark, noting some things that can’t be protected by copyright.
The answer is solid but came almost entirely from the US Copyright Office’s FAQs. Seeing how the AI adds more value than going directly to the FAQ is tough.
My Grade: A
Even though Gemini’s answer consists of rewritten passages from the USCO, I have to credit the AI for finding a decent source and its correct sections. This is a helpful answer, even if it doesn’t add much value over simply going to the source.
Q4: What are the Benefits of Copyright Registration?
With this question, Gemini doesn’t pull from the USCO at all. Instead, it pulls from three articles on other legal sites and pays for it.
Gemini lists five benefits, only two of which are wholly accurate. While it is true that a copyright registration is proof of ownership and gives you standing to file a lawsuit, it’s more dubious if it is a deterrence. Most infringers don’t consider copyright registration when choosing what works to target.
However, while copyright registration may have some practical deterrence, the final two listed items are flatly wrong. Once you create a work, you automatically have the sole authority to reproduce it and can license it—no registration is required.
The answer also omits actual rights granted under the law, such as claiming both statutory damages and attorney’s fees in a copyright case.
My Grade: F
The sources Gemini pulled from were fine, but the system severely misinterpreted them. As a result, 60% of this answer is false and the accurate parts omit critical information. There’s no way that I can’t give this an F.
Q5: What is the DMCA?
Here, Gemini gets the answer essentially correct. It identified what “DMCA” stands for, when it passed and identified two of the significant changes it brought.
Namely, Gemini identified the notice-and-takedown system (including its protections for online service providers) and the anti-circumvention rules. It also highlighted an online service provider’s obligations under the law.
However, as the USCO notes on its DMCA page, the law has three sections. Gemini didn’t mention the protection of copyright management information (CMI). Granted, humans also routinely forget that one simply because it has had much less impact than the other sections.
Overall, it’s a solid overview of the most impactful parts of the DMCA.
My Grade: B
This answer is largely accurate. Gemini pulled heavily from the USCO and university sources, which explains its quality. Still, omitting a third of the act is a significant oversight, even if it hasn’t grabbed as much attention.
Bottom Line
When Gemini pulled from reliable sources and parsed them well, its answers were not “glue on pizza” wrong. On some questions, it even did reasonably well.
But when Gemini did well, it’s because it relied heavily on one or two quality sources. What is the benefit of Gemini versus simply going to those sources?
That question becomes especially dubious considering that Gemini got significant things wrong. Why rely on Google’s AI when reliable sources are a click away?
That said, I wouldn’t call this a complete disaster. At no point did Gemini tell me to do the copyright equivalent of eating a small rock per day. Its mistakes were strangely human.
Many criticisms of Gemini would be dampened if Google made two changes.
First, stop presenting the answer as authoritative. The tone, presentation and length of the answer combine to make it seem authoritative, the “Generative AI is experimental” disclaimer be dammed.
Second, the vast majority of sources are hidden behind dropdowns. Placing them as links in line with the text, like this article does, would make them more visible. Likewise, they could add “recommended reading” at the bottom to clarify the answer isn’t complete and guide the reader where to go next.
While it wouldn’t eliminate all criticism, it would help. Currently, Gemini is presented as an authoritative replacement for clicking on quality links. However, this is a disservice to web admins and users alike.
Shifting the focus to creating a tool that encourages users to learn more would help everyone greatly.
However, that doesn’t appear to be the tool that Google wants to create.
Want to Reuse or Republish this Content?
If you want to feature this article in your site, classroom or elsewhere, just let us know! We usually grant permission within 24 hours.