One of the factors they look at is how well the search engines are at ranking original texts over the duplicates. This, of course, is of great interest to anyone who posts original content online as, inevitably, it will be copied (legitimately and otherwise) and it’s important that the search engines do a reasonable job favoring the original author.

However, the results from AnalyzeThis.ru are, simply put, less than inspiriting. Google, according to the site, gets it right about 57% of the time according to the most recent survey. Even scarier, this is after a year of drastic improvement, up from under 10% in June of last year. Even more discouraging is that Google is easily the best, way ahead of Bing, which is currently hovering at 7%.

To be clear, I take these numbers with a good deal of skepticism. The site doesn’t clearly explain how it obtains its data but, even if the tests aren’t perfect, they highlight the point that Google isn’t either. Even after it’s updates, in these tests, Google still managed to barely get half of the cases right. Even if we assume this test generated 90% false positives in this area, that still leaves just under 5% of all works being mislabeled by Google as duplicates.

It’s easy to see how, with so many content creators and so many different works, this can become a very big problem very quickly.

Google’s Problem

For Google (and other search engines) the problem of duplicate content is a thorny one. When a user enters a search query. They don’t want to see ten pages with the same content, they usually want a variety of links to choose from. So, if a lot of pages on the Web have the same content, Google has to decide which one is the original and give it preferential treatment.

However, finding the original is easier said than done.

Google uses a variety of factors to determine which is the original such as the trust it has with the domain, the age of the site/page and the number of inbound links. However, these metrics aren’t perfect and, inevitably, Google gets it wrong from time to time.

Spammers, in turn, count on this. They often copy large amounts of content hoping that Google will believe at least some of it is original. Sadly, it’s a numbers game and the spammers, despite improvements, are still winning.

If these numbers are remotely accurate, it’s easy to see how, while Google has definitely made things harder on the spammers, they can still find great success copying content.

This further emphasizes how important it is for webmasters and content creators to be vigilant with their content and not put their faith in Google to keep them ahead of the bad guys.

Bottom Line

Just to be clear, I’m still very skeptical about these exact numbers. Without knowing more about the methodology, it’s impossible to be sure that they are accurate.

However, even with imperfect results, they highlight just how flawed Google is, even after the recent (and significant) improvements.

This makes it clear that, despite what Matt Cutts said, the scenario of a duplicate outranking an original work is not “highly unlikely” and seems to happen fairly regularly.

Obviously, this is a tough problem for search engines to crack and, until they do, webmasters need to be vigilant with their content.

The problem largely centers around the issue of what is known as duplicate content, or content that can be found at more than one place online. This is a problem because search engines typically like to display a variety of results for every query and routinely hide or drop down pages that are too similar to each other.

That being said, SEO is also often about linking, something that those who use Creative Commons works are usually obliged to do (when appropriate for the medium). This means that, while licensing your content via Creative Commons may raise a spectre of duplicate content, it also opens the doors to increased search engine trust.

So how does Creative Commons impact SEO? There’s no simple answer but there is definitely a lot of theory to pour over.

The SEO of Licensing Your Work

When you license your work under a Creative Commons License, or any open license for that matter, you’re putting it out there with the intention that others will use it. That, in turn, means multiple copies of your work floating around on the Web, each competing for the same attention from the search engines.

That is what brings about the aforementioned duplicate content issue. Google doesn’t want to give the same article (or a very similar one) two slots on the same results page so one has to either be pushed into the supplemental index or to a much lower ranking.

Usually, Google attempts to make a determination about which article is the original and which is the duplicate. However, as they recently admitted, they aren’t perfect at doing that as they sometimes index sites out of order. So, even though Google claims claim that the duplicate content “penalty” doesn’t exist, there’s still a real risk of being hurt in the rankings due to multiple copies of a work floating around.

However, Creative Commons uses of a work are typically supposed to come with a link back to the source as well as an acknowledgement of the license the content was used under. This is part of why Creative Commons can be such a powerful force to stop scraping.

These signs are pretty clear indications to the search engines as to which site is the original and that is why the SEO risk of legitimate Creative Commons use is much less than with scraping and/or plagiarism.

Also, since Creative Commons reuse is most often done by humans and not automated scrapers, the original has much more time online before the duplicates appear, increasing the likelihood it has been indexed and discovered as the original, further reducing the dangers.

For images, this risk is significantly less. The reason is that Google doesn’t do quite as good of a job spotting duplicate content in image search results and doesn’t put as much emphasis on originality. A simple image search will often turn up many different versions of the same image, often just different sizes.

That being said, Google is getting somewhat better at this and that may become an increasing fear as time goes on.

However, on the positive side, both images and text uses of CC-Licensed works can be used to generate inbound links and many SEO experts view Creative Commons as a great way to generate such links to your sites and profiles.

This assumes, of course, that everyone using CC-Licensed work is playing by the rules but, those that aren’t are committing copyright infringement the same as if the work wasn’t Creative Commons at all.

In the end, the risks are likely minimal but the value of the reward will depend on how much you can gain from the inbound links, something that will vary wildly from site to site.

The SEO of Using Creative Commons Content

On the flip side of the coin, you know instantly when you use a Creative Commons article or blog post that you’re getting something that’s been posted elsewhere and probably for some length of time.

As such, there’s probably little expectation that the article will be of much SEO value as Google will spot it as a duplicate and treat it as such. This is especially true since the Farmer/Panda updates that came down hard on content farms and other sites seen has having “low quality” and duplicate content.

That being said, images may provide some SEO benefit as they can often be picked up despite being duplicates. This is especially true if they are resized prior to being reposted.

However, if you use Creative Commons content on your site, SEO benefit probably should not be your primary motivation. Typically, it’s more about filling blank spots in your posting schedule and getting useful content out to your existing readers, rather than trying to obtain some SEO benefit.

For the purpose of SEO, original and compelling content is almost always best.

This isn’t to say you can’t rank well with duplicate content, but the odds are slim and the next shift in the winds at Google could blow away your efforts.

Instead, if you want SEO benefit from Creative Commons content, your best bet is, most likely, to find content you can create derivative works from and use the content as a building block to make something new.

That, in turn, will be much more compelling to the search engines than a mere copy/paste.

Bottom Line

In the end, the SEO risks of licensing your content via Creative Commons are fairly low. Though duplicate content is real concern, legitimate CC-licensed use is far less likely to harm your site than scraping, plagiarism and other forms of non-attributed use.

That being said, Google isn’t perfect and mistakes will happen, but most likely there’s not much harm that can come from licensing your content this way, at least from an SEO-only standpoint, and given the potential for inbound links there is at least the potential for some gain.

Still, the choice of using a Creative Commons License is a personal one and, even if the SEO risks are low it still isn’t right for everyone.

Basically, this is just one factor of many to weigh and, even then, for some the risks may not outweigh the rewards, especially for those who already have a ton of inbound links and have little use for a few more.

For example, if you search for “Phones” and add the /android slash you’ll only get results for related to the Android operating system. Likewise, you can use slashed to manipulate the results in various ways, including selecting a date range, a political slant or only certain kinds of sites (forums, blogs, etc.).

Two of the more interesting slashes are /duptext and /domainduptext, which supposedly will check either a page or a domain’s content to find where it is being duplicated and how it is being misused. For webmasters, this could mean a powerful new tool for tracking duplicate content on the Web and tracking down those who are misusing their work.

So, as with other systems, I put it to the test and was, in a word, disappointed with the results. Though I think Blekko has a lot of potential in other areas, it doesn’t seem that duplicate content detection is one of its better uses, at least not at this time.

How Blekko’s Plagiarism Checker Works

Using Blekko’s duplicate content detection system is actually fairly easy. All one has to do is search for the URL they want to check the content of and then add the /duptext tag to the end of the URL.

For example:

http://www.plagiarismtoday.com/2010/11/23/5-rules-for-the-next-plagiarism-scandal/ /duptext

You can do this with any page on the Web and the results are usually presented in a few seconds.

As you can see, it breaks out the information by hosts and URLs and, from there, based on those that are on-site and off-site. Below the chart is a list of links where the duplicate content is present.

You can also check an entire domain for duplicate content by looking for just the domain and adding “/domainduptext” to the end. For example.

plagiarismtoday.com /domainduptext

However, with this slash you get significantly less information, basically just a list of domains where your duplicate content is suspected of appearing and links to their SEO pages.

The question, however, is “How well does it work?” Unfortunately, after a few searches, the answer appears to be a disappointing one.

Testing it Out

As is typical with my tests, I decided to have Blekko do a duplicate content check on several works with a relatively known amount of plagiarism, 2 poems, one short story and one post on Plagiarism Today.

Here are the results of those tests:

Test 1: Poem 1

I tried out Blekko on an old poem of mine that I knew had seen widespread copying, both plagiarized and attributed. However, after performing the search, Blekko failed to find a single copy of the poem on any other site, even though a simple Google search finds about 40 results, though many are admittedly duplicates.

Blekko Results: 0 Google Results: 40

Test 2: Poem 2

Testing with another poem produced very similar results. However, this time Blekko didn’t even find duplicates on my site and instead simply indicated that there were no duplicates at all. However, once again, a simple Google search turned up about forty results though though, as with before, many were duplicates or copies on my domain.

Blekko Results: 0 Google Results: 39

Test 3: Story

Following the lack of luck with the two poems, I then tried an old short story of mine that had seen a small amount of copying. However, once again, Blekko failed to find any results that were not on my domain and a quick Google search turned a duplicate of the story on a DeviantArt account.

Blekko Results: 0 Google Results: 1

Test 4: PT Post

Finally, I tried an old, popular post from Plagiarism Today to see how well its content was detected. However, once again, Blekko failed to return any results and Google found a duplicate version of the piece on what appears to be a BlogSpot spam blog (one I was previously unaware of too).

Blekko Results: 0 Google Results: 1

Test 5: Whole Domain

Finally, in a bid to see what would happen if I ran my entire old literature domain through Blekko using the /domainduptext slash, it found only 6 offsite domains and 11 offsite URLs, even though many individual pieces see more reuse than that. It was missing many domains with widespread reuse of my work (legitimate and plagiarized) including blogspot.com, myspace.com and deviantart.com to name just a few.

Worse still, I couldn’t examine any of the individual links as clicking the link provided by Blekko just took me to the SEO page for that domain, not to a list of suspect URLs on the site or even to the domain itself.

In short, if I wanted to find out exactly how my content was used on these sites, it was up to me to find it.

Blekko Results: 11 Google Results: N/A

Other Issues

It became pretty clear that Blekko was missing a lot of duplicate content with its searches. My suspicion is that its because it tries to hone in only what it considers the best sites and cuts out spam blogs and other sites it deems to be of low value.

While this may be great for searchers, it creates a real problem when checking for duplicate content as these are often the exact sites you need to find.

However, that can’t be the only cause of the problem. If you use Blekko to do search for quotes from the relevant pieces, you get much more respectable results. Though the results aren’t nearly as good as Google in this area, they are definitely much more useful than via either of the slashes.

But the biggest problem is what one does after they find content reuse via Blekko. With the /domainduptext slash you can’t even access the individual URLs to investigate further. Using the /duptext slash is a much more robust tool, taking you to a page where the duplicate content is highlighted, but in the pages I did check the results were hit and miss, as many as half of the pages linked had no duplicate content at all.

All in all, as useful as Blekko is for other kinds of searches, or at least as useful as it might be, it doesn’t handle duplicate content searches very well, certainly no better than Copyscape or even regular Google.

Bottom Line

None of this is meant to be a slight against Blekko in any other regard. The other searches I did with it were actually pretty useful and, though I wasn’t swayed enough to change my default search engine, I did enjoy a lot of what Blekko had to offer and can see myself making some slashes for my use.

In the end though, it just isn’t a good tool for detecting plagiarism, copyright infringement or other kinds of duplicate content. Though the idea is solid and its integration with other SEO functions very appealing to some, it just isn’t accurate or complete enough at this time.

Still, as with other tools I’ve reviewed, there is hope for the future. But it remains to be seen if this will be a priority for Blekko, which is clearly targeting a more generic search audience. Duplicate content detection is a high-specialized skill and the tools to find a keyword on the Web aren’t the same as the ones to find an article on every site it appears.

As such, this will most likely remain a nice idea by a decent search engine that just isn’t practical.

photo credit: WallTea

Feeding Google’s insatiable appetite for content is on of the main reasons why infringers scrape and plagiarize content and also one of the biggest reasons why it is important to monitor and, in many cases, defend against it.

The logic is simple enough, the more copies of a work that appear, especially without proper attribution, the less likely that the search engines will give credit to the original source. This can erode search engine performance, especially for smaller and less-established sites or those in highly-competitive fields.

However, duplicate content doesn’t just come from plagiarists and spammers, it also comes from oneself and our own actions when dealing with our own content. Some of it is errors within our site, some of it is in how we approach social networking and social news.

So even as we are enforcing our rights elsewhere, we have to be careful about how we use our own works. Though it might not be infringement, it can certainly have a very negative impact on you and your site and is worth dealing with all the same.

Starting at Home

The first steps to dealing with duplicate content have to start on your own site or blog. Many people don’t realize how many opportunities there are to create duplicate content on your site, even by pure accident.

Consider the following examples from a simple blog:

1. Tag Pages: Tag pages have much of the same content as individual post pages and are generated by most blogging applications.
2. Archive Pages: Monthly, yearly and other archive pages, similar to tag pages, have the same content, or significant portions of it, repeated.
3. Category Pages: As with Archives and Tags, category pages repeat content.
4. Printable Pages: Many themes include printable versions of content pages that can be indexed as duplicates.
5. Comment Pages: Finally, depending on the way comments are set up, a separate page with duplicate content can be created for the version with comments.

Depending on how your blog is set up, it is entirely possible that your article appears six times or more on your site. Google, and other search engines, have to make a decision about which page is the best page and link to it. However, it doesn’t always make the right decision and, in extreme cases, can even decide that the site is spamming and either lower its ranking or remove it.

Thus, it is important to make sure that you keep this duplicate content to a minimum and do your part to let the search engines know what you want them to link to. Here are a few tips:

1. Show Summaries: When possible, only use article summaries and link to the full article. There is no reason for your tag, archive or category pages to display the full text of every entry.
2. Use Robots.txt: Use your robots.txt file to prevent search engines from indexing unneeded pages, such as your printable pages. However, use caution with this method.
3. Use Canonical Tag: Google, Yahoo and Bing all support the canonical tag, which tells search engines which page is the best to include in the index.

In short, be very clear what versions of your content are ideal and try to keep the duplicates to an absolute minimum. Doing so will greatly help search engines tell which page to link to, helping both you provide a better service.

Away from Home

The other problem with self-defeating content use lies away from the home site. Where once an entire person’s presence was in their home page, now it can be scattered all over the Web, including other sites they run and social networking sites that they integrate with and use.

While it may seem like a great idea to post your content on every site you take part in, it can confuse the search engines. You want your efforts in social media to support your search engine strategy, not replace your original site. However, many people unwittingly do exactly that.

Here are a few ways to avoid that:

1. Unique Content for Each Site: If you run multiple sites, you need to have unique content for each. You can use snippets of content to cross promote and certainly link between them, but don’t repost everything. It confuses search engines and readers alike.
2. Use Snippets: When posting your content on other sites, use snippets and link to the original works. The likelihood of this replacing your content, in human or search engine eyes, is slim to none.
3. Require Links: Whenever any content of yours appears on another site, even in snippet form, request links back to the original, specifically SE-friendly ones.

In short, be careful how you use your content. Though linked use isn’t likely to hurt you with the search engines, if you aren’t careful you can really eat up your own site by spreading your work too thin, too carelessly.

Bottom Line

When we think of content reuse, we think of what others do with our work. However, the fact is we are all the biggest reusers of our own work and, perhaps, the most important.

Though we can and should track how others use our content, as well as prevent uses that are against our wishes, it is also important to keep an eye on ourselves and make sure that our actions are working for us and with our strategy.

As with anything else in life, the best place to start your content strategy is by looking at yourself and your own actions, after all, you are your own biggest customer.

The site currently is just a “parked” domain page running ads for the domain’s registrar, GoDaddy. According to the whois information for the site, the domain was “updated” on the eighth, indicating that it possibly expired and was transferred to another owner.

Workfriendly attempted to disguise Web surfing as a Microsoft Word document by formatting Web pages to appear as text in a Word file while bordering the site content with a fake border designed to look like the application. This was supposed to make it “safer” to surf at work as it would raise less suspicion should anyone see your monitor.

The site created problems, however, when it allowed search engines to index its modified pages, injecting many thousands of of pages worth of duplicate content into Google. It also created headaches by not obeying certain meta tags, causing links to break on some sites and for Google to report those errors as broken links on the original domain.

It is unclear at this time if the outage is temporary or permanent, however, the site has been down for at least two days, making a temporary outage increasingly unlikely.

Hat tip: Special thanks to David Bradley of Sciencebase (stupid typos, thanks for the catch!) for letting me know that Workfriendly is not working

Even though a recent report by Attributor indicates that the search engine has done a terrible job separating originals from copies, the spammers don’t seem to be taking any chances.

Spam bloggers are no longer content on scraping entries and republishing, but are modifying them in a variety of ways. On The Blog Herald, another site I write for, editor Tony Hung wrote about a site that seemed to either be synonymizing or double translating content from the site. On tenforty, blogger Deb wrote about a case where her story was translated into another language before being reposted on a spam Blogspot blog.

What started out as a rare phenomenon is now turning into a regular occurrence. Unfortunately, as the tactics of spammers change, so must the tactics of those who seek to protect their content and this calls for a new look at how we protect our works on the Web.

Background

The technology behind modified scraping has been around for several years. I first wrote about it in December of 2005. Back then the problem was fairly rare and the concept was still somewhat new. However, it seems as if more and more spammers are catching on to the trick and, possibly, that new spam blog networks are cropping up to take advantage of the technique.

The idea is that posting verbatim copied works is dangerous. Not only are you likely to get caught and shut down, but Google and other search engines assign penalties to duplicate content, making it harder to get those search results needed to make spamming profitable.

Since editing content by hand takes too long and defeats the purpose of scraping, spammers have started creating ways of modifying or “spinning” content before reposting it in order to fool the search engines into thinking that spam site is actually both legitimate and original.

To achieve that, they use one of several techniques including, but not limited to, the following:

1. Synonymized Content: The most basic approach takes an article and swaps out occasional words for synonyms according to a built-in thesaurus. Such a system can actually create hundreds or thousands of articles from a single source by using different combinations of synonyms.
2. Translated Content: This approach runs the content through an automatic translation program similar to what you find on Babelfish. Though the translations are far from perfect and leave the work in a foreign language, it is usually intelligible by humans and search engines alike. This will likely become a more popular technique as the Web gains more of an audience in non-English-speaking countries and those markets become more valuable for spammers.
3. Double Translated Content: The same as translated content, but this kind translates the translation back into English. This produces a heavily modified and often unintelligible outcome that bears little resemblance to the original in many cases. This type of scraper is purely theoretically at this time but very likely does exist.

In all of the above cases, the outcome is the same, the scraped article bears little resemblance to the original, making it much more difficult to detect and to stop, both for the victims and for the search engines.

Changing Strategies

The problem for bloggers when dealing with this type of plagiarism is that the typical methods of detection simply don’t work. Copyscape, though drastically improved, will struggle with this kind of scraping as will all other plagiarism checkers that work by looking for verbatim copying. This includes high-end academic solutions.

Even Google Alerts can be thwarted by this if the phrase being searched for is modified in the process of spinning the article. Though Blogwerx is working on a product that can detect synonymized scraping, it is clear that any system to search for this kind of abuse is going to require a great deal of power and, most likely, some expense to the user.

The focus then becomes not abandoning the old ways of detecting plagiarism, but on adding new ways to guard against this threat. Those methods should include the following:

1. Digital Fingerprinting: I’ve been beating on the digital fingerprinting wardrum for some time but such fingerprints are the most natural defense against modified scraping. Fingerprints, if done right, have no synonyms and no translations. They will remain intact no matter how the article is spun and can easily be searched.
2. Uncommon Uses: Since FeedBurner doesn’t rely on text detection to determine who is using your feed and where, their tools will remain effective even if the post is modified, that is, so long as FeedBurner’s code is not removed.
3. Using Names: If you can’t use the digital fingerprint plugin you can create your own by either entering your own fingerprint in the footer of all your posts, editing your RSS template or simply using your name, if very unique, at the top of your works. Like fingerprints, names do not have easy translations or synonyms and are unlikely to be altered. Even better, vanity searches can let you know who else is talking about your work.

In short, it is important to find elements that do not have easy translations or synonyms and focus on searching for those. These methods can, and should, be used in addition to other searching techniques to ensure that more human plagiarists or other kinds of scrapers, such as search engine scrapers, are also detected.

Even though these methods do not provide a perfect defense to modified scraping, it is a step in the right direction.

Conclusions

The good news with this kind of spam blog is that the risk being penalized in the search engines for being a victim of scraping goes down drastically. Since spammers avoid any potential duplicate content penalty, you do too.

However, none of this says that the scrapers won’t target keywords similar to your own and then use your own content to beat you in the results. That type of abuse might, in fact, be more likely than ever considering that Google will also not recognize the spam blog as junk and discard it.

As a result, even if we discount the emotional reasons for fighting plagiarism, there is still a great deal of need to monitor our content and ensure that those who make use of it, do so in an acceptable way.

Unfortunately, that will likely remain a game of cat and mouse for many years to come as the plagiarists and scrapers are rapidly changing their techniques to adapt to new situations. Clearly, we have to do the same.

Fortunately, at this phase, the adaptations are not that difficult but the future remains much less certain.

They repeated my tests with roughly the same results and discovered that there is an issue with the Google API, which they use as their backend, that was limiting its usefulness at detecting plagiarism in works that had been lifted many different times.

I received an email from Greenspan yesterday telling me that they’ve made adjustments on their end and they have greatly improved Copyscape’s ability to detect widely plagiarized works.

Eager to see if it worked, I opened up my Copyscape Premium account and decided to give it another try. The differences were immediately clear.

A Definite Change

To retest Copyscape Premium, I decided to try the service again with two of the poems I used in my original experiment.

In my first run, neither poem produced any results and subsequent tests since then only turned up one or two copies. This is despite the fact that both works are widely available on the Web, both as legitimately licensed copies and as plagiarized works.

When I analyzed the first poem, using the print page to avoid scanning comments (at Greenspan’s suggestion), Copyscape picked up ten results. The second one, using the same method, picked up fifteen.

To call the difference noticeable would be an understatement. Copyscape had gone from being completely ineffective in my experiments to producing some very powerful results.

First Run:

Second Run:

Limitations

As great as the improvements to Copyscape are, the results do have a few caveats.

First, a text search for one of the poems only produced four results. Still, far more than previous, but far less than using the printable page. The reasons for this are unclear.

The second, and largest, caveat is that the results were still only a fraction of the actual copies available. When I did a Google search for the two poems, the first produced 25 results, the second produced 36.

All in all, that leaves over half of all the copies of the test works undetected by Copyscape.

However, according to Greenspan, the fact that my works were so heavily plagiarized and copied was, and still is, part of the problem. Since Google tends to put duplicates of a work in their duplicate content filter, which in turn seems to hide it from the Google API, works that are copied less or are paired with other content may fare much better.

Unfortunately, this appears to be an issue on Google’s end and is not something Copyscape can easily correct.

Conclusions

It is clear that the steps Copyscape took to tweak their service had a drastic impact. Though it is still somewhat limited in how it handles widely-plagiarized works, it may be effective enough for many to consider using.

After all, even though Copyscape does not seem to detect every single use of the work, it does offer greater convenience and tools to help monitor and track plagiarism cases. Whether or not this trade off is worth the cost involved in using the premium service is a decision best left up to individual Webmasters.

Personally, I’m going to continue to watch the service and test it. Greenspan has said there may be future adjustments that could improve it even more. For me, Copyscape is reaching the tipping point where the benefits start to outweigh the drawbacks, however, I still have to hold back from giving an unrestrained recommendation since so many copies were still undetected.

The best thing to do right now is consider the content you post, the nature of the plagiarism you’ve faced and see if Copyscape Premium is a good fit.

It may not be right for everyone, but there are now many people who it may be right for.