The problem is that phrase selection is not simple task. It can be difficult for human beings to determine what phrases or sentences to search for, let alone a simple algorithm. As a result, such simplistic plagiarism checkers often times either miss a large number of results by choosing phrases that don’t work well with the search engines or produce a slew of false positives by selecting too common or too short of terms.

Thus, when I read about a new SE-based plagiarism checker, this one by SEO content writing service 10 Dollar Articles (10DA), I was skeptical at best.

Though a cursory search proved many of my original suspicions, it also showed that the plagiarism checker isn’t quite as useless as many of its brethren. Though it has its flaws and certainly isn’t as useful as its marketing might say, it does have some interesting features and potential compelling uses.

How it Works

The 10DA checker works like many similar services. Users copy an article or piece of content that they want to check for plagiarism, they then choose up to three services to use and hit the submit button.

The service then selects a series of five or six snippets from the work and runs them through each of the search indexes checked against. When it’s done, it links to each of the results pages and the user can go through the results to see if there are any suspicious matches.

The site keeps track of which results pages you have already visited, turning those numbers to black, and also lets you recheck the article with a different set of search engines.

The Good

The real benefit of this system is that it is extremely simple to use and free. Since the product is in beta, anyone can paste text in and run it through the system.

The service is also stands out somewhat in that it allows users to run the search through different search engines, unlike others that focus solely on Google. With the 10DA checker, you can easily search MSN and Yahoo! as well as Bloglines and more. Though many of the choices seem superfluous, especially the multiple Google services normally covered underneath the main search (such as searching either Wikipedia or Google Knol) the addition of extra choices is an interesting one.

That being said, it isn’t the first checker to offer this service, others have been doing so for some time.

Though it is unclear how much benefit one gets from running the same article through three different engines, it is easy to see how those eager to be extremely thorough may be tempted by that feature.

The Bad

Where the 10DA checker struggles the most is in the value that it adds, or lack thereof. Where Copyscape compiles the results from the various Google queries it makes and displays them in a simple results page, 10DA requires users to click through to each individual results page and do the actual legwork themselves. At this time, the 10DA checker does not even provide indication of the number of matches in the specific results pages.

Due to this, the results that one gets from the 10DA checker could be easily replicated by going to the individual search engines and doing the searches for yourself. The 10DA checker does not even automatically select to view similar matches, meaning that the initial display only includes one or two copies of the work in question.

With no match highlighting, organization or other input from the system, essentially it is the same as performing 5-18 individual searches at once. Since only one search is usually all that is necessary to prove that a work is plagiarized, one has to wonder how useful this really is.

My Tests

As with most plagiarism checkers I review, I ran the site through a short series of tests to see how the results compared with stock Google searches. Since the system still primarily uses Google, this would be a true “apples to apples” comparison.

The first test involved a prose work of mine that I know has been plagiarized many times before. I ran it through the 10DA checker and the best result of the six phrases checked in Google was 26 matches. However, after tinkering with the search term, namely by shortening it and removing punctuation, I was able to improve it to 31 results.

10DA Results:
Tweaked Results:

The reason for this is that the phrases the 10DA checker chooses seem, to me, to be extremely long. Where I can usually find a good statistically improbable phrase between 7-9 words long, all of the phrases chosen by the 10DA checker were over a dozen words, some even grow as long as 19.

Though the longer strings do reduce false positives, choosing a good unique phrase is more important in that regard. This is something that the 10DA checker struggled with as some of the results had only one match, indicating that the phrase selected was of poor quality.

I also quickly tested the checker with a poem that I knew to be heavily plagiarized. However, many of the matches, due to an issue with apostrophes, came back as false negatives. Of those that did, the highest had 25 results but, once again, by tweaking the search term, I was able to increase that number 28. However, using my own phrase, I was able to find several hundred results.

10DA Results:
Tweaked Results:
My Phrase Results:

(Note: The high number of results from my phrase are likely due in large part to matches on the same domain. However, in a cursory check of the first few pages of results, I did see at least some positive matches that were not in the first two.)

The end result is that most people will find it pretty trivial to get better results than the 10DA checker. If they can look at the phrase selected, remove punctuation and pull out a good section of unique content, they can increase the effectiveness of the search.

However, why one would do that is a bit of a mystery. If you’re going through all of these motions and need the added matches that come from a better phrase, you’re probably going to find it faster and easier just to pull the phrase yourself directly from the content and then perform your own search.

Conclusions

Even though the site’s marketing material says that it is both a competitor and a compliment to Copyscape, Copyscape is by far a more useful service. Though 10DA seems to be about on par with the number of matches Copyscape catches, the usability of Copyscape is much higher and well worth the five cents per search in most cases.

Still, if you’re looking to do a quick plagiarism check of an article before you post it on your site, something my wife has to do as her company’s blog editor, it might be a useful service. If you don’t feel like setting up a Copyscape account or don’t mind the extra step of visiting the results, then it could be useful.

However, I can not recommend this service for checking for duplicate content of your site’s material. You can get more accurate matches by hand and the amount of energy that is saved by using the 10DA checker is pretty minimal. Even the free version of Copyscape provides good matching and a much higher usability.

But even that seems somewhat defeatist. With Fairshare bringing professional-grade matching technology and automatic updates to bloggers, there is no reason that bloggers or other RSS providers should be punching in their articles by hand to check for plagiarism.

Static content may have different needs, but with Google Alerts and CopyAlerts, there is little reason to manually check those results either.

In short, the age of copying and pasting textual content to see where it has appeared on the Web is fast ending. That is good news though as the easier to use and more automated the systems become, the more likely bloggers and other writers are to use them.

Hopefully, similar systems for images, audio and video are also fast coming.

We have multiple tales from the RIAA and their battle against file sharing, news that P2P is no longer the big bandwidth hog, Microsoft destroying DRM keys and a fashion designer suing a charity operation over a drawing of a handbag.

Finally, in weird copyright news, we have Professor Lessig drawing the ire of Rush Limbaugh over his tastes in mashups and watermarked art making its way to the cover of a brand- new Wii game.

Remember, as usual, this week’s linkroll is a “raw” link list. Some stories are duplicated, some do not point to their original sources and some may not be accurate. A great deal of refining goes into producing the show notes for the Copyright 2.0 Show.

However, determining which search engine is “better” for most kinds of searching is difficult. It doesn’t matter how many results a search engine returns, but rather, how relevant and helpful those results are. That is a very subjective standard and one person could be very happy with a result while the other completely unsatisfied.

But there is one area where more is generally seen as better, plagiarism detection. When looking for copies of your own work, you want to find as many matches as possible, regardless of the order they are put in.

Unfortunately, that is a fairly specialized area of detection and one that not all search engines are well equipped for. Your typical search engine user looks for simple keywords and general information, not long phrases and specific copies. As such, most search engines don’t invest heavily in improving these types results.

But for those interested in searching for the content and detecting plagiarism, this raises the question “Which search engine is the best?” To help decide that, I put five of the top search engines through a battery of tests and the results were surprising.

The Tests

To cover as much of the market as possible, I decided to test the four top search engines, Google, Yahoo!, MSN and Ask, as well as an upstart meta search engine, Clusty.

I first tested their ability to detect static content that was copied. I ran a total of nine search results through each of the search engines, each used a statistically improbably phrase from different works I’ve written over the years and measured the number of results returned.

The first three searched dealt with poetry, which used shorter phrases and involved very high amounts of plagiarism/copying. Then I tested using three short stories, which have significantly less copying but longer phrases. Finally, I tested with three shorter prose pieces, such as essays, that are average in both copying and in phrase length.

When searching for the phrases, I included all supplemental results and included different pages on the same site since they are often separate infirngements. I did, however, browse through the results in order to seek out and eliminate any sites that contained the phrase I searched for, but not the actual work. That turned out to be unnecessary though as all results were relevant.

After those tests, I did one final search to test the ability of the search engines to detect plagiarism of dynamic content by using my Digital Fingerprint.

These tests are not scientific and are not designed to be the end all of search engine effectiveness in this area. Rather, it is just a quick and dirty look at how many results they return for searches for my own content and it is designed to give an idea of which search engines might be worth considering.

Your results may vary.

Static Content

The results of the static content searches are below:

When looking at the results, several things become clear. First is that Yahoo! was the winner, both in terms of number of rounds won and in terms of the total number of results returned. Google performed very well in all of the tests, save those for the short stories. This is interesting because Google failed to even return my own site as a result on all three occasions and no amount of tweaking the phrase (removing punctuation, shortening, etc.) seemed to help. Quick tests for other short stories on my sites had similar results, either returning just one or no hits.

The surprise of the test results was Clusty, which performed reasonably well in all tests, and even won or tied in more rounds than Google. Though its total results returned was significantly lower than both Yahoo! and Google, it established itself as a top-tier candidate in this area.

MSN and Ask, however, performed poorly all around. MSN returned less than half the results of Yahoo! and Ask returned less than one third. The best MSN could do was tie in one of the story rounds while Ask was unable to win or tie in any of the searches.

In the end, Google and Yahoo! finished neck and neck, with Yahoo! taking a slight lead due to Google’s strange performance in the story rounds. Clusty is right behind both of them and MSN and Ask are both left in the dust.

Dynamic Content

I decided to do a similar test using my digital fingerprint, as string of semi-random eight characters, to see which search engine could find more copies of my content from this site on the Web. In light of the previous results, these were very shocking.

In terms of pure count, Google blew away the competition scoring over six times the number of results found than its nearest competitor, Ask. With almost 300 results, Google stood tall in this division. Ask was able to locate about 48 results, Clusty managed to find 19 copies but Yahoo! and MSN both were only able to locate six pages with my digital fingerprint.

However, this trouncing comes with a caveat. Google’s results were filled to the brim with pages that, most likely, did not belong in the search engine index at all. Aside from spam blogs and other regular sources of content theft, there were a lot of legitimate sources such as other search engines, blog indexes and caches that should not have had their sites indexed by Google.

It is going to take a more thorough analysis to see if Google actually caught more scraping or if they are simply indexing more sources of legitimate reuse.

Though I agree more is typically better, it seems likely to me that the Google result is highly inflated and the sites I visited seem to indicate that.

“Bonus” Round

In one final test, I ran a line from Shakespeare’s “Hamlet” into the five search engines to see how many copies of the work it detected. Once again, the results were surprising.

This time around, the results more closely mirrored the first round of testing but instead showing Yahoo! with a much larger advantage. The other three, were left in the dusty with Clusty, this time, picking up the rear.

It is important to note though that, with these kinds of numbers, the results counters are notoriously unreliable and this test was not intended to be serious in any way. It was merely a curiosity that drove me to see how one of the most copied works in history was being used on the Web.

The one thing it did prove conclusively is that, no matter what, I am no Shakespeare.

Caveats

A mentioned earlier, these tests were not designed to be scientific, but rather, were simply “quick and dirty” checks to see how the different search engines handled the same queries. Different people will likely get different results and the conclusions that one can draw from this are, admittedly, very narrow.

Any researchers interested in replicating my study can contact me and I’ll send them the exact queries used and works checked for. I don’t wish to post them here as doing so could further skew future tests should this post be scraped.

All in all, the goal of this test is not to set any rules about who to trust with your plagiarism searches, but rather, where to start looking.

Conclusions

Personally, looking at these test results, I would say that both Google and Yahoo! are acceptable solutions for detecting static content. However, since Google offers Google Alerts, an email service that automates most search functions, it is safe to say that, for most, it will remain the default for plagiarism searching, even though Yahoo! produced slightly more results.

However, there is obviously a benefit to broadening your search horizons and incorporating both Yahoo! and Clusty into your efforts. Both sites produced better results than Google on many tests and often caught sites that Google missed.

Another thing that becomes clear is that MSN and Ask, though likely great search engines in other regards, both lag far behind in this area and in none of my searches did I notice Ask or MSN picking up copies that were clearly missed by the other sites. They might be worth experimenting with further, but are likely not the best places to start.

In the end, these tests seem to have achieved their goals by giving us an idea of where to begin and what services we should likely avoid. Your mileage may vary, but these results do seem to speak for themselves.

Many of these hosts have been copyright headaches for Webmasters. They are prime targets for spam blogs and scrapers and some have played a huge role in rise of the “splogosphere”.

Without the help of these hosts, the copyrights of bloggers will be almost impossible to protect, especially for smaller rightsholders that can not afford attorneys to go after plagiarists. That makes the DMCA/copyright policies of these hosts a matter of critical importance to the rest of us on the Web.

So how do they mneasure up? Let’s take a look at seven of the leaders and find out.

Format: Email
Email Address: amac at google dot com
Location of Policy: Google’s DMCA Policy
Registered with USCO: Yes
Comments: Every time I do one of these articles, it seems that Google owns one of the seven leading properties and I’m forced to cover them again and rehash the same complaints. To summarize, their requirement of a handwritten signature needlessly complicates the process of filing a DMCA complaint and, most likely, does not comply with the law, in particular the ESIGN Act. Until they are able to accept email complaints without PDF trickery, they will be a major thorn in Webmaster’s sides and, in this case, a great target for spam bloggers/scrapers.
Grade: D

Format: Email
Email Address: jkweston at microsoft dot com
Location of Policy: Microsoft’s Copyright Policy
Registered with USCO: Yes
Comments: Be grateful you read this site and be sure to bookmark this article. Otherwise, the odds of you quickly finding the copyright policy for MSN Live Spaces are slim to one. Their report abuse page gives you a drop down to report copyright infringement. However, sending a report there, even with a full DMCA notice, only results in an autoreply directing you to follow the guide above (as reported by visitors of this site). You can also find a mention of the policy on their legal page, but the link to the actual policy isn’t even clickable. You have to literally copy and paste the URL to use it. The policy itself is fairly complete, containing all of the necessary information, but finding it is a pain. You can’t even do a Google search for it. It is as if Microsoft is deliberately hiding this page. It may not be illegal, but it is some of the worst service I have ever seen.
Grade: C-

Format: Form/Email
Email Address: aolcopyright at aol dot com
Location of Policy: AOL’s Copyright Infringement Policy
Registered with USCO: Yes
Comments: AOL may be considered a dinosaur on the Web, but they do exhibit some forward thinking in this area. They provide a very convinient from (linked above) for filing complaints of copyright infringement that takes care of most of the dirty work for you. It’s definitely one of the easiest ways to report an infringement I’ve seen. The only problem with AOL’s policy is that it is also very hard to find. To get to that form you have to find a link buried in their terms of use to get to their infringement policy and, from there, click their “Copyright Notice” link to get to the form itself. They also don’t provide an email address, save on their USCO registration but otherwise is complete, giving all of the necessary information to file a notice. Though I am irked by the location of the policy and the roadblocks in finding it, it is overall a solid way to handle such matters.
Grade: B+

Format: Email
Email Address: dmca at automattic dot com
Location of Policy: Automattic’s DMCA Policy
Registered with USCO: No
Comments: Though Automattic is definitely on the offensive against spam, their DMCA policy leaves a lot to be desired. While the actual policy is very complete, though omitting a fax number, finding it is a pain. The link to it is buried in the terms of service and they have not registered with the USCO meaning that there is no alternative way to look up the information. Though they have a great reputation for handling such issues once notified, their front end and user-friendliness could definitely use some work. Though the policy is likely within the bounds of the law, the lack of USCO registration and the difficult location discourage me greatly and raise some potential legal issues.
Grade: D+

Format: Email
Email Address: copyright at typepad dot com
Location of Policy: Typepad’s Copyright Policy
Registered with USCO: Yes
Comments: One of two SixApart services we’re going to cover, Typepad is a breath of fresh air when stacked up to its competitors. The copyright policy is well-written, robust and complete. It is linked at the bottom of most SixApart-controlled pages, including the home page, and is very easy to find. They are also registered with the USCO and their information there is up to date. Though some issues exist on the backend, in particular with other SixApart services, they definitely understand how to comply with the DMCA and make the process as painless as possible for the end user.
Grade: B+

Format: Email
Email Address: copyright at livejournal dot com
Location of Policy: LiveJournalal’s Copyright Policy
Registered with USCO: Yes
Comments: The other SixApart service shares its policy with its brother. In fact, at first glance, the text of the two policies appear to be identical. As with TypePad, the copyright policy is linked at the footer of nearly every LiveJournal-controlled page, including the home page, and offers the same level of completeness. Though I would prefer the policy, along with other abuse information, be available on all pages, including individual blogs, the benefit of that would be minimal as most people will simply visit the home page.
Grade: B+

Format: Email?
Email Address: legal at blogsome dot com?
Location of Policy: Blogsome Terms of Service
Registered with USCO: No
Comments: Blogsome talks one of the toughest games when it comes to copyright infringement. Their modest-lengthed terms of service uses the word “copyright” ten times. However, there’s no teeth to this policy. There is no address given to contact about copyright infringement. Though they are an Irish site and do not have to follow the DMCA, there are EU regulations at play and, without a clear means of contact for reporting abuses, I am very worried about how this site might be misused. Hopefully, they will update their policies soon to make it more clear where they should take such matters, the address above is the only account I could find used in relation to the terms of service in any way, shape or form.
Grade: F

Wrap Up

It is an Alice in Wonderland moment for me. SixApart, who has drawn criticism for their copyright polices elsewhere, has the best public DMCA policies of all the services listed. WordPress, the dedicated spam fighter, has one of the worst.

Equally strange, the big companies, Microsoft and Google, both have obstructionist policies while the dinosaur AOL and small business SixApart have much more effective and open ones.

Over all though, I was very disappointed in the policies of the major blog hosts. Most have shown little interest working with rightsholders and some are downright uncooperative. With one F, two Ds, a C- and three B+s, two belonging to SixApart, this is easily one of the worst genres I have reviewed.

Conclusions

In the end, none of the blogging companies earned an A. They all had at least some issues with their policy and, despite being some of the largest, most important hosts on the Web, only one showed any kind of evolution or forward-thinking in this area.

When stacked against the video sharing sites, it becomes clear what a difference the threat of lawsuits has on a company. Where the video sharing sites have taken great care in crafting their policies, the blog hosts have largely just thrown theirs together, working only to meet the minimum standards they feel they should be held up to.

Looking at these policies and going back over my personal experiences with many of these hosts, there is no wonder why and how the spam blog problem grew to the proportions that it did.