It is the time of year when term papers are coming due and students are worrying more than ever about being accused of plagiarism. It’s the busy season for plagiarism detection services both for teachers and for students and, as is typical, I get asked my opinions on them.

One email recently asked me my thoughts on the Viper Anti-Plagiarism Scanner, a free application and service provided by Scan My Essay. I’ve been familiar with the service for a while but never bothered to test it. However, since it was requested I decided to put the application through a few of my paces.

Unfortunately, the results were less than spectacular and, truth be told the application fell almost completely flat. The only question is whether the problems were glitches caused by a temporary problems or something more chronic with the program.

How it Works

The idea behind VIper is that you download the small application, less than 1MB, and register for an account with the service. Once you’ve done that, you simply select the file or files you want to check for plagiarism and send Viper on its way. Viper, after some processing, will come back with the results.

Though the process is simple, and familiar to anyone who has used a plagiarism checker in the past, it does have a few interesting features. One of the biggest being its ability to match against a local database, the Web or both. This means that, if you have a pool of content you want to test against, you can do that with or without also checking the broader Web.

Also, the results page also uses a very effective layout, showing the uploaded work side-by-side to the suspected matches. This is very convenient for analyzing the match and makes developing an opinion about whether an element is plagiarized or not very simple.

Beyond those two features, both of which can actually be found in other applications or services, the rest of the application is fairly straightforward. While that is not a bad thing in and of itself, the problem is that it doesn’t seem to do the job it set out to.

My Tests

Setting up the application proved difficult. It took several tries to get the application to install correctly and almost 15 minutes to figure out how to create an account (Hint: You have to click the link in the program itself.). Though it took a while, about 40 minutes, I was eventually able to get the application up and start testing.

As with any test of a plagiarism checker, I start out by having it search for a work where there is a known amount of plagiarism. In this case, I started with an article that I had submitted to Who Is Hosting This? but still had the old RTF for. The work has not have been widely plagiarized, but does appear on the site so Viper should have registered the RTF as a 100% plagiarism.

Unfortunately, after letting Viper chew on the file for some time, it came up with nothing but a few short quotes, each a few words long, that were coincidence or properly cited. Though the work is a perfect “plagiarism”, Viper found only minor and incidental matching, all if it less than a few percent.

I decided to wait a few days before trying again and did so this morning, starting with an article I had written for the European Journalism Centre. This article, much like the previous one, only exists on the one site. However, it should still come back as 100% plagiarized.

Unfortunately, I never got results from this article. After uploading it and letting it spin for over ten minutes, nothing happened. The analysis of the article simply froze.

I tried it again repeatedly with the same article but received the same result.

For my last test, I tried an old poem of mine that I knew was widely copied, both with and without permission. I uploaded this one to the service but the first time it completed it found nothing. I tried again and the process froze up, even crashing the application. I tried it one more time and, finally, got an affirmative result.

Viper, after nearly two hours of setup and failed searching, finally had detected a single case of “plagiarism” spotting the URL where the poem can be found on the Web.

Needless to say though, this small victory has me much less than impressed.

More Problems

In the two hours I had allotted to test Viper, I had only been able to search for three documents and only one of those searches, after many retries, was successful. In the same amount of time, I could have processed many dozens of documents using virtually any other means. In fact, my search engine showdown post was compiled in about the same time I spent testing Viper and it required some 45 searches.

But in addition to Viper being slow and unreliable, it also has me a bit creeped out. The application, on Windows 7 at least, requires special permission to modify content on the hard drive. Though I don’t believe it is a virus or has any malicious intent, reviewers at CNet have warned that it messed with their Word settings and suffered many crashes. Not the kind of program I want having broad access to my computer.

In short, I would not recommend installing this program on your computer at this time. Looking at the decidedly mixed reviews on CNet, it seems as if my experience was fairly typical though others have had even worse problems.

Bottom Line

From what I can tell, Viper is an application that feeds into a Web service. Why it needs a downloaded app when the real work takes place online is unclear, but it seems likely that the problems are with the server end, not the app itself (other than the installation issues).

If they can correct their server problems, Viper has a lot of potential for doing decent plagiarism checking.

However, for students seriously worried about their papers, would urge you to go ahead and spend the small amount of money and use WriteCheck. Not only does it use the same database as most colleges and high schools, the Turnitin one, it doesn’t index your paper and it has access to private libraries and collections Viper can’t see. Furthermore, the matching technology, while imperfect, seems to be better.

As an alternative, you can use Copyscape, Plagium or simple Google queries to check for accident plagiarism. All will work faster and better than Viper in its current form.

The site, according to Klug, was getting about 2,000 visits per day when it was forgotten but is almost certainly doing much better now as it has taken off, attracting countless Twitter Tweets and other social news attention. Librarians and teachers are especially captivated by this site.

But is “The Plagiarism Checker” worth using? Is it as powerful of a tool as some, although not the site itself, have made it to be? The sad answer is no, but it could, with a few simple tweaks, become a much more useful service for teachers and bloggers alike.

How it Works

The basic premise of the minimalist site can be summed up by its instructions:

Cut & paste your students paper or homework assignment into the box below, and click the “check” button. This free plagiarism detector will find plagiarized text in homework and other essays/reports.

In short, you take an essay, article or other lengthy prose work, paste it into a textbox and hit “check”. From there, the site extracts several strings of text, runs them through Google and compiles the result, determining whether plagiarism is probable.

In that regard, the idea is actually very similar to Copyscape, which also uses Google via their API, to process results. However, where Copyscape’s keeps the “magic” hidden from the user, the “Dustball” plagiarism checker includes links to the Google results, encouraging users to click through and research the case for themselves.

That alone is a big part of the problem Webmasters, and many teachers, will have with the service. Where Copyscape, as well as academic tools such as TurnItIn, provide very simple and colorful results, The Plagiarism Checker is a very bare-bones approach, requiring the user to perform a large amount of research on their own.

Still, a bit of research will be welcomed if the service produces great results, unfortunately, it seems that the service performs only lukewarm, at best.

My Tests

To test the service, I decided to run it through a similar battery of tests that I had run Copyscape through and then watched as they improved upon the initial results.

The first test was to run an old poem of mine through the system, one that allegedly has over 300 matches in Google. However, that test was thwarted as The Plagiarism Checker refused to even look at the work, saying that it could not function with such short text strings.

I then shifted gears and started using prose works, the first being one that had 36 matches in Google at the time I did the search. The result was stunning.

Despite the fact Google had reported three dozen matches on test snippets from the work itself, the “Dustball” checker was unable to find anything. To make matters worse, using some of the sample quotes from the test, I was able to locate other copies of the work, such as with the first quote.

Clearly, The Plagiarism Checker was missing results that Google was finding, meaning it was discarding them for whatever reason.

A similar test for another prose work only returned one sentence that was matched against anything and the results for it were all false positives. This work, in Google, has six results.

The only search using the service that seemed to work remotely well was when I ran the Declaration of Independence through it. Every search term, in this test, came back positive.

It appears that text that is not widely distributed around the Web may or may not show up as plagiarized in this work, something that has me very worried as many are starting to rely on this plagiarism checker as their main tool for detecting both copyright infringement and the plagiarism of students.

The Sad Truth

Simply put, any and all of these search results should have come back as being plagiarized. Even if there were no other matches of the content, these works existed on my site and are available through Google there. There is no reason that any of these works should have come back as anything short of 100% plagiarized since this site can not know I was the one submitting them.

For teachers, this is not good news. Is a student plagiarizes material from obscure sources, they are likely to escape detection. Likewise, Webmasters and those that might want to use this tool to track their own content, will likely be disappointed that it doesn’t seem to pick up when the infringement is only a few dozen sites.

This can most likely be fixed through tweaks in the algorithm, but as it sits right now, it doesn’t appear that it has much to offer teachers or Webmasters, especially when Copyscape is relatively effective and cheap to use.

Simply put, at this moment, Copyscape is easier, more effective and faster than The Plagiarism Checker and, at only five cents a search, is affordable too.

However, the best technique still appears to be taking the time to select good phrases from a work and manually searching for those. It returns the most results and seems to work well nearly all of the time.

The Big Picture

My issue with The Plagiarism Checker has less to do with the service itself and more to do with how others have been promoting it. The site itself is actually fairly humble about what it can do, but bloggers and Twitter users have been advertising it as if it were a silver bullet to detect plagiarism. Clearly, that is not the case.

With a few tweaks and fixes to the algorithm, I don’t doubt that this service, much like Copyscape, could become a very powerful tool. However, even if the results were on par with Copyscape, the latter remains faster and easier to use, meaning that there will not be much reason to use the “Dustball” checker.

To make matters worse, most teachers and professors have access to services such as TurnItIn that are far more accurate and covers a much larger breadth of sources than “The Plagiarism Checker”. Considering the ease of us and added features, there is not much that can be gleaned from a Google-only search, that can’t be gleaned from the more automated service (Though Copyscape did top Turnitin in a recent plagiarism detection study).

In short, I don’t see much usefulness for this tool, even if its accuracy improves, and I and more than a little confused as to why so many seem to have promoted it so heavily.

Conclusions

More than anything, this is a case against the reliance on any one plagiarism checking service. Even the best services will let results slip through the cracks. Furthermore, just because a service is popular does not mean that it should be trusted above all.

However, I find it very difficult to fault The Plagiarism Checker for this confusion and these problems. It is clear that the service was as much an experiment as anything, it is promoted humbly and was actually abandoned for approximately six years. It was others, perhaps desperate for some way to more effectively detect plagiarism, that gave it an unjustified reputation.

If anything, this case shows the need and the potential market for such services and illustrates why some companies have made millions in this field. People are eager for a solution and are excited by any promise of one.

Sadly though, this site is not the one people are looking for.