Defending against scraping, however, is incredibly difficult. Though some plugins and tolls like Bad Behavior for WordPress and simple blocking of bots can help, they aren’t perfect or complete solutions and in some cases, can deeply drain both your time and your site’s resources.

However, the team over at Distil thinks they have found a better way. By acting as an intermediary between the Web and your site, they claim to not only be able to filter out most scrapers and infringers, but also to speed up your site and improve its performance.

How it works is by combining the their anti-scraping and bad bot technology with a robust content delivery network, this enables them to not only filter out threats to your site, but also serve much of your static content quickly and from servers located nearest to your visitors.

But is Distil worth the time and money? I decided to give it a trial and see what I found.

What is Distil?

The closest comparison one can make to Distil is Cloudflare as both use DNS changes to better protect and speed up your site.

With Distil (or Cloudflare) you edit your DNS settings, which can usually be found at your domain registrar or in your site’s control panel, to direct visitors not to your server, but to a custom nameserver from Distil. Visitors will then query Distil for your site, which first filters out any malicious users and then delivers any content it can from its servers, which are spread all across the world. Anything it can’t deliver, it queries from your server and then provides to the user directly.

The end result, if all goes well, is that most of the content of your site is delivered directly from Distil’s servers, which should be faster than coming from your own, and most malicious users, including scrapers, are filtered out before they ever reach your site or your content. Best of all, the process is completely invisible to end users (other than the potential speed increase).

To find out, if it works as advertisers, I switched Plagiarism Today over to Distil last weekend and, as of this writing, have been using it for the better part of a week.

Setting Up and Using Distil

To start using Distil, you have to first sign up for an account and have it activated. Once that’s done, you’ll be given an address that, using your DNS settings, you will direct both your www.domain.com and domain.com (as well as any other subdomains you want to redirect).

Then, after the DNS servers propagate, you should be using Distil’s service. From there, you can log into the Distil dashboard, which lets you configure a variety of options including:

• Site Acceleration Settings (if available)
• Rate Limiting
• Blocking Known Violators
• Blocking Bad User Agents
• Browser Integrity Checks
• Filter By Country
• Block Bad Referrers
• Whitelist/Blacklist
• WWW/Non-WWW Routing

You also get a bevy of statistical data including information about the number of unique sessions, the total number of requests, total human requests and the total bot requests. Bot requests are then further broken down by the number of search engine requests (which are always allowed) and the number of blocked requests (as well as the reasons for being blocked). The blocked bots are then further broken down by bot type, IP address and more.

The result is that you get an overall perspective of what’s going on with your site, both in terms of human traffic but, more directly, the security threats you’re facing.

But does that make Distil worth trying? A lot of it depends on your needs and what you’re looking to get out of it.

The Good of Distil

The one thing that immediately struck me about Distil is the granular level of control it gives you over security issues. Though Cloudflare offers a good deal of site security, it’s focused on spammers and attackers and only lets you set a broad level of security (low, medium, high or basically off). With Distil, you can set individual options to your liking both to target the threats most relevant to your site and, more importantly, make sure you don’t interfere with legitimate users.

Over the past few days I’ve had no reports of legitimate visitors being hassled by Distil, something that was an occasional problem with Cloudflare, especially for visitors from outside the U.S. and Europe.

So, even though Distil did not block as many bots as Cloudflare (likely because I have the security settings for most features turned down or off), it did a better job staying out of the way and still seemed to stop the most egregious offenders. Over time, I plan on slowly increasing the settings to see if they block more and continue to be non-intrusive.

Beyond security, my first concern after switching to Distil was that my site might take a performance hit. Having been a Cloudflare user for many months, I was used to the power of a robust CDN. However, I did a series of tests both before and after the change and found that Distil was usually slightly faster than Cloudflare, often shaving off 30% of the site’s loading time.

Compare these two example results, first before:

And then after:

(Note: While this example isn’t an apples-to-apples test due to differing endpoints, the results were consistent regardless of endpoint. Also, obviously there were other changes made in the four days between the tests, though no major alterations, frontend or back, were made.)

Finally, the support team at Distil is, simply put, the best of any company I’ve worked with. They answered every question I had very promptly, usually within 15 minutes and it didn’t seem to matter what time of the day I was asking it. This enabled me both to get my site set up quickly with Distil despite some confusion and questions and deal with an issue with Google Analytics (that turned out to be my own fault).

All in all, Distil did a good job in providing granular security control, a site performance boost and great support.

The Problems with Distil

The biggest initial problem with Distil is that, in its current form, it is not very simple to use. Not only do you have to wait for your account to be activated by a human, but the process of switching over your DNS is not as straightforward as Cloudflare.

If you aren’t comfortable working with DNS and aren’t familiar with how to edit CNAME and A records, the process is going to be intimidating. Sadly, unlike Cloudflare, there isn’t a great deal of hand holding unless you contact support. While I agree with Distil that’s better to not hand over total DNS control to a third party, as you have to do with Cloudflare, it’s also the much more difficult route for the user.

Another issue I have with Distil is the current pricing structure. The free account, which does not have content acceleration, offers only 5 GB of traffic per month, an amount even a modest blogger will likely blow through quickly. A site Plagiarism Today’s size fits (barely) under the cap for the small account, which offers 50 GB of transfer for $29 per month. However, Cloudflare’s free plan allows for unlimited traffic and it’s pro account, which offers additional statistics and monitoring, is only $20 per month. Other CDNs, such as MaxCDN, charge only $50 for 1 TB (1000 GB) of data.

Distil told me that they are considering restructuring their pricing in the coming weeks, a move that, most likely, will help with this problem.

For now at least, Distil is a terrible deal as CDN though its security features may help to make it more compelling to webmasters concerned about scraping and content misuse.

Finally, Distil, obviously, won’t be able to help with at least some kinds of scraping. RSS scraping likely won’t be blocked unless the bot doing it is already in the system and it is unclear just how many are. However, if you know the bot you can add it yourself in your control panel. Also, any human copying won’t be blocked because the system is designed precisely to allow humans to access your site.

Despite these limitations, there’s still a lot of webmasters who would likely benefit from Distil, even if that number could be a great deal larger down the road.

Bottom Line

Distil isn’t perfect. It’s a new company and it’s product certainly has its share of flaws. Right now, it’s aimed at a fairly niche market of webmasters who are technically savvy, want a great deal of granular control over their site’s security and are willing to pay extra to make it happen.

However, with some changes to its setup procedure, pricing and control panel, it could become a compelling option for many more sites.

In short, Distil is going to be a company to watch in the coming months and years. As it refines its tools and pricing, it could become a major force for helping content creators protect their work.

In the meantime though, other webmasters just wanting a CDN to improve their site’s performance will, most likely, want to look up other solutions, such as Cloudflare and MaxCDN as they are significantly cheaper and, in the case of Cloudflare, provides better analytics, easier setup and at some decent, if simplified, security features.

Still, if you’re in Distil’s niche, which is likely to grow, I can see why it would be a very powerful solution to a complex problem.

Have any suggestions for the 3 Count? Let me know via Twitter @plagiarismtoday.

1: New Version of SOPA Copyright Bill, Old Complaints

First off today, Rep. Lamar Smith (R-Texas) has announced that he has made several tweaks to the Stop Online Piracy Act (SOPA) to address “legitimate concerns” about the bill. Those changes include narrowing which sites could be targeted by rightsholder lawsuits, restricting which sites would be ordered to remove such sites from their queries and enable the removal of just a portion of a site. However, the heart of the bill remains, including giving copyright holders the ability to order ISPs, payment processors and advertising networks to block or stop doing business with so-called “Rogue” sites. As a result, the changes have not done much to appease critics of the bill. The bill is due for a vote in the House Judiciary Committee tomorrow where it widely expected to make it through.

2: Righthaven Backed Into a Corner; Copyrights to be Auctioned

Next up today, Righthaven, fresh off a string of defeats appears to be backed into a corner as a judge appears to be ready to auction off Righthaven’s copyrights, possibly marking an end to the company’s litigation campaign. Righthaven had sued some 275 defendants over using content from the Las Vegas Review-Journal and the Denver Post but filed as the copyright holder rather than the law firm representing the companies. They lost several cases based on a lack of standing to sue as it was revealed the newspapers had retained control over the copyright. The Review-Journal did eventually assign control in some of the works to help with Righthaven’s appeals but, after being ordered to pay legal fees for several defendants and unable to muster up enough cash, Righthaven may have those same copyrights auctioned off, giving it nothing to sue over.

3: Bratz Copyright Lawsuit Tossed

Finally today, one of the Bratz lawsuits was tossed but it wasn’t the big one between MGA and Mattel. Instead, it was the one that pitted photographer Bernard Belair against MGA, the makers of the Bratz line. Belair had created a series of advertisements in the late 90s featuring oddly proportioned dolls. A representative for MGA admitted that the ads were inspiration for the Bratz line but a judge ruled that the inspiration did not cross the line into infringement as it didn’t rise to “substantial similarity”. There is no word if Belair plans on appealing.

Suggestions

That’s it for the three count today. We will be back tomorrow with three more copyright links. If you have a link that you want to suggest a link for the column or have any proposals to make it better. Feel free to leave a comment or send me an email. I hope to hear from you.

Want the Full Story?

Tune in every Wednesday evening at 5 PM ET for the live recording of the Copyright 2.0 Show or wait and get the edited version Friday right here on Plagiarism Today.

The 3 Count Logo was created by Justin Goff and is licensed under a Creative Commons Attribution License.

Rather than talk about the bill itself (Note: Read Terry Hart’s CopyHype and Mike Masnick’s Techdirt for pro/con analyses on the issue) I wanted to talk about the technology behind it and what it means for copyright holders both large and small.

However, to do that, we first have to take a look at what DNS is, how it works and how, likely, how these bills would work with it.

Then, and only then, can we start to figure out what the likely impact is going to be.

What is DNS?

DNS stands for Domain Name System and it is a distributed system for converting the domain names we humans understand and the numbers that machines on the Web need.

Basically, every computer on the Web is identified by an IP address. Though multiple machines or sites can have the same IP, for another computer to get to yours it needs the IP address to get there. IP addresses, in the current version, are a series of four numbers from 0 to 255 separated by periods. If you go to WhatIsMyIP you’ll see what your public IP address is.

While this system is great for machines, humans would find it difficult to remember the IP address for every site they wanted to visit. DNS was created to let humans use easy-to-remember domain names (IE: plagiarismtoday.com) and translate those domains into machine-usable IP addresses.

To enable this, there are DNS servers positioned all over the world. These servers function like a telephone directory, converting the domain name into an IP address. When you type in a domain your computer hasn’t been to recently, your computer queries your designated DNS server, gets the DNS record and visits the site.

Generally, if you haven’t altered your DNS settings, you are using a DNS server provided by your ISP. That server, in turn, got its information from the root DNS servers, which sit atop of the DNS tree. Every time a change is made to a domain’s IP, the root servers are informed but it takes a while for that information to trickle down to the user-facing servers who typically cache DNS data for anywhere from 4-72 hours.

This is why, for example, when you move hosts it takes a while to show the changes on your home computer.

The main thing to remember though is that DNS functions like a phone book, listing domains and their IP addresses and converting the two for your computer. Without it, domains don’t work properly and you have to manually type in IP address information.

How DNS Blocking Works

What this has to do with the above bills is fairly simple. According to them, a rightsholder could get a court order that forces DNS providers in the U.S. to block access to a “rogue” website.

That would, most likely, work by having DNS providers simple remove the site’s domain from their list, similar to removing a name from the phonebook. Either that or the line in the database would be changed to point to the incorrect address, similar to changing the number in a phone book to direct callers elsewhere.

Similar blocking techniques are already fairly widely used. On the smallest level, many homes and businesses use similar DNS alteration techniques to block access to malware, pornography and other sites they don’t want those within visiting. OpenDNS, for example, does this for users for free.

On a grander scale, this kind of DNS filtering is part of many countries’ attempt at filtering the Web and is a key component of the Great Firewall of China.

So, while the tech has been used for good and evil in different situations, the more important question right now is “How effective would it be?”

The Effectiveness of DNS Filtering

Though DNS filtering sounds like a death blow to a website, it is anything but. After all, the site still exists and it does so at the same IP address, it just means a computer using that DNS service can’t get that IP using the domain.

This gives several options to a site that is blocked, including getting a new domain or directing visitors to use their IP. Users who want to get around such a block can do so easily as well by either switching to a different DNS service, possibly one located in another country, using the IP address directly or using a proxy (connecting to the Web via a third party computer) to bypass the DNS service.

In short, hardcore pirates would not likely be deterred, at least not greatly. A recent court order in the UK to force ISPs to block the site Newzbin 2, a Usenet scraper, has met with, at best, with mixed results as the audience of Newzbin 2 is already tech-savvy and less-than-casual in their piracy.

Casual pirates, however, might be more frustrated. Though it’s trivial to change DNS providers, a recent malware scam did it on some 4 million PCs without users knowing, it’s not something that most users know how to do or would feel comfortable doing.

Furthermore, pointing DNS to an untrusted third party can lead to other security risks as they can literally direct any domain to any server at will, raising issues of malware, identity theft and more. This makes the risks and effort higher than the reward for many casual pirates.

A lot of these workarounds could be mitigated by including IP address blocking with the DNS blocking, thus preventing anyone from accessing the IP address directly, but sites can and do change IP addresses regularly without much effort. Also, maintaining such a list would be a much greater challenge and greatly increases the risk of non-infringing sites being blocked as, at times, thousands of sites can share one IP address.

In the end, the question isn’t if someone will be able to get around these measures, it’s a matter of how many and what number of hurdles will they have to leap to make it happen.

What Does This Mean for Me?

It’s almost impossible to predict what the bill would mean for the Web if passed. There are predictions on all sides but much of the actual impact would depend on how courts approach the new law and how it is applied. That, as the Digital Millennium Copyright Act (DMCA) has showed, can be almost impossible to predict.

There are predictions on both sides ranging from sites like YouTube being blocked to a much more limited use where only a handful of extreme “rogue sites” being blocked. Those in favor of the bill claim it will only target the “worst of the worst” while those opposed to it claim it’s so broad it could impact almost any site.

Both of those predictions are theoretical until the law passes and the courts begin to wrangle with it.

One thing that is certain, however, is that smaller copyright holders probably will see no benefit from it. Even if the bill isn’t used just on the “worst of the worst”, getting a site blocked still requires a court order. Filing suit for copyright infringement is far too expensive for most smaller copyright holders, making it so that they will unlikely be able to use this bill in any way. (Note: Some versions of the bill have it so that rightsholders can force payment processors and advertisers to stop working with infringing sites on a notice-and-takedown basis rather than a court order).

In short, the most likely way a small-to-medium sized rightsholder will see anything from this bill is if they are visiting one of the site to get blocked.

Bottom Line

As hinted above, the bills have a lot more to them than just website filtering, they also include provisions for forcing payment processors, advertisers, search engines and other companies to stop doing business with or listing such sites. These provisions, in the long-run, could have a much larger impact on the Web.

One thing that is clear though is that these bills are a hot topic right now and have people lined up on both sides to either support or condemn them.

This makes it all the more important, to me, to understand how the technology would work and analyze what the likely outcomes of the tech would be.

While we can’t accurately predict how the bill would be applied, we can understand how it would work and that can give at least some insight to the legislation and its importance for the Web.

DNS Tree Image By: LionKimbro on Wikipedia – Public Domain

In 2007 I wrote an article entitled “Why I Embed My Images” that discussed how embedding images and other can provide greater security when you feel there is a risk someone might file a takedown notice. By separating your images from your server, should someone file a takedown notice over an image, your site will remain active and, with good backups, you can get your site back up more quickly.

It is a way to guard against misuse of the DMCA or fair use disputes.

However, since then I have backed away from that stance. Once I moved to my new VPS, I stopped hosting images remotely as I have a good relationship with my host and have no reasons to worry. That being said, in an effort to improve the efficiency of the site, I’ve also started toying with Amazon S3 to see if it can help improve the site’s speed (the images in this post will be hosted on S3 as part of the test).

It was at this point that I realized a problem. If I were malicious in my use of S3, or any similar service, it could be used as a method not to prevent complete site failure, but to avoid a DMCA altogether. It is possible, using these services, to trick users into filing complaints with the wrong hosts, delaying or even preventing anything from being done.

I immediately, using my own site as a test subject, began to seek a way around it and, fortunately, found a way to ensure that, no matter where a file is hosted, you’ll always be able to track down the host with reasonable accuracy.

The Nature of the Problem

If you right click on the images in this post and view their URL, you’ll see that they are hosted on a subdomain of Plagiarism Today named “files.plagiarismtoday.com”. This makes it appear, including to many automated tools, that the content is hosted on the same server as the rest of the site. The problem is that they are hosted on Amazon S3, clear across the country.

This trick is fairly trivial to do and only involves a minor tweak to DNS. There are many legitimate reasons for doing it, for example, hosting images on your domain while using a content delivery network to increase speed.

However, if a copyright holder decided one of these images were infringing, filing a DMCA notice would be difficult. The reason is that since the files are on a subdomain of plagiarismtoday.com most will assume it’s located on my server and act accordingly. This is due to a fluke in both the way we read URLs, where we routinely ignore subdomains, and the way networking tools routinely discard subdomain information.

Some copyright holders, especially those less familiar with DNS and networking, might not consider this and could inadvertently file a DMCA notice or other abuse complaint with the wrong host. This can result in a delay in getting a complaint resolved, in it being outright ignored or even causing it to be handled in a questionable way.

The good news is that there is a simple way around it and, as long as you are careful about how you gather your information, there is no need to make this mistake.

Dealing with Linked Files

When you’re dealing with an image file or any content that is linked into a Web page (not part of the actual HTML) it is important to make sure that you get the correct information about where that particular file is hosted, not just the page that it is on.

The solution is pretty simple:

1. Get the URL of the File: Rather than copying the URL of the page, right click the image or the link and copy the URL. Check and see if it is on the same site, a subdomain or another domain altogether.
2. Use Who Is Hosting This: Once you have the URL, delete the “http://” as well as everything including and after the first remaining “/” and process it through Who is Hosting This. Who Is Hosting This handles subdomains correctly, unlike Domain Tools, which strips out subdomain information in my testing.
3. Confirm the Results: You can then confirm the results by copying the IP address (you’ll have to actually copy the numbers on the site, not using the link) and then running it through Domain Tools. Once you’ve done that, you can then go forward and begin the work of finding the DMCA or abuse agent and contacting them.

Though this adds a few extra steps to the process, it is worth doing to ensure that you contact the correct party as doing so is the only way to guarantee the quickest and most reliable resolution.

Why This is Important

The reason that this is critical is because sending a DMCA notice to the wrong host, at the very least, will greatly slow down the process as the host has to research and figure out what is going on and then decide if they going to A) Disable the page anyway B) Forward the notice on or C) Do nothing.

Since the company that hosts the Web site does not host the image, their role under the DMCA is much less clear. Section 512(c), which usually deals with Web hosts and takedowns, only pertains to “the storage at the direction of a user of material that resides on a system or network controlled or operated by or for the service provider”. Since there is no storage, a regular DMCA notice doesn’t apply.

Section 512(d) does pertain to “information location tools” but in that case, it would be the site owner, not the host that is party for the notice. This section deals with sites, such as Google, that are “referring or linking users to an online location containing infringing material or infringing activity”. Since the host isn’t the one linking to the file, it is the user, the application of 512(d) doesn’t make as much sense.

This isn’t to say that hosts won’t deactivate sites or remove pages if the content is embedded or hyperlinked, especially if the site is spammy in nature or has other abuse issues, but the fastest way to secure removal of images or other media files is to go to the source.

It can be a bit tedious to do, but it is well worth the time.

Bottom Line

The simple truth is that the days of all of the content on a site being hosted on the same server have long since passed. Content embedding from photo sharing sites, video sites and elsewhere have made it much more difficult easily track down where a particular item is hosted.

Though sometimes, as with YouTube clips, where the content is hosted is obvious, other times, as with image hosts, it is much less clear.

Unless you are dealing with textual works, which are almost never embedded (unless you use a service such as Voxant Newsroom that embeds text via Flash and JavaScript), this is something you have to constantly watch out for.

Dealing with content theft issues is not difficult, but it does require a bit of detective work. However, knowing the challenges you face and the tools that can help you overcome them can keep the sleuthing required to a minimum.

But while AbouThiSite was interesting, its data is woefully incomplete, forcing me to continue relying on other sites for most of my information.

However, long-time reader Voyagerfan5761 alerted me to a similar, though more complete, service that had flown under my radar. The service, Quarkbase, promises to provide “Everything About a Site” and to be everything I wanted AbouThiSite to be.

So, I excitedly gave the site a whirl and learned quickly that it is a huge step in the right direction, but not quite the endgame I was hoping for.

What It Does

Quarkbase works very similarly to other sites in this field. Users either punch in the domain they are interested in or use their bookmarklet, to pull up information from dozens of resources about the domain a site is hosted on.

That information is then broken up into seven different categories, which can either be scrolled through on the default “All” page or quickly selected via the tabs at the top. Those sections are as follow:

1. Introduction: This includes basic information about the site including its name, common tags, contact information, logo and slogan.
2. Social Popularity: This analyzes how well the site has performed on various social sites including Digg, Reddit, Delicious, etc. Also pulls the subscriber account from FeedBurner if possible.
3. Traffic: This pulls in a variety of traffic estimates from Alexa. Though notoriously unreliable, it does provide clues as to which countries most frequently visit a site and a general idea of popularity.
4. People: Takes a best guess at the person or people that run the site. It is not clear where this information comes from.
5. Spotlight: This section attempts to glean who is “talking” about a domain, specifically by looking at Twitter. Though flawed in that it can’t parse TinyURLs, which are heavily used on the service, it still works surpringly well.
6. Company: Only shows up on reports of sites owned by a company. Offers company profile information and job postings for the company that owns the site.
7. Technical: This is the “meat” of the site’s information from an abuse standpoint, providing information on who is hosting the site, what the nameservers are and the location of the server.

Quarkbase is able to do this by bringing together information from a variety of sources including Alexa, Zoominfo and more.

The result of all of this information is that the results page is extremely large and, at times, slow loading. The initial page is also very cluttered, though the tab feature makes it much easier to cut to what you need.

All in all, the information that Quarkbase provides is very robust and very simple to use, however, there are a few hiccups that prevent me from making this service my default.

Small Roadblocks

The biggest issue I have with Quarkbase is that the information, in particular in the Technical section, is questionable at best. For example, when looking up Hostgator’s main site, the site doesn’t even fathom a guess as to the hosting provider. It correctly guesses that the ISP is “THEPLANET.COM INTERNET SERVICES” but even that is not completely accurate as Hostgator is its own host, just using ThePlanet’s servers.

This was an issue that has tripped up other services, including Domain Tools, and prompted a fix from WhoIsHostingThis. During many of my tests, Quarkbase refused to even guess about the host information, instead just leaving that line blank, and the ISP information was dubious at best.

Other information on the service was unreliable as well. Alexa’s traffic data is notoriously unreliable, though still an understandable choice under the circumstances. Also, information in the “Introduction” section is routinely either left blank or inaccurate, especially the contact information on non-company sites.

This limits the usability of the service, especially when competing sites such as WhoIsHostingThis have largely overcome many of the same challenges, but it still remains one of the most complete overviews of a site or domain that you can get, even with the hiccups and speedbumps.

Conclusions

Quarkbase is far from perfect, but what it does it does well. Though its results are not as technically-oriented as Domain Tools or as accurate as WhoIsHostingThis, its sheer breadth of data makes it great for a “quick overview” of a site’s information.

Though I again don’t think Webmasters and bloggers will get a lot of use from this tool when chasing down scrapers and plagiarists, especially since the site does not do subdomains at this time, it could provide some assistance with directly contacting infringers, locating the host and learning about the background of a site before moving in.

If nothing else, it might be worth seeing what Quarkbase turns up on an infringing domain just to see if there is anything you were unaware and can use, such as an email address or contact form.

I can pretty much promise you that you will learn something about every site you punch into this service, the question is how accurate and useful will that information be.

AbouThiSite attempts to be one of latter kind of mashups, providing valuable information about a target domain at the click of a button.

But how useful is it in the real world? The answer, sadly, is not very much. It won’t be a part of my arsenal, not unless it adds some additional data. Still, there is much that can be gleaned from it, if others are willing to listen.

What it Does

The idea behind AbouThiSite is very similar to WhoIsHostingThis and Domain Tools in that you punch in a domain and receive back vital information on it. But where Domain Tools is targeted at those who are familiar with networking tools and WhoIsHostingThis focuses on making the process of finding a site’s host simple, AbouThiSite attempts to provide a different set of information in an easy to approach manner.

This includes the following:

1. The IP Address
2. Other Sites Likely on the Same Server
3. The location of where the site is hosted.
4. The PageRank/Traffic of the Site
5. Information About the Colors and HTML of the Site

This information is then displayed in a colorful and easy-to read page that includes a Google Map of the estimated server location, a preview of the site and a link to subscribe to the site’s RSS feed, if it has one.

It is indeed incredibly easy to use, but, for Webmasters dealing with content theft or abuse issues, it is a fairly useless service. In fact, outside of some limited SEO purposes, I have a very difficult time imagining why anyone would favor AbouThiSite over other sites.

Missing Details

The most useful aspect of AbouThiSite is the SEO elements. Having the PageRank, rough traffic and IP information in one place is useful. Though the traffic stats seem to underestimate every site I punched in, the relationships between them made sense.

However, you can get most of this information elsewhere, the only advantage with AbouThiSite being that the information is very cleanly laid out and easy to read. Whether that is worth the trip is up to each Webmaster to decide.

For those wanting to deal with abuse issues, this site is missing critical information that one needs including Whois information, information about the actual host of the site (other than its location) and provides no easy means to obtain it.

Since all of the useful information can easily be procured off another site and you will have to go there regardless to get the information you need, there is little reason to make AbouThiSite a stop at all.

Lessons

This isn’t to say that AbouThiSite is a bad tool, just that it doesn’t fill any needs that I have. There is still a great deal it does right and others may find it useful.

What I sincerely hope is that other sites, especially Domain Tools, might take a look at the way AbouThiSite displays information and glean a few lessons from it, namely how to put a lot of information about a site in front of viewers in a clean, attractive manner.

Though appearance is definitely not everything when looking for tools to help you get the information you need, it does count, as WhoIsHostingThis has showed us. The easier a site is to read, the quicker we get the information.

Likewise, though WhoIsHostingThis is laser-focused and clean to use, it could also benefit from some additional information, such as the location of the host (at least the country) and, perhaps, the whois data.

The bottom line though is that AbouThiSite offers a glimpse of what a good domain information mashup could be without actually being that mashup.

Conclusions

The bottom line is that, if AbouThiSite has information that you find useful, then by all means use it. It’s a fast, user-friendly site that is easy to pick up and add to your toolbox. I, personally, don’t have much use for it nor do I see how others might.

That being said, it appears to me that the site is more of a proof of concept than a finished product and, with that in mind, the concept it does show is valuable.

The Web may not have a lot of use for this site, but there is a lot it could learn from it, if one is willing to listen.

The techniques for determining who a host is are, at best, complicated and somewhat geeky in nature. Though I wrote a guide on how to use some tools for finding the host, the process remains one of the most common questions I get asked about.

At least one site, WhoIsHostingThis, has attempted to simplify this process. Turning into a Google-style experience. Previously reported on here the site did a respectable job in most cases, though there were some peculiar results on some tests.

The idea is that the networking wizardry should be hidden from the user and the site should receive a domain (or bookmarklet click) and then simply return the host. A great theory, especially for the non-tech oriented, but due to the nature of the work it is not always reliable. Most who are familiar with the tools, myself included, tended to lean on more sophisticated sites, such as DomainTools.

However, an upgrade at WhoIsHostingThis is attempting to change that, by fixing the kinks and bugs and, potentially, making the site a one-stop shop for domain hosting and information.

Some Geek Stuff

The typical way to determine the host of a site is a tool called IP Whois. Basically, IP Whois works like this:

1. All servers on the Web (as well as all computers or routers facing the Web) resolve to an IP address, a set of four numbers from 0-255.
2. Those IP addresses are controlled and doled out by various Regional Internet Registries (RIRs) that are non-profit oversight boards that help control these limited resources. ARIN is the RIR for the United States and North America.
3. When RIRs assign IP addresses, they keep a registry of who is assigned what numbers. That information can be queried by an IP Whois.
4. The most common purchasers of IP addresses are Web hosts, such as GoDaddy, ISPs, such a your cable company, and academic institutions.
5. These institutions then allow their customers to use the IP address for accessing the Internet, hosting a site, etc. but usually do so only on their own network. Most of the time an IP address purchased by company X will point to a customer of their company.
6. Thus, an IP Whois can usually trace you back to who is hosting a particular site or at least who is responsible for the IP address at that particular location.

The procedure is far from perfect and, as we’ll explore there are ways it can be gamed. But it is far more accurate than other methods, such as looking at the DNS servers, which can be trivially changed by spammers and plagiarists.

It is also this method that has been largely utilized by WhoIsHostingThis with great results. However, where the site has struggled has been with exceptions to the rule, cases where the IP Whois is misleading or, worse still, downright wrong.

Though these are cases that can usually be corrected with other tools, such as traceroutes (which look at the path traffic takes to arrive at the destination) or the DNS information, that information has, traditionally, not been used by WhoIsHostingThis.

That is starting to change.

The “HostGator Problem”

In March of 2007, one of the largest moves in Web hosting took place as HostGator, the very popular budget Web host, moved much of its 500,000 plus domains into ThePlanet’s datacenter. Though the move made sense for both parties, it created an abuse reporting kludge that remains.

The problem is this, on those half million domains, the IP Whois information points to The Planet and not Hostgator since they are located within The Planet’s network. Thus many, myself included, have sent DMCA notices or spam reports to The Planet thinking that they were the host. This has created slow downs in addressing critical issues.

However, these problems are largely avoidable as the DNS servers, as well as other information, do point to HostGator as the host. The problem is that the information can be easily overlooked.

So, while this problem can be overcome by humans, it requires a fair amount of skill at reading networking and domain information and, even then, is prone to mistakes. WhoIsHostingThis is seeking to fix that problem by looking at multiple sources of information, including the DNS information, to determine who the host is.

In that regard it has already “fixed” the Hostgator problem, a search on the site for a HostGator domain reveals HostGator as the host, not The Planet. A similar result happens when you look for WordPress.com domains, as it shows WordPress as the host, not Layered Technologies.

Though the site provides the additional information below the main result, in case the results are mistaken, it is right in these cases.

Further Improvements

Though WhoIsHostingThis has already integrated many of the hosts that, like HostGator, have their IP addresses listed as being another service, this is not to say that they have all of them. The operators of the site admit that the site needs further improvements.

However, where the site was previously about 95% accurate with its information, it is now most likely well over 99%. These cases where the IP Whois was wrong were rare to begin with and the site has already fixed most of the larger outliers. This means that only a fraction of a fraction of domains should return any issues.

That being said, there are still issues and bugs to be worked out. For one, where the site does very well with U.S. and Canada-based hosts, international ones, especially those in languages other than English, seem to give the site trouble from time to time. Also, there are still at least some cases where the information might be technically correct, but does not provide a correct URL for the host or enough information to locate it.

However, as I said earlier, these are extreme outliers. For most cases, WhoIsHostingThis works very well and certainly good enough for those that don’t have the technical expertise to use traditional networking tools.

Conclusions

Personally, I’ve begun using the WhoIsHostingThis bookmarklet to help me determine the host of sites and only using DomainTools or other sites whenever I get a strange result. It’s worked very well these past few weeks (since the updates began) and I’ve been impressed with the work that they have done.

Though I’m never likely to use this site, or any other site, as my exclusive resource for this kind of information (best to have confirmation no matter what you use), the improvements at WhoIsHostingThis have really impressed me.

While there is clearly work to be done, the progress is clearly evident and I am very happy with the improvements they have been making.

The video, which lasts about five minutes, takes the user through the process of finding the host for this site, instructing on the use of the Domain Tools Web site and various features of it.

The video is my first attempt at such a screencast so it is far from perfect. There also seemed to be a minor encoding issue when I uploaded the video to Revver. I might try a different video sharing site in the future.

Please let me know what you think of the video. I’ve embedded it below.

So, I’ve started taking some time to rewrite and redraft this portion of the site and I’ve started with the two sections most sorely in need of updating.

New and Improved!

The first section to get an overhaul was the first chapter itself, How to Find Plagiarism. To that chapter, I’ve added two sections, one targeted at bloggers and RSS scraping as well as a section targeted at videographers.

Since spam blogging and video sharing sites were relatively new concepts when the draft was first completed almost three years ago, those sections were not included. However, the article also underwent something of a rewrite in the Non-blogging author section as well as other tweaks to the entire piece.

However, it was the third chapter, Finding the Host, that has undergone the biggest revision. The previous version was practically unusable. Sam Spade, the tool of choice when it was first penned, has been defunct for some time and the tips for determining the host of free Web sites was poorly written and hard to understand.

I’ve updated the page to use Domain Tools instead of Sam Spade and provide a much more detailed set of instructions, including screenshots.

This should bring that page into the modern times and and allow newcomers to the site to effectively use it.

Since editing these files and updating them is a surprisingly time-consuming process. I’m going to be doing this update over a period of a few weeks. If things go according to plan, I should have the entire series, along with other static pages on the site, updated by the end of the year or very early next.

Please let me know if you find any problems with them. I’m going to do my best not to let these pages wait so long before another update and they will be under more constant revision from now on.

According to a recent AP article, the Whois service, a series of databases with information about the individuals that register domains, has come under fire from privacy advocates and a new proposal seeks to do away with the service altogether.

Such a move would be a tremendous blow to law enforcement, lawyers and researchers that regularly use the database. However, it may also alleviate some of the spam and privacy concerns that come with the database in its current format.

No matter what is decided this Wednesday when a committee from the Internet Corporation for Assigned Names and Numbers (ICANN) meets, this will be a major issue to follow and one that will have a major impact on both the structure of the Internet and how Webmasters protect their content.

What Is Whois

When someone registers a domain such a .com or a .net, they provide a set of information including their name, email, address and phone number. This information is placed into a Whois database, which each domain registrar keeps, and is then searchable by using a Whois lookup tool, such as the one found on Domain Tools.

This means that, if you register a domain, anyone can look up the information that you provided for it. If you gave your personal information, that could include your home phone and address.

To alleviate these privacy concerns, Whois protection services such as Domains By Proxy have sprung up to keep Whois information secret. These services work as forwarders, letting you use their address as your own and forwarding you email sent to the anonymous account they created for you. They can also be compelled to give up your actual registration info in certain events, including legal matters.

However, such steps have done little to alleviate the concerns of some privacy advocates. They worry about the requirement of posting personal information in a public space in order to obtain a domain and want better protection for Webmasters.

To make matters even worse, spammers have seized on the service, using it to harvest thousands of addresses and send out bulk mail, often relating to domain names.

On the other hand, law enforcement uses Whois to track down criminals. Consumer watchdogs use it to spot scammers. Also, lawyers use it to locate copyright and trademark infringers and anti-spam groups use it to track and monitor spammers.

In short, it is a very powerful tool with both great limitations and serious drawbacks.

Fixing Whois

The problems with Whois are well-documented.

Beyond that above mentioned privacy issues, it is too easy to supply false information to the database. There is nothing to stop a scammer from just putting garbage into his Whois information and avoid detection.

You can report invalid Whois information but action is unlikely and slow. It is also, generally, limited to the revoking of the domain and does little to actually identify the person behind the site.

This has resulted in a great deal of inaccurate information in the Whois databases and that, in turn, has limited the usefulness of the tool. Because of that, ICAAN has started looking at ways to improve the tool and the organization is actively looking at proposals for fixing it.

However, most of the proposals are much more mild than the “sunset” proposal that has caused such a stir and would do away with Whois by the end of 2008. According to the AP article, a proposal encouraging more study of Whois abuse and the extent of personal registration is much more likely to pass.

But no matter what happens tomorrow when the committee meets, what is clear is that this will be an issue to watch and follow as the outcome of what is decided could drastically change the Web and how we identify the people behind the sites.

What it Means to Us

The good news for Webmasters dealing with copyright issues is that the Whois service is not the tool to use when locating the host of a site. Other tools, such as DNS and IP Whois, are much more valuable in gleaning that information.

However, while that is great for copyright holders that favor contacting hosts and sending DMCA notices, such as myself, it doesn’t bode well for those who prefer to contact the infringer directly. In many cases, the Whois database is the only source for that information as it is not always posted on the site itself.

If the Whois service does disappear, I expect the following things to happen in the realm of content theft.

1. DMCA Notices Will Become Much More Popular: Expect more and more Webmasters to turn to DMCA notices and other host contacts as the methods of contacting infringers directly become more limited.
2. More Subpoenas: Though the information would not be a in public database, the registrar would still have all of the pertinent information, especially if the individual paid with a credit card. Lawyers might not simply be able to look up who owns a domain, but they certainly could subpoena the information if needed.
3. Less Enforcement By Novice Webmasters: The Whois service is popular for tracking down plagiarists because it is very simple to use. You punch in a domain and out comes the information associated with it. Novice Webmasters often lack the knowledge to use more advanced networking tools and, without the Whois database, would have almost no recourse.

In short, veterans of dealing with content theft will likely barely notice the disappearance of the Whois service but less experienced Webmasters may find themselves lost without it.

Conclusions

The Whois service is a powerful tool that has some very large problems and raises some very serious concerns. There is little doubt that the service is both riddled with problems and has attracted unwanted attention from the dark side of the net.

However, that does not mean that one can simply ignore all of the good that it does and the potential uses for the Whois service. Proposing to kill off Whois because of its flaws is an extreme overreaction, especially when the depth of these flaws and the possibilities for remedy are not fully understood.

In the end, I have to agree that further study of the problem is needed before anything is decided, especially something as drastic as eliminating the service altogether. The problems that face Whois are great but trying to shut it down is just another non-answer, a way to avoid dealing with the issues.

Whois needs an overhaul, that much is very clear, but shutting it down and walking away not only does more harm than good, but fails to address the issues adequately and, instead, just shuffles them onto the individual registrars.

We, the Internet-using public, entrust ICANN to deal with the tough issues and not run away from them. Shutting down Whois, at this phase, would be exactly that.

We can only hope that ICANN will see this and do what is right, take a more measured approach to the problem and learn more about the issues at hand before leaping off the veritable cliff.

Related: Mashable

Update: The committee voted 17-7 to keep the Whois database as it is right now and shot down a proposal to allow “natural persons” to designate a third party agent. The “sunset” proposal was defeated 13-10. ICANN is instead comissioning additional studies on the issue and will likely be coming back to it in the coming years. With the closer margin on the sunset option, it appears more likely than previously thought.