Is Lijit Creating Spam Blogs?
By Jonathan Bailey • Feb 12th, 2009 • Category: Articles, Products
Lijit is a popular and well-respected search application for bloggers. Many sites, including this one, use it because it not only drastically improves upon the default WordPress search, but also because it allows searchers to pull content from sites within the blogger’s network, including social network sites and other blogs.
However, a new service of Lijit has been causing some controversy. It’s new content networks service has already been accused of being similar to scraper and spam blogs.
So I decided to take a look and see for myself. However, what I found very worrisome and gave me a great deal of reason to caution bloggers to pause and think before they made the jump to join or create a network on Lijit.
The Big Idea
The idea behind the content networks service is one that should be familiar to bloggers and Webmasters. In these networks, a group of bloggers in a similar subject come together and pool their content into a single site (or as Lijit calls it, “publication”) and, in exchange, content providers get their sites linked in the blogroll of the hub site and have the articles linked back to their source.
In short, content networks are not “networks” so much as “aggregators”. These sites act as a hub for all of the content on the member blogs, in some cases republishing the full content from the RSS feeds.
The theory is that, if member sites link to this hub, visitors will explore the network site, find new content to follow and this will increase readership for all of the blogs involved. It is very similar, in that regard, to other networks and traffic-generating schemes that have been tried over the years, usually without success.
However, the question remains, is this service a spam blog and should bloggers consider signing up?
The Spam Question
Determining whether or not these content networks push the boundaries of spam blogs or scraper blogs is pretty difficult as the definition itself itself is hard to nail down. What we can do is look at how the sites operate, the good and the bad, and make a determination. For this purpose, we will be looking at the network blog for their new Security Blog Network.
The first thing that most will notice is that the network site is scraping the whole content of the RSS feeds, including any footers, such as Feed Flares, that are added. Though other network sites, such as the Cycling Bloggers Network, use truncated feeds it is clear that the service is both capable of and often does use the full content.
The good news is that, once you look past the full content reuse, it is clear that the site does follow many of the content reuse best practices. On the site in question, clicking the headline of any given story will take you to the original story and the “Comments” link also goes to the original page as well. I was unable to find a “permalink” that did not reference back to the original source. Furthermore, all of the links to the original content are “dofollow” links, ensuring that the search engines will pick up the original sites.
On the flip side, images are hotlinked from the original source and the entire site can, and is, easily indexed by the search engines. Currently, the robots.txt file does not ban any search engines from any on the site (though only tag pages seem to be indexed right now as the permalinks point to the original content).
Though it is clear that these network sites could do more to minimize the negative impact they could have on the bloggers that join them, it is also clear that they are not trying to outright rip off the content of hapless Webmasters who sign up.
The end result is that, while I don’t think Lijit is doing something that is outright spammy, I also can’t advise anyone to consider joining one of these networks. Between duplicate content issues, network imbalance and oversaturation of content, it seems likely that many will find more drawbacks than benefits.
Conclusions
I recognize that many will disagree with me when I say that Lijit’s content networks aren’t outright spam. As the Mashable article pointed out, the appearance, presence of unwanted RSS content (such as Feed Flares) and bold advertising blocks seem to further that notion.
Indeed, if I found that one of these sites were scraping my content I would likely assume it was a spam blog and react accordingly. These sites look like spam blogs, scrape content like a spam blogs and will likely rank better than most spam blogs due to the large number of legitimate sites linking to them.
To some, they might not be spam, they are something worse.
However, it is important to remember that these networks are purely opt-in. Sites have to register and agree to let their content be used in this manner. If a blogger doesn’t want their material to appear on one of these sites, all they have to do is not register.
If bloggers want to allow their content to be used by a site that shares many features with a spam blog and provides what would seem to be little hope of an equitable return, this becoming more true the larger the original blog grows, then it is not my place to say no.
Though I would feel better if Lijit would add extra protections to ensure that search engines are not confused, such as mandating partial feeds or blocking the search engines from indexing the site, the end decision is the blogger’s.
Personally, I find that these sites border a little too closely to spam blog behavior for me to seriously consider joining. Unlike other content networks, such as BlogBurst, which add editorial value and a human element to the reuse, Lijit’s service is more about straight scraping and republishing.
However, the service, at this time, doesn’t sour me enough on the Lijit name and product to drop their search tool. I don’t think that Lijit is “evil” or “bad”, but that their new service, while well-intended, has some potentially ugly side effects.
 |
Protect Your Work. Subscribe to Plagiarism Today via Email or RSS. |
Jonathan Bailey is The Webmaster and author of Plagiarism Today, which he founded in 2005 as a way to help Webmasters going through content theft problems get accurate information and stay up to date on the rapidly-changing field. He is also a consultant to Webmasters and companies to help them devise practical content protection strategies and develop good copyright policies.
Email this author | All posts by Jonathan Bailey
If they're opt-in, I have no problem with them. This indicates the content owner is ok with it.
Jonathan, these are great points. Our intent is always to provide value to publishers. The content networks themselves are interesting, because they create a central repository of content around a specific topic area, while driving as much traffic back to the publishers as possible. We want everything to go back to the publishers themselves, comments, traffic, links, etc.
The main site is really a hub in the wheel. Its a place someone can go and discover great content on bloggers that may have been hidden on Google or otherwise.
At the same time, your comments around potential abuse are extremely valid, and we take them seriously.
Currently, we are reviewing each request to start a content network manually. We will determine if the manager of the network has the intent to provide value to the members of that specific network. We will periodically review networks to make sure that the managers havent abused the system, and if it appears that someone is doing bad things, we will remove them (or take the network itself down).
Lijit was founded on the principle that adding trust to search creates a better search experience. You can discover new content and learn more about a publisher based on their trust relationships. The content networks are a great way to leverage our technology to provide a real resource around specific topics.
Thanks again for being a Lijit publisher, and providing some real and valuable feedback! Contact me directly if you have any other thoughts – micah [at] lijit [dot] com
I agree with the opt-in nature of it, the only ethical question I whether the people who do opt in are fully aware of what is involved…
Thank you for your responses to this. I'm glad that Lijit is taking these issues seriously and working to minimize them. The fear that I have though is that these hub sites will become not intentional, but accidental spam blogs. Not such a much a case of abuse, but unintended consequences.
I'm going to email you shortly with more questions and thoughts.
Jonathan, Lijit started this service for the bloggers we serve. We take this subject very serious and do not support the creation of Spam blogs.
The terms of service creating a network on the infrastructure state that the site editor is responsible for recruiting the individual publishers to the network. Liijit has no interest in moderating the content produced by the network but we will research and remove any feed that the publisher himself did not agree to providing.
Since the press release went out we have been contacted by hundreds of blogger interested in the positive aspects of creating a community around their content, driving traffic, receiving revenue, and being part of the discussion. The concept is designed to be a win/win for everyone.
Thanks for taking a look and feel free to contact me at Lijit if you have any concerns.
I saw comments from Lijit team members which give a great viewpoint to the company's focus/intent on the Content Networks, but I figured I'd add some context from my lens as a content provider and member of one of these networks.
Aggregation is aggregation, but there is something unique here and that is Lijit itself and what they provide vis-à-vis the Blogger Network concept: sophisticated search and content discovery at such a focused level, it is an immense service to a Network’s readers. As a contributing member of the Cycling Bloggers network as an example, I am excited to have my content added side by side with other bloggers and authors I immensely respect, but what is on offer to our readers is absolutely unique. They get to experience a method of social-graph content search no simple aggregator can match. I enjoy the fact that a reader of the Network can search for and find a specific piece of content they desire woven through and across the contributing member social network..e.g. their content and peripheral social-graph. Moreover, we, as content authors, can help refine how our content can be used to expose unique external content, products and services we believe in to our Network. It’s immense. Readers win, the authors win and those we expose win.
So I’m stating the ‘obvious’ here about the uniqueness of the Network and what it offers. You’re right: if mishandled, there could be issues exposed. While this is ramping up, the content network managers need to review and elect all who want to become part of a Network or grow a net-new Network themselves….ensuring legitimacy, ethics, etc. This concept and foundation of trust will, however need to be broadened ATMO to ensure it is scalable to allow wide/global participation. I believe Lijit has this at the core of the Network’s ethos or I wouldn't be involved.
Hi Jonathan,
The Spam issue and Plagiarism is a critical component for Blog Publishers to both be aware of and consider, and your thoughts on Lijit are appreciated. Bloggers need to think before install of any recycling of content. There is also the simple viability issue concerning revenue based widgets like Lijit. The Bloggers Union currently advises Publishers currently using or considering embedding Lijit Search into their content: Lijit’s viability as an ongoing functioning entity is questionable, and to consider the impact a non functioning widget would have on any foundation incorporating it. It's only a 'win-win' when Publishers are treated as partners through transparency.
P.U.B. basis this alert these facts:
1. Public releases from Lijit and other news sources put Lijit’s total money raised beside live revenue at under $20 Million over the last 24 months.
2. Lijit has refused to offer transparency its financial condition or revenue from advertising on Publisher’s Blogs.
3. The current economic environment demands both transparency from revenue based providers like Lijit; and due diligence from the Bloggers Union on avoiding potential content disaster for Publishers.
P.U.B. continues to take heat asking hard questions for our Publishers, including threats of legal action against the bloggers union by organizations like Lijit, because it’s simply the right thing to do in this economic environment for publishers. P.U.B. would rather pro actively protect Publishers now, then be answerable to publishers after companies like Lijit fail and leave that result for Publishers to contend with concerning their content.
What to do:
1) Have a back up.
If you are a publisher either using Lijit Search or considering it, have a back up plan using a publicly traded search option on your blog or blog network. Publicly traded companies are obligated by law to have a degree of transparency concerning their financial position.
2) Ask the same hard questions of Lijit and other revenue based blog vendors.
Contact these companies seeking use of your content directly. Ask what their financial position is, and what their guidance moving forward for 2009 and 2010 and beyond is.
P.U.B. will continue to post on these issues for our Publishers, thank you for your blog on Plagiarism for our community.
Thank you for your feedback. I'm glad to know that Lijit is taking these issues seriously. I'm already in contact with Micah and will definitely reach out to you if I have any further questions.
I agree with you completely about the core ethics of the network and of Lijit. I am merely cautious of unintended side effects. As you pointed out, aggregation is aggregation and, though you can provide unique content that the network can not on your site, which I think is a great thing, it is up to the search engines to decide which is the most valuable and they are notoriously unreliable.
I'm not saying it is a bad idea and I think the network you are in is one of the ones that makes the most sense, fewer members, truncated feeds, etc. But there's a lot of ways this can go very bad. Not all networks will be the same here as they will all have different feed usage rules and different membership set ups. From what I see, your network is much better set up than the security one.
I guess what I want to urge is caution. There's a lot of ways this can head south and Webmasters need to think hard before they leap…
Their spider hit my blog 31830 times between the dates of December 9th and 10th. I've emailed their support to see why in the world their spider would do that. Seems sort of fishy to me.
Sounds like a bug to me, albeit a very ugly one. I hope they get it fixed…
I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.
Miriam
http://www.craigslistguide.info
I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.
Sarah
http://www.craigslistdecoded.info
I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.
Miriam
http://www.craigslistguide.info