net.wars: The 5 percent solution

by Wendy M Grossman | posted on 07 August 2009

So much has been said about Australia's Internet filtering this year that nearby New Zealand's project has mostly escaped notice. The plan is to implement filtering sometime in the next couple of months. Unlike the UK, where the blocklist is maintained by the Internet Watch Foundation under a voluntary arrangement, in New Zealand the list is being administered by the Department of Internal Affairs.

Wendy M Grossman

It turns out that the technology New Zealand is putting in place is coming into use in the UK, courtesy of Watchdog International, which recently signed a deal to supply it to Talk Internet.

Watchdog's managing director, Peter Mancer, says the idea for the technical implementation comes from Sweden.

"I was impressed at the cooperation of police and NGOs," he said of the work he observed there, "but I don't like DNS poisoning. It's not effective enough and it's too broad a brush, and my ten-year-old can bypass it by putting someone else's DNS servers in the browser settings. But it's easy to employ from the ISP's point of view." DNS poisoning – or rather, blocking selected domains – is, of course, what is implemented in the UK through BT's Cleanfeed.

The system Mancer was shown by the Swedish royal technical college and now supplies via his company relies instead on Border Gateway Protocol, or BGP, the core routing protocol of the Internet. Users don't interact with it directly; it's used among ISPs to route traffic correctly. In New Zealand's case, the necessary servers are all managed and hosted by the government. Mancer's explanation: "All ISPs connect to those servers via Internet tunnels using BGP, so the URL list is managed independently of the ISPs, and there is very little cost to the ISP – a few configurations and they're connected to it."

The point for the UK: Cleanfeed requires implementation effort from the ISP. If you're Virgin or another huge ISP, you have sufficient resources and in-house expertise to do it. But the difficulty and expense is, says Mancer, one of the reasons why smaller ISPs haven't adopted it – and why the percentage of British consumer broadband users covered by the IWF blocklist has remained stuck at 90 to 95 percent for years.

Smaller ISPs, says Mancer, "find it quite a challenge. Cleanfeed is not suitable for a lot of ISPs, and there's no commercially available system." So, he says, to the "remaining 5 percent tail which the Home Office and the government keep jumping up and down about a commercially available solution is more attractive." Watchdog's system starts at €2,000 per year, or about £200 per month, and the cost per user goes down as the number of users goes up. Despite the horrid economics of running a small ISP, 5p per customer per month ought in theory to be affordable.

All of this leads back to the question we posed in a panel at this year's Computers, Freedom, and Privacy conference: can the Internet still route around censorship? Images of child abuse (the IWF's preferred term) are illegal in most countries.

Even the US is beginning to show signs of moving in the hotline-voluntary blocklist direction. Last year, for example, Qwest began blocking access to a list of sites that the National Center for Missing and Exploited Children has identified as containing child pornography. (This is not, by the way, a violation of the First Amendment right to free speech as far as I can make out. The First Amendment says, "Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances." It does not prohibit private companies like Qwest from making their own rules, a reality that seems to be widely misunderstood.)

Mancer himself is passionate on the topic: "I sat on a Swedish hotline and took some of the reports and looked at sites. It really does impact you, and it's worth fighting against." He adds, "We're a bit frustrated. We believe we have a good solution that's affordable, but a lot of ISPs are sitting on the fence." There isn't, he concludes, enough pressure.

Given some odds and ends of possible failures – the link to Watchdog's servers has to stay up, the ISP has to configure its systems correctly – Watchdog's system seems likely to be hard for Web users to bypass, although Richard Clayton, the expert in these matters, queries whether the technology will be able to track changes fast enough to deal with the fast-flux technology in use on botnets.

But Clayton also sugests that blocking Web sites is becoming quaintly old-fashioned.

"The IWF list is down to c. 400 sites (from 1500+, of which about 1/3 are 'free' sites – ie: a single phone call would remove the material)," he said by email. In other words, the Web may not be able to bypass the technology – but things like TOR, Freenet, closed peer-to-peer networks, and that wacky darknet-in-a-browser project showed off at Black Hat last week probably can because they were deliberately created to bypass the domain name system entirely. The Web is not the Internet. The Web may no longer be able to route around censorship, but the Internet still can in the time-honored way: by changing technologies. Originally, John Gilmore's aphorism referred to…Usenet.

Technorati tags:   
Filtering; any tips? - You can discuss this article on our discussion board.

Wendy M. Grossman’s Web site has an extensive archive of her books, articles, and music, and an archive of all the earlier columns in this series. Readers are welcome to post here, at net.wars home, follow on Twitter or send email to netwars(at) (but please turn off HTML).