Andrew Cuomo, New York’s Attorney General, is fighting child pornography, a noble battle indeed. His office conducted a recent investigation of Usenet, and found that 88 groups out of over 100,000 contained child porn. Last week, “Cuomo announced that Verizon Communications, Time Warner Cable, and Sprint would “shut down major sources of online child pornography.”
The agreements resulted from an eight-month investigation and sting operation in which undercover agents from Mr. Cuomo’s office, posing as subscribers, complained to Internet providers that they were allowing child pornography to proliferate online, despite customer service agreements that discouraged such activity. Verizon, for example, warns its users that they risk losing their service if they transmit or disseminate sexually exploitative images of children. After the companies ignored the investigators’ complaints, the attorney general’s office surfaced, threatening charges of fraud and deceptive business practices.
All three companies agreed to purge child porn from web sites that are hosted on their servers. Each will pay a fine of 1.125 million dollars. Time Warner is dropping Usenet services altogether, Sprint is cutting out the entire alt.* hierarchy, and Verizon is eliminating all except the “Big-Eight” and the Verizon.* hierarchies from their newsservers. Verizon announced to its customers last Thursday, “On 06/24/2008, Verizon will be modifying its Newsgroup offerings to only offer groups in the Big-8 Newsgroup hierarchies, which are listed below. Users will not be able to post or download from any other newsgroups.” Ostensibly in order to eliminate child pornography, these companies are eliminating tens of thousands of groups from their newsservers.
There has been little reaction from the tech community to date; many (like me) may not have noticed the issue until today. A post at Slashdot identifies several concerns. Blogger Declan McCullagh asserts at CNet, “No law requires Verizon to do this. Instead, the company (and, to varying extents, Time Warner Cable and Sprint) agreed to restrictions on Usenet in response to political strong-arming by New York State Attorney General Andrew Cuomo, a Democrat.” TechDirt went further, “The ISPs are also giving Cuomo’s office over a million dollars, ostensibly to help wipe child porn off the internet. If that’s Cuomo’s goal, this isn’t the best way to do it — though, it will get him plenty of press coverage for bullying companies into doing something they aren’t required to do under law.”
This is important. Well…. not like Boumediene vs. Bush is important. But it will certainly have an impact beyond child porn and may be part of one of those slippery slopes with further ramifications for the tension between Internet regulators and providers. I suspect, however, that the ISPs’ decision to roll over here was influenced by several factors having nothing to do with child pornography. And unfortunately, I don’t believe it will have much of an impact on online child porn availability. First, some background for those of you who aren’t familiar with this technology. If you use Usenet, you can skip to the last couple of paragraphs. Usenet is…. umm… Usenet is OLD. Think of it as a hierarchy of blogs, each devoted (more or less) to a particular topic, idea or frame of reference. As with Bulletin Board Systems, everyone who reads a particular group can read what anyone else says and conversations sometimes resemble the Comments section of a particularly busy and partisan blog. Usenet was developed before there was even email. It was originally used by scientists to communicate with each other around the academic world.
There is no central repository or server which holds all of the data. Instead, newsservers around the world send their new posts in a feed to other newsservers, who pass more new content to other servers constantly, until all servers theoretically have all of the posts in which they’re interested… except no two newsservers ever have exactly the same content at the same time. It’s a process.
The central hierarchy, the “Big Eight,” has rules, developed over the years by consensus within the community; the group names start with comp.*, humanities.*, misc.*, news.*, rec.*, sci.*, soc.* and talk.*. Other hierarchies have been added which are not regulated by this particular set of rules, including commercial sets of groups (e.g. microsoft.*, symantec.*, borland.*), country groupings (e.g. fr.*, japan.*, italia.*) and many, many others (e.g. alt.*, biz.*, fido.*, free.*, utexas.*). Some groups outside of the Big-Eight have rules; but any existing rules outside of commercial hierarchies are self-enforced, if at all. Some group names might give you a flavor of the structure.
- comp.sys.mac.games.strategic
- comp.sys.ibm.pc.soundcard.advocacy
- talk.religion.christian.orthodox.greek
- talk.philosophy.epistemology
- soc.genealogy.african
- soc.support.fat-acceptance.moderated
The alt.* hierarchy is the elephant in the living room; it is the reason that it is said “The first rule of Usenet is don’t talk about Usenet.” The alt.* groups started by attracting people who wished to start conversations that were not approved by the nebulous Usenet community for inclusion in the Big Eight. On the newsservers I use, it looks like roughly a third of the groups are in the alt.* hierarchy. You’ll note the newsgroup names are somewhat… different.
- alt.fan.jadzia.dax.slug.slug.slug
- alt.dunderheaded-dolts.dowap
- alt.2600.hackers.programming
- alt.activism.children.cps-issues
- alt.alien.vampire.flonk.flonk.un-flonk
- alt.animals.dogs.collies.open-forum
Sorting out the wheat from the chaff based on newsgroup title is impossible. If you look inside the alt.* groups, you’ll find that many (perhaps most) of them are empty, but for spam. There are few barriers to starting up a new group, and many useless ones hang around forever. A complete list of groups available cannot exist, but most show up here at Internet Systems Consortium (ISC) or you can search or browse for groups here.
Over the years this technology, first developed to disseminate text messages around the world, was adapted to carry file attachments. In the early 1990s, people began to send pictures through Usenet, followed by music and software. A convention developed in most groups and attachments came to be restricted (mostly) to the alt.binaries.* hierarchy. Each time available consumer bandwidth increased, Usenet’s alt.binaries.* jumped in size. In 1990 I was running a BBS and downloading 1 Megabyte of data took about an hour. Today, users are are downloading 1 Gigabyte in an hour, a 1,000-fold increase in 18 years.
Storage technology has increased exponentially also. But newsservers have always struggled to keep up with Usenet volume increases. One newsserver estimated daily traffic at 3.8 Terabytes (3800 Gigabytes) in April. Last I checked, Verizon retained on the order of 10 days of traffic in most of the alt.binaries.* groups, requiring perhaps 40 Terabytes of online storage, not including backups. And commerical standalone newsservers advertise up to 40 days or so of storage capacity.
Here’s the dirty little secret that doesn’t get much publicity: virtually every CD and DVD ever made has been available at some time or another in Usenet’s alt.binaries.* groups. That includes software, books, music, pictures, TV shows and movies. And while the alt.binaries.pictures.erotica.* hierarchy probably contains the most groups (and much of the pornography, child and otherwise), the movie.* and multimedia.* groups transfer most of the volume.
Copyright enforcers have periodically cracked down on those who post. But many posters live outside of the US, and volume is too large to expect anyone to police the whole thing. They’ve never been successful at going after those who download the material. Part of the reason Usenet hasn’t been targeted by law enforcement is assumed to be that the volume is lower than that of other Internet file-sharing technologies. While this is probably true, I’d guess that several million people occasionally use Usenet to access materials that may be covered by copyright.
Ten years ago, most ISPs (Verizon, Sprint, AT&T, AOL, etc) provided Usenet service for their customers, just as they provide email and web access. They either maintained their own newsserver or bought access for their users from another provider. As daily traffic increased from an estimated 4.5 Gb per day in 1997 to over 3.8 Tb per day this year, some large ISPs invested significantly in their infrastructure, but most eliminated the service altogether. Since the early days of the Internet’s commericialization, Usenet users have represented a small and declining percentage of the total online community.
Time Warner has presumably maintained AOL’s Usenet infrastructure. But AOL was always the laughingstock of Usenet; the AOL users were the butt of all of the jokes by the tech-savvy crowd. (To this day, those using email addresses @AOL.com are treated derisively by long-term Internet users.) This was partly because of the flood “clueless newbies” who flooded all online venues in the mid-90s. But additionally, AOL’s newsservers were always awful. Their binaries were incomplete and retention was rarely more than a couple of days. So AOL never attracted a subscriber-base who were knowledgeable users of Usenet. I don’t know in what ways this may have changed since the growth of cable. But it doesn’t surprise me that Time Warner would choose to abandon Usenet altogether in response to Andrew Cuomo’s pressure. They don’t have a history of a strong presence in this market, and their cable subscribers have fewer options for changing high-speed providers if they are unhappy with Time Warner’s decision.
Further, the cable model has more of a problem with heavy traffic than DSL. ISPs’ generally assert that on the order of 5% of users consume 50% of the bandwidth. Last month, Time Warner began testing usage-metering their cable Internet service, a move which in the past has been used to try to eliminate heavy bandwidth customers. Those Usenet users who participate in the alt.binaries.* groups tend to be some of the largest bandwidth hogs. Considering all of these factors, Time Warner may be making a very sensible business decision here to stop providing free Usenet service.
I know nothing of Sprint’s participation in Usenet. I haven’t seen evidence that they provide ISP or Usenet services to consumers. CNet reported that “Sprint is cutting off the alt.* hierarchy, Usenet’s largest, which will primarily affect its business customers.”
But Verizon’s decision to virtually abandon its Usenet business surprised me. I have used Verizon’s DSL service for over 8 years. For most of that time, the performance of their newsservers has been outstanding. They have been one of the few ISPs who have maintained a high-quality Usenet service, free to their customers. They have invested significantly in their Usenet infrastructure, and in keeping it available 24/7. They have many subscribers who are heavy users of Usenet, but their network doesn’t suffer from heavy usage in the same way that cable networks do.
By retaining only their Big-Eight groups, Verizon is eliminating over 99 percent of its customers’ existing Internet usage. It seems odd that they are bothering to retain even these. Perhaps they find that preferable to notifying all customers that they are reducing services, by modifying their TOS. Since the Big-Eight hierarchies are relatively well-behaved and self-moderating, they can probably run mostly on autopilot, possibly allowing Verizon to reduce support staff.
Although the record may not be consistent, Verizon has pushed back at times against unlawful and burdensome tactics sometimes employed by those seeking to enforce copyrights under the DMCA. But fighting these causes can be costly, in both money and staff time, and sometimes in public relations. Perhaps this particular suit from New York’s Attorney General was just one more hassle than upper management wanted to tolerate, particularly while they are fighting the FISA immunity issue with Congress.
The admirable goal of eliminating child pornography cannot explain the decisions announced by these major Internet providers. I don’t know what fraction of kiddie porn on the Internet flows through Usenet, but I doubt that its very large. The use of Web, peer-to-peer and Instant Messaging technologies are all far more popular. I’m not trying to excuse Usenet by saying this; eliminating any conduit is a sensible thing to do. But all three of these companies seem to me to have made business decisions about their future involvement with Usenet services that could not have been primarily motivated by Andrew Cuomo’s concerns. So to associate these actions with “Cuomo announced that Verizon Communications, Time Warner Cable, and Sprint would “shut down major sources of online child pornography,” seems disingenuous.
So in order to comply with Cuomo’s wishes the companies are: 1) paying New York so that the problem goes away, 2) enforcing their own terms of service by denying hosting services to web sites they host which contain child porn, and 3) reducing or eliminating Usenet services for their own business reasons, under PR cover of fighting child porn. I’m not so worried about the “slippery slope” theory at this time; since the providers didn’t agree to do any website blocking (as was initially mistakenly reported by the NYT). But pretending that their decisions regarding Usenet were somehow a victory for anti-porn forces may have unintended consequences.
Is this a harbinger of the long-heralded end of Usenet? Maybe. But, netizens have been forecasting Usenet’s doom at least since I was first aware of it in 1995. There aren’t very many 28-year old communications technologies still in use today. Having defied the predictions for many years, I’d bet it still has years of life left in it. It’s a shame that pornography and illegal activities that are probably keeping it alive.
