Usenet (), USENET, or "in full", User's Network, is a worldwide distributed discussion system available on computers. It was developed from the general-purpose UUCP dial-up network architecture. Tom Truscott and Jim Ellis conceived the idea in 1979, and it was established in 1980. From Usenet to CoWebs: interacting with social information spaces, Christopher Lueg, Danyel Fisher, Springer (2003), , Users read and post messages (called articles or posts, and collectively termed news) to one or more topic categories, known as Usenet newsgroup. Usenet resembles a bulletin board system (BBS) in many respects and is the precursor to the that have become widely used. Discussions are threaded, as with web forums and BBSes, though posts are stored on the server sequentially. The jargon file v4.4.7 , Jargon File Archive. Chapter 3 - The Social Forces Behind The Development of Usenet , Netizens Netbook by Ronda Hauben and Michael Hauben.
A major difference between a BBS or web message board and Usenet is the absence of a central server and dedicated administrator or hosting provider. Usenet is distributed among a large, constantly changing set of that store and forward messages to one another via "news feeds". Individual users may read messages from and post to a local (or simply preferred) news server, which can be operated by anyone, and those posts will automatically be forwarded to any other news servers Peering with the local one, while the local server will receive any news its peers have that it currently lacks. This results in the automatic proliferation of content posted by any user on any server to any other user subscribed to the same newsgroups on other servers.
As with BBSes and message boards, individual news servers or service providers are under no obligation to carry any specific content, and may refuse to do so for many reasons: a news server might attempt to control the spread of spam by refusing to accept or forward any posts that trigger Email filtering, or a server without high-capacity data storage may refuse to carry any newsgroups used primarily for file sharing, limiting itself to discussion-oriented groups. However, unlike BBSes and web forums, the dispersed nature of Usenet usually permits users who are interested in receiving some content to access it simply by choosing to connect to news servers that carry the feeds they want.
Usenet is culturally and historically significant in the networked world, having given rise to, or popularized, many widely recognized concepts and terms such as "FAQ", "flame", "sockpuppet", and "spam". In the early 1990s, shortly before access to the Internet became commonly affordable, Usenet connections via Fidonet's dial-up BBS networks made long-distance or worldwide discussions and other communication widespread, not needing a server, just (local) telephone service.Pre-Internet; Usenet needing "just local telephone service" in most larger towns, depends on the number of local dial-up Fidonet "nodes" operated free of charge by hobbyist "SysOps" (as FidoNet echomail variations or via gateways with the Usenet news hierarchy. This is virtual Usenet or newsgroups access, not true Usenet.) The participating SysOps typically carry 6–30 Usenet newsgroups each, and will often add another on request. If a desired newsgroup was not available locally, a user would need to dial to another city to download the desired news and upload one's own posts. In all cases it is desirable to hang up as soon as possible and read/write offline, making "newsreader" software commonly used to automate the process. Fidonet, bbscorner.com
fidonet.org, Randy_Bush.txt
The name Usenet comes from the term "users' network". The first Usenet group was NET.general, which quickly became net.general. The first commercial spam on Usenet was from immigration attorneys Canter and Siegel advertising green card services.
On the Internet, Usenet is transported via the Network News Transfer Protocol (NNTP) on TCP Port 119 for standard, unprotected connections and on TCP port 563 for SSL encrypted connections.
The articles that users post to Usenet are organized into topical categories known as Usenet newsgroup, which are themselves logically organized into hierarchies of subjects. For instance, news:sci.math and news:sci.physics are within the sci.* hierarchy. Or, news:talk.origins and news:talk.atheism are in the talk.* hierarchy. When a user subscribes to a newsgroup, the news client software keeps track of which articles that user has read.Lehnert, Wendy G.; Kopec, Richard (2007). Web 101. Addison Wesley. p. 291.
In most newsgroups, the majority of the articles are responses to some other article. The set of articles that can be traced to one single non-reply article is called a thread. Most modern newsreaders display the articles arranged into threads and subthreads. For example, in the wine-making newsgroup rec.crafts.winemaking, someone might start a thread called; "What's the best yeast?" and that thread or conversation might grow into dozens of replies long, by perhaps six or eight different authors. Over several days, that conversation about different wine yeasts might branch into several sub-threads in a tree-like form.
When a user posts an article, it is initially only available on that user's news server. Each news server talks to one or more other servers (its "newsfeeds") and exchanges articles with them. In this fashion, the article is copied from inter-server and should eventually reach every server in the network. The later peer-to-peer networks operate on a similar principle, but for Usenet it is normally the sender, rather than the receiver, who initiates transfers. Usenet was designed under conditions when networks were much slower and not always available. Many sites on the original Usenet network would connect only once or twice a day to batch-transfer messages in and out.. Carnegie Mellon School of Computer Science. This is largely because the POTS network was typically used for transfers, and phone charges were lower at night.
The format and transmission of Usenet articles is similar to that of Internet e-mail messages. The difference between the two is that Usenet articles can be read by any user whose news server carries the group to which the message was posted, as opposed to email messages, which have one or more specific recipients.Kozierok, Charles M. (2005). The TCP/IP guide: a comprehensive, illustrated Internet protocols reference. No Starch Press. p. 1401.
Today, Usenet has diminished in importance with respect to , , and social media. Usenet differs from such media in several ways: Usenet requires no personal registration with the group concerned; information need not be stored on a remote server; archives are always available; and reading the messages does not require a mail or web client, but a news client. However, it is now possible to read and participate in Usenet newsgroups to a large degree using ordinary since most newsgroups are now copied to several web sites.One way to virtually read and participate in Usenet newsgroups using an ordinary Internet browser is to do an internet search on a known newsgroup, such as the high volume forum: "sci.physics". Retrieved April 28, 2019 The groups in are still widely used for data transfer.
Not all ISPs run news servers. A news server is one of the most difficult Internet services to administer because of the large amount of data involved, small customer base (compared to mainstream Internet service), and a disproportionately high volume of customer support incidents (frequently complaining of missing news articles). Some ISPs outsource news operations to specialist sites, which will usually appear to a user as though the ISP itself runs the server. Many of these sites carry a restricted newsfeed, with a limited number of newsgroups. Commonly omitted from such a newsfeed are foreign-language newsgroups and the hierarchy which largely carries software, music, videos and images, and accounts for over 99 percent of article data.
There are also Usenet providers that offer a full unrestricted service to users whose ISPs do not carry news, or that carry a restricted feed.
With the rise of the World Wide Web (WWW), web front-ends (web2news) have become more common. Web front ends have lowered the technical entry barrier requirements to that of one application and no Usenet NNTP server account. There are numerous websites now offering web based gateways to Usenet groups, although some people have begun filtering messages made by some of the web interfaces for one reason or another. Google Groups is one such web based front end and some can access Google Groups via news: protocol links directly.
Historically, a mod.* hierarchy existed before Usenet reorganization. Now, moderated newsgroups may appear in any hierarchy, typically with .moderated added to the group name.
Usenet newsgroups in the Big-8 hierarchy are created by proposals called a Request for Discussion, or RFD. The RFD is required to have the following information: newsgroup name, checkgroups file entry, and moderated or unmoderated status. If the group is to be moderated, then at least one moderator with a valid email address must be provided. Other information which is beneficial but not required includes: a charter, a rationale, and a moderation policy if the group is to be moderated. Discussion of the new newsgroup proposal follows, and is finished with the members of the Big-8 Management Board making the decision, by vote, to either approve or disapprove the new newsgroup.
Unmoderated newsgroups form the majority of Usenet newsgroups, and messages submitted by readers for unmoderated newsgroups are immediately propagated for everyone to see. Minimal editorial content filtering vs propagation speed form one crux of the Usenet community. One little cited defense of propagation is canceling a propagated message, but few Usenet users use this command and some news readers do not offer cancellation commands, in part because article storage expires in relatively short order anyway. Almost all unmoderated Usenet groups tend to receive large amounts of spamming.
RFC 850 was the first formal specification of the messages exchanged by Usenet servers. It was superseded by RFC 1036 and subsequently by RFC 5536 and RFC 5537.
In cases where unsuitable content has been posted, Usenet has support for automated removal of a posting from the whole network by creating a cancel message, although due to a lack of authentication and resultant abuse, this capability is frequently disabled. Copyright holders may still request the manual deletion of infringing material using the provisions of World Intellectual Property Organization treaty implementations, such as the United States Online Copyright Infringement Liability Limitation Act, but this would require giving notice to each individual news server administrator.
On the Internet, Usenet is transported via the Network News Transfer Protocol (NNTP) on TCP Port 119 for standard, unprotected connections and on TCP port 563 for SSL encrypted connections.
See also the Great Renaming.
The alt.* hierarchy is not subject to the procedures controlling groups in the Big Eight, and it is as a result less organized. Groups in the alt.* hierarchy tend to be more specialized or specific—for example, there might be a newsgroup under the Big Eight which contains discussions about children's books, but a group in the alt hierarchy may be dedicated to one specific author of children's books. Binaries are posted in , making it the largest of all the hierarchies.
Many other hierarchies of newsgroups are distributed alongside these. Regional and language-specific hierarchies such as .*, .* and ne.* serve specific countries and regions such as Japan, Malta and New England. Companies and projects administer their own hierarchies to discuss their products and offer community technical support, such as the historical .* hierarchy from the Free Software Foundation. Microsoft closed its newsserver in June 2010, providing support for its products over forums now. Microsoft Responds to the Evolution of Communities , Announcement, undated. , CNET, May 4, 2010. Some users prefer to use the term "Usenet" to refer only to the Big Eight hierarchies; others include alt.* as well. The more general term "netnews" incorporates the entire medium, including private organizational news systems.
Informal sub-hierarchy conventions also exist. *.answers are typically moderated cross-post groups for FAQs. An FAQ would be posted within one group and a cross post to the *.answers group at the head of the hierarchy seen by some as a refining of information in that news group. Some subgroups are recursive—to the point of some silliness in alt.*.
The oldest widely used encoding method for binary content is uuencode, from the Unix UUCP package. In the late 1980s, Usenet articles were often limited to 60,000 characters, and larger hard limits exist today. Files are therefore commonly split into sections that require reassembly by the reader.
With the header extensions and the Base64 and Quoted-Printable MIME encodings, there was a new generation of binary transport. In practice, MIME has seen increased adoption in text messages, but it is avoided for most binary attachments. Some operating systems with metadata attached to files use specialized encoding formats. For Mac OS, both BinHex and special MIME types are used. Other lesser known encoding systems that may have been used at one time were BTOA, Xxencode, BOO, and USR encoding.
In an attempt to reduce file transfer times, an informal file encoding known as yEnc was introduced in 2001. It achieves about a 30% reduction in data transferred by assuming that most 8-bit characters can safely be transferred across the network without first encoding into the 7-bit ASCII space. The most common method of uploading large binary posts to Usenet is to convert the files into RAR archives and create Parchive files for them. Parity files are used to recreate missing data when not every part of the files reaches a server.
Binary newsgroups can be used to distribute files, and, as of 2022, some remain popular as an alternative to BitTorrent to share and download files.
Binary newsgroups are only able to function reliably if there is sufficient storage allocated to handle the amount of articles being added. Without sufficient retention time, a reader will be unable to download all parts of the binary before it is flushed out of the group's storage allocation. This was at one time how posting undesired content was countered; the newsgroup would be flooded with random garbage data posts, of sufficient quantity to push out all the content to be suppressed. This has been compensated by service providers allocating enough storage to retain everything posted each day, including spam floods, without deleting anything.
Modern Usenet have enough capacity to archive years of binary content even when flooded with new data at the maximum daily speed available.
In part because of such long retention times, as well as growing Internet upload speeds, Usenet is also used by individual users to store backup data. While commercial providers offer easier to use online backup services, storing data on Usenet is free of charge (although access to Usenet itself may not be). The method requires the uploader to cede control over the distribution of the data; the files are automatically disseminated to all Usenet providers exchanging data for the news group it is posted to. In general the user must manually select, prepare and upload the data. The data is typically encryption because it is available to anyone to download the backup files. After the files are uploaded, having multiple copies spread to different geographical regions around the world on different decreases the chances of data loss.
Major Usenet service providers have a retention time of more than 12 years. This results in more than 60 (60000 ) of storage (see image). When using Usenet for data storage, providers that offer longer retention time are preferred to ensure the data will survive for longer periods of time compared to services with lower retention time.
ISP-operated Usenet servers frequently block access to all groups to both reduce network traffic and to avoid related legal issues. Commercial Usenet service providers claim to operate as a telecommunications service, and assert that they are not responsible for the user-posted binary content transferred via their equipment. In the United States, Usenet providers can qualify for protection under the DMCA Safe Harbor regulations, provided that they establish a mechanism to comply with and respond to takedown notices from copyright holders.
Removal of copyrighted content from the entire Usenet network is a nearly impossible task, due to the rapid propagation between servers and the retention done by each server. Petitioning a Usenet provider for removal only removes it from that one server's retention cache, but not any others. It is possible for a special post cancellation message to be distributed to remove it from all servers, but many providers ignore Cancel message messages by standard policy, because they can be easily falsified and submitted by anyone.Microsoft knowledgebase article stating that many servers ignore cancel messages For a takedown petition to be most effective across the whole network, it would have to be issued to the origin server to which the content has been posted, before it has been propagated to other servers. Removal of the content at this early stage would prevent further propagation, but with modern high speed links, content can be propagated as fast as it arrives, allowing no time for content review and takedown issuance by copyright holders.
Establishing the identity of the person posting illegal content is equally difficult due to the trust-based design of the network. Like SMTP email, servers generally assume the header and origin information in a post is true and accurate. However, as in SMTP email, Usenet post headers are easily falsified so as to obscure the true identity and location of the message source. ...every part of a Usenet post may be forged apart from the left most portion of the "Path:" header... In this manner, Usenet is significantly different from modern P2P services; most P2P users distributing content are typically immediately identifiable to all other users by their network address, but the origin information for a Usenet posting can be completely obscured and unobtainable once it has propagated past the original server.
Also unlike modern P2P services, the identity of the downloaders is hidden from view. On P2P services a downloader is identifiable to all others by their network address. On Usenet, the downloader connects directly to a server, and only the server knows the address of who is connecting to it. Some Usenet providers do keep usage logs, but not all make this logged information casually available to outside parties such as the Recording Industry Association of America. The existence of anonymising gateways to USENET also complicates the tracing of a postings true origin.
Newsgroup experiments first occurred in 1979. Tom Truscott and Jim Ellis of Duke University came up with the idea as a replacement for a local announcement program, and established a link with nearby University of North Carolina using Bourne shell scripts written by Steve Bellovin. The public release of A News was in the form of conventional compiled software, written by Steve Daniel and Truscott.LaQuey, Tracy (1990). The User's directory of computer networks. Digital Press. p. 386. In 1980, Usenet was connected to ARPANET through , which had connections to both Usenet and ARPANET. Mark Horton, the graduate student who set up the connection, began "feeding mailing lists from the ARPANET into Usenet" with the "fa" ("From ARPANET") identifier. Usenet gained 50 member sites in its first year, including Reed College, University of Oklahoma, and Bell Labs, and the number of people using the network increased dramatically; however, it was still a while longer before Usenet users could contribute to ARPANET.Hauben, Michael and Hauben, Ronda. " Netizens: On the History and Impact of Usenet and the Internet, On the Early Days of Usenet: The Roots of the Cooperative Online Culture ". First Monday vol. 3 num.August 8, 3 1998
In addition to UUCP, early Usenet traffic was also exchanged with Fidonet and other dial-up BBS networks. By the mid-1990s there were almost 40,000 FidoNet systems in operation, and it was possible to communicate with millions of users around the world, with only local telephone service. Widespread use of Usenet by the BBS community was facilitated by the introduction of UUCP feeds made possible by MS-DOS implementations of UUCP, such as UFGATE (UUCP to FidoNet Gateway), FSUUCP and UUPC. In 1986, RFC 977 provided the Network News Transfer Protocol (NNTP) specification for distribution of Usenet articles over TCP/IP as a more flexible alternative to informal Internet transfers of UUCP traffic. Since the Internet boom of the 1990s, almost all Usenet distribution is over NNTP.Huston, Geoff (1999). ISP survival guide: strategies for running a competitive ISP. Wiley. p. 439.
AOL discontinued Usenet access in 2005. In May 2010, Duke University, whose implementation had started Usenet more than 30 years earlier, decommissioned its Usenet server, citing low usage and rising costs. On February 4, 2011, the Usenet news service link at the University of North Carolina at Chapel Hill (news.unc.edu) was retired after 32 years.
In response, John Biggs of TechCrunch said "As long as there are folks who think a command line is better than a mouse, the original text-only social network will live on"."." TechCrunch. August 1, 2008. Retrieved on May 8, 2011. While there are still some active text newsgroups on Usenet, the system is now primarily used to share large files between users, and the underlying technology of Usenet remains unchanged.
In 2008, Verizon Communications, Time Warner Cable and Sprint Nextel signed an agreement with Attorney General of New York Andrew Cuomo to shut down access to sources of child pornography.Rosencrance, Lisa. . Computer World. June 8, 2008. Retrieved on April 30, 2009. Time Warner Cable stopped offering access to Usenet. Verizon reduced its access to the "Big 8" hierarchies. Sprint stopped access to the alt.* hierarchies. AT&T stopped access to the hierarchies. Cuomo never specifically named Usenet in his anti-child pornography campaign. David DeJean of PC World said that some worry that the ISPs used Cuomo's campaign as an excuse to end portions of Usenet access, as it is costly for the Internet service providers and not in high demand by customers. In 2008 AOL, which no longer offered Usenet access, and the four providers that responded to the Cuomo campaign were the five largest Internet service providers in the United States; they had more than 50% of the U.S. ISP market share.DeJean, David. "Usenet: Not Dead Yet." PC World. Tuesday October 7, 2008. . Retrieved on April 30, 2009. On June 8, 2009, AT&T announced that it would no longer provide access to the Usenet service as of July 15, 2009.
AOL announced that it would discontinue its integrated Usenet service in early 2005, citing the growing popularity of weblogs, chat forums and on-line conferencing.Hu, Jim. "." CNet. January 25, 2005. Retrieved on May 1, 2009. The AOL community had a tremendous role in popularizing Usenet some 11 years earlier.
In August 2009, Verizon announced that it would discontinue access to Usenet on September 30, 2009.Bode, Karl. . DSLReports. August 31, 2009. Retrieved on October 24, 2009."" Verizon Central Support. Retrieved on October 24, 2009. JANET announced it would discontinue Usenet service, effective July 31, 2010, citing Google Groups as an alternative. Ukerna.ac.uk Microsoft announced that it would discontinue support for its public newsgroups (msnews.microsoft.com) from June 1, 2010, offering web forums as an alternative.
Primary reasons cited for the discontinuance of Usenet service by general ISPs include the decline in volume of actual readers due to competition from , along with cost and liability concerns of increasing proportion of traffic devoted to file-sharing and spam on unused or discontinued groups.
Some ISPs did not include pressure from Cuomo's campaign against child pornography as one of their reasons for dropping Usenet feeds as part of their services. ISPs Cox and Atlantic Communications resisted the 2008 trend but both did eventually drop their respective Usenet feeds in 2010.
In 1985, two news archiving systems and one RFC were posted to the Internet. The first system, called keepnews, by Mark M. Swenson of the University of Arizona, was described as "a program that attempts to provide a sane way of extracting and keeping information that comes over Usenet." The main advantage of this system was to allow users to mark articles as worthwhile to retain. The second system, YA News Archiver by Chuq Von Rospach, was similar to keepnews, but was "designed to work with much larger archives where the wonderful quadratic search time feature of the Unix ... becomes a real problem." Von Rospach in early 1985 posted a detailed RFC for "archiving and accessing usenet articles with Index term lookup." This RFC described a program that could "generate and maintain an archive of Usenet articles and allow looking up articles based on the article-id, subject lines, or keywords pulled out of the article itself." Also included was C code for the internal data structure of the system.
The desire to have a fulltext search index of archived news articles is not new either, one such request having been made in April 1991 by Alex Martelli who sought to "build some sort of keyword index for the." In early May, Mr. Martelli posted a summary of his responses to Usenet, noting that the "most popular suggestion award must definitely go to 'lq-text' package, by Liam Quin, recently posted in alt.sources."
The Alt Sex Stories Text Repository (ASSTR) site archives and indexes erotic and pornographic stories posted to the Usenet group alt.sex.stories.
The archiving of Usenet has led to fears of loss of privacy. An archive simplifies ways to profile people. This has partly been countered with the introduction of the header, which is itself controversial.Strawbridge, Matthew (2006). Netiquette: Internet Etiquette in the Age of the Blog. Software Reference. p. 53.
Google Groups hosts an archive of Usenet posts dating back to May 1981. The earliest posts, which date from May 1981 to June 1991, were donated to Google by the University of Western Ontario with the help of David Wiseman and others,Wiseman, David. "Magi's NetNews Archive Involvement" , csd.uwo.ca. and were originally archived by Henry Spencer at the University of Toronto's Zoology department.Mieszkowski, Katharine. "", archive.salon.com (January 7, 2002). The archives for late 1991 through early 1995 were provided by Kent Landfield from the NetNews CD seriesFeldman, Ian. , "TidBITS" (February 10, 1992) and Jürgen Christoffel from GMD. (December 21, 2001)
Google has been criticized by Vice and Wired contributors as well as former employees for its stewardship of the archive and for breaking its search functionality.
|
|