(Note: This article once appeared on the GNN site. GNN is no more, so I put up a copy of the article here.)

The Collapse of misc.invest?

by Christopher Lott
March, 1995

The explosive growth of the number of users on the Internet over the past three years seems to have resulted in the near collapse of misc.invest. By collapse I simply mean that it appears the signal (i.e., the useful information) is in danger of being totally subsumed by the noise (i.e., the nonsense that comes across the wire). As I write this in March of 1995, the high volume alone is discouraging, and the noise level is high.

In this article, I'd like to mention some problems as I see them and offer my opinions about information exchange via open outcry. I'll also tell some stories stemming from the misc.invest FAQ, and propose some solutions to the problems that I see. To put the disclaimer up front: although I do compile a list of frequently asked questions (FAQ) for the misc.invest newsgroup, the people who mistake the compiler of a FAQ for some sort of person in charge are wholly mistaken. I read the group in the hopes of gleaning useful bits of information that will help me towards early riches and retirement :-) just like many other readers. The only thing I'm in charge of is watering my plants, and what follows are just one man's opinions.

I'll begin with the big picture. Today we have a setup in which any new user on any host can post any message s/he wants to one of the news groups. In other words, NetNews is constructive anarchy (if that's not a hopeless oxymoron). Still, I think newsgroups, or possibly their multimedia equivalent, will be with us for some time, simply because of the sheer power of the Net (which repeatedly amazes me). The ability for individuals to make contact across the zillions of kilometers of fiber-optic cable in order to exchange information is worth so much, and has been so successful, that I think the anarchy will survive for quite some time yet. Being able to post a question in some group and get replies, sometimes within hours, has saved us enormous frustration many times, especially in matters related to system administration.

However, the open-outcry system as practiced in these groups is under pressure. Open outcry survives in the marketplace on the Chicago Board of Trade & Mercantile Exchange, but that's the exchange of products, with a limit on the number of players. The net only exchanges information, and there's no limit on who can play -- not yet, anyhow.

I think that newsgroups can only survive if the volume can be kept in check. The good news is that I believe the volume is self limiting. Although increasing participation will encourage the newsgroup hierarchy to get deep so as to limit the scope of any one group, I suspect the growth has limits that are not imposed by the technical details of the news systems. After all, if misc.invest hits one thousand articles a day, who's going to keep reading? So people will unsubscribe when traffic becomes unmanageable, the volume will dip again, and eventually people will pop back in. However, I don't know where the natural limit for a group like misc.invest might be. I suspect it's above my threshold.

One important difference in character between the misc.invest groups and other newsgroups is the possibility of personal monetary gain. In most other groups, the gain is in information geared towards making life easier at home or work. For example, there are groups that can help me choose a mountain bike, train a misbehaving pet, or find out how to configure a new laser printer. However, if I'm following misc.invest and I glean some information about a stock, I may be able to use that information to make money. A darker angle on this is that if I am somehow able to convince a few people to buy my favorite penny stock on the Vancouver stock exchange, there's the chance to profit from a scam. I believe that this particular character of misc.invest tends to exaggerate some of the problems.

So what are the specific problems in misc.invest? I think they're pretty obvious to anyone who tries to follow the group, but here's my listing. I've paired a possible solution or two with each problem.

Problem: Noise. I see a relatively high incidence of trolls (inflammatory articles posted to get a reaction, any reaction), advertisements, and other nonsense posts.

Solution: Intolerance on the part of users to see this crap in newsgroups and, more importantly, mail sent to the appropriate system administrators tends to eliminate the most blatant cases rather delightfully. But in general, I see no solution here. As long as NetNews maintains its state of anarchy, people will post nonsense from time to time.

Problem: Ignoring the basic charter of the group. For example, a poster asks about a specific stock in misc.invest, or begins discussing strategy in misc.invest.stocks. Although this is not nearly as obnoxious noise as that mentioned above -- the people are basically looking for investment information -- it's not helpful.

Solution: Our version of the Pnews newsreader presents about 40 columns of descriptive text about each newsgroup in an effort to remind people about the topics expected in that group. That's nice, although hardly sufficient, but I suspect we won't do much better.

Problem: Ignoring the FAQ. OK, maybe this issue lies a little too close to home for me to be objective about it, but I'll try. I find it frustrating to see people post novice questions that are nicely covered in the FAQ. A few weeks ago a scenario played out that epitomized some of the problems with NetNews: someone posted a note with a fairly trivial question and admitted that s/he (I honestly forget the name, let's call him/her "Newbie") had not read the FAQ. Newbie got flamed for not reading the FAQ, which I thought was also in order -- after all, Newbie admitted to knowing about the existence of FAQs, and should have tried just a little harder. Then it got interesting: Newbie's flamer got flamed by someone who felt that anyone should feel free to post questions, even the ones already answered in the FAQ. The final sputtering flames (hope I haven't lost anyone yet!) were directed against the person who was all for repeats of questions. The last post in this little flame-fest closed the circle by trying to make it clear that new users really, really should read the FAQ before posting, because it would probably answer most of their questions. Whew!

Solution: This is a tough one. However, if newsreaders could locate a FAQ for any group quickly -- i.e., make finding the FAQ easy -- I think most people would read the FAQs. Just fantasizing wildly, I wonder if the NetNews community could found some consensus on a text marker (i.e. tag) that could appear in the header of any FAQ posting. The newsreader would merely have to search for that tag, ideally using capabilities of a News Overview Server (NOV) or whatever other goodie is out there to make the search fast. Of course, once the FAQ article expires, this scheme falls flat on its proverbial face, but it might be a step in the right direction.

Problem: High volume. misc.invest receives in the neighborhood of 100 new articles per day, every day. Basically I find it demoralizing to try to plow through that much news every day, and have from time to time given up trying for a week or two.

Solution: Again there are no panaceas. When ever-increasing numbers of people participate, you get ever-increasing numbers of articles. However, some things might help.

I have found that "trn", a threaded newsreader that lets you ignore entire sequences of articles (i.e., threads), really saves me time. With it, I avoid reading articles that don't interest me; well, to be precise, that have subject lines that don't interest me.

Subdividing the group may bring useful results. For example, for misc.invest.stocks, I had hoped to find inspiration from Value Line. As it turns out, Value Line divides their stock ratings into about 100 categories, and I don't think we're ready for 100 new groups under misc.invest.stocks! And I don't know what to propose for misc.invest.

Adding a moderated group "misc.invest.moderated" (or some reasonable name) has been much discussed in the group. Unfortunately, much of the discussion degenerated into a lot of knee-jerk responses from readers who thought the ideas was to convert *misc.invest*, rather than the addition of a new group. They posted rather outraged notes with (borrowing from Dave Barry and the NRA) many correctly spelled words that all basically said "I'll give up free access to misc.invest when you pry the keyboard out from under my cold, dead fingers." Clearly, converting misc.invest is a nonstarter, but adding a moderated group could be highly conducive for discussions about strategies, tax code, and other issues that are not so timely as yesterday's X point drop in stock YZ. I'm thinking along the lines of comp.risks. The obvious problem is that someone has to moderate, and no, I'm not volunteering.

At the risk of being accused of patting myself on the back, I'd like to mention a few success stories related to the FAQ and misc.invest. A few weeks ago, a fellow expatriate named Derrick Reagins (he's just a few klicks away in France) wrote to thank me for reminding him of security issues related to on-line trading as well as for the listing of discount brokers. He said I had saved him a bit of money. Naturally I offered to relieve him of worrying about all that money he saved but strangely enough he didn't take me up on that offer :-). Recently Bill Rini incorporated the FAQ into his WWW site, and Karthikeyan Guruswamy wrote a converter that automagically transforms the text version of the FAQ into a HTML version for use with browsers such as Mosaic or Netscape. And the FAQ has been included on one of Walnut Creek's CD-ROMs. So I think it's safe to say that quite a number of people have access to the information, and clearly some have found it useful. Unfortunately I have no way of counting! I think this is a good spot to thank all my fellow netters who have contributed time and effort in information, programs, comments, corrections, and other input for the FAQ. Without them, there would be no FAQ. Thanks, people.

In conclusion, I think reports of the death of newsgroups would be premature at this point, including misc.invest. However, I also think the days when discussions were focused and the signal/noise ratio was high (if there ever were those days), are long behind us. Dealing with the traffic on the Net in the late 90's will require both patience and good tool support, but considering the gains to be had in information, I'm confident people will not give up trying to use the newsgroups.

This article is copyright © 1995 by Christopher Lott.