Sunday, October 23, 2005

The PC Industry Reference List

As I mentioned in one of my earlier articles, I used to have an automated process that scanned a large list of company websites looking for their new press releases. I was using the Netscape browser since it kept it's favorites in a single html file instead of multiple individual files like Microsoft's Internet Explorer. This made it really easy to write various utilities to manipulate the data from that list for other purposes. In particular, I could keep my current reference list of websites in my favorites file for Netscape and then using a utility program that I wrote automatically generate a reference website for everyone to access that listed all the links organized in various groupings.

One of the tools I used was a shareware program that would automatically scan all the links in my favorites file and check for two things: what press release pages were updated and what web links had moved since the last time I scanned the links. The utility recorded additional information in the favorites html file for the next scan (like the date of the last scan) and used certain website header information to find moved pages or sites. The scanning tool created a log file as it ran that made it really easy to find new press releases and manage the link list in my favorites file.

The other tool that actually created my PC industry weblink reference site I wrote myself in C. It used the Netscape favorites html file and created the fancy website pages organized into various categories. My listing of web resources included online publications, organizations, events, company/product indexs, distributors and more from throughout the PC industry. Plus it listed all the current press release pages for various companies from throughout the industry.

I originally made the weblink reference list available on AOL and then later to everyone via the web. I actually kept the reference list goning for quite some time after my area on AOL was canceled. The website was called the PC Industry URL Reference List and even thought it hasn't been online for many years, there are actually copies still available in the Wayback Machine at archive.org -- click here for a sample! Just keep in mind most of the links listed in the archive are no longer valid since this was back in 1999 and has not been maintained since.

Sunday, October 16, 2005

The PC Products Database

This database turned out to be one of my biggest undertakings ever! It started with trying to catalog the various Amiga products in the early days which led to publishing several product indexes for Commodore. Then I started adding some PC related products with the advent of the Amiga Bridgeboard that allowed running PC applications on the Amiga. Eventually I stopped adding Amiga information as the market died and finally concentrated on the PC products.

The original database used a program called Superbase that started on the Commodore C-64, then evolved to the Amiga where is was greatly enhanced and eventually was ported to the PC market. Superbase supported truly relational databases with extensive programmability. I wound up writing my own application that never seemed to stop expanding and growing. I developed a data structure that allowed cataloging all sorts of information about the PC industry and I could perform all sorts of queries with sometimes very interesting results. In the end there were over 250,000 records of data in the database with information on well over 50,000 products and tens of thousands of companies.

One of the programs I wrote in Superbase allowed generating a bulk mail distribution to newly added companies, printing envelopes in bulk mail sort order all set for bundling complete with my bulk mail permit. I hate to think how many envelopes were printed on my old LaserJet II printer! I used to send a letter to every new company when it was added in the database to introduce my newswire service, requesting press releases and information for my database.

My database actually consisted of 6 or 7 interlinked files with the main file containing the basic company information: the company name, mailing address, various phone numbers, plus a website URL and email addresses. There were a number of flag fields that indicated the company's basic products being hardware, software, accessories, peripherals, books/magazines and more. There was also a flag to indicate a shareware publisher. Every company entered in the database had a unique company ID number assigned when they were entered plus a date field to indicate the last time the company's information was updated.

There was also an alternate address file that allowed recording any number of additional addresses, phone numbers, email addresses, website URLs, etc. for a particular company. Each record in this file also had a comment field where the information could be labeled or identified. Each alternate address record in this file was linked back to the main company record in the main file by the company ID number.

A Comments file allowed added general text comments for a particular company and simply had one text field and a company ID field to link the comment back to the company information. This came in handy when there was some little tid-bit of information that just didn't fit in any of the regular data file records.

Now for the meat of the database - the Product records allowed entering the product name and brief description (60 characters maximum) with a link back to the company ID number plus a unique product ID number that was assigned to ever product record. There was another file that contained thousands of product categories with a 50 character text description and a unique product code. Then a Product Index file had two fields to link a given product using it's unique product ID number with a particular product category. This structure allowed associating a particular product with any number of product codes without any limits what-so-every!

The really fantastic part of the product indexing was the product categories that were divided into seven major groups: (0xx) special attributes, (1xx) hardware, (2xx) software, (5xx) publications & information services, (6xx) organizations, (7xx) services plus (8xx) accessories & supplies. I used five groups of numbers to identify each particular product category, for example: where each grouping identified a more detailed subgroup as you moved right in the number. For example, all 130.xxx categories were Hardware Peripheral Devices, 130.60.xxx might be hard-copy output peripherals, 130.60.10.xxx might be printers, might be dot matrix printers and might be 24-pin dot matrix printers. Additional codes could be added at any level as needed to refine the data as needed.

The first group of product codes starting with a zero were reserved for special attributes where I could flag a discontinued or retired prdouct, shareware, freeware or public domain software, special media like CD-ROM or videodisc, age groups to identify educational material for specific groups or adult material, or flag products for specific operating systems like GEOS, PICK, UNIX, etc.

The way the product indexing developed it was extremely easy to expand and add additional information as it was needed. When searching on product codes, you just had to search on the highest product code in the tree and everything else that was categorized with a more refined product category code was also included. This made it really flexible and easy to use!

The database was published on AOL using their Rainman process but was very limited since I could only publish simple text articles with a very basic text index. So there was an article generated for each company in the database with all the information for that company listed in that article. The online search would let you search by company or product name but you couldn't do much with product categories. The date field in each company record allowed me to track what company information needed to be updated on AOL at any point in time so I didn't have to reload the entire database all the time, only the changes. Whenever I had to do a complete reload it would swamp AOL's Rainman process and no one else on the service could do anything until it completed. Needless to say that wasn't done very often!

Eric Englemann of EMS Professional Software in Maryland wrote a dBase interface for my data and published my entire database on CD-ROM for many years that closely resembled my Superbase capabilities. I would generate a copy of the database and ship it on floppy disks (lots of disks!) every month if you can believe it and he would convert everything to dBase and publish it on his next CD-ROM release. At one point we won a Software Star Search award from Computer Reseller News in March of 1992. I think Eric still has some copies of the last PC Products Database that we did 8 or 10 years ago that he still distributes, but he really concentrates on buying and selling old development software now. If interested, you can find his current website at: http://www.emsps.com/

No Starch Press saw the database and had me do a stripped down version that we published in book form in 1996 as the Computer Phonebook. I think we sold a few thousand copies but it never really took off and was only published that one year. The only good thing was that another reference book publisher, Omnigraphics saw the No Starch Press book and flew my wife and I down to Florida to talk with them. They wound up licensing a copy of my entire database to merge into their databases to create reference books for libraries and universities. If anyone is really interested I may still have a few copies of the original Computer Phonebook still around!

Eventually the PC Products Database was put to rest. It was really hard to stop after all those years but I eventually moved on to other things and wound up concentrating on my full time job for the next few years after all the writing and online work came to an end.

Sunday, October 02, 2005

The New Product News

The newswire on AOL was ultimately called the New Product News, as mentioned in an earlier posting. Being a programmer, I was writing all sorts of programs to make the work easier to handle the news flow. Early on I developed a standard text file format that could easily be converted to whatever format was needed for each news distribution platform.

Basically there was a standard three line header at the start of every file that identified the major news category and the actual article title. My programs would then convert all the articles I was ready to publish for each of the online services - AOL, Delphi, BBS Press Service, etc. Some services wanted free flowing text that would be formatted online so there couldn't be any imbedded carriage returns or line feeds within each paragraph. Others wanted fixed formatted text within a certain line length with the associated carriage returns and line feeds. So my tools had to do some reformatting of the text besides creating the article header info for the upload control files. Sure beat typing things online like in the past.

For all my news distributions the articles were typically categorized within 16 news groups:

Software News

  • Business & Productivity Software
  • Professional & Engineering Software
  • Personal & Educational Software
  • Games & Entertainment Software
  • System Software & Utilities
  • Miscellaneous Software

Hardware News

  • PC Systems
  • Hardware Expansion Products
  • Peripherals
  • LAN/WAN Networking Hardware
  • Macintosh Specific Hardware
  • Miscellaneous Hardware

General Interest News

  • Books & Publications
  • Online & Internet Related News
  • Miscellaneous News

Besides making it easier to find specific news articles of interest this arrangement also helped keep more news online for a longer period of time. Each news group had a maximun number of articles that could be present, with older articles scrolling off the newswire. So by breaking them up this way, more of the articles stayed online longer.

Besides the main newswires, I also saved all the news articles distributed each month and put together a news archive at the end of each month that was distributed on the internet. Again I wrote a program that combined all the articles into an easy to access news archive with an index. I started with standard text files, with a simple text index in a standard zip archive file. Later I added an html index that could be used with any web browser. Plus I even created a WinHelp version of the news archive as well. For the WinHelp version I wrote another program that created the input code for the WinHelp compiler and then ran the standard compiler to create the final standard WinHelp file.

Both news archive files were uploaded to various download libraries (like SIMTEL and others) each month as sort of a way to help publicize my newswire and also attract more news input. I just recently found copies of all my old news archive files and have one of the later ones available for anyone interested in seeing what they looked like. You can click on the following links for a copy of the June 1997 news archive in text format or the WinHelp format (which you'll need to download and view on a Windows system). The text format news archive is a standard zip archive file with separate text files for each article. Just open one of the index files to start, either the text file (index.txt) or the html file (index.htm) with your web browser. The WinHelp file should be directly useable on any Windows operating system. These files are about 4MB in size!

By the way, the original name of the WinHelp news archive was WinNews but Microsoft objected and I wound up having to change the name a few months after I started distributing those files. It just wasn't worth fighting Microsoft's legal team even though I knew I was the first to use that name. Anyway, you still might find a few of those early files floating around the internet somewhere, I think there are still a few files on SIMTEL and other archive sites.