The Way of the great learning involves manifesting virtue, renovating the people, and abiding by the highest good.

2008年12月14日星期日

An Early History of Lynx: Multidimensional Collaboration

This is a brief history of the curses-based WWW browser called Lynx, developed within Academic Computing Services at the University of Kansas. It is part of the WWW History Project exploring the origins of Web technology. For more infomation see http://www.cc.ukans.edu/about_lynx.
This history will not be accurate; it will not be complete. Lynx was developed in a dynamic environment involving frequent collaboration, as well as independent work, and identifying who did exactly what and when is very difficult. Thomas Kuhn, the well known philospher of science comments in "The Historical Structure of Scientific Discovery:"

If the study of discovery has a surprise to offer, it is only that, despite the immense energy and ingenuity expended upon it, neither polemic nor painstaking scholarship has often succeeded in pinpointing the time and place at which a given discovery could properly be said to have "been made".

Lynx was developed primarily by Michael Grobe, Charles Rezac and Lou Montulli, and members of the "Internet community" by an iterative process of exploration, interaction, hacking and evaluating.

Prior to becoming a Web browser Lynx was a distributed hypertext browser based on the client/server model with its own intermachine and intradocument link tags. In addition, to support remote database applications, Lynx could serve as a kind of text-based X Window display server.

This document includes excerpts from early Lynx documentation, usually from 1992. Early documentation is presented in blue font, and surrounded by text markers for Lynx users, of course.

The problem
Lynx development was inspired by the late Herb Harris, Assistant Director of User Services at the University of Kansas, who wanted a "campus-wide information system" (CWIS) around 1989. Charles says he really wanted an organization wide information environment ( OWIE).
I was not excited by this idea to start with. It's utility seemed limited. In my view Herb wanted a CWIS mostly because "everyone else had one", and we needed to "keep up." Of course, these days "everyone" DOES have one though we had not the wildest expectation of such ubiquity.

The first solutions
Todd Gochenour made a first try at it using a DEC "Videotext" product.
Charles Rezac made a second try exploring Gopher, etc. and working on a PC client for Gopher. Charles ran into the Web project during this time, and he knew it did something similar to what we wanted, but he didn't follow up on it since he understood it to be limited to the Next platform. Much later, we encoutered the CERN linemode browser and found it too "ungraceful" to pursue, compared with Lynx's screen-oriented hypertext.

We advertised for a student programmer to develop a UN*X- based, menu-oriented system. A young EECS student named Lou Montulli applied, and the rest is . . ., well, . . . not quite history. We hired another student, Vijay Kolli, instead.

And we developed a simple menu-based system while Lou worked for another group within User Services and moonlighted with us.

A better solution: hypertext
Lou discovered Earl Fogel's version of Peter Scott's Hytelnet system based, apparently, on Neil Larson's DOS-based Hyperrez product (which is apparently a hypertext-based outlining system he developed for his company MaxThink), and felt he could make use of in our project. I had played around with an AI hypertext system, called Knowledge Pro, developed by Bev & Bill Thompson of Knowledge Garden during grad school, and the InfoExplorer from the KnowledgeSet Corp. on our IBM AIX system, and quickly approved the hypertext approach.
So we made a few changes and began to use Earl's code as a simple hypertext display tool. The earliest versions of Lynx employed their own link (tag) syntax, and simple URL scheme, assumed all documents were preformatted text (though we considered using ROFF source). Lynx also had an "owner tag", developed to identify the document creator (a forerunner to the mailto: URL), as implemented in the UNICORN CWIS system developed by Neal Erdwien of Kansas State University.

Here is a simple example document taken from 1992 documentation:

[Original Lynx documentation from 1992 begins here.]

This is roughly equivalent to the following HTML:

[Original Lynx documentation from 1992 begins here.]






CS 200 Introduction to Computer Science
MWF 3:30
104 Strong Hall A Lovelace


CS 600 Introduction to Data Structures
TR 2:30
210 Snow Hall A Turing

The file itself is considered to be "owned" by a user named Michael Grobe, whose electronic mail address is grobe@kuhub.cc.ukans.edu. Users may mail messages to the file owner within Lynx and Lynx itself may mail messages to the owner in response to certain error conditions, such as a link within the file proving to be inaccessible.

When Lynx displays the file, the link pointers will not be displayed. Instead, the screen will look something like:


[Original Lynx documentation from 1992 ends here.]

Still better: "distributed hypertext"
Meantime, Gopher was growing in popularity and we discussed ways to use both tools to meet our needs. Finally, one day in 1992 I had the idea of grafting the Gopher client net management routines onto Lynx to merge the two tools. Lou and Charles thanked me for finally catching on, pointing out that they had been suggesting just that for a week.

So Lou grafted and by July of 1992 we had what I took to calling a "distributed hypertext" system, and we immediately began presenting the system locally and at Midnet conferences. For a while we billed ourselves as "Mike and Lou's Asphalt Resurfacing Company". We used Gopher servers to house our Lynx-style hypertext documents. Lynx also became the best text-based Gopher client as a side-effect.

Here is a diagram of the distributed hypertext idea:


Here is an example showing the format of the early Lynx resource pointer (analogous to a URL):

[Original Lynx documentation from 1992 begins here.]

The following version of the timetable file shown above, directs Lynx to take course catalog information from a Gopher server running on a system whose Internet address is ukanaix.cc.ukans.edu:


LINK_DELIMITER = <
END_LINK_DELIMITER = >
OWNER = Michael Grobe
OWNER_ADDRESS = grobe@kuhub.cc.ukans.edu
OWNER_INFO = Academic Computing Services,University of Kansas
PATH = catalog/cs/

The Computer Science Department is offering the
following classes during the Spring Semester of 1992


CS 200 Introduction to Computer
Science MWF 3:30
104 Strong Hall A Lovelace

CS 600 Introduction to Data
Structures TR 2:30
210 Snow Hall A Turing

The general syntax for the name of a file held in a Gopher server is, then


pathname@hostname port_number

where the pathname is appended to the PATH variable to obtain the path to a file relative to the base Gopher directory on the remote server. port_number is the TCP/IP port on which the Gopher server is listening. The default port number for Lynx links of this format is 70 (which is also the default Gopher server port).

[Original Lynx documentation from 1992 ends here.]

We also did anchors with targets, as in:
[Original Lynx documentation from 1992 begins here.]
Aside from specifying files to be displayed, links can also specify the number of characters following the link to be highlighted. In addition, they can include a target string that determines which portion of the designated (target) file to display first. For example the link




specifies that the the first 41 characters following the link should be highlighted, and instructs Lynx to search for the string "CS 200" in the target file.

[Original Lynx documentation from 1992 ends here.]

This is roughly equivalent to:



. . . 41 characters of stuff . . .


[Original Lynx documentation from 1992 begins here.]

The courses file being used with this timetable file might look like:


TARGET_DELIMITER = [*
END_TARGET_DELIMITER = *]
OWNER = Michael Grobe
OWNER_ADDRESS = grobe@kuhub.cc.ukans.edu
OWNER_INFO = Academic Computing Services,University of Kansas

[*CS200*]CS 200 Introduction to Computer Science

CS 200 introduces students to the basic concepts of
computing. Topics include algorithm development, computer
organization, and the syntax and semantics of a high-level
programming language.


[*CS600*]CS 600 Introduction to Data Structures

CS 600 presents data structures and algorithms commonly
used in computer programming. Linked lists, arrays, trees,
and graphs are covered. Algorithms for creating, filling,
searching, sorting, traversing these structures are covered.

[Original Lynx documentation from 1992 ends here.]


We could execute commands locally from Lynx:
[Original Lynx documentation from 1992 begins here.]
Lynx can start programs when links are activated. For example, the link


Find info on printing

would start man on the local system if activated. man would operate as usual and control would return to Lynx when man terminated. In a similar approach the program is executed and all program output is placed in a temporary file. (On UN*X systems this file is created in /tmp with a unique name based on the PID of the executing program). The temporary file is treated as a normal Lynx hypertext file, and may be searched for targets specified in the link. For example, a link like:


LYNX :21:print>Find info on printing

instructs Lynx to execute the command


man -k print

and capture the resulting output in a temporary file. Lynx will then display the file beginning with the first segment containing the string "print" and underline every occurrence of that string. The temporary file will be deleted when Lynx exits.



The ability to execute some local commands is so useful, however, that Lynx provides several standard local file execution links. These standard links include support for telnet, tn3270 and rlogin, and are defined by specifying one of the strings "TELNET", "TN3270", or "RLOGIN" as a program_path argument in the local execution link syntax presented above. Such links should look something like:






where the square brackets([]) indicate that an argument is optional. For example, the port_number arguments are not necessary (and the square brackets never actually appear in a link).

[Original Lynx documentation from 1992 ends here.]


Lynx: an animal that eats Gophers
[Original Lynx documentation from 1992 begins here.]

Lynx utilizes Gopher servers primarily to support distributed hypertext file access. In addition, Lynx can function as a Gopher client itself when it encounters links that specify gopher file types. For example, the link


<1/@gopher.micro.umn.edu>Connect to UMN Gopher Server

appearing in a hypertext file allows a user to begin a Gopher session with the Gopher server at gopher.micro.umn.edu. The Lynx interface to Gopher presents Gopher files as Lynx hypertext documents. For example, users can select a file from a Gopher directory by using the usual interactive arrow key sequences to highlight the desired file.

[Original Lynx documentation from 1992 ends here.]

More better solution: Integrating databases
Throughout this project we "discussed" ways of integrating databases with our system. Our University Relations group had an event calendar for the University that we set up to allow event catagory and date searches, and used that for proof-of-concept. There were several approaches to this database over the years. For example, we developed telnet, rlogin, and tn3270 hyperlinks to get to it, and we added the ability to act as a display server for screen-oriented text sent by a process running on another system. This is basically th X Windows of the text world. I liked to call it V-Windows, though "streaming text" would have been good. (Vijay Kolli actually wrote the first version of this display server code within Lynx.)
[Original Lynx documentation from 1992 begins here.]

Lynx employs two standard approaches to remote program execution. First, Lynx supports standard TCP/IP socket communications techniques to start and exchange information with remote servers. Each server listens on a defined port on a server system until it receives a request to establish a connection from a Lynx client process. Lynx can open connections to programs listening on specific TCP/IP ports when processing links like:


General Campus Events

When such a link is activated, Lynx makes a connection to the specified port and assumes that any text sent through the connection is intended for a VT100 display. If Lynx can determine that the user is using a VT100 display device, the VT100 commands are simply passed on to the user. Otherwise, Lynx converts VT100 screen control commands to commands suitable for the display device actually being used by the user (via calls to the UN*X curses library).

For example, the file below gives users access to a collection of event calendars spread across a number of servers:


Welcome to the Campus Event Calendar Menu
Please select an event database from the list below:


General Campus Events

Important Academic Dates

Linquistics Department Colloquies

City and Regional Calendar


Activating the General Campus Events link would then invoke a screen that provided campus events info, displayed by the event server running on ukanaix.cc.ukans.edu. The user may choose a new date range, or a subset of the event categories supported. When the user leaves this program, by typing a "q", control is returned to Lynx. (The Lynx history list is unavailable while the the event program is executing.)

This process is diagramed in the following image:


[Original Lynx documentation from 1992 ends here.]

Grafting wwwlib
Wes Hubert, manager of another User Services group attended INTEROP in October of 1992, where he attended a BOF on the Web. He heard about wwwlib and upon his return suggested Lou add it to Lynx. Lou had been interested in doing this for some time for a variety of reasons, and came to me for corroboration. I advised it would "just kludge up the code." (Several readers have suggested to me after the fact that it might have already been too late :- ) He returned a week later, having done the deed, and we began a wild ride on the Web.
During July of 1993, Lou and I attended what I like to call the Oth World-Wide World Wide Web conference in Cambridge, MA organized by Dale Dougherty of O'Reilley and Associates (who also provided some financial assistance for our Lynx and DosLynx projects). There we encountered 30-some Web developers, including Berners-Lee then at CERN, Hardin, Andreesen, Totic, Mittelhauser, etc. from NCSA, Tom Bruce (author of Cello) from Cornell Law School, among others.

The biggest challenge
Integrating databases engendered the most "passionate" discussions throughout the project. Each of us had a favorite solution beyond the V Windows approach. Lou wanted to build a "forms" interface capability. I thought that was too restrictive and favored moving small programs written in an interpreted language down to the client to execute and return queries to database servers. (We considered Perl, Tcl, and SafeTcl, but we expected to create desktop clients somewhere down the road, and weren't aware of PC environments for these languages.) Charles wanted to move complete objects around; move the data with methods to display it locally.)
Since Lou was doing the coding, you can guess how this worked out. Lou implemented a set of form tags for this database, making it one of the first, if not THE first, form implementations. As Charles puts it, Lou's approach got done, mine is becomming common via Java, etc., and his is the way of the future.

The Problem with Success
Lou left for Netscape in 1994, Garrett Blythe, who authored our DosLynx system took over, but left soon after to join Lou at Netscape. Charles left for greener pastures (but not quite AS green), and Jeff Porter took over as Lynx developer. Jeff had hoped to rewrite Lynx to get rid of that "hacked" look, but was unsuccessful and returned to graduate school just as a campus-wide hiring freeze hit leaving Lynx unsupported except for the community of users organized around lynx-dev and led by Foteos Macrides of Worcester Foundation for Experimental Biology.
There was some consideration of taking Lynx commercial, and there were several offers (by 3 letter companies) to license Lynx. Unfortunately there were numerous contributors (We counted some 18 during one search through the code.) during the early history of Lynx, making "Statement of Originality" a nightmare.

I moved to put the KU portions of Lynx into the public domain in 1996, but the lynx-dev group preferred GNU status, so Lynx was declared to be under GNU General Public License in 1996?

Besides that, good authority assured me that it was impossible to make money on a client. (They apparently didn't consider making money on the hope engendered by a client.)

Where are they now?
Lynx development continues organized around lynx-dev@sig.net.
Lou went on to give the world the amazing fish cam and the BLINK tag from within Netscape, to enjoy the American dream of working for "god's gift to startups," and, one day in a galaxy far, far away, he will finally be awarded his undergraduate degree.

Charles built the KU CWIS, which we named KUfacts, using the old Lynx hypertext format, modified it to HTML as we moved to the Web, and it remained pretty much the same until September 1996 when it was finally replaced with a fancier graphics-oriented version. Charles has since moved on to the KU Medical Center, where he designed their OWIE, known as Pulse, and is "currently entertaining job offers and marriage proposals."

I lingered on at the Computer Center, as Manager of Distributed Computing support and (now ex-) Director of our statewide network (KANREN), encouraging the use of Internet technology. In the short term, I would like to participate in building the "real web", a network of communicating document databases, so we can stop using file systems as document databases, and Intenet 2.

What did we actually accomplish?
I like to say that we "invented a Web." Rather than the Web, of course. It seems to me that Tim Berners-Lee defined a much better version than ours, and conversion was comparatively easy, since we had already developed similar ideas.
Tim balanced what was useful with what was do-able in an incredibly elegant way. To me the Web is the "big 3" that Tim "invented": URL syntax, HTML (choosing a "markup approach"), and HTTP in the context of the Client/Server model.

I note that each of these

still works and is in use today,
has limitations,
can be significantly improved usually only with strenuous effort and limited payoff, and
has served as a flexible foundation for many variations on the theme (images, CGI scripting, and applets, new version of HTML, etc.).
Limitations notwithstanding, the platform is clearly extremely powerful and a remarkable piece of software engineering.


Michael Grobe
April 10, 1997

没有评论: