[Lispweb] www.fast-index.com

Dr Nick Levine n.levine at anglia.ac.uk
Mon Jun 11 04:29:43 CDT 2001


   Hi Nick,

Hi Marc.

   > I'd be interested in people's opinions of this site.

   I played a little with it. Nice, simple, easy to understand. Too much
   "Please register" nagging ;-)

It really isn't that obstrusive now - it gets mentioned on the welcome
pages but you can just ignore that, then the __first__ time a user
accesses text they are asked to consider registering (and they're
given a clear opportunity to ignore the request and go straight on to
the page they wanted to read).

Actually this is a tough nut. I kept toning it down more and more, but
I wanted to leave something in so that people could access their
bookshelf on a future visit. I have found out the hard way that if you
require people to register before using the site then they simply
won't use it, and that if you don't make it very clear that you don't
need a password off them then most of them still aren't prepared to
use the site. The above paragraph was written defensively. I would
welcome opinions as to how to jolly users into giving enough of a hook
that you can identify them next time they visit, but without scaring
them off.

   [snip]

   a few questions:

   -Why didn't you use an html-gen like approach for writing html instead of
   the with-* approach ?
   That is

   (with-titled-html
	 (with-body
	   (with-heading
	    (title)
	    ....
   instead of

   (:html
    (:body
     (:head
      (:title "hello"))
     (:p "today is the " *current-day*
      ...

   If you want to hide the HTML syntax you could use your own tags instead of
   :html, :body etc...

Stylistic preference? I found it more natural that way to mix html
with pure lisp. 

(if (member volume (user-bookshelf user) :key 'bookshelf-entry-volume)
    main-list
  (cons (with-bullet
	 (with-main-link
	  ((url-from-template "$user-id/shelf.html?action=$url&datum=$volume-id"
			      (list user-id
				    (standard-encode url)
				    volume-id)))
	  "Add to bookshelf"))
	main-list))

   -What performance do you get with your Lisp search engine vs a classic one

No idea, I have't tested it. I took a look at the freely available
engines and decided that none of them met my needs; also if the
commercial project had gone ahead we would definitely have used a
purpose-built engine. So I wrote my own.

   -As I've written mod_lisp which works the same way as you do with Apache, I
   would be interested to hear from you about this. Do you reuse the socket ?
   (I do this now in mod_lisp 2.0 and found a very big performance
   improvement), what protocol do you use for the Lisp <=> Apache
   communication?

I do not reuse the socket. I have never had enough traffic to warrant
the effort. I am aware of the overheads, but they're not great. The
vital thing is to keep the SAME LISP IMAGE SERVING EVERY REQUEST,
rather than the naive CGI approach of starting a different CGI
executable for every request (ie serving /cgi-bin/foo/bar/wombat
starts executable "foo" and passes it (bar, wombat) as
parameters). You'll note that there is no "cgi-bin" string in my
URLs. The Apache configuration for the whole site (apart from the
initial page http://www.fast-index.com/ and the gifs for the Dancing
Men, all of which are served from static content) is:

ScriptAliasMatch ^/FastIndex/(.*) "c:/Program Files/Apache Group/Apache/cgi-bin/client.exe"

The C layer is really thin and the protocol about as simple as it
could be. It reads the request from REQUEST_URI in the environment; it
opens a socket to the lisp and sends the entire request to the lisp
as-is; the lisp generates html with enough http headers to persuade
Apache to generate the rest (Status and Content-Type are all you
need); the lisp sends the result back; the C client prints the string
as-is to stdout and exits. I only use GET requests. I think that if I
used POST, I would handle that in the C layer so that the protocol
between C and lisp remained fool-proof.

Using Apache rather than handling port 80 myself was a key
decision. It meant the bulk of http processing was handled by the
"professionals", leaving me to get on with the application. It also
insulates my code from most of the hack attacks you get. 

I have encountered a socket-based system in the past where the socket
was kept open and reused and it was not a happy experience. You have
the problem that if anything unusual happens then instead of fouling
up one page for one user, you have a rogue C client which is still
alive but in an unpredictable state, fouling up everyone's
communications for a long time to come (in particular you can't
guarantee it's back at the head of its top-level-loop at the right
time).

I am always happier allowing the system to manage resources rather
than trying an ammateur approach. It's a lot of work to do a better
job than the professionals.

   Regards,

   Marc

All the best, and thanks for the feedback. I look forward to any
rebuttals. 

- nick




More information about the lispweb mailing list