[Lispweb] www.fast-index.com

Marc Battyani marc.battyani at fractalconcept.com
Tue Jun 12 02:56:11 CDT 2001

Hi Nick,

> Actually this is a tough nut. I kept toning it down more and more, but
> I wanted to leave something in so that people could access their
> bookshelf on a future visit. I have found out the hard way that if you
> require people to register before using the site then they simply
> won't use it, and that if you don't make it very clear that you don't
> need a password off them then most of them still aren't prepared to
> use the site. The above paragraph was written defensively. I would
> welcome opinions as to how to jolly users into giving enough of a hook
> that you can identify them next time they visit, but without scaring
> them off.

Yes, it would be nice if we could use cookies... But as I generally disable
them (and I clear them often), I know it's not reliable.
Using URI encoding for session tracking works well excepted when the URI is
bookmarked or archived in a search engine.
I would also welcome some ideas on the subject....

>>    -Why didn't you use an html-gen like approach for writing html instead
>>    the with-* approach ?

> Stylistic preference? I found it more natural that way to mix html
> with pure lisp.



>>    -As I've written mod_lisp which works the same way as you do with
Apache, I
>>    would be interested to hear from you about this. Do you reuse the
socket ?
>>    (I do this now in mod_lisp 2.0 and found a very big performance
>>    improvement), what protocol do you use for the Lisp <=> Apache
>>    communication?

> I do not reuse the socket. I have never had enough traffic to warrant
> the effort. I am aware of the overheads, but they're not great. The
> vital thing is to keep the SAME LISP IMAGE SERVING EVERY REQUEST,
> rather than the naive CGI approach of starting a different CGI
> executable for every request (ie serving /cgi-bin/foo/bar/wombat
> starts executable "foo" and passes it (bar, wombat) as
> parameters). You'll note that there is no "cgi-bin" string in my
> URLs. The Apache configuration for the whole site (apart from the
> initial page http://www.fast-index.com/ and the gifs for the Dancing
> Men, all of which are served from static content) is:
> ScriptAliasMatch ^/FastIndex/(.*) "c:/Program Files/Apache
> The C layer is really thin and the protocol about as simple as it
> could be. It reads the request from REQUEST_URI in the environment; it
> opens a socket to the lisp and sends the entire request to the lisp
> as-is; the lisp generates html with enough http headers to persuade
> Apache to generate the rest (Status and Content-Type are all you
> need); the lisp sends the result back; the C client prints the string
> as-is to stdout and exits. I only use GET requests. I think that if I
> used POST, I would handle that in the C layer so that the protocol
> between C and lisp remained fool-proof.

> Using Apache rather than handling port 80 myself was a key
> decision. It meant the bulk of http processing was handled by the
> "professionals", leaving me to get on with the application. It also
> insulates my code from most of the hack attacks you get.

I quite agree with this. It's easy to process HTTP requests in Lisp, but why
bother? Apache does this very well and in the Lisp process I want to focus
on the applicative part not serving jpegs. But I think it's better to avoid
an external C client and have a direct Apache -> connection without having
to create a new process each time.
There are also commercial reasons to use Apache. It's really easier to
propose Apache based solutions than Lisp only ones and to integrate into an
existing server.

> I have encountered a socket-based system in the past where the socket
> was kept open and reused and it was not a happy experience. You have
> the problem that if anything unusual happens then instead of fouling
> up one page for one user, you have a rogue C client which is still
> alive but in an unpredictable state, fouling up everyone's
> communications for a long time to come (in particular you can't
> guarantee it's back at the head of its top-level-loop at the right
> time).

Could you give more info about this? It's not my experience.
In mod_lisp 2.0, I keep the Apache <-> Lisp socket open and I haven't found
any problems so far. As it's a direct Apache -> Lisp connection, if the Lisp
process is dead then your server is dead but it's the same in your case. If
it's only the processing of a request that is dead then this page is stuck
but the server is still alive for other requests. When the user try to
reload the page, a new connection is made. As we are in Lisp and not in
lesser language we can correctly trap errors in the Lisp process and reply
an error page without closing or blocking the Apache connection.
I've found a very big improvement by not closing the socket. On the
benchmarks I've done, I've got a factor going from x 40 to x 80 !

All the best,


More information about the lispweb mailing list