[Lispweb] debugging and serving web pages at the same time in lisp (specifically clisp)

Lawrence Au laau at erols.com
Thu Oct 5 17:55:22 CDT 2006


Yinso,
    The single function Save-application worked great in MCL (macintosh 
common lisp).
It produced a double-clickable executable with the saved state of the 
entire Lisp environment.
Unfortunately I've yet to fully master the more complex equivalent in 
Lispworks.
My systems are always in source code form,  because I only run code in 
the Listener
after copying and pasting it from my source code files.  So if I 
rebuild,
I just recompile from my latest source files.  Typically my source code 
files
consist of defvar global variables,  defstruct structure definitions 
and defun function definitions,
each followed by specific,  commented-out sets of test cases for unit 
testing.

    Yes,  Lisp supports a completely different and more unified 
methodology than
other environments.  There is essentially a single runtime phase and 
all states
and abstractions are directly accessible from that phase.  Syntactic 
and semantic bugs
are caught while compiling functions and structure definitions but 
these compilations
are performed directly within a runtime context.

    As far as rolling back and verifying changes,  Lisp has unique 
advantages
which make source control seem like a giant afterthought.  Since the 
nested parentheses
and optional type declarations of Lisp supports a pure tree-structured 
syntax,
re-factoring of function calling trees is far simpler than other 
languages.
Consequently I can make deep changes without the immense
re-factoring cost which causes systems to be run half-modified,  
half-modifications
which require source control to keep track of.   This all holds true as 
long as I
keep passing most of my data through the function arguments.
In Lisp it's unusually easy to pass data through function arguments.

    Consequently making deep changes to a system is much easier,
especially with complex systems.  That said,  I still prefer to make 
deep changes
to a test system,  simply because I make typos when I code,  and I don't
always think of all the test cases needed to root them out.  But I've 
never had
to resort to a source control system to rollback changes in Lisp,  
because as long
as I pass most of my data through function arguments and results,  the 
test cases needed to
verify code changes are very simple and I've never had to rollback to 
more
than a previous version of a function.

    Don't rush to grok all this at once.  I used to program in Fortran, 
C, C++,  Java and SQL
before learning Lisp properly from Paul Graham's book.  I really had to 
read his book
carefully to free myself of encumbering Fortran, C, C++,  Java and SQL 
notions.
For instance,  I used to pass data through global variables when they 
should have been
passed through function arguments.  This made my code harder to test,  
harder to re-factor
and much more error-prone,  like a typical Fortran, C, C++,  Java or 
SQL program.

    The parentheses of Lisp make re-factoring of function
arguments in Lisp so much easier than Fortran, C, C++,  Java and SQL 
that almost all
function inputs can come via calling arguments,  and almost all 
function outputs can return in the
function results,  without causing the kind of syntactic overhead that 
makes
global variables attractive.

Lawrence Au
Q-Phrase LLC


On Oct 5, 2006, at 6:11 PM, Yin-So Chen wrote:

> Thanks Ivan, Richard, and Lawrence for responding - this is all very 
> intriguing.  Sounds like CGI is the only sure way to go with Clisp if 
> I am looking to run multiple concurrent sessions?
>
> Also - Lawrence, I am assuming you have a way to capture your changes 
> in life system back into source code form easily?  I've heard the 
> save-image capability, but would think you still need to have source 
> code so you can rebuild image again if you can, is this correct?
>
> Richard - you meant you can (trace) h-r-r in SBCL, right?  In CLISP, 
> once (host-serve-events) is called, it blocks REPL, so I can't do 
> trace just-in-time without killing off the lisp session (I can't tell 
> whether there is a way to accept a control-character within araneida).
>
> BTW Ivan - I did find a blog article by BillClementson discussing the 
> same topic at http://bc.tech.coop/blog/040224.html.
>
> Based on all the descriptions so far, I am wondering if the 
> development "methodology" is completely different with lisp.  I so far 
> lived in the compile-link-install-test world, where most of syntactic 
> bugs and some semantic bugs are caught during the compile phase. And 
> that leaves the bugs that are harder to find in live systems.  Of 
> course, examining trace files is horrible compared to live traces, but 
> I've found that a good majority of the bugs found in production 
> requires non-trivial changes (even with decent abstraction layers) and 
> hence my experience is that without going through another test cycle, 
> changing is extremely risky.
>
> Unless... there is a way to quickly verify and rollback changes easily 
> in lisp?
>
> It would be great if someone with experiences in both worlds (I am 
> sure many of you do ;p) can share some light on whether lisp's 
> development cycle is completely different than c/c++/java/c# world.  
> I've started to write lisp code, but my limited experience does not 
> allow me to grok this yet.
>
> Thanks to all for help,
> yinso
>
> On 10/5/06, Lawrence Au <laau at erols.com> wrote:
>>       I run PortableAserve on top of Lispworks and I fix hot live
>> production
>> bugs all the time.  It's the fastest way to fix bugs;  I inspect the
>> actual
>> production data in the Listener to diagnose and then redefine actual
>> production
>> functions and data in the Listener to fix the bugs.  My web pages are
>> all served from a set of structures rooted in global variables which
>> define multiple concurrent session states,
>> so individual sessions state is fully accessible from any Lisp thread,
>> such as the Listener.   The individual web pages are computed
>> from these structures for each http-get,  so on the client web 
>> browser,
>> just flushing cache and reloading the page to displays the fixed
>> behavior.
>>
>>      This avoid having to replicate bugs before fixing them,
>> and avoids time consuming turnover actions.  I fix actual problems
>> directly,  instead of approximations of those problems.  Most bugs
>> are just typos,  which are great to fix this way.  Of course if the 
>> bug
>> required a serious rewrite,  I'd do the rewrite on a test server.
>> However,  the data I can pull from a production Listener is really
>> valuable in setting up the test cases on the test server.  Since Lisp
>> structure print out as loadable versions of their contents,  I can
>> just cut and paste the full session state to a test server to 
>> replicate
>> specific problems there.  Of course it's important to drive
>> PortableAserve's
>> http handlers with functions that take a single structure argument,
>> (whose structure fully describe session state)  otherwise that cut and
>> paste
>> would have multiple pieces which would have to be loaded together
>> by hand.
>>
>>     Paul Graham is right.  I have found Lisp to be about 3x more 
>> elegant
>> than
>> any other web serving technology.  By this I mean that one person
>> can easily do the work of three.
>>
>> Lawrence Au
>> Q-Phrase LLC
>>
>>
>>
>> On Oct 5, 2006, at 1:41 AM, Yin-So Chen wrote:
>>
>> > Hi lisp gurus-
>> >
>> > sorry if this (newbie) question has been asked before - I tried to
>> > search for the information on the web but can't (probably because 
>> it's
>> > hard to tell what the keyword is). I am wondering if it is possible
>> > to debug and serve webpage at the same time in lisp (specifically
>> > clisp) and verify my underestanding.
>> >
>> > The reason I am wondering is that I've read Paul Graham's note on 
>> his
>> > experience in viaweb he was able to diagnose a problem while the
>> > customer is on the phone. His statement implies that he was running 
>> a
>> > separate instance of the website (so not the same one as the
>> > customer's looking at), but he attempt to reproduce the problem and
>> > can fix it at the same time (all appearing without having to 
>> shutdown
>> > the production image).
>> >
>> > You can find the excerpt here:
>> > http://lib.store.yahoo.net/lib/paulgraham/bbnexcerpts.txt
>> >
>> > After playing a bit with araneida's example - it seem that with 
>> clisp
>> > this is not possible - because (host-serve-events) blocks repl. I
>> > didn't try this on SBCL, but assuming it has multi-threads I can see
>> > it being a possibility.
>> >
>> > Another way I can think of is that he built an abstraction layer 
>> that
>> > allow him to work from repl to reproduce a problem discovered 
>> through
>> > a web server request, so he can work in session to fix the problem,
>> > but I can't see how he can update the running clisp image without
>> > having to kill the current web server session (assuming he has
>> > something that works similar to araneida) and reload code.
>> >
>> > Please let me know if I am missing something obvious. While I don't
>> > know if I agree with the practice of directly changing production 
>> code
>> > in real-time, the concept is intriguing, and certainly would be 
>> great
>> > to know how it's done.
>> >
>> > Oh - I am using clisp now as I am doing work on windows. I might 
>> move
>> > to sbcl on linux but certainly would love to understand how it's 
>> done.
>> >
>> > Thanks for any comments,
>> > yinso
>> >
>> > _______________________________________________
>> > lispweb mailing list
>> > lispweb at red-bean.com
>>  > http://www.red-bean.com/mailman/listinfo/lispweb
>>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/enriched
Size: 9637 bytes
Desc: not available
Url : http://www.red-bean.com/pipermail/lispweb/attachments/20061005/52256927/attachment-0001.bin 


More information about the lispweb mailing list