The Singularity Institute just announced the release of the Open Cognition Framework developed by Ben Goertzel.

The Open Cognition Framework (OpenCog) is software for the collaborative development of safe and beneficial Artificial General Intelligence.

OpenCog provides research scientists and software developers with a common platform to build and share artificial intelligence programs. The framework includes:

• a flexible and highly optimized in-memory database for knowledge representation,
• a plug-in architecture for cognitive algorithms and a cognitive process scheduler,
• a built-in LISP-like programming language, and
• other components to support artificial intelligence research and development.

Programs written or adapted for OpenCog may be combined and used in concert with one another for experimentation or to achieve better results compared to their stand-alone counterparts.

Markus Völter interviewed Dick Gabriel at OOPSLA 2007. Don't miss Dick's explanation of Lisp's EVAL! [via Mike Ajemian]

Dick initiated the idea of a Master of Fine Arts in Software and my recent experience with the software creation process, if you dare call it a process, in the financial industry indicates, we should make it mandatory for any software architecture position.

And in closing let me quote from Dick's essay The Art of Lisp & Writing:

Lisp is the language of loveliness. With it a great programmer can make a beautiful, operating thing, a thing organically created and formed through the interaction of a programmer/artist and a medium of expression that happens to execute on a computer.

Links:

• Innovation Happens Elsewhere by Ron Goldman and Richard P. Gabriel
• Poem a Day project by Richard P. Gabriel
• Eternal Flame/God Wrote in LISP interpreted by Julia Ecklar
• The Art of Lisp & Writing by Richard P. Gabriel (the foreword to Successful Lisp by David B. Lamkins)
• Markus Völter's homepage

Reading Greg Linden's blog I realized that the technology I developed for medigist five years ago was an excercise in computational advertising.

Here's a project overview:

• Web-crawl a list of URLs and fetch new medical news content [Common Lisp]
• Clean up news using source specific filters and store into database [Common Lisp]
• Automatically tag news according to medical themes using a statistical text classifier [Common Lisp]
• Web subscription interface for medical topics [PHP]
• Manage advertising campaigns and related medical topics and upload PDF ads [PHP]
• Create monthly medical journal (PDF) for each user containing personalized news and ads [Perl, TeX]
• Track click-through rate for news and ads using a trampoline for each embedded article and ad [PHP]
• Do post analysis on click-throughs to improve content and ad selection for each user [Common Lisp]

Recently Dan Weinreb mentioned a paper about ObjectStore he co-authored with Charles Lamb, Gordon Landis and Jack Orenstein in 1991. It's a bit hard to find unless you have an ACM account. I've made it available here. Enjoy!

After reading Dan Weinreb's posting I ordered three OLPC XO notebooks. I've met Walter Bender some years ago through a licensing deal with MIT Media Lab. He is now heading the OLPC initiative. This is quite likely the most significant project since Tim Berners Lee's invention of the WWW.

Some nice features of the XO-1:

• Comes with source code for all applications and most of the operating system. There's even a View Source button on the keyboard that shows the code of what's running.
• Based on Linux, but application code is in Python
• 1200x900 7.5" diagonal LCD that works without backlight in bright daylight
• Wireless networking capable of mesh mode
• 2- or 5-cell LiFePO4 battery pack
• Built in color camera

Links:

• Review of the XO-1 by a 12 year old blogger
• Great technical info in the video of the talk at Google by Ivan Krstic, who is the architect of Bitfrost but talks about all aspects of the system

Marc Andreessen writes about the three different levels of web platforms in this recent blog posting.

I'd like to point out that, as far as I know, Viaweb, an online shop system build by Paul Graham and Robert Morris, was the first level 3 web platform according to the definition given by Andreessen:

A Level 3 platform's apps run inside the platform itself -- the platform provides the "runtime environment" within which the app's code runs.

When one of the customer support people came to me with a report of a bug in the editor, I would load the code into the Lisp interpreter and log into the user's account. If I was able to reproduce the bug I'd get an actual break loop, telling me exactly what was going wrong. Often I could fix the code and release a fix right away. And when I say right away, I mean while the user was still on the phone.

Such fast turnaround on bug fixes put us into an impossibly tempting position. If we could catch and fix a bug while the user was still on the phone, it was very tempting for us to give the user the impression that they were imagining it. And so we sometimes (to their delight) had the customer support people tell the user to just try logging in again and see if they still had the problem. And of course when the user logged back in they'd get the newly released version of the software with the bug fixed, and everything would work fine. I realize this was a bit sneaky of us, but it was also a lot of fun.

Links:

• Beating the Averages by Paul Graham
• The Other Road Ahead by Paul Graham
• Lisp in Web-Based Applications by Paul Graham

David McClaim released the source code for his Lisp-based signal, image-processing and modeling workbench SigLab. The talk he gave at ECLM06 was spectacular. He explained why he uses Lisp (multiple layers of abstraction using macros), why you need a text based system description and not a graphical (GUI) description and his demo of real-time processing was just mind-blowing.

Links:

• SigLab Release announcement
• SigLab download

Richard Greenblatt commented on my improvised video of the Powerpoint slides of his presentation at ILC'02 and suggested to publish the video recording of his talk. I'm to busy at the moment to convert it from WMV into a decent format, but if someone converts it to Quicktime I'd be happy to host it. Thanks to Richard Greenblatt for making the video available!

Video of Richard Greenblatt's presentation at ILC'02 [70MB]

I'm on my way back from the European Common Lisp Meeting in Hamburg. Edi Weitz and Arthur Lemmens put together an incredible conference. Everything was well organized, the location at the Gastwerk hotel was a beautiful setting, the food was delicious and the speakers gave some amazing talks. I've recorded all talks on DAT and will make them available for download as soon as the speakers have reviewed the recordings of their talks. I've put a couple of images up at Flickr.com.

1. 9:30 Jans Aasman: AllegroGraph

• The semantic web
• Intelligent search depends on meta data
• Why we need a triple database
• XML is actually not relevant, but reasoning is
• Missing constraints for IDF documents and contained definitions: OWL
• An IDF reasoner can reason about IDF documents
• The intelligence community was first do demand reasoning tools
• Other customers: telecom companies, biotech
• ISP (configuration mgt): 2 billion triples
• Telecom datamining over VOIP CDRs: Is there a circle of friends? -> almost impossible to do with relational databases!
• Movie database: Kevin Baken number, relationship mining
• First Franz product using Lisp inside a black box with a http and Java interface (no interface for other Lisps?)
• Reification:
  • put a trust metric as meta data on the graph
  • meta data about triples
• burst rate 14000 triples per second (saving and indexing)
• treat AllegroGraph as a Prolog database
• for governments it's important to say things about triples (trust metric)
• you can use this meta information inside of queries
Questions:
• How do you add new/correcting information to the triple
• fuzzy queries? Yes, every triple comes with a trust value and you can use a reasoning engine to do queries.

2. 11:12 James Anderson: Time series analysis for financial markets

• First get prediction with internal market data working
• You cannot beat the market with internal data only
• mix in external data (news headlines)
• timestamp the data on the feeds in multiple time dimensions so you can go back in time an analyze again
• 250MB per day input feed (one minute resolution) = 5 megapixel snapshot of the market
• evaluating several million ticks per day
• solve problems with differences in timing/timestamping between different servers
• develop the aibility to go back in time an clean up data (in case of interruption in the feed)
• Now predicting "market sentiment" not "nearest neighbour"
• used for ESF trading (future of Dow Jones), traded in Chicago 24h/day
• you need to do synchronous analysis to get rid of the discrepancies btw. servers
• impedance: code complexity issues even with Lisp
• Create a DSL with special operators to deal with the problems
• do data introspection at a higher level
• create a "generic function graph"
• give the analysts a language to express the way they think about the problem set
• visualize data in 3D using an abstraction layer that is device independant
• our target marktet is day-traders

3. 12:00 Arthur Lemmens: Rucksack

• persistence library for CL, because no one else was doing it
• deal with serializing shared objects
• write your own serializer to speed up serialization (orders of magnitude faster)
• use the MOP to check when a slot is referenced/accessed
• use a proxy for referenced objects that have not yet been accessed (memory forwarding pointer)
• cache: put loaded objects into a hash table, put changed objects into a separate hash table
• manual memory managment is a sin and it isn't even a pleasant one
• there should not be a delete instance in Rucksack= no dangling pointers!
• build a disk based garbage collector for the store
• copying collector for structured data
• mark and sweep collector more efficient for big blobs
• use a commit file to recover from crashes
• simple query language
• support schema evolution: class definitions are immutable, keep a list of versions of class definitions
• all schemas are kept in the schema table and the table is kept on disk
• about 2MM effort implementing it

7. 14:30 Jim Newton, Thomas F. Burdick, Peter Herth, Björn Lindberg (Cadence): Electronic Design Automation

Jim Newton:

• 5 billion per year industry
• 25% market share by Cadence
• physical design kit at the core
• Lisp library "Skill"
• Can't keep Cadence IP and customer IP in same repository for revision control (because of IP law)
• develop a "Meta-Version Control" to separate the code
• PCMan database is an implementation of this

• client server model of the PCman system (pronounced Pacman)
• create a CLOS wrapper for the database scheme

• assume that the clients are hostile
• only the server knows about the database
• clients only know about objects
• we have a first class transaction method
• first committer wins and all other clients are forced to update to the latest version of an object

• use Ltk to connect Lisp to Tk GUI
• use the Ltk remote extension to start the Gui remotely
• Generic Browsers

• How do you pick the modules to be used for a release? A release engineer decides which modules go into a release.
• Show the macro expansion of the database stuff...

3. 15:28 Martin Cracauer (ITA): QPX - Low fare search engine

• the airline industry separates the prices from travel information
• the booking problem is complicated by the number of prices created by the airline companies
• there's a combinatoric explosion of possible intiniaries
• Boston-Hamburg -> search 2.5 mio prices
• 10^28 solutions is not unusual!
• "In a way we have the worlds most sophisticated compression program for this data"
• "We need the raw speed from Lisp..."
• unecessary initialization of data structures is a problem with dynamic languages (and Lisp)
• we need bitfields
• 5 GigaBytes of maximally compressed flight data
• efficient datastructures are key
• we need to mmap anyway (because of the size of the data structure)
• share the data between instances of the search processes (one instance per processor)
• we need to read the 32bit of a C-struct directly from Lisp without a function call in between
• manual control of inlining is a very important feature for us (for 32 bit integer operations)
• one of the big benefits of Lisp to us is, that you can compile with different presets for speed and safety
• regression tests use the range check in safe compiled test versions to catch overflows
• we always leave array bounds checking on (because it is very hard to debug problems with array overruns) even when compiled in fast mode

• Why still use Lisp? Because we have very complicated algorithms in the actual search engine. We use macros (several million lines of code after macro expansion)
• how do you deal with negotiated fares? We use dedicated machines for customers with negotiated fares.
• How do you deal with availability checks?

4. 16:34 Klaus Harbo (Mu): cl-muproc

• Algorithmic trading platforms
• arbitrage in online sports betting markets
• MP is here to stay
• implement the important aspects of Erlang in Lisp
• muprocs exchange mumsg messages
• everything goes on in a single Lisp instance
• clients block on reads until they get a timeout
• we can generate all the message passing and the client only has to call a function
• supervisor for server processes enforce the policy (like automatic restarting services)
• 2500 lines of code
• BSD licensed

6.David McClain (Refined Audiometric Laboratory): Signal Processing SigLab

• Lisp-based numerical signal processing
• connect blocks using virtual wires
• we can send functional closures down the wire
• run simulations of signal processing
• model systems in Lisp
• Why Lisp?
• Exceedingly powerful macrology!
• most elegant OOP: CLOS
• dynamically typed
• ad-hoc design of new networks
• performance
• using Lisp augmentation (C-array substrate) to speed up processing
• hand coded high performance vectorized math routines
• aspect oriented programming extensions
• I've reverse engineered the BBE-Processing filter (~40 lines of code)
• discovered the EarSpring Equation which describes normal human hearing
• might make it available for people who are interested

Arthur Lemmens and Edi Weitz suggested a ECLM-themed t-shirt to celebrate the European Common Lisp Meeting 2006. The text on the front side reads:

Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.

European Common Lisp Meeting 2006 Hamburg, April 29/30