A Day for the Smalltalk Programming Language and its Friends
Sunday 10 April 2016
8am – 4pm
The Box SF
1069 Howard Street (between 6th and 7th)
three blocks from Moscone Center, SoMa
San Francisco, USA
the event is free
a request for presenters will be announced in a future month
a presentation called An Introduction To Smalltalk will promoted to the Ruby, Python and Objective C communities
the latest details will be available at http://www.ooplu.com
Contact Person: Chris Cunnington
Email: brasspen (at) gmail.com
Adam Spitz did a presentation on Clojure in January. He made an immutable collections library, which you can find here:
I now have the bug to try to better understand functional programming. Adam mentioned a video by Rich Hickey where he makes it clear that “place” is bad.
What does that mean? What is “place”? Why is it bad? Thinking about this led me to a brain pop moment.
Rich Hickey reveres Chris Okasaki’s book “Purely Functional Data Structures.” And Okasaki disparages “destructive updates,” which are why “place” is bad. You have one location and one copy in the location, which you destroy to replace with another. Functional data structures don’t want to do that. They want “persistence,” which is copying what came before, and then changing the copy to leave the original untouched. There is no “place” where the data is.
So I started to think and my thinking became figurative and literal and odd.
A car in a numbered parking lot can be identified by the lot number, but the car still has an inherent identity. If X = 10, then X is a location that contains the number 10. But 10 has an identity regardless of the variable assignment. The car is still a car for not being in the parking lot.
The parking space is a “place.” How can a functional language access something if it has no place and no label? Where is it?
Let’s say a police station is at an intersection. You identify that specific police station by its place, by its address. Let’s say the police station now has no place. Let’s say the police station floats like a balloon above the neighbourhood. How could anybody access it? How would they know where it was? It would no longer be useful, because it could not be found or accessed.
Then I thought of a a car. (I was in a bus at the time.) A car can have no specific place. You worry if a police station has no specific place and is floating over the neighbourhood like a balloon. But you will not worry about a car. A car is supposed to move. It is not supposed to be in once place. If it’s on the highway between places, then it’s doing what it’s supposed to be doing. Moving.
And then it all collapsed into a picture in my head. In a functional program, the data is moving. You don’t worry about the “place” of a thing that is supposed to move. It’s the behavior that is staying still. Functional programs create static behavior that creates pipes that carry data forward from station to station. Questions about “place” mean nothing. If I ask “How can I access 10, if it is not assigned to X?” does not mean anything. It’s water going through a pipe. Drop a ping pong ball into clear plastic pipe and watch it go.
And so that made me realize that scope, in a functional program, is reduced to almost nothing. There is only enough scope for the data/car/water/ping pong ball to move forward. Scope in real world terms is lots and lots of space. A police station with no specific “place” can not float anywhere, because the scope of a functional program would shrink to contain it almost alone. Clojure is big on namespaces. And Rich Hickey thinks data moving over the wire though a TCP socket is an ideal. How much scope does an HTTP message have travelling over the wire? Not a lot, I don’t suppose.
You don’t need “place” in a functional language, if your scope is so constrained that there can be no confusion about the pipe your functions have created for your data to move through. Your data mutates as it travels. Persistence in the functional sense allows for continuous copying and mutation of an original datum, while leaving that datum unchanged.
I had the idea that a plugin from one VM could be dragged into another VM’s directory and that it could be used just by starting up the image. I’ve done a little experimenting and it seems more of an idea of a reality. The functionality is there, but VM developers over the years have not seen this as a priority. Typically plugins and their VMs are compiled together. Or a person adds one by recompiling a VM compilation rig they already have.
As I had a rig for the Interpreter VM I decided to compile a plugin and see if I could drag it around to other VM directories for use. Most times it didn’t work.
I compiled a TheUniversalAnswer plugin which returns 42. I moved it from a 4.14.1-3430 VM to a 4.14.1-3414 VM. I could get it to work if I did not use the squeak.sh start script. That is, I dragged so.vm-sound-null, so.TheUniversalAnswer and so.vm-display-X11 into the same directory as the VM binary. That worked.
The only way to see the external plugins is with #listLoadedModules. But, irritatingly, modules are loaded as needed, so once you have proof of using the primitive from the plugin, yea, it will appear as a result of that Smalltalk listLoadedModules message. So, it’s not that useful.
I dragged so.TheUniversalAnswer around to other VMs such as 4.0.3-2202 and 4.13.10-3268 without success. And the unload selectors #forgetModule: and unloadModule:, which both use primitive 571 don’t appear to work on my Ubuntu 15.04.
So, I don’t know if a community of plugins passed around is in the offing. What I have learned is that many of the answers are in sqUnixExternalPrims.c. The issues are really where does the VM look for primitives and what does it do when it finds one. It appears to me, after reading both the non-Cog and Cog versions of that file, that this an interesting area poorly documented that VM developers alter in haste to get on to something else.
It’s pretty important stuff, though. Do you start the VM with the binary on the path? Do you use squeak.sh? Where is the VM looking for stuff. Where will FFI look for stuff. And so on.
I just compiled a 64-bit Spur Stack VM and I’m a bit confused, because it works. This is contrary to my experience. I’m jotting down what I did to remind myself.
Download the following:
sudo svn co http://www.squeakvm.org/svn/squeak/branches/Cog/src
While doing the following, open up the preferences for the files. I execute “chmod -R 777 directorynamehere” into each thing downloaded.
Put the first three downloads in a single directory called StackSpurBuild. Put the src directory aside for a moment. We’re only ransacking it for parts.
Go to spurstack64src and delete the examplePlugins.ext and examplePlugins.int files.
Go to the src directory you downloaded and drag the plugins directory, examplePlugins.ext and examplePlugins.int into spurstack64src. You can now move the src directory into the trash.
Open examplePlugins.ext and delete the BochsIA32Plugin line. You’re not building a simulator.
Go to /build.linux32x86/squeak.stack.spur/build/mvm and open it up. We need to make some changes.
Go down to the “test -f config.h” line. Ensure that you have the following:
Those two flags should now gibe with the directory dragging we’ve been doing.
Drag the /squeak.stack.spur/build directory onto the Terminal. Use Ctrl+A to go to the front of that line, write “cd” and execute, so we are in that directory. It’s our current directory. At this point its a good idea to have the /squeak.stack.spur/build directory open on your desktop, so you can see the explosion of directories on the left, while configure and compilation details scroll in the Terminal on the right of the screen.
Execute “sudo ./mvm”. Give your password and say yes to a “clean build”.
Wait. If all goes well you have a new directory in StackSpurBuild called /products. It has a /stkspur64linuxht/squeak script you can drag on to a Terminal to start an image. The actual binary is deeper in the directories at /stkspur64linuxht/lib/squeak/5.0-3411/squeak.
The image to run on this is at http://mirandabanda.org/files/Cog/SpurImages/
You need trunk46-spur-64.image and trunk46-spur-64.changes. You’ll also need a SqueakV41.sources file. You can get one at http://ftp.squeak.org/4.5/
This was on an Ubuntu 15.04 systemd laptop.
Dependencies for compiling an 64-bit vm ought to be less noisome than for 32-bit. It’s been a while since I loaded this laptop but I added some of the following to compile 32-bit vms. Are any of these necessary? I don’t remember.
apt-get install lib32z1-dev -y # install 32-bit libs
apt-get install libsm-dev:i386 # for Squeak vm
apt-get install libglu1-mesa:i386 # X11 for Squeak
apt-get install uuid-dev:i386 # for Squeak vm
apt-get install mesa-common-dev:i386 # add gl.h
Naturally, these do need to be 64-bit related. And I do know you need uuid-dev. That’s important. This is the official HowToBuild document. It’s 32-bit centric.
Two years ago I posted that I’d put a stripped down HTTPView2/Seaside into Cuis. At the time there was no response. I was asked on Wednesday if I still had the image. Not likely. Since then I’ve probably bought two new laptops and had two TimeMachines fail on me. So could I rebuild it? Ah…maybe.
I rebuilt it and posted a link (and script for Squeak) on both the Cuis and Squeak mailing lists. Look for a Chris Cunnington post dated 29 May if you want to download the image for Cuis or install using a script for Squeak.
The interest in Cuis for a web framework comes from recent advances in porting Andreas Raab’s WebClient to Cuis. This prompted Edgar to ask me if I still had a framework. I must say I was surprised. Just about everything I build sinks without a trace of interest.
I need to document some things about what I’ve done. Or more importantly, what I haven’t done. This can better allow the Cuis community to either use GreenNeon or leapfrog it with something better.
I’ll address that in four areas: where GreenNeon came from; what it is; how to use it; and, how I ported it to Cuis.
That ends the general specification of this blog post. From here I’m going to go down a rabbit hole of details.
About twelve years ago Avi Bryant and Goran Krampe worked together to bring the HTML domain into Squeak Smalltalk. Their first creation was a Builder like the WAHtmlBuilder in old Seaside images. The big idea was to have one selector for each HTML tag. They built it, it worked, and they found the limitations of that idea fast. Those of you who remember David Shaffer’s blog documentation on Seaside will remember the long selectors that would take six arguments at a time. This wasn’t ideal, so they decided to work on another coded representation of the HTML domain in Smalltalk. This lead them to the Canvas like WAHtmlCanvas.
The big idea here was to have a class for each tag instead of a selector for each tag. The Seaside 2.6 image came with both the WAHtmlBuilder and the WAHtmlCanvas. There was a selector for choosing which to use called #rendererClass, a builder or a canvas. But let’s step back to where Avi and Goran created the Builder.
Avi created Seaside and Goran created HTTPView2. The main difference was the dispatching mechanism. By that I mean how a URI in a request finds the objects it needs to return a response. Continuations in Seaside required a 24-character token. HTTPView2 was more RESTful in that it used conventional looking URIs (i.e. /page/secondtierpage/resource.jpg).
HTTPView2 never adopted a canvas. If you download it into Squeak using SqueakMap you will see it still uses a builder called HVBuilder. A half completed canvas is there, but was never used. As a result, the page descriptions are messier, because one of the virtues of a one-tag-one-class canvas system is a neater looking method. I should point out that the canvas is what Seaside uses to this day.
Goran was ahead of Avi in wiki creation. Avi borrowed some code to create a simple wiki maker in Seaside 2, which stored the text in a method as a string.
Avi’s Seaside went on to be used for SqueakSource and other projects. HTTView2 was used only once and you can still find it online as the SqueakMap server at http://map.squeak.org.
There’s one more piece of background I need to give you. Andreas Raab created a replacement of the Comanche server called WebClient. It’s the success of Cuis programmers to port WebClient to Cuis that porting GreenNeon to Cuis is possible.
HTTPView2 in the SqueakMap server online uses the Comanche server, the HVBuilder for rendering the HTML domain in Smalltalk, and a RESTful dispatcher focused in HVTransientView>>#dispatchOn:.
I took the HTTPView2, replaced the HVBuilder with a WACanvas from a Seaside 2.6. Then I created an adaptor (i.e. GNAdaptor) to replace the Comanche server with WebClient.
Let’s sum up. GreenNeon can do anything that SqueakMap server can do. If you are familiar with Seaside page creation in #renderContentOn: methods, then you are familiar with how GreenNeon adds tags. The part you are not going to be familiar with is the dispatching mechanism adopted from HTTPView2. How does GreenNeon relate the URI to an object with tags to produce an outgoing stream?
Let’s start simple. Say you have “foo.com”. It’s implied that you are accessing the “/” URI. Then the dispatcher looks for the #default selector in the class with your pages in it.
Hold it. I need to back up again.
Let’s say you want to create a Hello World class. You need to subclass from HVTransientView. Let’s call that SuperDuperHelloWorld. We will return to how to add tags in a second. Go to a Workspace and execute “SuperDuperHelloWorld startOn: 9981”. This will start a listening process, which you will see in the ProcessBrowser. (Incidentally that is where you need to kill GreenNeon processes. There are no #start or #stop selectors.)
We have a class. It’s running in a process. The request will be routed to the correct class which we have called SuperDuperHelloWorld. Because we are requesting “foo.com” it will default to a selector called #default. That’s where we put our tag statements like this:
default html := self canvas. html buildHead. html p: 'Hello World !'. ^html
If you are a Seaside user this will look familiar and weird at the same time. In the usual #renderContentOn: the canvas and head are built somewhere else. And the selector does not terminate with a return caret (i.e. “^”). In Seaside, your selector is one selector inside a class of a graph of classes sent to be rendered with the Chain Of Responsibility pattern. You’ve used composition to create a Composite pattern, decorated it with the Decorator pattern and render the lot to a stream. GreenNeon is a bit simpler.
Each selector is its own page. If in SuperDuperHelloWorld we have selectors for #foo, #bar, #red and #car, then we can make requests like foo.com/foo, foo.com/bar, foo.com/red and foo.com/car. One selector for each page.
This poses an obvious problem. How do we get to second tier pages? How do we reach foo.com/bar/sega or foo.com/bar/genesis? HTTPView2 uses double dispatch. You pass the request from one class where the selectors are whole pages to another class where the selectors are whole pages. I’ll spare myself explaining that for the moment. If there’s any interest, I can blog on that sometime.
What did I add to Cuis to port GreenNeon from Squeak to Cuis?
The Url hierarchy including Url, GenericUrl, FileUrl, etc. All subclasses of Url.
Some methods to String>>#squeakToIso,#utf8ToIso, BlockClosure>>#renderOn:, BlockContext>>#renderOn: and Object>>#asSeasideString,#convertSeasideString:,#renderOn:,#contentsChanged.
I added ValueHolder.
It think that’s about it. That list is not exact or complete.
Cuis renamed WebClient as HTTPClient. It’s included in the core packages in /Packages/Protocols-HTTP.pck.st. HTTPServer is in GNAdaptor>>startOn:.
Five years ago Lukas Renggli and team took SushiStore out of the base Seaside distribution. That annoyed me no end. To me, it meant they had removed the sole clear example of a website in Seaside. The rest were just examples of tiny widgets that didn’t add up to a full website. Five years have past and I have a new view on that decision. It’s more nuanced than my opinion of five years ago.
What I couldn’t see five years ago was that while the website was comprehensible to an end user, to a programmer looking at how it was implemented, there were some problems. In short WAStore, the foundation class or root class of SushiStore, lead a person on a convoluted journey. A lot goes on in a very small area. Now that I’m looking at Lisp in a book called “Land Of Lisp” SushiStore has the density of a Lisp program. Avi was a Lisp programmer, so maybe he was used to packing a staggering amount in a small area.
The foundation of any Seaside website is going to be some kind of Compsite pattern straight out of the Design Pattern Smalltalk Companion. Seaside follows that pretty closely. It also adds Decorations. Though, it’s worth noting that the DPSC does not say much about how to implement the Composite and Decorator patterns. It will not tell you how to write #addDecoration: or #decorationChainDo: or #childrenDo:. The DPSC does describe the Decorator pattern using VisualWorks of about twenty years ago. Squeak has no native implementation of the Decorator pattern that I can see in Morphic or anywhere, so I posit that the Decoration implementation came from VisualWorks to Squeak.
At any rate, the Composite pattern is the foundation of any Seaside application. You create a root. Perhaps you extend the tree structure downward and outward with #children. Or you add a widget that is rendered from your root with #render (or with div id: ‘foo’; with: which does the same thing). But here’s where it gets hairy: what kinds of nodes are you going to have?
SushiStore starts with a node that renders some HTML. Then it has a node that contains a subclass of WATask. In this case it’s WAStoreTask. That’s a kind of Strategy pattern node, I think. It contains the algorithm or steps of your website/program. Then there are other nodes called to fill ivars like task, main, cart, and cartView. And then, on top of that, you are calling continuations on some of these ivars. In terms of following the Composite pattern that can be a surreal experience, because the Composite pattern just … stops… and then picks up somewhere else. This leap is managed by decorations added with #isolate:
So, in terms following a textbook Composite pattern, this is quite a lot going on in quite a small space. I imagine this is the kind of thing that made Dijkstra upset about the GOTO command. This morning I’m looking in “Land Of Lisp” and there’s a long disclaimer about the loop and format commands. The author tells at length that though these commands are impure, but they sure are useful. Ah, OK. As far as I’m concerned a hand on a hammer is a hand on a hammer.
And I guess I cannot be too aloof of this ideological concern, because I can see now that SushiStore is extremely dense. It’s disorienting. And who had some solutions to the fruit cup chaos of Sushi Store, why, Lukas Renggli did. Did things get clearer? Yes the did. Was there less complexity. Nope. The fusion of Pier and SmallWiki created a much more complex beast than SushiStore. But if you have some DPSC patterns under your belt, you are amazed at how cleanly Lukas makes patterns work together. It’s remarkable. And yet, I sort of wonder if the cure isn’t just as bad as the disease.
As far as I can tell, Lukas saw a number of problems with SushiStore and came up with amazing solutions for each of them. But what sort of gets me about Pier is how the original problems are so totally obscured by staggeringly innovative solutions.
The Composite pattern in Pier is uniform. If SushiStore’s Composite pattern is a mash of behaviour and state, then the Composite pattern in Pier is purely state. It does one job. All the behaviour has been pulled out and put in Visitor pattern classes that iterate over the clean state-only Composite. That’s pretty nifty.
Yet, it abandons continuations entirely. I have Pier3 loaded in a Squeak image and there are no senders of #call: anywhere. I don’t think that clean Composite pattern with its Visitor pattern can use continuations at all. The commands are executed with Command patterns. And I think you can add Composites inside the main Composite pattern in the form of structures and widgets and such.
The WATask Strategy class? Gone. It has no place in the Composite pattern in Pier. My knowledge of Pier is only starting, so I guess there is lots of space around the clean, state only Composite that allows for different algorithms.
Looking at SushiStore, I think that what is needed is a generic Composite structure for a beginner to make a Seaside website with. Perhaps there would be some standard ivars that could be nodes. Then those nodes could be filled with widgets. And those same ivars would be the pedestals continuations could stand on to move orthogonally above the Composite pattern. A nice map of a standard Composite pattern with ivars/nodes to extend as needed would be a great map for a person starting with Seaside to use. As usual, Lukas saw something similar, and as usual he went about three steps better.
He created the Environment page. Using the SmallWiki tools of SmallCC and Pier syntax, you have a base, bottom, “under page” presented to you. He saw that a Seaside application needs a bottom page or map to build on top of and he made it changeable like any wiki page. Very clever. At bottom, any Seaside application is one page – the one you start on. From there you’re swapping components in and out of ivars or calling continuations. Lukas sees the problems and creates amazing solutions. And yet, it would be nice if the problems were elucidated clearly. Instead, the dazzling solutions erase the original problems altogether.
My ultimate point is that by revisiting the original problems, I think I see a point in the evolution of Seaside where it reached a crossroads. That moment was somewhere in 2008, I think. There are a number of problems to getting started with Seaside. There are almost too many possibilities. It can be overwhelming. For each problem, Lukas came up with a solution. I want to emphasize that I believe they weren’t the only solutions. It’s hard to argue against such a brilliant programmer. Yet I think to revisit the evolution of Seaside is to see the problems that grew into Pier with fresh eyes. There is the Lukas way of Seaside. Yes. But I don’t think it was ever the only way.
A standard Composite pattern where every node is either behaviour or state is viable. Seemed to work for Avi. Seaside adoption hasn’t exactly exploded in the past five years. I think that’s because the evolution toward Pier as the paramount solution made things more overwhelming for a beginner, not less.
At the Toronto Smalltalk Users Group last night I presented on the article The Early History Of Smalltalk by Alan Kay. You can Google it or buy a beautiful copy from the Digital Library at the ACM.
My intent was to see what Smalltalk was like without a compiler, image, browser, or byte codes. The problem I’ve been facing for years is to look at the vm or byte codes or a CompiledMethod and not be able to look past the immediate context of what I’m looking at to see how it relates to other parts of the system.
I was intrigued by the part of the story where Dan Ingalls implemented the first interpreter in a Data General Nova in BASIC to make 3+4 tally. I thought why not do this in BASH? I found the St-72 simulator Dan made for the 3.2 image and that idea was quickly put aside. You can download the image above. You’ll need to use a vm from Squeakland. Then just read the instructions in the Transcript window.
Smalltalk-72 is very different from Smalltalk as we know it. It has no utility and is some kind of semiotics engine. The objective was to build better children with this medium of computing. Alan Kay talks about how he was influenced by Piaget, Bruner, and Papert. He’s not fooling around here. The difference between Smalltalk-72 and Smalltalk-76 cannot be overstated. If Dan Ingalls had not refused to “burn the disk packs” at the three day retreat in Pajaro Dunes in late January 1976, then you would never have heard of Smalltalk. They went separate ways. Alan started work on NoteTaker. Dan came up with a new version of Smalltalk with byte level implementation that ran x180 faster and did work you as a programmer could appreciate. Alan came around when he saw the new system seven months later.
Even still, Smalltalk-72 is a blast. You can see the REPL from Lisp directly when you type “show ev” and it produces (repeat(cr read eval print)). That looks a lot to me like Interpreter>>#interpret. And Smalltalk-72 fits my agenda of wanting to see the interpreter in a simple, whole form that I can explore. All the pieces of future implementations are there in embryo. It’s great to be able to comprehend the whole system in the palm of my hand and then map it onto future evolutions.
Basically, you have an array called a vector that acts as the message stream. Vector understands the message eval. I won’t put the hash mark in front – # – because that wasn’t in use yet. At that time they used the “eyeball” or Celsius degree sign. So eval wound not be #eval but ˚eval. Get it? Eyeball. You send a message to vector and it “looks” to see if it understands that message. If it does then ˚eval would return true. True is the receiver of the new message which is sent >> which is an if/than conditional. If the answer is true, then you execute the content in the vector (i.e. the selector in the compiled method) of this object. If the answer is false then it falls through and looks a the next selector token. It looks like this:
The object read has a message “of”. If ˚of evaluates to true, then the if/than conditional ˚>> will execute the code in the vector (:str. CODE 2). If the eyeball did not see the selector token name, then it returns false and the vector is passed over to check the next selector token. It helps to have a copy of the Smalltalk-72 Instruction Manual pdf, which isn’t hard to find.
Interestingly, David C. Smith of Croquet fame did his doctoral thesis at Stanford on Smalltalk-72 and “iconic programming”. He wrote a twenty page program called Pygmalion in 1975.
The Early History Of Smalltalk is a great read and the Smalltalk-72 interpreter is fun. It sort of makes you feel like a kid, which I suspect is its point. I mean, if you spend your time working on a system for children, then isn’t that a way of reminding yourself what it was like to be a child, because that’s a place you want to be?
At any rate, if the virtual machine is completely incomprehensible to you as it is to me, then the place to start is the interpreter. And a simple interpreter in the the Smalltalk-72 interpreter described in Appendix II of Alan’s article is a great place to start. The 60s live.
“Computer power to the people! Down with the cybercrud!”