What programming language features are well suited

问题:

I would like to build a "live coding framework".

I should explain what is meant by "live coding framework". I'll do so by comparing live coding to traditional coding.

Generally put, in traditional programming you write code, sometimes compile it, then launch an executable or open a script in some sort of interpreter. If you want to modify your application you must repeat this process. A live coding framework enables code to be updated while the application is running and reloaded on demand. Perhaps this reloading happens each time a file containing code is changed or by some other action. Changes in the code are then reflected in the application as it is running. There is no need to close the program and to recompile and relaunch it.

In this case, the application is a windowed app that has an update/draw loop, is most likely using OpenGL for graphics, an audio library for sound processing ( SuperCollider? ) and ideally a networking lib.

Of course I have preferred languages, though I'm not certain that any of them would be well suited for this kind of architecture. Ideally I would use Python, Lua, Ruby or another higher level language. However, a friend recently suggested Clojure as a possibility, so I am considering it as well.

I would like to know not only what languages would be suitable for this kind of framework but, generally, what language features would make a framework such as this possible.

回答1:

Clojure has pretty much everything you are likely to want as a live coding language. Main highlights:

• Interactive REPL - so you can interact directly with your running program. Even when I'm doing "traditional programming" I tend to write code interactively and copy the bits I like into a source file later. Clojure is just designed to work this way - pretty much everything in your program is inspectable, modifiable and replaceable at runtime.
• Great concurrency support - you can kick off concurrent background tasks trivially with code like (future (some-function)). More importantly, Clojure's STM and emphasis on high performance immutable data structures will take care of the more subtle concurrency aspects (e.g. what happens if I update a live data structure while it is in the middle of being rendered??)
• Library availability - it's a JVM language so you can pull in all the audio, visual, IO or computational tools you require from the Java ecosystem. It's easy to wrap these in a line or two of Clojure so that you get a concise interface to the functions that you need
• Macros - as Clojure is a homoiconic language you can take advantage of the Lisp ability to write powerful macros that extend the language. You can effectively build the exact syntax that you want to use in the live environment, and let the compiler do all the hard work of creating the complete code behind the scenes.
• Dynamic typing - the benefits of this can be argued both ways, but it's certainly a huge benefit when trying to write code quickly and concisely.
• Active community with a lot of cool projects - you're likely to find a lot of people interested in similar live coding techniques in the Clojure community.

A couple of links you might find interesting:

• Paul Graham on Lisp - beating the averages
• Live Clojure coding example with the Overtone sound synthesizer (a frontend to SuperCollider)

回答2:

I have implemented a live coding feature in Lua as part of the ZeroBrane Studio IDE. It works exactly as you described by reloading the application when a change in the code is made. I'm working on possible improvements to modify values at run-time to avoid full reload of the application. It's a pure Lua-based solution and doesn't require any modifications to the VM.

You can see the demo of the live coding as currently implemented here: http://notebook.kulchenko.com/zerobrane/live-coding-in-lua-bret-victor-style.

In terms of language features used/required, I rely on:

1. the ability to interrupt/resume a running application (this is based on debug.hook and error() calls),
2. the ability to interact with the (unmodified) application remotely (this is done based on debug.hook, TCP interactions with select() support to detect if a new request is being sent from the host machine, as well and on coroutines to switch between the main application and the live coding module), and
3. the ability to inject new code into the application (this mechanism is also using co-routines, but I'm sure there are alternatives). There is also a possibility to inject just a modified fragment, but it need to be at the level of a function, and if this function is a local to some other function, you need to include that too and so on.

回答3:

The only thing that’s necessary to make this work is a form of dynamic binding, e.g., message passing in Erlang or eval in many other languages.

If you have dynamic binding, then you can change the target of a message without affecting the message, or a message without affecting the target—provided that a target is defined when you try to send a message to it, and a message is defined for the targets to which you send it, when you send it.

When changing a target, all you have to do is serve messages to the previous version until the new version is in place, then do a small locked update to transition to the new version. Similarly, when changing a message, you just serve the old version till the new one is available.

Readily hot-swappable code must still be designed as such, however—the application must be modular enough that replacing the implementation of a component does not cause an interruption, and that can only come from careful programming.

回答4:

It's well and good to have 'live coding' on your dev box, but a way to directly interact with a deployed server takes it a lot closer to being 'real'. For this you need a network aware REPL.

clojure provides this nicely in the form of a socket repl. This allows you to remotely attach to the running version of your code on your deployed tomcat server (for instance). You can then attach your favorite swank-enabled development tool and hack away.

回答5:

Tcl has such a thing already. For example, you can write a gui program that creates a separate window that has an interactive prompt. From there you can reload your code, type in new code, etc.

You can do this with any gui toolkit, though some will be much harder than others. It should be easy with python, though the indentation thing -- IMHO -- makes interactive use challenging. I'm reasonably certain most other dynamic languages can do this without too much trouble.

Look at it this way: if your toolkit lets you open more than one window, there's no reason why one of those windows can't be an interactive prompt. All you need is the ability to open a window, and some sort of "eval" command that runs code fed to it as a string.

回答6:

python on google appengine has repote_api_shell.py. this is not a full live-coding suite - clojure on emacs w/ swank-clojure has had much more real-life use as far as integrating 'livecoding' into everyday development rhythms - but a lot of people don't realize this is possible in certain python environments.

$ PYTHONPATH=. remote_api_shell.py -s dustin-getz.appspot.com
App Engine remote_api shell
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]
The db, users, urlfetch, and memcache modules are imported.

dustin-getz> import models
dustin-getz> models.BlogPost(title='a modern take on automated testing', link='https://docs.google.com/document/pub?id=1DUxQogBg45rOTK4c5_SfEHiQcvL5c207Ivcy-gDNx2s', dont_publish_feed=False).put()

dustin-getz> items = models.BlogPost.all().filter('dont_publish_feed =', False).order('-published_date').fetch(100)

dustin-getz> len(items)
58

dustin-getz> for item in items[:5]: print item.title
a modern take on automated testing
Notes: Running a startup on haskell
the [un]necessity of superstar middle management in bigcos
"everything priced above its proper value"
stages of growth as a software engineer

回答7:

Smalltalk is probably the best bet for this. As unlike the others, it has a whole IDE for live coding, not just a REPL

回答8:

I'm working on a live coding feature for PyDev's Python editor. It was inspired by Bret Victor's Inventing on Principle talk, and I've implemented a program state display as well as turtle graphics. They both update as you type your Python code in Eclipse.

The project is hosted on GitHub, and I've posted a demo video, as well as a tutorial.

The main features of Python that I used were abstract syntax trees and dynamic code execution. I take the user's code, parse it into a tree, then instrument any assignment statements, loop iterations, and function calls. Once I've instrumented the tree, I execute it and display the report or draw the requested turtle graphics.

I haven't implemented the swapping feature that other answers discuss. Instead, I always run the code to completion or a time out. I envision live coding as an enhancement to test-driven development, not as a way to hack on a live application. However, I will think more about what swapping out pieces of a live application would let me do.