Parse string with “read” and ignore package namesp

Question

I am writing a program that opens a lisp file, calls "read" on the stream until the stream is empty, and does things with the lists it collects.

This was working quite nicely until I discovered that "read" will perform package lookup, for instance if it encounters some-package:foo it will complain that Package SOME-PACKAGE does not exist.

Here is an example showing what I mean:

(read (make-string-input-stream "(list 'foo :foo some-package:foo)"))

So I now I would like one of three things:

1. Make it so "read" will ignore package namespaces so I can convert arbitrary source files to lists of symbols.
2. Use some other parsing library with similar behavior to "read" but that only gets plain symbols, either by mangling the : or ignoring the colon and everything before it.
3. Pre-processing the file and use regex or such to package lookups and replace them with plain names, such as converting "some-package:foo" to simply "foo"

The purpose of all of this in the first place was to make a function call dependency graph. I'm aware there exists things of that nature of much higher quality that exist, but I wanted to do it myself for fun/learning. However, I have hit a snag with this problem and don't know how to proceed.

Answer 1

For your use case, you could handle the package-error condition by creating the required package and restarting. That would also preserve the symbol identities. Note that you need to handle in-package forms when you encounter them.

Answer 2

The simplest answer is to tell the Lisp reader to read colon #\: as is:

(defun read-standalone-char (stream char)
  (declare (ignore stream))
  char)

(defun make-no-package-prefix-readtable (&optional (rt (copy-readtable)))
  "Return a readtable for reading while ignoring package prefixes."
  (set-syntax-from-char #\: #\Space rt)
  (set-macro-character #\: #'read-standalone-char nil rt)
  rt)

(let ((*readtable* (make-no-package-prefix-readtable)))
  (read-from-string "(list 'foo :foo some-package:foo)"))
==> (LIST 'FOO #\: FOO SOME-PACKAGE #\: FOO) ; 33

The obvious problem is that this will read FOO:BAR and FOO :BAR identically, but you might be able to work around that.