Redirect test results for tcltest

Question

I like use tcltest in my daily testing tasks. But test results i can only see in console, so after each test run i need to switch to that console, to find results. Аccording to tcltest documentation, there is option outputChannel. This option allow redirect test results to pipe or channel and then display anywhere i want. So i try to create this channel:

set logger [ open "|bash -c \"while true;do sleep 1; while read mess;do  debug=\\\"\$debug \n \$mess\\\";done; if \\\[\\\[ \$debug != \\\"\\\"  \\\]\\\];then notify-send \$debug; debug=\\\"\\\";fi; done \" " r+]

Next i configure tcltest like this:

configure  -singleproc 1 -verbose t -match ** -outputChannel $logger

Then i try to send test messages in my channel:

puts $logger  "Test message 1st line \n test message 2 line"

This script works, but show no test messages ,and no tcltest output in notifications. How i can create my logger channel??

Answer 1

That seems like quite a complicated script. With such things, doing it in stages makes life much easier. The first part is to put your Bash script in braces and to store it in a variables:

set notifySendScript {
    while true; do
        sleep 1
        while read mess; do
            sleep 1
            debug="$debug\n$mess"
        done
        if [[ $debug != "" ]]; then
            notify-send $debug
            debug=""
        fi
    done
}

Then you can run your script rather more simply and it is much clearer what is going on (I've switched to list so that it automatically quotes things for us):

set logger [ open [list |bash -c $notifySendScript] r+]
puts $logger "aaaaaaaa\n bbbbbbb"

Now that we've split these pieces apart, we can see that there are problems in your bash script. In particular, it reads from standard input repeatedly (because of the loop) until it gets EOF, which your code never sends as it doesn't close the pipeline. Even more fun, it puts that in a loop so that your code will continue to try to read from standard input repeatedly after EOF which is very unlikely to be what you want. There's other issues too (e.g., not reading from that read-write pipe) but I think that the big problem is that your code was a horrible one-liner when it didn't need to be, and that was concealing all the problems behind a wall of backslashes and unreadability. I strongly encourage trying to keep sub-scripts much neater (either as separate files or at least as braced sections such as I did above) as that stops you from going crazy.

Since you're trying to redirect substantial streams of output to it you need a smarter script:

set script {
    while read mess; do
        while read -t 1 tail && [ -n $tail ]; do
            mess=`printf '%s\n%s' $mess $tail`
        done
        if [ -n $mess ]; then
            notify-send $mess
        fi
    done
}

set pipeline [open [list |bash -c $script] "w"]
# This could also be an issue; want line buffering because that's what the
# bash script expects, and the default is full buffering (as for *all* channels)
fconfigure $pipeline -buffering line

# Demonstration
puts $pipeline "aaaaaaaa\nbbbbbbb"
after 5000
puts $pipeline "cccccccc\nddddddd"
after 5000
close $pipeline

The trick is that I'm using the -t option (for timeout) to read but only for the inner loop that is accumulating extra lines. Also, it treats blank lines as an excuse to send the message. Finally, the outer loop will terminate when it gets an EOF. That's important to allow you to shut the whole thing down correctly.

---

The other problem that was there (and which would have been more of a problem in testing than when deployed IMO) was that it was a line-oriented script in a pipeline with default buffering, which is full buffering. The fconfigure in the second part of my answer is how to fix that; it lets you tell Tcl to send on each line to the pipeline implementation as soon as it is ready instead of waiting for a full 4–8 kB of data.

Answer 2

You might want to simplify your setup and stay in Tcl? Redirecting stdout (which is used by tcltest under the hood) using Tcl's channel transforms is a convenient option; and has been covered here before.

There are many variations to this theme, but you might want to get started by:

Step 1: Define a Channel Interceptor

oo::class create TcltestNotifier {
    variable buffer
    method initialize {handle mode} {
        if {$mode ne "write"} {error "can't handle reading"}
        return {finalize initialize write}
    }
    method finalize {handle} {
        # NOOP
    }

    method write {handle bytes} {
        append buffer $bytes
        return $bytes
    }

    method report {} {
        # sanitize the tcltest report for the notifier
        set buffer [string map {\" \\\"} $buffer]
        # dispatch to notifier (Mac OS X, change to your needs/ OS)
        append cmd "display notification \"$buffer\"" " "
        append cmd "with title \"Tcltest notification\""
        exec osascript -e $cmd
    }
}

The above snippet was derived/ stolen bluntly from Donal.

Step 2: Register the interceptor with stdout around your tcltest suite

package req tcltest
namespace import ::tcltest::*

set tn [TcltestNotifier new]
chan push stdout $tn

test mytest-0.1 "Fails!" -body {
    BARF!;
} -result "?"

chan pop stdout
$tn report

Some remarks

• You can vary the granularity, request notification for each output line rather than the test report (then you have to dispatch to your notifier in the write method). But I doubt that this makes sense.
• The example is built to run on Mac OS X (osascript), you have to modify it to your *nix tooling.