How to redirect stderr in Python? Via Python C API

Question

This is a combination of my two recent questions:
[1] Python instance method in C
[2] How to redirect stderr in Python?

I would like to log the output of both stdout and stderr from a python script.

The thing I want to ask is, to create a new type according to [1] seems fairly complicated. Does it simplifies the things if there was no need to expose the new type to Python, i.e. it would only exist in C?

I mean, when Python prints something it goes to "Objects/fileobject.c" and there in "PyFile_WriteObject" it check whether it is possible to write to its argument:

writer = PyObject_GetAttrString(f, "write");
if (writer == NULL)
...

Also, it is possible to get stdout and stderr like this:

PyObject* out = PySys_GetObject("stdout");
PyObject* err = PySys_GetObject("stderr");

My question is then, is it somehow possible to construct necessary PyObject which satisfies the above 'PyObject_GetAttrString(f, "write")' and is callable so I can write:

PySys_SetObject("stdout", <my writer object / class / type / ?>);

http://docs.python.org/c-api/sys.html?highlight=pysys_setobject#PySys_SetObject

This way, there would be no need to expose the new "writer type" to the rest of Python script so I thought it might be a bit simpler to write the code...?

Answer 1

Just make a module object (you're doing that anyway, if you're using the C API!-) and make it have a suitable write function -- that module object will be suitable as the second argument to PySys_SetObject.

In my answer to your other question I pointed you to xxmodule.c, an example file in Python's C sources, which is a module with a lot of examples including types and functions of various kinds -- you can work from there even if (mysteriously to me) you consider the "make a new type" part too difficult;-).

Edit: here's a trivial working example (aview.py):

#include "Python.h"
#include <stdio.h>

static PyObject *
aview_write(PyObject *self, PyObject *args)
{
    const char *what;
    if (!PyArg_ParseTuple(args, "s", &what))
        return NULL;
    printf("==%s==", what);
    return Py_BuildValue("");
}

static PyMethodDef a_methods[] = {
    {"write", aview_write, METH_VARARGS, "Write something."},
    {NULL, NULL, 0, NULL}
};

PyMODINIT_FUNC
initaview(void)
{
    PyObject *m = Py_InitModule("aview", a_methods);
    if (m == NULL) return;
    PySys_SetObject("stdout", m);
}

Once this aview module is properly installed:

$ python
Python 2.5.4 (r254:67917, Dec 23 2008, 14:57:27) 
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import aview
>>> print 'ciao'
==ciao====
==>>> 

...any string emitted to standard output is written with == signs around it (and this print calls .write twice: with 'ciao', and then again with a newline).

Answer 2

Based on Alex's answer, here is fully working C code, without the Python "import aview", in Python 3 (so no Py_InitModule), and with stderr redirection :

#include <functional>
#include <iostream>
#include <string>
#include <Python.h>


PyObject* aview_write(PyObject* self, PyObject* args)
{
    const char *what;
    if (!PyArg_ParseTuple(args, "s", &what))
        return NULL;
    printf("==%s==", what);
    return Py_BuildValue("");
}


PyObject* aview_flush(PyObject* self, PyObject* args)
{
    return Py_BuildValue("");
}


PyMethodDef aview_methods[] =
{
    {"write", aview_write, METH_VARARGS, "doc for write"},
    {"flush", aview_flush, METH_VARARGS, "doc for flush"},
    {0, 0, 0, 0} // sentinel
};


PyModuleDef aview_module =
{
    PyModuleDef_HEAD_INIT, // PyModuleDef_Base m_base;
    "aview",               // const char* m_name;
    "doc for aview",       // const char* m_doc;
    -1,                    // Py_ssize_t m_size;
    aview_methods,        // PyMethodDef *m_methods
    //  inquiry m_reload;  traverseproc m_traverse;  inquiry m_clear;  freefunc m_free;
};

PyMODINIT_FUNC PyInit_aview(void) 
{
    PyObject* m = PyModule_Create(&aview_module);
    PySys_SetObject("stdout", m);
    PySys_SetObject("stderr", m);
    return m;
}


int main()
{
    PyImport_AppendInittab("aview", PyInit_aview);
    Py_Initialize();
    PyImport_ImportModule("aview");

    PyRun_SimpleString("print(\'hello to buffer\')");
    PyRun_SimpleString("make a SyntaxException in stderr");

    Py_Finalize();

    return 0;

}

Note, though, that if you plan to have several distinct interpreters, this won't be enough, because aview_write won't be able to know which buffer to append into. You'll need something like that.

Here is an awesome reference on how to add new modules and types, btw.