Way to have compiled python files in a separate fo

Question

Is it possible to have Python save the .pyc files to a separate folder location that is in sys.path?

/code
    foo.py
    foo.pyc
    bar.py
    bar.pyc

To:

/code
   foo.py
   bar.py
/code_compiled
   foo.pyc
   bar.pyc

I would like this because I feel it'd be more organized. Thanks for any help you can give me.

Answer 1

I agree, distributing your code as an egg is a great way to keep it organized. What could be more organized than a single-file containing all of the code and meta-data you would ever need. Changing the way the bytecode compiler works is only going to cause confusion.

If you really do not like the location of those pyc files, an alternative is to run from a read-only folder. Since python will not be able to write, no pyc files ever get made. The hit you take is that every python file will have to be re-compiled as soon as it is loaded, regardless of whether you have changed it or not. That means your start-up time will be a lot worse.

Answer 2

In the dark and ancient days of 2003, PEP 304 came forth to challenge this problem. Its patch was found wanting. Environment variable platform dependencies and version skews ripped it to shreds and left its bits scattered across the wastelands.

After years of suffering, a new challenger rose in the last days of 2009. Barry Warsaw summoned PEP 3147 and sent it to do battle, wielding a simple weapon with skill. The PEP crushed the cluttering PYC files, silenced the waring Unladen Swallow and CPython interpreter each trying to argue its PYC file should be triumphant, and allowed Python to rest easy with its dead ghosts occasionally running in the dead of night. PEP 3147 was found worthy by the dictator and was knighted into the official roles in the days of 3.2.

As of 3.2, Python stores a module's PYC files in __pycache__ under the module's directory. Each PYC file contains the name and version of the interpreter, e.g., __pycache__/foo.cpython-33.pyc. You might also have a __pycache__/foo.cpython-32.pyc compiled by an earlier version of Python. The right magic happens: the correct one is used and recompiled if out of sync with the source code. At runtime, look at the module's mymodule.__cached__ for the pyc filename and parse it with imp.get_tag(). See the What's New section for more information.

TL;DR - Just works in Python 3.2 and above. Poor hacks substitute for versions before that.

Answer 3

If you're willing to sacrifice bytecode generation altogether for it, there's a command line flag:

python -B file_that_imports_others.py

Can be put into IDE's build/run preferences

Answer 4

There is ongoing pep that will enable building bytecode to magic directory.

Basically all python files will be compiled to directory __pythoncache__.

Answer 5

I disagree. The reasons are wrong or at least not well formulated; but the direction is valid. There are good reasons for being able to segregate source code from compiled objects. Here are a few of them (all of them I have run into at one point or another):

• embedded device reading off a ROM, but able to use an in memory filesystem on RAM.
• multi-os dev environment means sharing (with samba/nfs/whatever) my working directory and building on multiple platforms.
• commercial company wishes to only distribute pyc to protect the IP
• easily run test suite for multiple versions of python using the same working directory
• more easily clean up transitional files (rm -rf $OBJECT_DIR as opposed to find . -name '*.pyc' -exec rm -f {} \;)

There are workarounds for all these problems, BUT they are mostly workarounds NOT solutions. The proper solution in most of these cases would be for the software to accept an alternative location for storing and lookup of these transitional files.

Answer 6

There is PEP 304: Controlling Generation of Bytecode Files. Its status is Withdrawn and corresponding patch rejected. Therefore there might be no direct way to do it.

If you don't need source code then you may just delete *.py files. *.pyc files can be used as is or packed in an egg.