Update all pip packages that don't come from c

Question

On my windows 7 system I try to install as many packages using conda. These are easy to update with

conda update all

Unfortunately some packages don't appear in conda but are available through pip and so for those I install them using pip. Updating all pip packages on windows seems more difficult but

for /F "delims===" %i in ('pip freeze -l') do pip install -U %i

is one way I found.

However, this attempts to update all packages, even those installed by conda I believe.

Is there some way to update only those packages installed by pip?

Answer 1

This is tricky as Pip packages are different than conda packages. Anaconda adds pip as an install choice and puts them in the environment but it does not manage them. Pip is still without an easy command to upgrade all but some suggestions are as you tried and this is another:

pip freeze --local | grep -v '^\-e' | cut -d = -f 1  | xargs pip install -U

Answer 2

Here is my attempt at a shell script that will parse the output of conda env export and upgrade any PIP packages:

#!/bin/sh

###############################################################################
# Script to scan an Anaconda environment and upgrade any PIP packages.
#
# Usage:
# $ ./upgrade_pip_packages.sh 
# or explicitly give it an environment file to parse:
# $ ./upgrade_pip_packages.sh <environment.yml file>
#
# Author: Marijn van Vliet <[e-mail]>
#
# Version: 1.0 (29-09-2017)
#   - Initial version of the script.

# Check for optional command line argument
if [ "$#" = 0 ]
then
    ENV_OUTPUT=`conda env export`
elif [ "$#" = 1 ]
then
    ENV_OUTPUT=`cat $1`
else
    echo "Usage: $0 [environment file]" >&2
    exit 1
fi

PIP=0  # Whether we are parsing PIP packages
IFS=$'\n'  # Split on newlines
PIP_PACKAGES=""  # PIP packages found thus far

# Loop over the output of "conda env export"
for line in $ENV_OUTPUT
do
    # Don't do anything until we get to the packages installed by PIP
    if [ "$line" = "- pip:" ]
    then
        PIP=1  # From this point, start doing things.
    elif [[ "$line" = prefix:* ]]
    then
        PIP=0  # End of PIP package list. Stop doing things.
    elif [ $PIP = 1 ]
    then
        # Packages are listed as "   - name==version==python_version"
        # This is a regular expression that matches only the name and 
        # strips quotes in git URLs:
        REGEXP='^  - "\?\([^="]*\)"\?.*$'

        # Find PIP package name (or git URL)
        PIP_PACKAGES="$PIP_PACKAGES `echo "$line" | sed -n "s/$REGEXP/\1/p"`"
    fi
done

# From now on, split on spaces
IFS=' '

echo "The following packages are marked for upgrade using PIP:"
echo
for package in $PIP_PACKAGES
do
    echo " - $package"
done
echo

read -r -p "Shall we proceed with the upgrade? [y/N] " response
case "$response" in
    [yY][eE][sS]|[yY]) 
        # Upgrade each package
        for package in $PIP_PACKAGES
        do
            pip install --upgrade $package
        done
        ;;
    *)
        echo "Aborting"
        ;;
esac

Answer 3

This is an another simple script using output of conda list which contains list of pip packages.

conda list | grep "<pip>" | cut -d " " -f 1 | xargs pip install --upgrade