Pip regular expression search

Question

I need to find all packages on PyPI that match a particular regular expression:

^django-.*?admin.*$

Basically, the package name should start with django- and have admin word after. For example, the following packages should match:

django-redis-admin
django-admin-ckeditor 
django-admintools-bootstrap

I can do pip search django-, but there is a huge amount of packages that I'm not interested in.

Does pip provide a way to find packages by a regex? Or, should I just pipe the results of django- to grep to filter out irrelevant packages?

Also, probably an "intersection" of pip search django- and pip search admin would help too.

Answer 1

aleckxe, I believe this is the one-liner you are looking for.

pip search django | grep -P "^django-(?=[-\w]*?admin)[-\w]+"

As suggested by chromate in the comment below, you could easily pipe to sort for a sorted list if you wished.

pip search django | grep -P "^django-(?=[-\w]*?admin)[-\w]+" | sort

Let me know if you'd like any tweaks.

Explanation:

After the pipe | which redirects the output of the pip command to <stdin> for the grep command, we enter grep in Perl mode -P. This is necessary, otherwise we would not be allowed to use a lookahead.

We anchor the pattern at the beginning of the string with ^ and immediately match django- as a literal. We then assert (lookahead) that at this position we would be able to match any number of dashes or word characters (which include digits and underscores), followed by the literal string admin.

Having made this assertion (which is a form of validation), we now match as many dashes and word characters as we can, which should take us to the end of the module name.

There are several ways of expressing this and for this simple pattern the variations are largely a matter of preference or mood.

If you ever wanted to change this to match django- patterns that contain someword, just replace admin with someword.

The output:

    django-smoke-admin        - django-smoke-admin tests that all admin pages for all registered models responds correctly (HTTP 200).
    django-adminskin          - UNKNOWN
    django-admin-exporter     - Simple admin actions to download/export selected items in CSV, JSON, XML, etc.
    django-treeadmin-fork-alt-storage - Tree UI for mptt-managed models, extracted from FeinCMS. This is a fork with support for alternative storage engines
    django-relatedadminwidget - Get edit and delete links in your django admin. A utility class to let your model admins inherit from.
    django-admin-langswitch   - Adds easy language switch in admin
    django-authy-admin        - A drop in replacement for django's default admin site that provides two-factor authentication via authy's REST API.
    django-frontendadmin      - A a set of templatetags to allow an easy and unobstrusive way to edit model-data in the frontend of your page.
    django-admin-app-names-singleton - Django admin enhancer
    django-mobileadmin        - The Django admin interface for mobile devices.

(The list goes on.)

By the way, looking at the pip search documentation, I don't see a way of doing this without the pipe.

Answer 2

One approach is the above mentioned method that pipes the pip search results to grep. I recommend to use this if you want to use regex search once or rarely.

However if you need this feature regularly you should check out yip which is a package I wrote to accomplish regex search alongside other useful additions that pip's search can't do like displaying extra info(size, upload time, homepage or license) or colorizing the output for readability.

Answer 3

Seems PyPI search method doesn't support regexp. Pip use xmlrpc for method calls but I coudn't find any info for search(or any other) method not in docs or even with methodSignature or methodHelp for PyPI XML-RPC server. You can create you own script as alternative for grep(But it isn't a lot of sense). Fast example:

from xmlrpclib import ServerProxy
import re

URL = 'https://pypi.python.org/pypi'
TEST_RE = r"^django-.*?admin.*$"
TEST = 'django-'

def main():
    pypi = ServerProxy(URL)
    res = pypi.search({'name': TEST})

    for r in res:
        m = re.match(TEST_RE, r['name'])
        if m:
            print(m.group(0))


if __name__ == '__main__':
    main()