ImportError: No module named google.protobuf

Question

I am following this guide (https://developers.google.com/protocol-buffers/docs/pythontutorial) and using the exact sample of addressbook.proto.

Post the content of compiler generated addressbook_pb2.py file as well. When I run the following simple program, there is error saying, cannot find google.protobuf, any ideas how to resolve this issue? Thanks.

BTW, using Python 2.7 on Mac OSX.

from addressbook_pb2 import Person

p = Person()
p.email = "abc"

print p.email

Here is the automated generated file addressbook_pb2.py,

# Generated by the protocol buffer compiler.  DO NOT EDIT!
# source: addressbook.proto

import sys
_b=sys.version_info[0]<3 and (lambda x:x) or (lambda x:x.encode('latin1'))
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from google.protobuf import reflection as _reflection
from google.protobuf import symbol_database as _symbol_database
from google.protobuf import descriptor_pb2
# @@protoc_insertion_point(imports)

_sym_db = _symbol_database.Default()




DESCRIPTOR = _descriptor.FileDescriptor(
  name='addressbook.proto',
  package='tutorial',
  syntax='proto2',
  serialized_pb=_b('\n\x11\x61\x64\x64ressbook.proto\x12\x08tutorial\"\xda\x01\n\x06Person\x12\x0c\n\x04name\x18\x01 \x02(\t\x12\n\n\x02id\x18\x02 \x02(\x05\x12\r\n\x05\x65mail\x18\x03 \x01(\t\x12+\n\x05phone\x18\x04 \x03(\x0b\x32\x1c.tutorial.Person.PhoneNumber\x1aM\n\x0bPhoneNumber\x12\x0e\n\x06number\x18\x01 \x02(\t\x12.\n\x04type\x18\x02 \x01(\x0e\x32\x1a.tutorial.Person.PhoneType:\x04HOME\"+\n\tPhoneType\x12\n\n\x06MOBILE\x10\x00\x12\x08\n\x04HOME\x10\x01\x12\x08\n\x04WORK\x10\x02\"/\n\x0b\x41\x64\x64ressBook\x12 \n\x06person\x18\x01 \x03(\x0b\x32\x10.tutorial.Person')
)
_sym_db.RegisterFileDescriptor(DESCRIPTOR)



_PERSON_PHONETYPE = _descriptor.EnumDescriptor(
  name='PhoneType',
  full_name='tutorial.Person.PhoneType',
  filename=None,
  file=DESCRIPTOR,
  values=[
    _descriptor.EnumValueDescriptor(
      name='MOBILE', index=0, number=0,
      options=None,
      type=None),
    _descriptor.EnumValueDescriptor(
      name='HOME', index=1, number=1,
      options=None,
      type=None),
    _descriptor.EnumValueDescriptor(
      name='WORK', index=2, number=2,
      options=None,
      type=None),
  ],
  containing_type=None,
  options=None,
  serialized_start=207,
  serialized_end=250,
)
_sym_db.RegisterEnumDescriptor(_PERSON_PHONETYPE)


_PERSON_PHONENUMBER = _descriptor.Descriptor(
  name='PhoneNumber',
  full_name='tutorial.Person.PhoneNumber',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  fields=[
    _descriptor.FieldDescriptor(
      name='number', full_name='tutorial.Person.PhoneNumber.number', index=0,
      number=1, type=9, cpp_type=9, label=2,
      has_default_value=False, default_value=_b("").decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='type', full_name='tutorial.Person.PhoneNumber.type', index=1,
      number=2, type=14, cpp_type=8, label=1,
      has_default_value=True, default_value=1,
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
  ],
  extensions=[
  ],
  nested_types=[],
  enum_types=[
  ],
  options=None,
  is_extendable=False,
  syntax='proto2',
  extension_ranges=[],
  oneofs=[
  ],
  serialized_start=128,
  serialized_end=205,
)

_PERSON = _descriptor.Descriptor(
  name='Person',
  full_name='tutorial.Person',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  fields=[
    _descriptor.FieldDescriptor(
      name='name', full_name='tutorial.Person.name', index=0,
      number=1, type=9, cpp_type=9, label=2,
      has_default_value=False, default_value=_b("").decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='id', full_name='tutorial.Person.id', index=1,
      number=2, type=5, cpp_type=1, label=2,
      has_default_value=False, default_value=0,
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='email', full_name='tutorial.Person.email', index=2,
      number=3, type=9, cpp_type=9, label=1,
      has_default_value=False, default_value=_b("").decode('utf-8'),
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
    _descriptor.FieldDescriptor(
      name='phone', full_name='tutorial.Person.phone', index=3,
      number=4, type=11, cpp_type=10, label=3,
      has_default_value=False, default_value=[],
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
  ],
  extensions=[
  ],
  nested_types=[_PERSON_PHONENUMBER, ],
  enum_types=[
    _PERSON_PHONETYPE,
  ],
  options=None,
  is_extendable=False,
  syntax='proto2',
  extension_ranges=[],
  oneofs=[
  ],
  serialized_start=32,
  serialized_end=250,
)


_ADDRESSBOOK = _descriptor.Descriptor(
  name='AddressBook',
  full_name='tutorial.AddressBook',
  filename=None,
  file=DESCRIPTOR,
  containing_type=None,
  fields=[
    _descriptor.FieldDescriptor(
      name='person', full_name='tutorial.AddressBook.person', index=0,
      number=1, type=11, cpp_type=10, label=3,
      has_default_value=False, default_value=[],
      message_type=None, enum_type=None, containing_type=None,
      is_extension=False, extension_scope=None,
      options=None),
  ],
  extensions=[
  ],
  nested_types=[],
  enum_types=[
  ],
  options=None,
  is_extendable=False,
  syntax='proto2',
  extension_ranges=[],
  oneofs=[
  ],
  serialized_start=252,
  serialized_end=299,
)

_PERSON_PHONENUMBER.fields_by_name['type'].enum_type = _PERSON_PHONETYPE
_PERSON_PHONENUMBER.containing_type = _PERSON
_PERSON.fields_by_name['phone'].message_type = _PERSON_PHONENUMBER
_PERSON_PHONETYPE.containing_type = _PERSON
_ADDRESSBOOK.fields_by_name['person'].message_type = _PERSON
DESCRIPTOR.message_types_by_name['Person'] = _PERSON
DESCRIPTOR.message_types_by_name['AddressBook'] = _ADDRESSBOOK

Person = _reflection.GeneratedProtocolMessageType('Person', (_message.Message,), dict(

  PhoneNumber = _reflection.GeneratedProtocolMessageType('PhoneNumber', (_message.Message,), dict(
    DESCRIPTOR = _PERSON_PHONENUMBER,
    __module__ = 'addressbook_pb2'
    # @@protoc_insertion_point(class_scope:tutorial.Person.PhoneNumber)
    ))
  ,
  DESCRIPTOR = _PERSON,
  __module__ = 'addressbook_pb2'
  # @@protoc_insertion_point(class_scope:tutorial.Person)
  ))
_sym_db.RegisterMessage(Person)
_sym_db.RegisterMessage(Person.PhoneNumber)

AddressBook = _reflection.GeneratedProtocolMessageType('AddressBook', (_message.Message,), dict(
  DESCRIPTOR = _ADDRESSBOOK,
  __module__ = 'addressbook_pb2'
  # @@protoc_insertion_point(class_scope:tutorial.AddressBook)
  ))
_sym_db.RegisterMessage(AddressBook)


# @@protoc_insertion_point(module_scope)

Edit 1,

Tried pip install protobuf, met with the following error,

Requirement already satisfied (use --upgrade to upgrade): protobuf in /Users/foo/miniconda2/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /Users/foo/miniconda2/lib/python2.7/site-packages/six-1.10.0-py2.7.egg (from protobuf)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /Users/foo/miniconda2/lib/python2.7/site-packages (from protobuf)

Here is the output of python version,

python -V
Python 2.7.11 :: Continuum Analytics, Inc.

**Edit 2 **

Post exact error message,

Traceback (most recent call last):
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 1531, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/pydevd.py", line 938, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/Users/foo/personal/featureExtraction/protobuf_test.py", line 1, in <module>
    from addressbook_pb2 import Person
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_bundle/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
  File "/Users/foo/personal/featureExtraction/addressbook_pb2.py", line 6, in <module>
    from google.protobuf import descriptor as _descriptor
  File "/Applications/PyCharm CE.app/Contents/helpers/pydev/_pydev_bundle/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
ImportError: No module named google.protobuf

Edit 3,

error message when import google,

Edit 4,

Output of which pip,

which pip
/Users/foo/miniconda2/bin/pip

Output of sys.executable,

/Users/foo/anaconda/bin/python

Edit 5,

foo-mn1:featureExtraction foo$ sudo /Users/foo/miniconda2/bin/pip install protobuf
Password:
The directory '/Users/foo/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/foo/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): protobuf in /Users/foo/miniconda2/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): six>=1.9 in /Users/foo/miniconda2/lib/python2.7/site-packages/six-1.10.0-py2.7.egg (from protobuf)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /Users/foo/miniconda2/lib/python2.7/site-packages (from protobuf)
foo-mn1:featureExtraction foo$ sudo /Users/foo/miniconda2/bin/pip install google
The directory '/Users/foo/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/foo/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): google in /Users/foo/miniconda2/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): beautifulsoup4 in /Users/foo/miniconda2/lib/python2.7/site-packages (from google)

Answer 1

Had the same issue and I resolved it by using :

conda install protobuf

Answer 2

I encountered the same situation. And I find out it is because the pip should be updated. It may be the same reason for your problem.

Answer 3

The reason of this would be mostly below evil command pip install google. I was facing issue for google-cloud the same steps are true for protobuf also as protobuf also get installed in google > protobuf namespace.

If you have executed this command like me then you are at correct place actually google package is not owned by google which can be confirmed by command pip show google output of which should be somewhat as below

 Name: google
 Version: 1.9.3
 Summary: Python bindings to the Google search engine.
 Home-page: http://breakingcode.wordpress.com/
 Author: Mario Vilas
 Author-email: [e-mail]
 License: UNKNOWN
 Location: <Path where this package is installed>
 Requires: beautifulsoup4

So because of above google namespace is reserved for this package and coincidentally google-cloud also expects namespace google > cloud and it results in collision of namespace of these two packages.

See in below screenshot namespace of google-protobuf as google > protobuf

Solution :- Unofficial google package need to be uninstalled which can be done by using pip uninstall google after this you can reinstall google-cloud using pip install google-cloud or protobuf using pip install protobuf

FootNotes :- Assuming you have installed unofficial google package by mistake and you don't actually need it along with google-cloud package. If you need both unofficial google and google-cloud above solution won't work.

Let me know if this solves your particular issue.

Answer 4

When pip tells you that you already have protobuf, but PyCharm (or other) tells you that you don't have it, it means that pip and PyCharm are using a different Python interpreter. This is a very common issue, especially on a Mac, with no standard Python package management.

The best way to completely eliminate such issues is using a virtualenv per Python project, which is essentially a directory of Python packages and environment variable settings to isolate the Python environment of the project from everything else.

Create a virtualenv for your project like this:

cd project
virtualenv --distribute virtualenv -p /path/to/python/executable

This creates a directory called virtualenv inside your project. (Make sure to configure your VCS (for example Git) to ignore this directory.)

To install packages in this virtualenv, you need to activate the environment variable settings:

. virtualenv/bin/activate

Verify that pip will use the right Python executable inside the virtualenv, by running pip -V. It should tell you the Python library path used, which should be inside the virtualenv.

Now you can use pip to install protobuf as you did.

And finally, you need to make PyCharm use this virtualenv instead of the system libraries. Somewhere in the project settings you can configure an interpreter for the project, select the Python executable inside the virtualenv.

Answer 5

On Mac OS - Installing tensorflow 1.3 - it will automatically remove other protobuf installs and install protobuf 3.4. However, this does not work and neither does installing or downgrading to any other protobuf version.

However I found a solution. Not sure why this works - but on Mac OS this solved it.

pip install google

Answer 6

You should run:

pip install protobuf

That will install Google protobuf and after that you can run that Python script.

As per this link.