How to fix Google-cloud-sdk 156.0.0 “There are too

Question

I just installed Go 1.6.4 and google-cloud-sdk 1.56.0.0 on osX.

When I attempt to run the local dev_server I get the following warning.

/Users/Bryan/go/src/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/mtime_file_watcher.py:156: UserWarning: There are too many files in your application for changes in all of them to be monitored. You may have to restart the development server to see some changes to your files.

How do I fix this?

Here is the project, there are only 4 files:

$ ls
total 32
drwxrwxrwx   6 Bryan  staff   204 May 23 15:57 .
drwxr-xr-x  10 Bryan  staff   340 May 25 18:39 ..
-rwxrwxrwx@  1 Bryan  staff    69 Mar 22 09:40 app.yaml
-rw-r--r--   1 Bryan  staff   554 Mar 28 09:26 index.yaml
-rwxrwxrwx@  1 Bryan  staff  3015 May 25 18:36 skincare.go
-rw-r--r--   1 Bryan  staff   870 May  3 09:18 skincare_test.go

I'm also having a problem that "go get" attempts to compile the code instead of downloading the remote packages "google.golang.org/appengine".

$ go get

# import "google.golang.org/appengine" is ignored and "go get" yields this error:
./skincare.go:31: undefined: appengine.Context

EDIT additional info: I attempted to to force the update with the -u flag:
$ go get -u

package skincarereview: directory "/Users/Bryan/go/src/skincarereview" is not using a known version control system

This is the error output when I attempt to run the local server:

$  /Users/Bryan/go/src/google-cloud-sdk/bin/dev_appserver.py app.yaml
INFO     2017-05-30 14:27:31,722 devappserver2.py:692] Skipping SDK update check.
INFO     2017-05-30 14:27:31,785 api_server.py:272] Starting API server at: http://localhost:60703
INFO     2017-05-30 14:27:31,789 dispatcher.py:205] Starting module "default" running at: http://localhost:8080
INFO     2017-05-30 14:27:31,793 admin_server.py:116] Starting admin server at: http://localhost:8000
/Users/Bryan/go/src/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/mtime_file_watcher.py:156: UserWarning: There are too many files in your application for changes in all of them to be monitored. You may have to restart the development server to see some changes to your files.
  'There are too many files in your application for '
ERROR    2017-05-30 14:27:36,123 go_runtime.py:182] Failed to build Go application: (Executed command: /Users/Bryan/go/src/google-cloud-sdk/platform/google_appengine/goroot/bin/go-app-builder -app_base /Users/Bryan/go/src/skincarereview -arch 6 -dynamic -goroot /Users/Bryan/go/src/google-cloud-sdk/platform/google_appengine/goroot -gopath /Users/Bryan/go -nobuild_files ^^$ -incremental_rebuild -unsafe -binary_name _go_app -extra_imports appengine_internal/init -work_dir /var/folders/mw/0y88j8_54bjc93d_lg3120qw0000gp/T/tmpEOVMVuappengine-go-bin -gcflags -I,/Users/Bryan/go/src/google-cloud-sdk/platform/google_appengine/goroot/pkg/darwin_amd64_appengine -ldflags -L,/Users/Bryan/go/src/google-cloud-sdk/platform/google_appengine/goroot/pkg/darwin_amd64_appengine skincare.go skincare_test.go)
/var/folders/mw/0y88j8_54bjc93d_lg3120qw0000gp/T/tmpEOVMVuappengine-go-bin/skincare.go:31: undefined: "google.golang.org/appengine".Context

2017/05/30 10:27:36 go-app-builder: build timing: 0×skip (5ms total), 11×compile (2.128s total), 0×link (0 total)
2017/05/30 10:27:36 go-app-builder: failed running compile: exit status 2

Answer 1

I only used an older version of appengine sdk, but I believe below is still true.

The main issue is that you have google-cloud-sdk inside your GOPATH. This should not be necessary. I have it installed in my path. But even if not installed it should be resolving it's location relative to the executable or script you are running and should find all required sdk files properly. At the same time SDK is roughly 7500 files and you only need to have more than 10000 (approx) in your GOPATH to get that "to many files" warning. So, if you have another (e.g. older) copy of SDK or other big project inside same GOPATH, you are pretty much guaranteed to exceed the threshold. Just locate /Users/Bryan/go in Finder and use "Get Info" to count files.

So, either move appengine SDK outside of your GOPATH. Or make a new folder, export it as your new GOPATH and move your project files (but not SDK) to that new folder. This should fix the "too many files issue".

The other issue is that your app actually fails to build. And as you have already figured out yourself the (immediate) issue is missing appengine package.

But the correct way of getting google.golang.org/appengine is as follows: go get google.golang.org/appengine

Though keep in mind that GOPATH must be set correctly before running the above command. It will fetch google.golang.org/appengine and all it's dependencies.

Once you have retrieved all required packages (and fixed errors, if any) it should compile and run without issues.

Answer 2

The corresponding error occurs when mtime_file_watcher.py tries to monitor all files under "GOPATH" but has too many files, so use the skip_files option of app.yaml to check vendor, node_modules and arbitrary number of files Can be avoided by excluding many projects to be monitored.

https://cloud.google.com/appengine/docs/standard/python/config/appref#skip_files

skip_files:
- .*node_modules
- .*vendor
- .*project-name-having-too-many-files

Answer 3

With my current sdk version (Google Cloud SDK 171.0.0) dev_appserver.py has one option named enable_watching_go_path.

Setting it to False removed the warning.

The command I use is: dev_appserver.py --enable_watching_go_path=False server/app.yml

Answer 4

I'm using just the App Engine SDK for Go (1.9.54), not the Cloud SDK, but have the same problem [Goapp serve - Warning: There are too many files.

I solved it by moving large packages and repositories to vendor/ dir.