How to authenticate a private Go Module using go 1

问题:

I've been updating my entire go gae standard project to use go 1.11's modules.

Main directory structure

app.yaml
app.go
go.mod
go.sum

app.go

package main

import "bitbucket.org/myPrivateRepo"

func main() {
    myImportantModule.Run()
}

go.mod

module myProject

require bitbucket.org/myPrivateRepo v0.0.1

The Error

If I try to gcloud app deploy:

ERROR: (gcloud.app.deploy) Error Response: [9] Cloud build <GUI> 
status: FAILURE.
Build error details: go: bitbucket.org/myPrivateRepo@v0.0.1: 
https://api.bitbucket.org/2.0/repositories/myPrivateRepo?fields=scm: 
403 Forbidden

(Note: obviously the repo I'm using has a real name).

So can I do it this way? I'll admit to not fully understanding the migration documentation, particularly when it talked about "Moving files to your GOPATH". https://cloud.google.com/appengine/docs/standard/go111/go-differences

I mean, I thought one of the benefits of the new module system is that you don't need everything under the go path. When I read https://github.com/golang/go/wiki/Modules for example, it very early on says "Create a directory outside of your GOPATH:"

So, to be clear, right now all of my code is outside the go path, but everything builds locally just fine.

I think it all works becausego automatically downloads and caches things within the go path when I run go mod tidy / go build etc.

Yet it fails when I try to gcloud app deploy. How would the google cloud build system ever have access to my private repositories anyway? I'm obviously missing something important. I also read you are not supposed to combine vendoring with the new module system so that can't be it.

I will be very happy if this works, as using DEP forced me to use goapp deploy very awkwardly.

Thanks!

回答1:

My solution:

Instead of dealing with credentials, I'm using go's module replace functionality to point GAE to use my local code. This is working well.

Directory structure:

myService/
    src/
        service.go  // has a run() function to set up routers etc.
        go.mod      // depends on my private module in bitbucket and other things
        …           // other source files
    build/
        gae/
            src/        // simlink to ../../src
            modules/    // git ignored, I clone or copy my modules in build scripts.
            app.go  // see below…
            go.mod  // has main() which calls service.run() and appEngine.Main()
            app.yaml

Method

I use git module replace so that GAE uses my local code. Before building I parse myService/src/go.mod to find the correct version of my private module, then I clone it into the modules folder. I also made an option to copy wip module source code for debugging locally without committing to my module repositories.

go.mod from gae directory:

module myServiceGAE

require (
    bitbucket.org/me/myService v0.0.0
    google.golang.org/appengine v1.4.0
)

replace bitbucket.org/me/myService => ./src

replace bitbucket.org/me/myModule => ./modules/utils

Pros

The package under myService has no references or knowledge of GAE, so I can easily build it into a docker etc. I think parsing the service go.mod files would be like creating my own dependency manager, defeating the benefits of go modules.

Cons

If I had a private module which depended on another private module, I think things would get too complicated.

回答2:

Set git credentials before deploying:

git config credential.helper '!f() { sleep 1; echo "username=${GIT_USER}\npassword=${GIT_PASSWORD}"; }; f'

export GIT_USER=put_git_user_here
export GIT_PASSWORD=put_git_password_here

gcloud app deploy