Minikube weird behavior under VPN

Question

I was just curious if there are known workarounds for the Minikube Vpn issue apart from clean wipe of the minikube from the machine and reinstallation. Minikube works just fine when it is connected to either office network or home network but goes dead when connected to a vpn. When I do a kubectl get pods gives an i/o timeout. I did read couple of git issues in their Issues section but none of the solutions seemed comprehensive.

Using Mac OS Minikube V0.25.Any suggestion is truly appreciated!

Answer 1

I had this problem when the Cisco VPN was connected to our business network. I was able to stop the VPN, then start the minikube and could run kubectl version with the VPN stopped. I then started the VPN with minikube running and reissued the kubectl version command and it worked without timeout. I have not tested beyond this however.

minikube version: v0.28.2
kubectl version
Client Version: version.Info{Major:"1", Minor:"11", 
GitVersion:"v1.11.2", 
GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", 
GitTreeState:"clean", BuildDate:"2018-08-08T16:31:10Z", 
GoVersion:"go1.10.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10",     
GitVersion:"v1.10.0", 
GitCommit:"fc32d2f3698e36b93322a3465f63a14e9f0eaead", 
GitTreeState:"clean", BuildDate:"2018-03-26T16:44:10Z", 
GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

I had also upgraded VirtualBox to Version 5.2.18 r124319 (Qt5.6.3) prior to performing the reinstallation of minikube with the VPN off. However, this may not have had any affect as I was on newer versions of VirtualBox than others had recommending to upgrade to.

Answer 2

I suggest not using VPN with Minikube. The work-arounds do not work for me always or when I upgrade my VPN software or reboot my laptop.

I observed that VPN drops ingress traffic on unknown interface (with unknown IP address, port and DNS name) from Minikube into the laptop for security reasons.

Answer 3

There are some issues reported around VPN for minikube.

In particular, issue 2344 shows the exact error message:

$kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.0", GitCommit:"925c127ec6b946659ad0fd596fa959be43f0cc05", GitTreeState:"clean", BuildDate:"2017-12-15T21:07:38Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
Unable to connect to the server: Gateway Time-out

Not sure any good solution to correct routing but there is remedy for this to disable TLS verification.

From issue 2325:

On a side node, on Windows 10 1709 update, Hyper-V comes with a build in "Default Switch".
If use it in minikube start, then everything works in minikube. All Host VPN (I have 2, Cisco Anyconnect and ArraySSL VPN) works inside minikube.

Some networking settings are suggested in "vpn or proxy for easier local development (issue 38)".

Issue 1413 suggests:

In order to remedy VPN issues, it is helpful to specify the hostonly nic used by minikube.
This way other VMs on that NIC will have no issues talking to each other even when Cisco wrecks the local network. Docker Machine tdocker-machine create) exposes this property:

--virtualbox-hostonly-nictype "82540EM"   

This is not yet available with minikube though.

As a workaround, specifying IP addresses in the same CIDR as minikube will set put the VM on the same host only network.
For example, applying an ip address 192.168.99.XX will use the same host only network that minikube uses with its IP address 192.168.99.10X.