Integration testing and debugging many microservices can be painful. Often, I need to debug a service on a staging environment. This article shows how we use Docker for Mac with Weave (an overlay network) to connect our local machine to our remote staging environments.
In my workflow, I usually create a WIP git commit, push it to staging, and try to debug with the Ubuntu server’s limited tools. I could set up a bunch of SSH tunnels to connect to all the remote services, but our stack changes too frequently and finding IP addresses for each service is a pain. My dev machine is a Mac, so most of the tools we use locally don’t work on Linux.
I wanted something better that would speed up my dev and debugging flow. The first thing I tested with the Docker for Mac beta was Weave integration. Weave creates an overlay network to connect containers across multiple hosts together, which is very useful when distributing containers across a Swarm cluster.
Remote docker-engine. I used the latest engine (v1.10.3).
Weave binary on local and remote machine:
sudo curl -L git.io/weave -o /usr/local/bin/weave
sudo chmod a+x /usr/local/bin/weave
Docker for Mac installed locally (similar steps can be followed if connecting from a Linux machine).
Note: You don’t need to have Weave running beforehand; we can dynamically attach the Weave network to any existing container.
Step 1: Set up Weave network routers and proxy
The first step is to set up our Weave overlay network so that we can connect to it from our local machine.
Launch Weave router: 
weave launch-router --password <password>
This will pull the latest Weave images and start the Weave router container. The password field is optional but recommended to prevent others from accessing your network while you are debugging.
Launch Weave proxy:
This proxy will automatically attach your IP address. More info here.
Test network connectivity of Weave ports:
nc -z <remote_host_ip_address> 6783
Ensure your firewalls and security groups allow Weave’s control port (TCP 6783) and data ports (UDP 6783/6784) on the remote machine to be accessible from your local machine. If you need to create an intermediate link, refer to footnote #2.
Launch Weave router on your local machine:
weave launch-router --password <password> <remote_host_ip_address>
If you set a password on the remote machine, ensure the same password is set here.
remote_host_ip_addressshould be the IP address of the remote machine you set up above.
Verify Weave connection:
weave status connections
Launch Weave proxy on the remote machine:
If we were connecting from a Linux machine, we could use
weave launch-proxy. However, we need to change this for Docker on Mac. The workaround is to tell Weave to listen on an HTTP interface:
WEAVEPROXY_DOCKER_ARGS="-p 9999:9999" weave launch-proxy -H tcp://0.0.0.0:9999
WEAVEPROXY_DOCKER_ARGStells Weave to publish to port 9999 so the host has access to it, and the
-H tcp://0.0.0.0:9999tells Weave to listen on port 9999. If port 9999 is used on your machine, you can change it to any port, but ensure you replace 9999 with your port throughout the following steps.
Step 2: Launch containers into Weave network
Now that we have the Weave routers and proxy set up, we need some containers to talk to. In this example I will use a MongoDB container.
Configure Docker client to go through the Weave proxy.
eval $(weave env)
That command will set the
DOCKER_HOSTenvironment variable to point to the Weave proxy. Note: You’ll have use the same terminal session, or that variable will get unset.
Run MongoDB container
docker run -d --name=remote-mongo mongo
--namevalue is important here. It will be the hostname used to connect to this container. Notice that I did not expose any ports. Exposing ports to the host is not needed since all traffic to this container will be going through Weave. If you want to connect to an existing container, refer to footnote #3.
Configure our Docker client to go through the Weave proxy.
We also need to setup the Mac Docker client to point to our proxy. This is a bit tricky since we did a custom proxy configuration.
ping -c1 docker.local
docker.local_ip_addressshould be the IP address returned from the ping command. Same note from above — you have to say in the same terminal session or that variable will get unset.
Create a port forward container
docker run -e REMOTE_HOST=remote-mongo -e REMOTE_PORT=27017 -p 27017:80 -d djfaze/port-forward
In order to gain access to the remote machine, we need to set up a simple port forwarding container on our local machine. I used this image, which requires the following options to be passed to the
remote_container_nameis set to the hostname or Weave IP you want to forward to. Since we started the remote container with
--name=remote-mongo, Weave DNS will resolve that name to the Weave IP address.
remote_container_portis set to the port you want to connect to.
host_portis set to the port on your local machine you want to map to the remote container.
The above example maps the local machine’s port 27017 to the remote container’s MongoDB port 27017.
Step 3: Connect and Play
After everything is set up,
docker.local:27017 will point to the remote MongoDB container. I can now run my sample app server locally with all my local tools and simply configure my application to
docker.local:27017. Integrating this with Runnable allows me to develop on my local machine with my entire isolated production stack without affecting anyone else, speeding up my feature development and bug bashing.
Using this technique, you can easily connect your local development machine to a staging environment and debug services just like you’d debug code locally.
1: Weave’s default subnet is
10.32.0.0/12. If your subnet overlaps, you will run into some issues. Add
--ipalloc-default-subnet <subnet> and
--ipalloc-range <subnet> to the Weave launch command where
subnet is a CIDR for a non-overlapping network. ↩
2: If your application is on a private network, you can create an intermediate link with a server on a public network that can talk to your private network. On the intermittent server, run
weave launch --password=<your password> <ip_addr_of_private_host> and on your local, run
weave launch --password=<your password> <ip_addr_of_intermediate_server>. ↩
3: If you already have a container running and would like to attach to it, use
weave attach <container_ID>. Note that DNS will not work. You will have to use the IP address returned from the
attach command to connect. When using the port forwarder, use
-e REMOTE_HOST=<weave_ip> where
weave ip is the address returned from the
weave_attach command. ↩