Notes From The Homelab: Configuring A Local Docker CLI To Work With Remote Podman

With the school year activities winding down and brief lull before I kick off a number of house activities, I of course decided it was about time to sit down and clean up all of these computers I've got haphazardly laying around into some sensical shape that I can call a homelab (re: I stuffed them all in a rack and plugged them in). So at this point, I've got a bunch of Kubernetes clusters, two Macs that I use on a daily basis and a beefy Fedora Silverblue dev machine where I can jettison a bunch of builds, random utility containers, etc. to.

This is about the point where I realized the docker commands that I've regrown accustomed to running started to bite me. As Fedora naturally comes installed and functional with Podman, I didn't want to have to go through the obstacle of installing Docker there, but I also didn't want to change my existing workflow and install something like Podman Desktop on my local machines.

And this is where you'll all tell me about why I should just use Podman for all the things. And that's fine. But you don't need to cite the dark magic to me here friends. I was there. I know the benefits. It's just in the ways and places that I work now.. these are the ways that I can remain a functional member of the team. If you are without these constraints, I am very happy to hear that.

But for those of us who are happy to dance this jig, there are times where we may find ourselves living in both worlds. We may just want our docker CLI to work with all the backends - whether they're Docker or not. Specifically if I'm using the Docker CLI and I've got a machine somewhere on the backend that's actually just got podman available. It makes things simple. Ideally as little additional configuration/installation as possible was my goal as I was going through this.

As someone who has actually done this enough times now that it should be burned into my memory - I was actually surprised to see that the garbled mess that is Google AI Summary didn't just pop up the steps needed here and even a brief skim of the search results didn't lay this out directly. So I figured maybe I would scream something out into the ether so that something useful could pop up next time I find myself needing to do this.

Setting Up Your Docker Context

So as I'm driving a lot of this from either of the Mac's that I'm working from on a daily basis - I'll need to get my Docker CLI talking to the other backends - which themselves are either Docker or Podman. We're going to focus specifically on Podman here though. To do this, we'll set up a different Docker context, which tells us that we're going to be talking to Docker in some other space. In this case, it will be Podman running on my separate machine.i

To do this, we can run:

docker context create silverblue --docker "host=ssh://me@silverblue"

Now what this does is allow me to specify using a context that will interact with my remote machine running podman. It does this by telling it to go run some docker commands against the remote host we specified via SSH.

Note: If you're not already setup to be able to connect to your remote machine via ssh, you can set this up with the following (where you replace me and silverblue with your own username and hostname respectively).

ssh-copy-id me@silverblue

Once our context is setup, we can then run docker context use silverblue to tell docker to start running all the commands I specify against this specific host/context.

So of course, when I went to run something like docker ps I expected it to just work! Of course, I was wrong:

error during connect: Get "http://docker.example.com/v1.44/containers/json": 
command [ssh -o ConnectTimeout=30 -l me -- silverblue docker system 
dial-stdio] has exited with exit status 127, please make sure the URL is valid,
and Docker 18.09 or later is installed on the remote host: stderr=bash: 
line 1: docker: command not found

While the above is a whole lot of words, the last handful actually tell us what's going on. On the remote host, the docker command isn't there. Naturally, because of course it's not. It's a podman machine. Why would it have a docker CLI present?

Configuring Our Podman Host

Alright. So that's the issue. In order to test the remote connection, Docker on the local machine expects there to be a Docker CLI on the remote machine. For this case, there exists the podman-docker package. It's available in most distributions, so you can install it using the standard installation methods. For Fedora/Fedora Silverblue, that looks something like:

Fedora:

sudo dnf install -y podman-docker

Fedora Silverblue:

rpm-ostree install -y podman-docker

Once this is installed, what you have is basically an alias setup so that whenever something calls docker, it actually calls podman. So anything expecting to be able to use a Dmocker command will be able to work as expected. So now this should work right! RIGHT!?

> docker ps
Cannot connect to the Docker daemon at http://docker.example.com. 
Is the docker daemon running?

Of course there's still more to do. One of the big differences between Docker and Podman is that Docker expects there to be a socket that it can connect to. However since we're trying to connect over to a Podman host - there's nothing there to connect to. So our Docker CLI is confused and just thinks that the service must not be started. To give Docker something to hook into we can enable the podman.socket service. Well that's simple enough, so the first thing that you'll probably want to do is run

sudo systemctl enable --now podman.socket

And of course, we would be wrong. I can tell you this because that's exactly what I did and I was still seeing the same error about the docker daemon running that we saw above. And why is that? Well because I started that service with sudo, which means it's running as root. And, unless we set up our context to connect as root, that will not be available to us. So if we want this socket to be available specifically to the user that we set up our Docker context to use, then we'll want to run this (as that specific user) on your remote host:

systemctl --user enable --now podman.socket

At this point, the socket will be online and then we can finally test this by running the following from our local host (in my case one of the Mac's):

> docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

I of course don't have anything running because we've just set this up. But we have a connection now! And now all of my machines can work together happily without me having to think about what commands I need to run.

Hopefully this is helpful! But if nothing else, hopefully this helps out future me next time I need to re-do this again!