If you are like me, and you have just learned how docker and docker compose worked, you will probably head out to rebuild your own little server, or start one anew, watch some tutorials, look for some tools, and end up with a little compose file to start your infrastructure.
It probably has a reverse proxy like nginx, traefik or Caddy, a little static website and maybe something for fun, a little self made project, a node app, a lil' blazor app, maybe just some open source link shortener you found (obligatory Just Short It! mention). A total of a hundred lines, not bad, manageable.
But it doesn't stay like that, does it?
Rise of the stacks
Now, you may get interested in some more advanced tech. Like, let's say, a blogging engine (like Wave), but that doesn't just take a container for a website, it also comes with a postgres database, a redis cache, maybe even more (at point of writing, Wave is still in alpha), so there goes another block in your compose, another 80 lines or so, now we got 200, not great, still okay I guess?
But now your thinking, you need some utilities to have a good look at everything, maybe add a dashboard, uptimekuma, maybe grafana? Three more containers can't hurt can they? 300 lines, should be fine, right?
But then you are me, create GeeksList, have a couple websites, a couple tools like JSI, FreshRSS, MastoGraph, your own docker repository with a web frontend to browse it, two matrix instances, a fedi instance, dpaste why not, a Minecraft Server, sure. And now we got to the 1000+ line docker compose file, and you realize you got a problem.
Sure it works, it's fine, but doesn't it feel icky? Surely there is a better way…
External Networks, Include and Extends
There sure is buddy.
For once, there is extend (docs.docker.com - extends). This allows you to centralize certain configuration, like when you got a couple static websites, you can put in your nginx or something. Next, there is include (docs.docker.com - include), which allows you to split up your compose into multiple files, and have a root file that combines them all. Personally, I've chosen a secret third option, and that is not having them work together at all. You see, when I separate, I want a clear separation, I have a folder /docker with folders for all my app stacks, completely isolated from each other, so if I want to move one, I just move that folder, and here I want to also have my compose, independent from everything else.
Docker does allow this, very easily even. The trick is, to have one central point of communication with the outside world, your reverse proxy. So I created a central container, in my case a Caddy server, that I hooked up to an external network I created previously, I called it Internet, easily created with docker add network Internet, if I recall correctly. Then in the compose, you just add the proxy to that network, and declare it as external like so:
networks: Internet: external: true
Now we have a network on our server, that we can include in any other compose or container whenever we want to be reachable by the outside world.
Next I want to set up a stack, I go into one of my folders, like /docker/Wave, and I create my compose (in this case, just copy it from the github repository and fill in the blanks). At the top level, I add name: wave, at the bottom, the external Internet network, and on the web container, I add that network.
Now when I run docker compose up -d in that network, it will create the container wave-web-1, wave-database-1, wave-redis-1. Nice names, easy to recall and predict, and wave-web-1 can communicate with my proxy, even tho they are in two completely unrelated compose projects, so now I can just add a little reverse_proxy instruction to my Caddyfile and everything works nicely. And this is basically what I have to do with all my projects, set up their individual compose file, update my Caddyfile, done. Neat right?
Now I just have to actually do that with anything other than Wave… (I get to it I swear)