Post #1843613

@voidplush@dorm.social Sorry if this is coming off "heated" I'm genuinely not trying to argue in bad faith here. I'm just frustrated cause this is a real road block i'm hitting in development right now. I don't quite know what you mean by "has conditionals in build-logic+compose allows it too", my best guess is that you're trying to hint at combinations of ARG/ENV in dockerfile, compose's profiles / variable interpolation or override files? but if that's what you mean, that's not really conditionals, you're not branching, it's not comparable to conditionals like in starlark for example. re dependency graphs, what tooling? docker history?. that's not gonna help you much. in terms of caching semantics, you get layer memoization, but that's hardly real caching semantics, i don't think there's a way to change how they're invalidated. I'll show you a quick example that hopefully illustrates my problem. Say I have 3 services: "Apple": Admin panel "Banana": Backend (stateless, may be deployed multiple times) "Cherry": Some sort of Gateway that routes stuff to the correct Backend Apple and Banana share "cryptohelper", which is just some thin validation layer that's important to the service. Banana and Cherry share a protobuf api schema that defines the interactions between the gateway and the backend. so you have (imagine a file tree here): libs: [cryptohelper] protos: [api.proto] services: ["apple/", "banana/", "cherry/"] With docker, you're now practically limited with a few options, and all of them are kinda terrible (and this sort of problem just scales as your project grows): You can copy the sources of libs/ & protos/ into the dependants, but in that case you end up rebuilding each of those individually for each dependant, if there was 100 dependants of e.g. cryptohelper, you're rebuilding the entirety of cryptohelper 100 times, docker has no way to stop this from happening. So then you think "okay, i know how to do this properly" and you start going for one (or multiple, more on that later) base-image(s), except now you have to version your baseimage as an intermediate image and make sure the CI pulls it before building anything else, you're basically starting to roll your own artifact registry because there's no real dependency graph. And even then, if you start to pool two dependencies that are used by different services into the same base-image, you need to know, for every small change that affects the libs in the base-image, which services are affected. you're basically back to inventing a graph, all by yourself. (or you just say fuck it and rebuild everything). you can't nicely do this with "multiple" base-images either, because the "graph"/inheritance is single-inheritance, so you're either: picking from one of them as the base, copying from the other (but then the two containers need to have the same base-system and build environment, which may not be given). or, you're gonna build a third base image, which copies from both. (you're back to inventing a graph). so now, for a breath of fresh air, consider something like bazel/buck (all of these would look kinda similar): you define nodes for each input ("libraries") whatever_library(name="cryptohelper", srcs=..., visibility=["//services:..."]) # and proto_library(name="api_proto", srcs="api.proto", visibility=["//services:..."]) and you simply build all your binaries in one go whatever_binary(name="apple", srcs=..., deps = [".../cryptohelper"]) whatever_binary(name="banana", srcs=..., deps = [".../cryptohelper", ".../api_proto"]) whatever_binary(name="cherry", srcs=..., deps = [".../cryptohelper"]) it can cleanly figure out what to rebuild, because it knows about the graph. there's even tooling that can build containers from bazel, you just define it as another output that contains the binaries. I'm now in the progress of migrating towards something like this.