Post #2506777

2025-11-30 07:00 UTC

@yosh@toot.yosh.is Uh, uhm. I took a PhD course on how to model this and arrive roughly at that number (long story on how I was allowed to do this, even though I only hold a Masters). We did this on server-client systems, but what is a train but a request traveling through the internet. Essentially, you can take timed petri-nets (a form of state machine with multiple states being active at once - think "trains on routes, in stations") and use a statistical function to model how long it takes for an token (train) in that net to go to the next state (every edge has a different function, runlength). You can then actually reason _locally_ about states at every node using markov chains. ("how many trains are at this station at any time with which probability") and then _back_-calculate that into delay probabilities. Biggest learning: this isn't linear. It's almost linear, until the point it breaks, where delays grow rapidly and catastrophically over the network. Which is also why 70% system load is about the moment where you should start thinking about upgrading your servers. Sorry for the brain dump :).

Replies (6)

@msfjarvis@androiddev.social 2025-11-30 07:58

@skade@hachyderm.io @yosh@toot.yosh.is That's super fascinating! Do you have a recommended resource where I can read about this?

Open ##2506778
@hipsterelectron@circumstances.run 2025-11-30 09:24

@skade@hachyderm.io @yosh@toot.yosh.is this was a remarkable explanation!

Open ##2506790
@cvtsi2sd@hachyderm.io 2025-11-30 10:49

@skade@hachyderm.io @yosh@toot.yosh.is very interesting, this seems somehow related to other research I read about queuing theory, where an important result is that you have to have extra capacity to handle statistical fluctuations in requests duration, otherwise, while average throughput may remain constant, average queue length/waiting time tends to grow catastrophically

Open ##2506792
@PeterSommerlad@mastodon.social 2025-11-30 12:29

@skade@hachyderm.io @yosh@toot.yosh.is thank you for triggering memories of my university time in the 1980s. The crucial thing I remember is that high usage tends to block everything. Design for slack!

Open ##2506793
@jssfr@zombofant.net 2025-11-30 14:32

@skade@hachyderm.io That's actually quite interesting. Do you have references for further reading?

Open ##2506794
@clew@ecoevo.social 2025-11-30 21:48

“It's almost linear, until the point it breaks” Could there be a more seductive characteristic for bad managers? Or, the other phrasing, “went bankrupt in the usual way — slowly, and then suddenly.” @skade@hachyderm.io @yosh@toot.yosh.is

Open ##3042604