Post #2962838

@jenniferplusplus@hachyderm.io i've been playing around with one hosted on my own hardware for a little while and if anything this is making me less hopeful on using them for anything that does not require constant babysitting and can't accept "ok let's drop this and we'll try it tomorrow when i think of a better prompt or i get a luckier seed" as an outcome maybe it's because i'm using a relatively small (which in this context still means 25-35 Billion parameters, with 3 Billion active at any given time) model at a low quantization because that's as much as i can fit, compared to whatever hell the corporate hosts are doing, but from what i can tell all this really means is they start breaking down faster, earlier in the process (so if anything it's less of a false sense of security)