← Feed
@jbert@hachyderm.io
Post #2280770
2025-10-30 09:42 UTC
@sminez@hachyderm.io I'm not sure I'm understanding the problem with streaming + parallel execution.
I'm guessing the issue is knowing whether there is still a match "pending" on received data before you can commit to running the actions/substitutions?
If that is the case, is it possible to chunk the input and at the end of each chunk, ask each matcher if it is waiting on more input to see if it should match? (Alternative api - ask each matcher for the offset up to which there is no ambiguity for whether it is matching or not).
i.e. if the input stream is currently:
aaaaa
abaaa
then the matcher /ab/ could report an offset of 9. It doesn't (yet) know if the last 'a' is the start of a match or not.
That would allow you to process up to the minimum offset which all matchers are OK with.
Sorry if this is nonsense. Interesting stuff though.
> Ideally I'd like to be able to run "structex" scripts
Or perhaps "stawk"? :-)
Replies (1)
-
@jbert@hachyderm.io not nonsense at all!
The problem is when you have parallel actions that would trigger on the output produced by one another. This is the simplest example of the problem:
, {
x/Alice/ c/Bob/;
x/Bob/ c/Alice/;
}
"Replace all occurrences of "Alice" with "Bob", replace all occurrences of "Bob" with "Alice".
You need to know which "Bob"s are from the original input and which came from being replacing an "Alice", otherwise everything will end up as "Bob".
You can get around this by interleaving the execution of each branch but the combinatorics of that quickly get fiddly if you have compound expressions. Breaking things into two passes (locate all matches, apply actions) ends up being a lot easier.
Open ##2280771