Robin Kipp

@jcsteh@aus.social Last year I did something similar with a Minisforum mini PC and its HDMI output connected to my Windows desktop using an Elgato capture card. Except I then used OBS Studio's full screen projector so it was showing the HDMI video stream, which I then passed along to Gemini using the Google AI Studio website and the screen sharing option found there. Combined with the voice chat feature, this meant I was actually able to talk to the AI which also had access to the video stream, enabling me to ask Gemini what was currently visible, how to navigate to a certain option using arrow keys, whether it was currently on or off, etc. It was still kinda slow but ended up working incredibly well, once it was going it honestly was pretty mindblowing stuff.

@mysk@mastodon.social Please also call out the fact that apps can query which accessibility features are enabled on an iOS device, such as VoiceOver. Not only can this be used for fingerprinting, but it can also be used to make good guesses on which physical disabilities / impairments a user might have.

Alrighty, here’s a quick and dirty demo of my Flipper Zero accessibility project, navigating the main menu, using it to scan an NFC tag and emulate my TV’s power button using infrared. Enjoy, any feedback, suggestions and comments more than welcome!

@jcsteh @menelion @tailscale Tailscale is absolutely amazing, and yes they do care about accessibility. Opened a GitHub issue once for some VO accessibility bugs, and they got fixed within only a couple weeks. So yes, can very strongly recommend as well!

@jakobrosin Wow, this is incredibly cool and totally, totally awesome!!! Congrats to you for pulling that off, what a neat idea!

@jakobrosin You are most welcome! Same here actually, the live streams are great but not being able to view the live data has always been such a bummer. I even had the idea of building an iOS app for this at some point as that's where I’m most experienced, but didn't know whether it would be possible to gather up all this telemetry data in the first place. One suggestion, would you be willing to consider adding web push notifications for things like start of crew phases and key mission events? That would be very cool to have, and probably not too hard to do with Claude or whichever AI platform you're using.

@jakobrosin Oh, yes... That's actually what I meant.

@jakobrosin Wow, thanks!!! Have just turned them on on my Mac using Safari, let’s see what happens.

@jakobrosin Also just confirmed that notifications can be enabled in iOS as well if you add the web app to your home screen first, awesome!