that would be really interesting, like a old school PDA but with AI. If only we could locally host a helpful AI like ChatGPT, then we could build it to be compatible with ThunderBird, the LibreOffice suite, calenders, etc.
2 Likes
python book out there - automate the boring stuff would be an improvement for me… issue i have with Devterm right now is a sleep recovery issue - screen driver locks and unable to access the unit without an external screen or ssh… if this is not an issue for the uConsole then i could see running it persistent. as it stands i use the Devterm for a bit then shut it down.
I was able to get the smallest GPT-2 model working on a Raspberry Pi 4, and it was about as slow as molasses. Took 90 seconds to generate a couple semi-coherent sentences.
That’s running locally on the device, though. You might be able to do something with the API if you have a reliable internet connection.
3 Likes
90 seconds… I hope there would be better way to reduce the time.
Heard an interview by an AI scientist and a physiologist… they don’t fully understand how the AI language solver works! Also building the parameters for an AI is going to be like an occupation in few years!
I’d take that claim with a healthy grain of sodium chloride. Researchers at Google designed transformers. We know how they work. The calculations they make on any given prompt are just complicated and not particularly human-readable.
The thing about generative AI is the underlying technology is cool, but people desperately want it to be smarter than it actually is. And if you ask it to do tasks beyond its capabilities (essentially predicting the next plausible word), it will play along.
1 Like
You can play around with it, but the version of GPT-2 that can run on the Raspberry Pi is so much worse than ChatGPT at everything that I’d only recommend it for tinkering.
1 Like
It’s a bit of a stretch to say “we don’t know how it works”. These generative models were theorized by Minsky et all back in the mid 80s; modern computers are just getting to the point where experimenting with them is practical. It’s not laid out like normal code that you can modify and debug, but there’s decades of research into “how it works”.
Be careful about hype!
3 Likes
with llama.cpp you probably could get somewhere between 1-0.2 tokens/s locally with, say, LLaMA 7B
1 Like
does anyone own a RPi 4B to test this? would be interesting. Of course you’d want to sand box it so it doesn’t tattle back to FaceBook.
LLaMA can run totally offline, and TMK it’s been done before on a 4B, though not very quickly. been a while since i heard about it running on one so it might be faster by now
That is very cool! Coincidentally, I have the same SDR. What are you using for ADS-B? (I’m very new to SDR.)
Edit: Found it: it’s called “dump1090”. It’s by Antirez, he’s really cool!
Been writing software for a long time. My gpd micropc died, and I need something new to program on with my thumbs. Been running a lil cloud vm, moshing into it, and attaching to tmux. It’s not the same. I miss physical keys, rugged hardware, a full Linux distro. Something that’s not my phone. Something I can just hack around with. Professionally I do api, database, and web dev. Hobby I enjoy some of that in addition to game and system dev. I just wanna chill and code on a lil handheld.
2 Likes
Did you ever experience problems with termux and git? I get a lot of corrupted files when pulling my GitHub repo on my Android phone using termux. This makes it virtually useless, and my solution is to use something with a Linux distro and not emulated storage.
I’ve never heard of termux
Android terminal emulator.
Ah okay yeah I looked it up. I’m on a 
phone . Blink shell is the best I’ve come across.
Have you ever done anything with Defold? I have been waiting to build something with Lua to use across platforms. I know it is a game engine, but wanted to make some kind of GUI application with Lua and other assets.
Looks cool. I’m gonna mess around with this for sure
2 Likes
I actually had a really cool idea to set up jack2/ardour/guitarix and plug the uConsole into an audio interface to use as a portable FX pedal/recording device for jamming with my friends.
3 Likes