Anyone else running local LLM? What do you think?

Joined
Nov 17, 2019
Messages
244
Location
Alberta
Ive been messing with Ollama and some models. Mostly Gemma3. Its pretty neat stuff. I dont know if it provides anything more than a bit of research can provide but it can answer questions quickly. I dont code so I dont have use for them in that case.

Im a bit of a privacy nut so really have no use for online or intgrated AI. I run it at home and access it through tailscale if I want to use it.

If anyone here knows a good front-end ios app or something to link back to ollama I'm all ears.
 
I setup one a while back but didn't really get into playing with it much. Just haven't had the time. I threw it together from an old couple of decommed servers we were throwing out from work. Its a a little old but I managed to get 128GB of RAM into it and a few TB of hard drive space as well. Couple of old enterprise level graphics cards too. It works but again, haven't played with it enough to really get anything interesting out of it. Just been way to busy with work and life stuff lately. I used Ollama 3.2 for my setup, currently only have it setup for text only mode.
 
I setup one a while back but didn't really get into playing with it much. Just haven't had the time. I threw it together from an old couple of decommed servers we were throwing out from work. Its a a little old but I managed to get 128GB of RAM into it and a few TB of hard drive space as well. Couple of old enterprise level graphics cards too. It works but again, haven't played with it enough to really get anything interesting out of it. Just been way to busy with work and life stuff lately. I used Ollama 3.2 for my setup, currently only have it setup for text only mode.

Yeah I am text only mode. i have a 9070xt and was trying to get StableDiffusion running with AMD but ran into some issues. I tried AMD supported pic generation models. They work okay but lots of room for improvement.
 
I wouldn't mind running a security focused LLM and letting it chew on syslog from my (home) network. Don't know if anything exists like that. Haven't really looked into it much.

I could feed it external blacklists and such and tell it to look for hits.
 
Back
Top Bottom