I really like Jan, especially the organization's principles: https://jan.ai/
Main deal breaker for me when I tried it was I couldn't talk to multiple models at once, even if they were remote models on OpenRouter. If I ask a question in one chat, then switch to another chat and ask a question, it will block until the first one is done.
Also Tauri apps feel pretty clunky on Linux for me.
Yep. I really see them as an architecture blueprint with a reference implementation and not so much as a one size fits all app.
I stumbled upon Jan.ai a couple of months ago when I was considering a similar app approach. I was curious because Jan.ai went way beyond what I considered to be limitations.
I haven’t tried Jan.ai yet, I see it as an implementation not a solution.
> Also Tauri apps feel pretty clunky on Linux for me.
All of them, or this one specifically? I've developed a bunch of tiny apps for my own usage (on Linux) with Tauri (maybe largest is just 5-6K LoC) and always felt snappy to me, mostly doing all the data processing with Rust then the UI part with ClojureScript+Reagent.
I think at this point it's fair to say that most of the stuff Ollama does, is closed source. AFAIK, only the CLI is open source, everything else isn't.
Not exactly. OWUI is a server with a web app frontend. Jan is a desktop app you install. But it does have the ability to run a server for other apps like OWUI to talk to.
Tried to run Jan but it does not start llama server. It also tries to allocate 30gb that is the size of the model but my vram is only 10gb and machine is 32gb, so it does not make sense. Ollama works perfect with 30b models.
Another thing that is not good is that it make constant connections to github and other sites.
Yes it is a bold claim that I defend well though. I have been looking for smth full privacy lik HugstonOne but I couldn't. Like it or not I am proud of it. You still couldn´t mention one good GUI and call me shameless. I don´t really care if is fully native or not as long as it works for my research. I use it mostly for personal medical/health enhancement and my data/history and pics stay mine, there is nothing wrong in that. I accept every challenge to prove that HugstonOne is worth the claim.
1 It has no memory (HugstonOne does a great one).
2 Closed model ecosystem, restricted to built in models
3 Limited model support
4 Not ideal for coding
5 Not multimodal
6 Limited Advanced Settings
7 It is only for mac for god sake, you need to pay to breath.
You lost this one so badly. As I said challenge accepted.
Check Github for what HugstonOne does.
And I would Appreciate you or whomever to stop using your karma to downvote me (is called abuse of power.
P.S. I would try your app but I will never buy a mac and I would like to use open source like Jan and friends rather :)
Edit: Your app is remarkably well done though, I am jealous of native apps.
I did? None of your points talk about privacy which was your original argument.
I’ll remind you,
> If you looking for privacy there is only 1 app in the whole wide internet right now, HugstonOne (I challenge everyone to find another local GUI with that privacy).
Heck, if you look at the original comment, it clearly states it’s macOS and iOS native,
> I've been selling a macOS and iOS private LLM app on the App Store for over two years now, that is:
> a) is fully native (not electron.js) b) not a llama.cpp / MLX wrapper c) fully sandboxed (none of Jan, Ollama, LM Studio are)
How do you expect it to be and cross platform? Isn’t hugstone windows only?
So, what are your privacy arguments? Don’t move the goal post.
I am honest, I don't know how your app works, speaking privacy. I can´t even try it. You are free (like literally free) to try mine. How do users of mac/ios own their data is unknown to me. I didn't want to make a point on that as I have already big techs against, and didn't want to hijack the session further.
Now for real, I wish to meet more people like you, I admire your professional way of arguing, and I really wish you all the best :)
The app has a licence well visible when you install the app. The rest is written in the website and in github. Then about: requires an activation code to use, ofc it is made for ethical research purposes, so yes I am distributing it responsibly. And you can see the videos in the youtube channel for how it works. But the most important point is that you can try it easily with a firewall to see that it do not leak bytes like all the rest there. That´s what i call privacy, It has a button that cut all connection. You can say what you want but that´s it that´s all.
Closed source, without 3rd party independent review and people should just trust you? As if your app cannot start sending data away in a month or attempt to detect monitoring software, to name a couple
> But the most important point is that you can try it easily with a firewall to see that it do not leak bytes like all the rest there.
Great to hear! Since you care so much about privacy, how can I get an activation code without sending any bytes over a network or revealing my email address?
This is from webui website docs: Once saved, Open WebUI will begin using your local Llama.cpp server as a backend!
So you see Llama server not CLI. That´s a big flag there. I repeat no app in the whole world takes seriously privacy like HugstonOne. This is not advertisement, I am just making a point.
I'm not sure what you're talking about. Llama.cpp is an inference server which runs LLMs locally. It has a built-in web UI. You can't get more private than the inference server itself.
I tried downloading your app, and it's a whopping 500 MB. What takes up the most disk space? The llama-server binary with the built-in web UI is like a couple MBs.
With all respect you do seem to not understand much of how privacy works. Llama-server is working in Http. And yes the app is a bit heavy as is loading llm models using llama.cpp cli and multimodal which in itself are quite heavy, also just the dlls for cpu/gpu are huge, (just the one for the nvidial gpu is 500mb if I don't go wrong).
Unless you expose random ports on the local machine to the Internet, running apps on localhost is pretty safe. Llama-server's UI stores conversations in the browser's localStorage so it's not retrievable even if you expose your port. To me, downloading 500 MB from some random site feels far less safe :)
>the app is a bit heavy as is loading llm models using llama.cpp cli
So it adds an unnecessary overhead of reloading all the weights to VRAM on each message? On some larger models it can take up to a minute. Or you somehow stream input/output from an attached CLI process without restarting it?
> With all respect you do seem to not understand much of how privacy works. Llama-server is working in Http.
What in the world are you trying to say here? llama.cpp can run completely locally and web access can be limited to localhost only. That's entirely private and offline (after downloading a model). I can't tell if you're spreading FUD about llama.cpp or are just generally misinformed about how it works. You certainly have some motivated reasoning trying to promote your app which makes your replies seem very disingenuous.
I am not here to teach cybersecurity Tcp/ip protocols or ML. HTTP = HyperText Transfer Protocol
The standard protocol for transferring data over the web. CLI = Command-Line Interface. Try again after endless nights of informatic work please.
Please do try it out again, if things used to be broken but they no longer are, it's a good signal that they're gaining stability :) And if it's still broken, even better signal that they're not addressing bugs which would be worse.
No, but maybe that their shared opinion will be a lot more insightful if they provide a comparison between then and now, instead of leaving it at "it was like that before, now I don't know".
For me, the main difference is that LM Studio main app is not OSS. But they are similar in terms of features, although I didn't use LM Studio that much.
Main deal breaker for me when I tried it was I couldn't talk to multiple models at once, even if they were remote models on OpenRouter. If I ask a question in one chat, then switch to another chat and ask a question, it will block until the first one is done.
Also Tauri apps feel pretty clunky on Linux for me.
I stumbled upon Jan.ai a couple of months ago when I was considering a similar app approach. I was curious because Jan.ai went way beyond what I considered to be limitations.
I haven’t tried Jan.ai yet, I see it as an implementation not a solution.
All of them, or this one specifically? I've developed a bunch of tiny apps for my own usage (on Linux) with Tauri (maybe largest is just 5-6K LoC) and always felt snappy to me, mostly doing all the data processing with Rust then the UI part with ClojureScript+Reagent.
Can't make it work with ollama endpoint
this seems to be the problem but they're not focusing on it: https://github.com/menloresearch/jan/issues/5474#issuecommen...
- Cloud Integration: Connect to OpenAI, Anthropic, Mistral, Groq, and others
- Privacy First: Everything runs locally when you want it to
That's a tall claim.
I've been selling a macOS and iOS private LLM app on the App Store for over two years now, that is:
a) is fully native (not electron.js) b) not a llama.cpp / MLX wrapper c) fully sandboxed (none of Jan, Ollama, LM Studio are)
I will not promote. Quite shameless of you to shill your electron.js based llama.cpp wrapper here.
> I accept every challenge to prove that HugstonOne is worth the claim.
I expect your review.
1 It has no memory (HugstonOne does a great one). 2 Closed model ecosystem, restricted to built in models 3 Limited model support 4 Not ideal for coding 5 Not multimodal 6 Limited Advanced Settings 7 It is only for mac for god sake, you need to pay to breath.
You lost this one so badly. As I said challenge accepted. Check Github for what HugstonOne does. And I would Appreciate you or whomever to stop using your karma to downvote me (is called abuse of power. P.S. I would try your app but I will never buy a mac and I would like to use open source like Jan and friends rather :)
Edit: Your app is remarkably well done though, I am jealous of native apps.
I’ll remind you,
> If you looking for privacy there is only 1 app in the whole wide internet right now, HugstonOne (I challenge everyone to find another local GUI with that privacy).
Heck, if you look at the original comment, it clearly states it’s macOS and iOS native,
> I've been selling a macOS and iOS private LLM app on the App Store for over two years now, that is: > a) is fully native (not electron.js) b) not a llama.cpp / MLX wrapper c) fully sandboxed (none of Jan, Ollama, LM Studio are)
How do you expect it to be and cross platform? Isn’t hugstone windows only?
So, what are your privacy arguments? Don’t move the goal post.
Now for real, I wish to meet more people like you, I admire your professional way of arguing, and I really wish you all the best :)
It's not open source, has no license, runs on Windows only, and requires an activation code to use.
Also, the privacy policy on their website is missing[2].
Anyone remotely concerned about privacy wouldn't come near this thing.
Ah, you're the author, no wonder you're shilling for it.
[1]: https://github.com/Mainframework/HugstonOne
[2]: https://hugston.com/privacy
Great to hear! Since you care so much about privacy, how can I get an activation code without sending any bytes over a network or revealing my email address?
Llama.cpp's built-in web UI.
I tried downloading your app, and it's a whopping 500 MB. What takes up the most disk space? The llama-server binary with the built-in web UI is like a couple MBs.
>the app is a bit heavy as is loading llm models using llama.cpp cli
So it adds an unnecessary overhead of reloading all the weights to VRAM on each message? On some larger models it can take up to a minute. Or you somehow stream input/output from an attached CLI process without restarting it?
What in the world are you trying to say here? llama.cpp can run completely locally and web access can be limited to localhost only. That's entirely private and offline (after downloading a model). I can't tell if you're spreading FUD about llama.cpp or are just generally misinformed about how it works. You certainly have some motivated reasoning trying to promote your app which makes your replies seem very disingenuous.
I first used Jan some time ago and didn’t really like it but it has improved a lot so I encourage everyone to try it, it’s a great project