It all depends on the size of the model you are running, if it cannot fit in GPU memory, then it has to go back and forth with the host (cpu memory or even disk) and the GPU. This is extremely slow. This is why some people are running LLMs on macs, as they can have a large amount of memory shared between the GPU and CPU, making it viable to fit some larger models in memory.
- 1 Post
- 12 Comments
Thanks for sharing, I find it hard to discover new “lemmy spaces”??? On here
mlflexer@lemm.eeto Linux@lemmy.ml•Is there a downside to sticking to iptables over ufw?English111·30 days agoI thought nftables where replacing iptables?
mlflexer@lemm.eeto Mechanical Keyboards@programming.dev•GitHub - jonboh/ulp-dactyl: A Dactyl mounting Ultra Low Profile switches :DEnglish3·30 days agoLooks awesome, how are the ULP switches compared to chocs or mx?
I tired it once with my buddy and it seemed to work fine on the element client. Not sure if this was placebo, but it felt like the unencrypted video call had better quality, however it wasn’t very noticeable. Though it might be a bigger problem with many people in the call, haven’t tested it though
Matrix? I think you can setup text channels and also do voice/video/screen sharing in the channels as well if you’re using element, though I havn’t been able to convince my friends to jump ship yet, so don’t know how it compares to discord
mlflexer@lemm.eeto Programming@programming.dev•Ubuntu 25.10 Looks To Make Use Of Rust Coreutils & Other Rust System ComponentsEnglish10·1 month agoThey don’t seem to have 100% pass of the tests, but I might be missing something?
Would love to take the jump, but I think I’ll wait until they pass all tests
mlflexer@lemm.eeto Buy European@feddit.uk•Firefox based European browser alternativesEnglish3·1 month agoI was never a browser poweruser, but zen has made me one. Really loving it so far coming from Firefox.
mlflexer@lemm.eeto Buy European@feddit.uk•Danish grocery chain to distinguish European from US goods – DW – 02/27/2025 - Danish people, have they started doing this already?English9·1 month agoAs a Dane, I’m looking foreward to this change although I haven’t seen it implemented yet
You should only need to have Java and then download the server and open the port if they want to play vanilla mc. If they want modded then idk.
You might also want to check this out, haven’t used it myself but it looks cool if you don’t like wasting server resources: https://github.com/timvisee/lazymc
Oh, I thought you could get 128gb ram or more, but I can see it does not make sense with the <24gb… sorry for spreading misinformation, I guess, in this case a GPU of the same GB ram would probably be better