I just spent a good few hours optimizing my LLM rig. Disabling the graphical interface to squeeze 150mb of vram from xorg, setting programs cpu niceness to highest priority, tweaking settings to find memory limits.

I was able to increase the token speed by half a second while doubling context size. I don’t have the budget for any big vram upgrade so I’m trying to make the most of what ive got.

I have two desktop computers. One has better ram+CPU+overclocking but worse GPU. The other has better GPU but worse ram, CPU, no overclocking. I’m contemplating whether its worth swapping GPUs to really make the most of available hardware. Its bee years since I took apart a PC and I’m scared of doing somthing wrong and damaging everything. I dunno if its worth the time, effort, and risk for the squeeze.

Otherwise I’m loving my self hosting llm hobby. Ive been very into l learning computers and ML for the past year. Crazy advancements, exciting stuff.

Cool, page assist looks neat I’ll have to check it out sometimes. My llm engine is kobold.cpp, and I just user the openwebui in internet browser to connect.

Sorry I don’t really have good suggestions for you beyond to just try some of the more popular 1-4bs in a very high quant if not full f8 and see which works best for your use case.

Llama 4b, mistral 4b, phi-3-mini, tinyllm 1.5b, qwen 2-1.5b, ect ect. I assume you want a model with large context size and good comprehension skills to summarize youtube transcripts and webpage articles? At least I think thats what the add-on you mentioned suggested was its purpose.

So look for models with those things over ones that try to specialize in a little bit of domain knowledge.

Haha yeah it sure did! Thanks for sharing. I find it funny the first and only example is using a * rope* to pull it away.

Hermes CoT: “no, quantum wormhole wont work either, maybe antimatter propulsion drive. But then again that can’t be…”

NeMo: “Rope. Temperature resistant. I dunno get a big space truck and haul it. You do the math.” taps forehead

The average of all different benchmarks can be thought of as a kind of ‘average intelligence’, though in reality its more of a gradient and vibe type thing.

Many models are “benchmaxxed” trained to answer the exact kinds of questions the test asked which often makes the benchmarks results unrelated to real world use case checks. Use them as general indicators but not to be taken too seriously.

All model families are different in ways that you only really understand by spending time with them. Don’t forget to set the rigt chat template and correct sample range values as needed per model. Openleaderboard is a good place to start.

DeepHermes 24B CoT thought patterns feels about on par with the official R1 distill Ive tried. Its important to note though my experience is limited to the deepseek r1 NeMo 12B distill as thats what fit nice and fast on my card.

All the r1 distill thought process internal monolouge humanisms “let me write that down” “if I remember correctly” “oh, but wait that doesnt sound right lets try again” are there. the multiple 'but wait, what if’s" before ending the thought to examine multiple sides are there too. It spends about 2-5k tokens thinking. It tends to stay on track and catch minor mistakes or hallucinations.

Compared to the unofficial mistral-24b distills this is top tier for sure. I think its toe to toe with ComputationDolphins 24B R1 distill, and its just a preview.

Could have just taken a break from Lemmy to focus on life stuff. Those stats for comments and post in 1.8y takes a lot of time and energy.

Hope they are doing well also.

What!

In the fictional trolly Im definitely undergrad think its way funnier that way especially if the solution actually works out.

In reality im neither an undergrad nor a post doc nor anything in between. I just passionately enjoy understanding the behaviors of the universe and have extensively studied various topics including quantum field theory as well as theoretical particle physics in my free time for many years.

I did not spin it myself, though it sounds like a fun thing to try!

Before picking up crochet I was already a big fan of hemp. I’m a pot smoker first and foremost so thats where the interest stems from. I vaporize the plant for medicine, might as well see if I can wear its fibers too.

As I learned more about industrial hemp and its many pros as a natural fiber material, I became more interested from an ideological and material consumer perspective.

Hemp is stronger than steel in tensile strength, so anything you make with it is incredibly resistant to wear. Its a material that wears in like denim jeans so its fantastic for bedding sheets and clothing in the long term as it smooths and softens. This is incredibly appealing to me as a material property. I got so sick of all my cheap textile things from a store wearing down so its refreshing to make something with durability by my own hand as a “fuck it I’ll do it myself”. The hand towels are never going to seriously fray or come undone or turn rough and thinned out through normal wear.

I don’t like plastic fibers either. A lot of crochet yarn is either acrylic or a blend of acrylic and natural. I don’t want to introduce any more microplastics into my enviroment.

Hemp is a replenishing crop that heals the soil, puts nutrients in, captures a shit load of Co2, and grows like a weed without need for lots of water or fertilizer. compared to cotton which depletes soil, drinks water like crazy, and needs constant fertilizer. By buying hemp I’m voting with my wallet and saying I want to support ecologically friendly sources of my textiles which is a feel good kind of thing.

Hemp isnt a perfect material, though. Remember how I said its tough like steel? Its a real bitch to work with when trying to get a loop inside another loop in crochet. Absolutely no give at all which made it a real pain when the tolerances werent quite right. Lots of undoing and redoing the same loops. I imagine acrylic is much more workable and forgiving when trying to force it through.

Its also not a plush soft velvety texture. Its a rough and tough type of fiber that needs to get worn in before its really enjoyable to touch or use as a body scrub.

Sorry about the infodump, hope this helps you understand my reasoning.

I wasn’t referring to you specifically about wearing clothes and being antisocial. Those were just common personality and style examples i listed which ive subjectively experienced in my friends and others in my life who embraced the 'not like everyone else, im not into what everyone else is" mentality that went hard on trying to define themselves through hating what everyone else liked. I’m sorry to unintentially implicate you personally. Every time I read someones comments on the internet about priding themselves on not being like everyone because they only listen to unpopular music (its always the music for some reason) I think about how alike these people are to eachother and how that realization might piss them off if they ever saw it themselves.

You are spot on, everyone is special and unique when examining their life experiences and particular mental complexity as a whole. However people also like to fall into behavioral archetypes, their sense of individuality is often based around arbitrary and shallow things (especially as a teenager) and most idealogical beliefs people think is their own was ultimately formulated by someone else centuries ago. A five paragraph twitter bio of all a persons yucks and yums is not the same thing as developing genuine individuality through life experience/breakthroughs in understanding yourself.

Ironically the ‘nonconformist rebel who thinks themselves a more unique individual than their peers because they don’t listen to the same music/dress the same way/have a niche hyperobsession/ act antisocially’ is one of the most common and popular teenage archetypes.

Using generic punk identity signifers like dyed hair and piercings and only being into underground non-mainstream artist to show your not like everyone else, only serves to indicate that you are in fact predictably generic in identity seeking like everyone else.

So I guess my point is that maybe nothings changed about kids, its the same old same old. Most of them follow the trends and normalize to fit into tribal groups, the ones that don’t pride themselves on nonconformity while paradoxically adopting tribal markers to distinguish themselves as a group that conforms to nonconformity.

Evanescence: bring me to life

Ez. I put myself into a box with the track lever and a bomb. The bomb is activated by an XOR gate with one input being 50% chance activated if the lever is pulled, and the other input is attached to an uncollaped wave function with a 50% chance of being measured or not.

This allows the lever, bomb, and myself to enter an undefined state beyond life and death, beyond position and momentum, to become the transitionary flow between something and nothing.

While nobody is watching, the box is equally likely to be inside the quantum trolley, and on the tracks, and on the side as an observer, at the same time.

All my separate possibility wavefunctions then may or may not pull the lever at the exact moment the quantum trolley hits a quantum eraser.

If performed correctly, the collective consciousness of the universe experiences what us humans would refer to as a aneurysm followed by violent seizures.

Finally, this neuroplastic wound bootstraps a emergency regenerative attempt to repair such devastating damage to the concept of causality.

A catastrophic explosion of tachyons froth at the epicenter of the incident. They act like antibodies fighting a virus, cascading into a a level-4∆ Retroactive Timeline Recalibration Event(RTRE-4∆)

Boom, The quantum trolley never even existed in the first place.

Q.E.D

taps forehead

Just double knot it.

I die in a four way sesh-to-the-death match between snoop dog, cheech, Ricky from TBP and myself.

Was your education worth it? Did it pay itself off?

Ah so thats what theyre called. I think I found a mitochondrion under my moms bed one time, must have been some kind of map.

W-what?

Its like the cinema version of brown-n-bloom from games of the 7th console generation. Filmographers discovered digital filters as a new tool so of course they are they going to use the new toy everywhere.

Cryptocurrency.

Kidding! Sorry, I couldn’t help myself. I’m sure that will ruffle someone’s feathers for a fraction of a second.

Does vegan based meat/milk alternatives count as a technology sector? The moment we figure out a way to actually seriously imitate high quality steaks and 2% cow milk with it being indistinguishable and half the price I think that will hurt factory farming which is a bit of a domino effect that would lead to less co2 and methane produced, less agricultural effort wasted on feeding animals, that kind of thing.