@xyster I'm having opus 5.5 do the exact same thing right now. The best thing I did was create a harness so that my apps can be driven and tested by it. Then having it run tests for multiple hours with a emulator running for Bluetooth connections.
@xyster Right now we're just testing with qwen 27b, but I'm waiting for some RAM to arrive so that we can try and make qwen flash 175b with some RAM offloading
@xyster I'm genuinely considering adding a fifth b70 to my new setup so that I can run the new qwen 120b, are you running six or more cards on a single board?
@xyster Dude this AsRock Rack WRX80D8-2T motherboard is a total fucking sleeper!!!!! Seven 16x PCI 4 slots!!!! It pretty much uses any type of ram as well, including rdimms or udimms!
This is pretty much a ddr4 final boss
@xyster Excellent. I'll look into those!
Here is my current setup.
Core i7 7700k 32. Gigs of RAM modded bios and my two b70s. I'm running Nemo lightning at around 42 tokens a second.
@xyster What mother board are you running this on? I'm running 2 b70 right now on a modded bios z270 board. Was looking at the x399 platform as the best upgrade path to 4 cards.
Great work btw!