I use the Qwens as a vision model for my DeepSeek V4 Flashes to handle. But the Qwens run on old RTX A6000 Ampere. Does anyone know if there's any news about INT4/AWQ quants for the RTX A6000?
Was recently thinking about doing something similar, do you basically just have the qwens describe what they see for the flashes?
Was considering adding a LoRa/vision head to Flash, but seems like it could take a while to get it right.
If DSv4 Flash was multimodal, I’d probably be done model shopping for a while
Same, with a multimodal DSv4 Flash I would just stop paying attention to things. Very smart, and at 260 tok/s it's too fast to care about anything else. If you ever graft something like that I would love to hear about it.
Yes, I have a very dumb flow. The harness has a describe_image tool that takes an image and a prompt and so DSv4 Flash uses it to get an idea of what it's looking at.
loading story #49301831