Rendered at 07:41:06 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
Neywiny 8 hours ago [-]
What does that mean "running in real time" here? This seems like a meaningless phrase in this context. What would this look like if it wasn't running in real time?
For example, a video decoder running in real time means it can output x seconds of output data within x seconds of wall time. But there's no time constraint on an LLM.
einsteinx2 7 hours ago [-]
From the thread on X it’s only 4.7 tokens/sec which personally I would find unusable for “real time” work
For example, a video decoder running in real time means it can output x seconds of output data within x seconds of wall time. But there's no time constraint on an LLM.