<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Vllm on David R. Longnecker - Converting Coffee to Code</title><link>https://drlongnecker.com/tags/vllm/</link><description>Recent content in Vllm on David R. Longnecker - Converting Coffee to Code</description><generator>Hugo</generator><language>en-US</language><lastBuildDate>Wed, 28 Oct 2026 09:00:00 -0600</lastBuildDate><atom:link href="https://drlongnecker.com/tags/vllm/index.xml" rel="self" type="application/rss+xml"/><item><title>Two Models, One Token Stream</title><link>https://drlongnecker.com/blog/2026/10/two-models-one-token-stream/</link><pubDate>Wed, 28 Oct 2026 09:00:00 -0600</pubDate><guid>https://drlongnecker.com/blog/2026/10/two-models-one-token-stream/</guid><description>&lt;p&gt;When software engineers first learn how modern GPU hardware runs autoregressive language models, their immediate reaction is disbelief.&lt;/p&gt;
&lt;p&gt;You provision a server equipped with an 80GB enterprise GPU capable of trillions of floating-point operations per second. You send a user prompt to a 70B parameter model, and text comes back at a pace that feels slow for hardware this expensive, on the order of tens of tokens per second.&lt;/p&gt;
&lt;p&gt;If you profile the hardware while that stream is running, you discover that the compute cores are mostly idle. The GPU spends those cycles streaming weights from memory into the cores, not doing math.&lt;/p&gt;</description></item></channel></rss>