Weight placement is a critical decision in on-device large language model (LLM) inference, determining how model layers are split between GPU and system memory when a model exceeds GPU capacity. In this paper, we conduct an empirical study of how this decision affects inference performance on memory-constrained consumer hardware. We examine the static, estimate-based placement used by widely deployed inference engines and compare it against placements selected by direct measurement on one 8 GB consumer GPU. We find that default placement is consistently conservative: across three models and quantization formats, measured placement improves decode throughput by 6.7% to 402%, a gap that varies too widely to be corrected by any fixed safety margin. Our analysis further reveals two challenges that static estimation cannot address. The first is context coupling: a placement chosen at one context length becomes invalid at a longer one as the key-value (KV) cache grows, which reduced throughput from 8.71 to 1.47 tokens/s across consecutive calls in one session. The second is unpredictable oversubscription: once physical GPU memory is exceeded, the driver silently spills into system RAM, causing non-deterministic slowdowns. Building on these findings, we present Ramer, a runtime layer that selects placements by measurement, keys them to context length, re-validates them at run time, and coordinates which model remains loaded when multiple agents share one GPU. Through end-to-end experiments we show that Ramer reduces the median completion time of four concurrent agents from 210.6 s to 117.6 s and completes an externally verified coding task 38% faster, while providing no benefit when tool latency dominates. Our findings indicate that weight placement on consumer hardware should be measured rather than estimated.

on-device inference, edge AI, large language models, GPU memory management, model offloading, autotuning, LLM agents