An Empirical Study of GPU Concurrency Scaling for Pathology Inference at the Clinical Edge
编号:4 访问权限:仅限参会人 更新:2026-11-10 16:07:25 浏览:9次 Online

报告开始:2026年11月21日 15:15(Asia/Kolkata)

报告时间:15min

所在会场:[S10] Mixed Track [S10-1] Mixed Track

暂无文件

摘要
Pathology models deployed at the clinical edge often share one GPU, so they cannot absorb a surge of requests by starting more replicas. What they can change is how many queued images each forward pass carries. We study three policies that set this micro-batch slot count (1-8, applied 10 s after a scale-up request) for a ResNet-50 PathMNIST service on an NVIDIA RTX 4060 Laptop GPU: a CPU-threshold rule, a reactive GPU-utilization rule, and a Prophet forecaster. Under light load (60 runs, at most 42.3 requests/s), every request succeeded and the policies were indistinguishable, with median client p95 latency between 23 and 33 ms (Kruskal-Wallis p=0.89). Under overload (48 valid runs, a 79.3 requests/s base with 116.3 requests/s peaks), Prophet kept a median p95 of 24 ms through a sudden spike, while the CPU and reactive policies reached 685 ms and 622 ms (p=0.025). Bursty and ramped traffic showed no consistent difference, and Prophet was slightly slower under steady overload. A 16-run ablation with a fixed two-slot control explains the spike result: two pinned slots gave 24.5 ms, Prophet 26.9 ms, and the other two policies about 1.4 s (p=0.0045). Prophet won because a conservative capacity estimate kept a second slot active before the spike began, not because it predicted the spike. The reactive rule seldom scaled, as GPU utilization stayed at 37-56% while requests queued. On a single edge GPU, keeping spare slots under heavy load delivered the whole benefit without a forecaster.
关键词
Edge Inference,GPU Micro-Batching,Autoscaling,Time-Series Forecasting,Digital Pathology,PathMNIST
报告人
Visshva R
Student SRM Institute of Science and Technology

稿件作者
Visshva R SRM Institute of Science and Technology
Babu K SRM Institute of Science and Technology *
G Senthil Kumar SRM Institute of Science and Technology
Ruthwik S SRM Institute of Science and Technology
Adithyan E SRM Institute of Science and Technology
Irfan Fidel Farhath V SRM Institute of Science and Technology
发表评论
验证码 看不清楚,更换一张
全部评论
重要日期
  • 会议日期

    11月20日

    2026

    至

    11月22日

    2026

  • 10月31日 2026

    初稿截稿日期

  • 11月07日 2026

    提前注册日期

  • 11月22日 2026

    注册截止日期

主办单位
United Societies of Science
承办单位
Pondicherry University
协办单位
IEEE Section
IEEE Madras Section
移动端
在手机上打开
小程序
打开微信小程序
客服
扫码或点此咨询