Recent Papers / arXiv:2606.01912
SMH-Bench: Benchmarking LLM Agents for Environment-Grounded Reasoning and Action in Smart Homes
Kuan Li, Shuo Zhang, Huacan Wang, Fangzhou Yu, Zecheng Sheng, Yi Gu, Weipeng Ming, Lei Xue, Chen Liu, Sen Hu et al.
Abstract
1,100 tasks across 7 categories in an executable smart-home simulator.
Frontier LLMs fail on automation scheduling and ambiguity handling as home complexity increases.
Tasks
editResults
No benchmark results recorded yet.
Benchmark results referencing this paper haven't been added to the registry yet. If you have a reproduction, submit it →
CodeSOTA extraction
Benchmark evidence
- SMH-Bench: Accuracy breakdown by task category and home complexity (extract from Table 2).