This paper proposes a multi-action transmission scheduling problem for remote state estimation in wireless networked physical systems under global energy constraints. Firstly, the system state estimation is obtained through Kalman filtering. Then, the estimated values are transmitted from sensors to a remote estimator through randomly fading channel. Unlike traditional transmission scheduling problems, the proposed system model allows sensors to choose from multiple power levels at each decision node. Additionally, the study considers global energy constraint and formulates the system model as a constrained Markov decision process to obtain the optimal transmission strategy that minimizes the average estimation error covariance at the remote estimator over an infinite time horizon. Through model transformation, a important conclusion is derived. As the average estimation error covariance at the remote estimator increases, the optimal transmission power exhibits an increasing trend. This conclusion extends the threshold structure of the optimal strategy to a monotonic increasing structure. Finally, the theoretical result is verified through numerical simulations.

Xu Fanghui. The multi-action transmission scheduling problem for remote state estimation under global energy constraint. Academic Journal of Computing & Information Science (2023), Vol. 6, Issue 13: 20-28. https://doi.org/10.25236/AJCIS.2023.061304.

[1] Zhang K, Shi Y, Karnouskos S, et al. Advancements in industrial cyber-physical systems: an overview and perspectives[J]. IEEE Transactions on Industrial Informatics, 2022.

[2] Pivoto D G S, de Almeida L F F, da Rosa Righi R, et al. Cyber-physical systems architectures for industrial internet of things applications in Industry 4.0: A literature review[J]. Journal of manufacturing systems, 2021, 58: 176-192.

[3] Xing W, Zhao X, Basar T, et al. Optimal transmission scheduling for remote state estimation in CPSs with energy harvesting two-hop relay networks[J]. Automatica, 2023, 152: 110963.

[4] Yuan F, Tang S, Liu D. AoI-Based Transmission Scheduling for Cyber-Physical Systems Over Fading Channel Against Eavesdropping[J]. IEEE Internet of Things Journal, 2023.

[5] Ho C K, Zhang R. Optimal energy allocation for wireless communications with energy harvesting constraints[J]. IEEE Transactions on Signal Processing, 2012, 60(9): 4808-4818.

[6] Ozel O, Tutuncuoglu K, Yang J, et al. Transmission with energy harvesting nodes in fading wireless channels: Optimal policies[J]. IEEE Journal on selected areas in communications, 2011, 29(8): 1732-1743.

[7] Knorn S, Dey S. Optimal energy allocation for linear control with packet loss under energy harvesting constraints[J]. Automatica, 2017, 77: 259-267.

[8] Leong A S, Dey S, Quevedo D E. Sensor scheduling in variance based event triggered estimation with packet drops[J]. IEEE Transactions on Automatic Control, 2016, 62(4): 1880-1895.

[9] Qi Y, Cheng P, Chen J. Optimal sensor data scheduling for remote estimation over a time-varying channel[J]. IEEE Transactions on Automatic Control, 2016, 62(9): 4611-4617.

[10] Wei J, Ye D. On two sensors scheduling for remote state estimation with a shared memory channel in a cyber-physical system environment[J]. IEEE Transactions on Cybernetics, 2021.

[11] Wang J, Ren X, Dey S, et al. Remote state estimation with usage-dependent Markovian packet losses[J]. Automatica, 2021, 123: 109342.

[12] Wei J, Ye D. Double Threshold Structure of Sensor Scheduling Policy Over a Finite-State Markov Channel[J]. IEEE Transactions on Cybernetics, 2022.

[13] Leong A S, Quevedo D E, Dolz D, et al. Transmission scheduling for remote state estimation over packet dropping links in the presence of an eavesdropper[J]. IEEE Transactions on Automatic Control, 2018, 64(9): 3732-3739.

[14] Salh A, Audah L, Alhartomi M A, et al. Smart packet transmission scheduling in cognitive IoT systems: DDQN based approach[J]. IEEE Access, 2022, 10: 50023-50036.

[15] Djonin D V, Krishnamurthy V. Q -learning algorithms for constrained Markov decision processes with randomized monotone policies: Application to MIMO transmission control[J]. IEEE Transactions on Signal Processing, 2007, 55(5): 2170-2181.

[16] Altman E. Constrained Markov decision processes: stochastic modeling[M]. Routledge, 1999.

[17] Le H, Voloshin C, Yue Y. Batch policy learning under constraints[C]//International Conference on Machine Learning. PMLR, 2019: 3703-3712.

[18] Zang W, Miao F, Gravina R, et al. CMDP-based intelligent transmission for wireless body area network in remote health monitoring[J]. Neural computing and applications, 2020, 32: 829-837.

[19] Sun C, Stevens-Navarro E, Shah-Mansouri V, et al. A constrained MDP-based vertical handoff decision algorithm for 4G heterogeneous wireless networks[J]. Wireless Networks, 2011, 17: 1063-1081.

[20] Miryoosefi S, Brantley K, Daume III H, et al. Reinforcement learning with convex constraints[J]. Advances in neural information processing systems, 2019, 32.

[21] Yang L, Rao H, Lin M, et al. Optimal sensor scheduling for remote state estimation with limited bandwidth: A deep reinforcement learning approach[J]. Information Sciences, 2022, 588: 279-292.

[22] Zhang J, Sun J, Lin H. Optimal DoS attack schedules on remote state estimation under multi-sensor round-robin protocol[J]. Automatica, 2021, 127: 109517.