PDF(5625 KB)
PDF(5625 KB)
PDF(5625 KB)
边缘云环境中收益最大化的在线服务功能链部署机制
Deployment mechanism for maximizing revenue from online service function chains in edge cloud environments
针对边缘云环境中的服务功能链部署挑战,特别是资源限制、延迟满足和收入成本之间的权衡,该文提出了在延迟、计算资源和通信资源约束下,边缘云环境中服务功能链在线部署收益最大化的问题,并设计了基于深度强化学习的算法SDRM-EC-DPR。首先,根据市场供求规律为计算设备构建了部署成本模型,以此消除设备异构性并评估请求是否值得被接受;随后,为模型构建Markov决策过程,结合对决型双重深度Q网络(dueling double deep Q-network,D3QN)算法来处理大规模且复杂的决策过程,为优化学习效率并防止陷入局部最优,在D3QN的基础上引入优先经验回放机制及随机网络蒸馏技术。仿真实验在不同设备规模和服务功能链请求数量组合下展开,结果表明,相较于对比算法,SDRM-EC-DPR算法在总收益上实现了8.82%~18.95%的增长,此外,SDRM-EC-DPR算法在提高请求接受率、降低端到端延迟、优化运行时间以及负载均衡方面也表现出了明显优势。
Objective: The combination of service function chain (SFC) and edge cloud environments presents a promising technical architecture. Edge computing, being close to data sources, can quickly process data locally, while cloud computing provides robust computational power and storage capacity. Therefore, the integration of edge and cloud computing ensures the efficient execution of real-time tasks and sufficient computing support for large-scale data processing. This setup is applicable to a variety of complex scenarios. However, deploying SFC in the edge cloud environment presents several challenges. First, VNF instances in SFC requests from different users require computing and communication resources during deployment. Second, during SFC deployment, meeting the computing and communication requirements of the SFC requests is essential, along with ensuring that computing, communication and queuing delays—from arrival to processing—satisfy the SFC's end-to-end delay requirements. Finally, the resource capacity of edge devices is limited. Therefore, given capital expenditures and operating costs, it is crucial to balance resource capacity with latency requirements to ensure the revenue of network service providers. As SFC requests arrive dynamically, the edge cloud environment must make immediate, irreversible deployment decisions for these requests. Methods: We address the problem of maximizing revenue from the online deployment of SFCs in an edge cloud environment, subject to constraints on latency, computing resources, and communication resources (SDRM-EC). To solve this problem, an algorithm based on deep reinforcement learning, SDRM-EC-PRP, is designed. First, we comprehensively modeled the physical network, SFC request, and deployment cost. In particular, the deployment cost model incorporates market supply and demand principles, accurately assessing the costs of each request based on the real-time remaining computing power of devices and available communication resources of the links. This approach eliminates device heterogeneity and helps evaluate whether the request is worth accepting. Subsequently, we formulated a Markov decision process for these models and integrated them with the dueling double deep Q-network (D3QN) algorithm to manage large-scale and complex decision processes. To optimize learning efficiency and improve sample utilization, we introduced a priority experience replay mechanism based on D3QN. Additionally, to achieve faster convergence, better stability, and enhanced adaptability to complex environments, we incorporated the random network distillation technique. Results: Simulation experiments were conducted with varying combinations of device size and SFC request quantity. The results demonstrated that, compared with the optimal offline solution, two deep reinforcement learning algorithms and one heuristic algorithm, the SDRM-EC-DPR algorithm, achieved a total revenue increase of 8.82%-18.95% and a reduction in SFC end-to-end latency by 12.5%-38.8%. Furthermore, the SDRM-EC-DPR algorithm showed significant advantages in improving the request acceptance rate, optimizing runtime, and enhancing load balancing. Conclusions: The SDRM-EC-DPR algorithm is highly effective in addressing the SDRM-EC problem, and this study demonstrates its practical value in efficiently deploying SFCs in complex edge cloud environments. This algorithm offers a practical and feasible solution for deploying service function chains in the current edge cloud landscape.
边缘云环境 / 服务功能链 / 收益 / 延迟 / 深度强化学习
edge cloud environment / service function chain / revenue / delay / deep reinforcement learning
| 1 |
|
| 2 |
|
| 3 |
|
| 4 |
|
| 5 |
|
| 6 |
|
| 7 |
|
| 8 |
|
| 9 |
|
| 10 |
王进文, 张晓丽, 李琦, 等. 网络功能虚拟化技术研究进展[J]. 计算机学报, 2019, 42 (2): 415- 436.
|
| 11 |
|
| 12 |
HARUTYUNYAN D, SHAHRIAR N, BOUTABA R, et al. Latency-aware service function chain placement in 5G mobile networks[C]// 2019 IEEE Conference on Network Softwarization (NetSoft). Paris, France: IEEE, 2019: 133-141.
|
| 13 |
|
| 14 |
|
| 15 |
|
| 16 |
|
| 17 |
|
| 18 |
|
| 19 |
|
| 20 |
|
| 21 |
|
| 22 |
|
| 23 |
邱航, 汤红波, 游伟. 基于深度Q网络的在线服务功能链部署方法[J]. 电子与信息学报, 2021, 43 (11): 3122- 3130.
|
| 24 |
|
| 25 |
LÜ F, CAI X Y, WU F, et al. Dynamic pricing scheme for edge computing services: A two-layer reinforcement learning approach[C]// 2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS). Oslo, Norway: IEEE, 2022: 1-10.
|
| 26 |
|
| 27 |
|
| 28 |
|
| 29 |
|
| 30 |
|
| 31 |
|
| 32 |
|
| 33 |
|
| 34 |
|
| 35 |
PAN P, FAN Q L, WANG S, et al. GCN-TD: A learning-based approach for service function chain deployment on the fly[C]// GLOBECOM 2020-2020 IEEE Global Communications Conference. Taipei, China: IEEE, 2020: 1-6.
|
| 36 |
|
| 37 |
WANG T F, FAN Q L, LI X H, et al. DRL-SFCP: Adaptive service function chains placement with deep reinforcement learning[C]// ICC 2021-IEEE International Conference on Communications. Montreal, QC, Canada: IEEE, 2021: 1-6.
|
| 38 |
|
| 39 |
MARTÍN-PÉREZ J, MALANDRINO F, CHIASSERINI C F, et al. OKpi: All-KPI network slicing through efficient resource allocation[C]// IEEE INFOCOM 2020-IEEE Conference on Computer Communications. Toronto, ON, Canada: IEEE, 2020: 804-813.
|
| 40 |
|
| 41 |
|
| 42 |
|
| 43 |
LI H, LI X, QIAN Z Z, et al. Resource-aware service function chain deployment in cloud-edge environment[C]// IEEE INFOCOM 2021-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). Vancouver, BC, Canada: IEEE, 2021: 1-6.
|
| 44 |
MAO Y L, SHANG X J, YANG Y Y. Joint resource management and flow scheduling for SFC deployment in hybrid edge-and-cloud network[C]// IEEE INFOCOM 2022-IEEE Conference on Computer Communications. London, United Kingdom: IEEE, 2022: 170-179.
|
| 45 |
|
| 46 |
|
| 47 |
|
| 48 |
AL-AZAD M W, SHANNIGRAHI S, STERGIOU N, et al. CLEDGE: A hybrid cloud-edge computing framework over information centric networking[C]//2021 IEEE 46th Conference on Local Computer Networks (LCN). Edmonton, Canada: IEEE, 2021: 589-596.
|
| 49 |
|
| 50 |
|
| 51 |
ZHONG Y H, ZHENG D Y, CAO X J. A DRL approach with network service deployment transformer for reliable SFC deployment[C]// ICC 2024-IEEE International Conference on Communications. Denver, CO, USA: IEEE, 2024: 177-182.
|
| 52 |
|
| 53 |
|
| 54 |
CHEN Y T, LIAO W. Mobility-aware service function chaining in 5G wireless networks with mobile edge computing[C]// ICC 2019-2019 IEEE International Conference on Communications (ICC). Shanghai, China: IEEE, 2019: 1-6.
|
| 55 |
|
| 56 |
|
/
| 〈 |
|
〉 |