边缘云环境中收益最大化的在线服务功能链部署机制

刘光远, 陈世莹, 庞紫园

清华大学学报(自然科学版) ›› 2025, Vol. 65 ›› Issue (8) : 1516-1529.

PDF(5625 KB)
PDF(5625 KB)
清华大学学报(自然科学版) ›› 2025, Vol. 65 ›› Issue (8) : 1516-1529. DOI: 10.16511/j.cnki.qhdxxb.2025.27.015
计算机科学与技术

边缘云环境中收益最大化的在线服务功能链部署机制

作者信息 +

Deployment mechanism for maximizing revenue from online service function chains in edge cloud environments

Author information +
文章历史 +

摘要

针对边缘云环境中的服务功能链部署挑战,特别是资源限制、延迟满足和收入成本之间的权衡,该文提出了在延迟、计算资源和通信资源约束下,边缘云环境中服务功能链在线部署收益最大化的问题,并设计了基于深度强化学习的算法SDRM-EC-DPR。首先,根据市场供求规律为计算设备构建了部署成本模型,以此消除设备异构性并评估请求是否值得被接受;随后,为模型构建Markov决策过程,结合对决型双重深度Q网络(dueling double deep Q-network,D3QN)算法来处理大规模且复杂的决策过程,为优化学习效率并防止陷入局部最优,在D3QN的基础上引入优先经验回放机制及随机网络蒸馏技术。仿真实验在不同设备规模和服务功能链请求数量组合下展开,结果表明,相较于对比算法,SDRM-EC-DPR算法在总收益上实现了8.82%~18.95%的增长,此外,SDRM-EC-DPR算法在提高请求接受率、降低端到端延迟、优化运行时间以及负载均衡方面也表现出了明显优势。

Abstract

Objective: The combination of service function chain (SFC) and edge cloud environments presents a promising technical architecture. Edge computing, being close to data sources, can quickly process data locally, while cloud computing provides robust computational power and storage capacity. Therefore, the integration of edge and cloud computing ensures the efficient execution of real-time tasks and sufficient computing support for large-scale data processing. This setup is applicable to a variety of complex scenarios. However, deploying SFC in the edge cloud environment presents several challenges. First, VNF instances in SFC requests from different users require computing and communication resources during deployment. Second, during SFC deployment, meeting the computing and communication requirements of the SFC requests is essential, along with ensuring that computing, communication and queuing delays—from arrival to processing—satisfy the SFC's end-to-end delay requirements. Finally, the resource capacity of edge devices is limited. Therefore, given capital expenditures and operating costs, it is crucial to balance resource capacity with latency requirements to ensure the revenue of network service providers. As SFC requests arrive dynamically, the edge cloud environment must make immediate, irreversible deployment decisions for these requests. Methods: We address the problem of maximizing revenue from the online deployment of SFCs in an edge cloud environment, subject to constraints on latency, computing resources, and communication resources (SDRM-EC). To solve this problem, an algorithm based on deep reinforcement learning, SDRM-EC-PRP, is designed. First, we comprehensively modeled the physical network, SFC request, and deployment cost. In particular, the deployment cost model incorporates market supply and demand principles, accurately assessing the costs of each request based on the real-time remaining computing power of devices and available communication resources of the links. This approach eliminates device heterogeneity and helps evaluate whether the request is worth accepting. Subsequently, we formulated a Markov decision process for these models and integrated them with the dueling double deep Q-network (D3QN) algorithm to manage large-scale and complex decision processes. To optimize learning efficiency and improve sample utilization, we introduced a priority experience replay mechanism based on D3QN. Additionally, to achieve faster convergence, better stability, and enhanced adaptability to complex environments, we incorporated the random network distillation technique. Results: Simulation experiments were conducted with varying combinations of device size and SFC request quantity. The results demonstrated that, compared with the optimal offline solution, two deep reinforcement learning algorithms and one heuristic algorithm, the SDRM-EC-DPR algorithm, achieved a total revenue increase of 8.82%-18.95% and a reduction in SFC end-to-end latency by 12.5%-38.8%. Furthermore, the SDRM-EC-DPR algorithm showed significant advantages in improving the request acceptance rate, optimizing runtime, and enhancing load balancing. Conclusions: The SDRM-EC-DPR algorithm is highly effective in addressing the SDRM-EC problem, and this study demonstrates its practical value in efficiently deploying SFCs in complex edge cloud environments. This algorithm offers a practical and feasible solution for deploying service function chains in the current edge cloud landscape.

关键词

边缘云环境 / 服务功能链 / 收益 / 延迟 / 深度强化学习

Key words

edge cloud environment / service function chain / revenue / delay / deep reinforcement learning

引用本文

导出引用
刘光远, 陈世莹, 庞紫园. 边缘云环境中收益最大化的在线服务功能链部署机制[J]. 清华大学学报(自然科学版). 2025, 65(8): 1516-1529 https://doi.org/10.16511/j.cnki.qhdxxb.2025.27.015
Guangyuan LIU, Shiying CHEN, Ziyuan PANG. Deployment mechanism for maximizing revenue from online service function chains in edge cloud environments[J]. Journal of Tsinghua University(Science and Technology). 2025, 65(8): 1516-1529 https://doi.org/10.16511/j.cnki.qhdxxb.2025.27.015
中图分类号: TP393.0   

参考文献

1
BERISHA B , MËZIU E , SHABANI I . Big data analytics in Cloud computing: An overview[J]. Journal of Cloud Computing, 2022, 11 (1): 24.
2
CARLSSON-WALL M , GORETZKI L , HOFSTEDT J , et al. Exploring the implications of cloud‐based enterprise resource planning systems for public sector management accountants[J]. Financial Accountability & Management, 2022, 38 (2): 177- 201.
3
LI X B , DARWICH M , SALEHI M A , et al. Chapter Four-A survey on cloud-based video streaming services[J]. Advances in Computers, 2021, 123, 193- 244.
4
SHARIF Z , JUNG L T , AYAZ M , et al. Smart home automation by internet-of-things edge computing platform[J]. International Journal of Advanced Computer Science and Applications, 2022, 13 (4): 474- 484.
5
NAIN G , PATTANAIK K K , SHARMA G K . Towards edge computing in intelligent manufacturing: Past, present and future[J]. Journal of Manufacturing Systems, 2022, 62, 588- 611.
6
MINH Q N , NGUYEN V H , QUY V K , et al. Edge computing for IoT-enabled smart grid: The future of energy[J]. Energies, 2022, 15 (17): 6140.
7
KRISTIANI E , YANG C T , HUANG C Y , et al. The implementation of a cloud-edge computing architecture using OpenStack and Kubernetes for air quality monitoring application[J]. Mobile Networks and Applications, 2021, 26 (3): 1070- 1092.
8
ARTHURS P , GILLAM L , KRAUSE P , et al. A taxonomy and survey of edge cloud computing for intelligent transportation systems and connected vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23 (7): 6206- 6221.
9
YANG Z M , LIANG B , JI W . An intelligent end-edge-cloud architecture for visual IoT-assisted healthcare systems[J]. IEEE Internet of Things Journal, 2021, 8 (23): 16779- 16786.
10
王进文, 张晓丽, 李琦, 等. 网络功能虚拟化技术研究进展[J]. 计算机学报, 2019, 42 (2): 415- 436.
WANG J W , ZHANG X L , LI Q , et al. Network function virtualization technology: A survey[J]. Chinese Journal of Computers, 2019, 42 (2): 415- 436.
11
MEDHAT A M , TALEB T , ELMANGOUSH A , et al. Service function chaining in next generation networks: State of the art and research challenges[J]. IEEE Communications Magazine, 2017, 55 (2): 216- 223.
12
HARUTYUNYAN D, SHAHRIAR N, BOUTABA R, et al. Latency-aware service function chain placement in 5G mobile networks[C]// 2019 IEEE Conference on Network Softwarization (NetSoft). Paris, France: IEEE, 2019: 133-141.
13
GAO T , LI X , WU Y , et al. Cost-efficient VNF placement and scheduling in public cloud networks[J]. IEEE Transactions on Communications, 2020, 68 (8): 4946- 4959.
14
YUE Y , CHENG B , WANG M , et al. Throughput optimization and delay guarantee VNF placement for mapping SFC requests in NFV-enabled networks[J]. IEEE Transactions on Network and Service Management, 2021, 18 (4): 4247- 4262.
15
WU Y T , ZHOU J H . Dynamic service function chaining orchestration in a multi-domain: a heuristic approach based on SRv6[J]. Sensors, 2021, 21 (19): 6563.
16
WANG P W , XU J , ZHOU M C , et al. Budget-constrained optimal deployment of redundant services in edge computing environment[J]. IEEE Internet of Things Journal, 2023, 10 (11): 9453- 9464.
17
ZHANG Q Z , LI C L , HUANG Y , et al. Effective multi-controller management and adaptive service deployment strategy in multi-access edge computing environment[J]. Ad Hoc Networks, 2023, 138, 103020.
18
WANG L L , DENG X H , GUI J S , et al. Microservice-oriented service placement for mobile edge computing in sustainable internet of vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24 (9): 10012- 10026.
19
LI G L , FENG B H , ZHOU H C , et al. Adaptive service function chaining mappings in 5G using deep Q-learning[J]. Computer Communications, 2020, 152, 305- 315.
20
WANG L , MAO W X , ZHAO J , et al. DDQP: A double deep Q-learning approach to online fault-tolerant SFC placement[J]. IEEE Transactions on Network and Service Management, 2021, 18 (1): 118- 132.
21
GU L , ZENG D Z , LI W , et al. Intelligent VNF orchestration and flow scheduling via model-assisted deep reinforcement learning[J]. IEEE Journal on Selected Areas in Communications, 2020, 38 (2): 279- 291.
22
PEI J N , HONG P L , PAN M , et al. Optimal VNF placement via deep reinforcement learning in SDN/NFV- enabled networks[J]. IEEE Journal on Selected Areas in Communications, 2020, 38 (2): 263- 278.
23
邱航, 汤红波, 游伟. 基于深度Q网络的在线服务功能链部署方法[J]. 电子与信息学报, 2021, 43 (11): 3122- 3130.
QIU H , TANG H B , YOU W . Online service function chain deployment method based on deep Q network[J]. Journal of Electronics & Information Technology, 2021, 43 (11): 3122- 3130.
24
KHOSHKHOLGHI M A , MAHMOODI T . Edge intelligence for service function chain deployment in NFV- enabled networks[J]. Computer Networks, 2022, 219, 109451.
25
LÜ F, CAI X Y, WU F, et al. Dynamic pricing scheme for edge computing services: A two-layer reinforcement learning approach[C]// 2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS). Oslo, Norway: IEEE, 2022: 1-10.
26
CHAI F R , ZHANG Q , YAO H P , et al. Joint multi-task offloading and resource allocation for mobile edge computing systems in satellite IoT[J]. IEEE Transactions on Vehicular Technology, 2023, 72 (6): 7783- 7795.
27
LI H P , KORDI M E . DSPPV: Dynamic service function chains placement with parallelized virtual network functions in mobile edge computing[J]. Internet of Things, 2023, 22, 100733.
28
SUN B Q , THEILE M , QIN Z Y , et al. Edge generation scheduling for DAG tasks using deep reinforcement learning[J]. IEEE Transactions on Computers, 2024, 73 (4): 1034- 1047.
29
TANG Q Q , XIE R C , FANG Z R , et al. Joint service deployment and task scheduling for satellite edge computing: a two-timescale hierarchical approach[J]. IEEE Journal on Selected Areas in Communications, 2024, 42 (5): 1063- 1079.
30
YU X , WANG R , HAO J , et al. Priority-aware deployment of autoscaling service function chains based on deep reinforcement learning[J]. IEEE Transactions on Cognitive Communications and Networking, 2024, 10 (3): 1050- 1062.
31
GUO C J , REZAEIPANAH A . Dynamic service function chains placement based on parallelized requests in edge computing environment[J]. Transactions on Emerging Telecommunications Technologies, 2024, 35 (1): e4905.
32
ZHAI D , MENG X R , YU Z H , et al. Reliability-aware service function chain backup protection method[J]. IEEE Access, 2021, 9, 14660- 14676.
33
XIE Y H , WANG S , DAI Y Y . Revenue-maximizing virtualized network function chain placement in dynamic environment[J]. Future Generation Computer Systems, 2020, 108, 650- 661.
34
ALGHAYADH F Y , RAMESH J V N , QURAISHI A , et al. Ubiquitous learning models for 5G communication network utility maximization through utility-based service function chain deployment[J]. Computers in Human Behavior, 2024, 156, 108227.
35
PAN P, FAN Q L, WANG S, et al. GCN-TD: A learning-based approach for service function chain deployment on the fly[C]// GLOBECOM 2020-2020 IEEE Global Communications Conference. Taipei, China: IEEE, 2020: 1-6.
36
FAN Q L , PAN P , LI X H , et al. DRL-D: Revenue-aware online service function chain deployment via deep reinforcement learning[J]. IEEE Transactions on Network and Service Management, 2022, 19 (4): 4531- 4545.
37
WANG T F, FAN Q L, LI X H, et al. DRL-SFCP: Adaptive service function chains placement with deep reinforcement learning[C]// ICC 2021-IEEE International Conference on Communications. Montreal, QC, Canada: IEEE, 2021: 1-6.
38
SON J , BUYYA R . Latency-aware virtualized network function provisioning for distributed edge clouds[J]. Journal of Systems and Software, 2019, 152, 24- 31.
39
MARTÍN-PÉREZ J, MALANDRINO F, CHIASSERINI C F, et al. OKpi: All-KPI network slicing through efficient resource allocation[C]// IEEE INFOCOM 2020-IEEE Conference on Computer Communications. Toronto, ON, Canada: IEEE, 2020: 804-813.
40
YANG S , LI F , TRAJANOVSKI S , et al. Delay-aware virtual network function placement and routing in edge clouds[J]. IEEE Transactions on Mobile Computing, 2021, 20 (2): 445- 459.
41
MA Y , LIANG W F , WU J , et al. Throughput maximization of NFV-enabled multicasting in mobile edge cloud networks[J]. IEEE Transactions on Parallel and Distributed Systems, 2020, 31 (2): 393- 407.
42
XU Z C , ZHANG Z H , LIANG W F , et al. QoS-aware VNF placement and service chaining for IoT applications in multi-tier mobile edge networks[J]. ACM Transactions on Sensor Networks, 2020, 16 (3): 3387705.
43
LI H, LI X, QIAN Z Z, et al. Resource-aware service function chain deployment in cloud-edge environment[C]// IEEE INFOCOM 2021-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). Vancouver, BC, Canada: IEEE, 2021: 1-6.
44
MAO Y L, SHANG X J, YANG Y Y. Joint resource management and flow scheduling for SFC deployment in hybrid edge-and-cloud network[C]// IEEE INFOCOM 2022-IEEE Conference on Computer Communications. London, United Kingdom: IEEE, 2022: 170-179.
45
SIASI N , JAESIM A , GHANI N . Service function chain provisioning schemes for multi-layer fog networks[J]. IEEE Networking Letters, 2020, 2 (1): 38- 42.
46
REN J K , YU G D , HE Y H , et al. Collaborative cloud and edge computing for latency minimization[J]. IEEE Transactions on Vehicular Technology, 2019, 68 (5): 5031- 5044.
47
PARK S H , JEONG S , NA J , et al. Collaborative cloud and edge mobile computing in C-RAN systems with minimal end-to-end latency[J]. IEEE Transactions on Signal and Information Processing over Networks, 2021, 7, 259- 274.
48
AL-AZAD M W, SHANNIGRAHI S, STERGIOU N, et al. CLEDGE: A hybrid cloud-edge computing framework over information centric networking[C]//2021 IEEE 46th Conference on Local Computer Networks (LCN). Edmonton, Canada: IEEE, 2021: 589-596.
49
ZHOU R T , LI Z P , WU C , et al. An efficient cloud market mechanism for computing jobs with soft deadlines[J]. IEEE/ACM Transactions on Networking, 2017, 25 (2): 793- 805.
50
LIU Z W , SHU Z G , CHEN S W , et al. Low-latency virtual network function scheduling algorithm based on deep reinforcement learning[J]. Computer Networks, 2024, 246, 110418.
51
ZHONG Y H, ZHENG D Y, CAO X J. A DRL approach with network service deployment transformer for reliable SFC deployment[C]// ICC 2024-IEEE International Conference on Communications. Denver, CO, USA: IEEE, 2024: 177-182.
52
HU M , XIE Z X , WU D , et al. Heterogeneous edge offloading with incomplete information: A minority game approach[J]. IEEE Transactions on Parallel and Distributed Systems, 2020, 31 (9): 2139- 2154.
53
HUANG M T , LIANG W F , SHEN X J , et al. Reliability-aware virtualized network function services provisioning in mobile edge computing[J]. IEEE Transactions on Mobile Computing, 2020, 19 (11): 2699- 2713.
54
CHEN Y T, LIAO W. Mobility-aware service function chaining in 5G wireless networks with mobile edge computing[C]// ICC 2019-2019 IEEE International Conference on Communications (ICC). Shanghai, China: IEEE, 2019: 1-6.
55
LI J , LIANG W F , MA Y . Robust service provisioning with service function chain requirements in mobile edge computing[J]. IEEE Transactions on Network and Service Management, 2021, 18 (2): 2138- 2153.
56
QIN H S , MENG T , CHEN K , et al. A comparative study of DQN and D3QN for HVAC system optimization control[J]. Energy, 2024, 307, 132740.

基金

国家重点研发计划项目(2019YFB1701403)

版权

版权所有,未经授权,不得转载。
PDF(5625 KB)

Accesses

Citation

Detail

段落导航
相关文章

/