OPE3: Experimentation and Performance Evaluation for 5G and IoT
Friday, 11 June 2021, 09:30-11:00, Zoom Room
Session Chair: Brecht Vermeulen (IMEC Ghent Univ., Belgium)
Performance Evaluation of COINS Framework for Wireless Network Automation
Ivan Boškov (Jozef Stefan Institute, Slovenia); Halil Yetgin (Jozef Stefan Institute, Slovenia & Bitlis Eren University, Turkey); Carolina Fortuna (Jozef Stefan Institute, Slovenia); Mihael Mohorcic (Jozef Stefan Institute & Jozef Stefan International Postgraduate School, Slovenia)
With the evolution of mobile communications towards fifth-generation (5G) and beyond, all layers of the wireless networks are increasingly virtualized and software-controlled using automated tools. DevOps tools enabling smooth and fast testing through continuous integration (CI) and rapid deployment through continuous delivery (CD) of the software in cloud environments are permeating also communication infrastructures, eventually leading to the so-called “zero-touch network automation”. Such automation frameworks have been actively used in wireless testing infrastructures, such as FIRE and GENI. For instance, the continuous integration in wireless (COINS) technology development framework has been used in the LOG-a-TEC testbed facility of the FIRE federation. In this paper, we propose a task-based operational structure of COINS for a thorough evaluation of the time for operation (TFO) performance of each task. Moreover, we analyse the impact of Albertdifferent experiments and the number of nodes participating in the experiment on the TFO performanAlbertce of the COINS framework. The results; i) demonstrate that the device tasks within COINS framework are the bottleneck occupying up to 92.6% of the total TFO, and ii) reveal that the TFO performances of the receive details task on the device side and the gather results task on the server side are significantly diminished with the increased number of nodes participating in the same type of experiment.
A Performance Comparison of Virtualization Techniques to Deploy a 5G Monitoring Platform
Ramon Perez (Telcaria Ideas, Spain); Priscilla Benedetti (University of Perugia, Italy); Matteo Pergolesi (Telcaria Ideas, Spain); Jaime Garcia-Reinoso (Universidad Carlos III de Madrid, Spain); Aitor Zabala (Telcaria Ideas S. L., Spain); Pablo Serrano (Universidad Carlos III de Madrid, Spain); Mauro Femminella and Gianluca Reali (University of Perugia, Italy); Albert Banchs (Universidad Carlos III de Madrid, Spain)
The constant search for solutions to improve performance and resource efficiency in Cloud platforms has led to the introduction of new virtualization technologies and deployment paradigms. From container-based virtualization to micro virtual machines, new virtualization solutions claim to offer a performance close to bare-metal, with quick deployment and startup times. Furthermore, serverless computing has recently emerged as an alternative deployment model for Cloud workloads, providing scalability and cost reduction without requiring any additional configuration overhead from developers. In this work, we present a performance comparison of well-known virtualization technologies, e.g., virtual machines or containers, together with new, serverless-oriented virtualization solutions, all of them applied to deploy a real monitoring architecture for multi-site 5G platforms. Based on the results obtained from this benchmark analysis, the suitability of these new virtualization technologies for production environments is also evaluated, extracting useful conclusions to lay the ground for future work related to this novel paradigm.
A Performance Measurement Platform for C-ITS over 5G
António Serrador (Polytechnic Institute of Lisbon & ISEL, Portugal); Carlos Mendes (ISEL, Portugal); Nuno Datia (ISEL – Instituto Politécnico de Lisboa & NOVA LINCS, FCT, Universidade NOVA de Lisboa, Portugal); Nuno Cota (Instituto Superior de Engenharia de Lisboa, Portugal); Nuno Cruz (Instituto Politecnico de Lisboa & Universidade de Lisboa, Portugal); Ana Rita Beire (SOLVIT – Innovation on Telecommunications, Portugal)
This paper aims to present a new performance measurement tool dedicated to mobile cellular networks, focussed on the new C-ITS applications and challenges. Called ISEL QoS – Network Performance Evaluation (IQ-NPE), developed under the European Union 5G-MOBIX project, it is a measurement performance tool that is able to extract a wide range of performance parameters at different levels: radio, network layers and application. This is a step further when comparing with classical tools, based on mobile terminals, because C-ITS applications take advantage of cellular modems using Multiple Input Multiple Output (MIMO) advanced systems, due to available space offered by vehicles. IQ-NPE is designed to test classical alongside with the new C-ITS applications (ITS-G5 based) performance running at the Multi-access/Mobile Edge Computing (MEC) node in the new 5G architecture. All the collected data can be exported, as integrated performance data, to further be analysed using business intelligence tools. Thus, this paper presents the IQ-NPE general architecture, its main components, probes (units composed by hardware and software subsystems), together with configuration input and output examples.
Reprogramming of Embedded Devices Using Zephyr: Review and Benchmarking
João Oliveira (Fraunhofer Portugal AICOS, Portugal); Filipe Sousa (Fraunhofer Portugal, Portugal)
The Internet of Things is driving a new technological revolution that use wirelessly connected devices to sense and actuate in many use-cases of our daily lives. The development of firmware and software for this kind of devices requires specialized developers, and to update them after deployment is still an open topic of the field. This paper aims to address these issues by researching the solution that can isolate functionalities and minimize the size of updates and facilitate the deployments. A set of solutions are compared in terms of performance, update time, power consumption and footprint. Results show that scripting based approaches allow easy programming of the devices but are 270x slower than native code while consuming 75% more power. Virtual machines, although not as slow, still incur a slowdown of around 200x. Dynamic loading of native code shows the best trade-off between versatility and performance since they are only 2x times slower than native code. All approaches offer massive reductions in update size and time, with improvements of 30x and 700x, respectively. The results allow us to conclude that all solutions are valid but target different challenges and the use-case for each one needs to be carefully studied due to severe trade-offs.