Implementation of Kubeflow MLOps with a distributed information system of mobile agents with increased functional stability

DOI: 10.31673/2412-9070.2023.062836

Abstract

Currently, Machine Learning is one of the main tools for solving complex tasks in various fields of activity. The popularity of Machine Learning is caused by such factors as access to large data sets, the low price of computing resources, ready-made services based on cloud technologies, and rapid progress in the fields of artificial intelligence.
To develop, test and support the infrastructure of systems with data, in particular, the concepts of machine learning and processes (Machine Learning and Operation, MLOps) with a set of techniques for implementation and automatic continuous integration are applied. The concept of MLOps is considered in terms of Kubeflow tools — a cloud native system with open source code running on the Kubernetes platform. The possibilities of using the MLOps approach to improve the development processes of machine learning information systems are investigated. A distributed information system based on mobile agents acting as unmanned aerial vehicle (UAVs) is designed and their potential practical implementation in real-time conditions is briefly described.
It has been demonstrated that writing model code is only a small part of the tasks of Machine Learning, which affects the need for automation — the presence of a full-fledged pipeline of continuous integration and delivery of the application to the end user. Conducted studies have shown that Kubeflow consists of a set of various open source components that have a high level of integration with each other through the Kubernetes platform. This allows them to be launched on a variety of devices, including mobile agents such as UAVs. The use of mesh networks to increase the functional stability of the distributed information system of mobile agents is proposed and substantiated. The architecture of the system concept was designed based on the Kubernetes k3s distribution, which allows you to work in the edge computing paradigm.

Keywords: Kubeflow; Kubernetes; MLOps; mobile agents; continuous learning; machine learning; Mesh network; management systems; management efficiency; distributed information system.