The training course is read for undergraduates of the 1st year of study.
<iframe src="https://www.youtube.com/embed/videoseries?list=PLGESve048UzTbAqDil1EfsU4Lre3B09bt" width="640" height="360" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen>Big Data Technologies</iframe>
The course includes the following topics:
<ul>
<li>Hadoop</li>
<li>Spark</li>
<li>Ignite</li>
<li>Kafka</li>
<li>NoSQL</li>
<li>Docker</li>
<li>Kubernetes</li>
</ul>
<img src="/content/courses/big-data-course-qr.png" alt="image">

Big Data Technology

The course is taught for bachelors of 3-4 years of study.
<h2>About the course</h2>
The course is dedicated to the practical study of modern DevOps technologies. You will master the basic knowledge and skills of building pipelines, automating software development and operation processes, containerizing applications, deploying continuous code integration systems, building application assembly pipelines, deploying container orchestration systems, building application deployment processes on the execution platform, creating infrastructure and application monitoring systems, collecting and analyzing application logs.
<h2>You will learn</h2>
<ul>
<li>deploy continuous code integration systems</li>
<li>collect and analyze application logs</li>
<li>deploy container orchestration systems</li>
<li>create infrastructure and application monitoring systems</li>
</ul>
<h2>Topics</h2>
<ol>
<li>
Fundamentals of application containerization technology
The specifics of deploying distributed high-performance applications. Introduction to cloud services (IaaS, PaaS, SaaS). DevOps philosophy, continuous integration and code delivery. Version control system. Git. Typical operations. Repository of software code artifacts. Docker technology. Delimiting process spaces in the operating system using cgroups. Technology composition: docker files, image registry, containers, Docker daemon. File access layers. A cluster of Docker hosts. Creating a cluster. Scaling of containers. Preparing Debian-based virtual machines. Writing Dockerfile and docker-compose.yaml for local launch of React-mobi application with node.a js backend and a PostgreSQL database.
</li>
<li>
Installation and configuration of GitLabCI
Purpose and component modules of the GitLab CI assembly automation system. The format of the project build descriptor in the file .gitlab-ci.yml. Ways to scale GitLabCI to speed up the build and testing process. Installing and configuring GitLabCI in a docker-compose environment. Deploying the registry of GitLabCI Docker images.
</li>
<li>
Building applications with GitLabCI
Components of the GitLab CI pipeline, a system of stages in the GitLabCI pipeline. The pipeline starts according to the schedule. Publishing artifacts of the assembly line operation. Handling build errors. Writing.gitlab-ci.yml for testing, building, publishing a Docker image to the GitLabCI registry and local deployment of the project.
</li>
<li>
Kubernetes Orchestration System
Orchestration of large-scale application services. Problems and challenges. The composition of the Kubernetes cluster. The main elements of the deployment (Pod, Service, Route). Kybernetes cluster deployment options (mini cube, k8s, k3s, k8s managed). Managing a Kubernetes cluster via kubectl and Kubernetes Dashboard. Deploy the k3s and Kubernetes Dashboard cluster in a prepared Debian-based virtual machine.
</li>
<li>
Deploying applications to Kubernetes using GitLabCI
Automatic deployment of the application as a result of a successful build, deployment outlines. Create application deployment reports. Creating yaml deployment descriptors to the k3s cluster. Creation of elements for consolidation of deployment modules (Service) and routing (Ingress). Passing parameters via Config Map. Creating a GitLabCI pipeline for deploying an application in a Kubernetes cluster using the created descriptors.
</li>
<li>
Helm Deployment Template Engine
The release of applications, the composition of the release, the order of deployment of releases along the contours of operation. The purpose and use of the Helm template engine. Installing Helm. Creating Helm charts. The structure of Helm charts. Setting parameters (values) for the Helm chart configuration. Configuring the installation and updating of releases using Helm. Rollback a release using Helm.
Creating a GitLab CI pipeline for deploying an application in a Kubernetes cluster using helm artifacts.
</li>
<li>
Prometheus and Grafana monitoring technologies
Purpose and composition of the Prometheus-based metrics collection system. The subsystem for sending Prometheus warnings. Long-term storage of metrics, the use of PromQL for the analysis and aggregation of metrics in Prometheus. Grafana's capabilities as a Prometheus metrics visualization environment. Scaling up the monitoring infrastructure. Setting up monitoring of the operation infrastructure and applications using Prometheus and Grafana. Deployment of Prometheus and Grafana servers, deployment of node exporter to send metrics of the infrastructure (cluster nodes) to Prometheus. Creating a Grafana Dashboard. Collecting application performance metrics. Creating a GitLab CI pipeline for deploying an application in a Kubernetes cluster using helm artifacts.
</li>
<li>
Centralized collection of application logs
The purpose of centralized logging systems in cluster systems. The composition of centralized logging systems. Kafka message broker in centralized logging systems. Collecting standard container output using beat utilities. Logstash as a system for polling, filtering and transmitting messages from source to receiver. OpenSearch full-text search system as a log analysis tool. Deployment of modules of the centralized log collection system based on Logstash and OpenSearch technologies.
</li>
</ol>

DevOps Technologies

The training course is read for undergraduates of the 2nd year of study.
<iframe src="https://www.youtube.com/embed/videoseries?list=PLGESve048UzTYWps-8JjTUDThnGHOflg2" width="640" height="360" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen>Hybrid supercomputing technologies</iframe>
The course includes the following topics:
<ul>
<li>MPI</li>
<li>OpenMP</li>
<li>CUDA</li>
</ul>
<img src="/content/courses/hst-course-qr.png" alt="image">

Hybrid supercomputing technologies

Decentralized data storage systems allow consumers to access their data and perform
autonomous control over it. Fluid Storage benefits from any centralized solution
in a situation of increasing insurance sanctions and blockages from regional operators,
since it does not have a central storage.
Popularity of decentralized cloud storage is growing rapidly as a new generation of
platforms increases the flexibility and security of cloud storage, while reducing
costs for end users. Some decentralized storage networks seek to replace existing
cloud storage giants, while others seek to work together with them, expanding and
improving their capabilities.
Compared to its closest competitor BTFS, Fluid Storage uses more advanced replication
and routing technologies to reduce file access time and storage costs.

Fluid Storage - Decentralized storage system

In this project, it is proposed to develop software add-on for Kubernetes - KubeSmart.
Implemented as K8S Custom Resource Definition (CRD), these software modules should
take into account the interaction scheme of modules hosted in the cloud with each
other in order to identify their relationships and more effectively plan the use
of cloud hardware resources using machine learning methods.
The relevance of this project lies in the fact that the modern schedulers
do not have intelligent algorithms to distribute containers on nodes.
They work on the principle of placing container in the "first suitable slot".

Research projects

Fluid Storage - Decentralized storage system

KubeSmart - Intelligent PaaS system for Docker-container orchestration

Educational projects

Big Data Technology

DevOps Technologies

Hybrid supercomputing technologies

Address

Contacts