Fluid Storage - Decentralized storage system

Purpose

Decentralized data storage systems allow consumers to exercise autonomous control over and access to their data. Fluid Storage wins over any centralized solution in the situation of growing country sanctions and blocking from regional operators, since it does not have a central storage.

Advantages

Compared to its closest competitor BTFS, Fluid Storage uses more advanced replication and routing technologies, allowing to reduce file access time and reduce storage costs.

Key benefits of Fluid Storage include: Storing data on user nodes, Using blockchain to store meta-information, Global interconnectivity, Fast routing using AI route optimization, Protection against data loss, Transparent data access (data loading on demand, speculative loading)

Relevance

Demand for decentralized storage is growing rapidly as a new generation of platforms increases storage flexibility and security while reducing costs for end users. Some decentralized storage networks aim to replace existing cloud storage giants, while others aim to work alongside them, expanding and improving their capabilities.

Technology

Fluid Storage is a storage system for Web 3.0:

As we know, computer networks are largely heterogeneous and represent a graph. In this case, as a rule, this is not a minimum spanning tree, but a graph with different connectivity indicators in different network segments. Thus, as a rule, there is more than one route between its individual nodes. At the same time, the speed of interaction may differ, and if we imagine this parameter as the weight of the graph edges, we can say that the task of remote access to data will be reduced to finding the optimal path in a weighted graph. However, in computer networks, we ourselves cannot directly control the routing of requests, this is done by the network itself. We can only place data on a particular node where the software under our control is installed and there are a sufficient number of resources. Additionally, it is worth considering that if we are talking about user nodes, and not about servers, then they can disappear from the network quite often, which will lead to deterioration in connectivity and loss of replicas of the data stored on them.

Fluid Storage is designed to work in heterogeneous (different traffic transfer rates) and unstable (some hosts may disappear from the network temporarily or permanently) networks to ensure seamless integration of user data. For example, an HPC cluster connected to the Fluid Storage network should easily access the requested data from other nodes (storage cluster nodes, user nodes), moreover, Fluid Storage AI algorithms, having recognized certain patterns of data access, should create an additional replica of them near the HPC consumer, or gradually move the data closer to the consumer if the network reliability does not require the creation of an additional replica

Fluid Storage includes:

• Etherium (ETH) nodes - designed to store meta-information about the location of file replicas, file access rights, status and characteristics of storage network nodes. ETH nodes can be deployed either separately or as part of client agent programs (Agent).
• Agent - a program installed on network nodes and comprising: an ETH client for receiving and storing meta-information from the Etherium network and the storage agent program itself, which manages the life cycle of files stored on nodes, executes user commands to save/delete/share files in the storage network and is responsible for exchanging data with other network agents. IPFS (libp2p) is used as a transport layer for replicating files across network agents.
• bootstrap servers - a "starter kit" of servers created by Fluid Storage developers for faster network deployment. Bootstrap servers are publicly accessible Agent nodes for creating new user agents with default connections. When Agent starts, it accesses bootstrap servers by default and receives from them the current state of the blockchain, which records all other network nodes and their availability
• relay servers - part of the Agent nodes, with good network availability characteristics and CPU/MEM power, which provide network connectivity for nodes from private networks (which are not directly accessible). Fluid Storage also uses the "port punching" mechanism on NAT routers for subsequent direct transfers after the initial data transfer using the relay server

The general approach to interaction from the user's point of view is as follows: the user sends the agent a file to save, the agent splits the file into parts, compresses and encrypts them. Using a smart contract, ETH requests suitable nodes in the khareniya network, determines the degree of replication depending on the reliability indicators and saves replicas on the network nodes. In case of loss of some replicas, FluidStorage automatically recreates them on other nodes. Agents also periodically optimize file placement based on the interaction pattern with them.

Technology Readiness Level

TRL 3: A proof-of-concept prototype of the technology has been developed to demonstrate its key features