Build your own StorJ node (1/9) - What is StorJ and how it works

Build your own StorJ node (1/9) - What is StorJ and how it works

By DanBlock | Tech Projects | 26 Apr 2020


What is StorJ and how it works

Welcome back to this series of articles on how to build your own super efficient StorJ node, the summary of this series can be found here.

StorJ is an opensource, decentralised file storage solution that was first conceptualised in 2014, the project aim to deliver durable and cost efficient enterprise-grade cloud storage.

Over the years they released different network versions, each one providing even better and faster service. Starting from V1 in 2014, V2 in 2016 and V3 in 2019.

The latest V3 network version delivers faster, more durable and cheaper cloud storage than its competitors like the giants AWS or Google Cloud.

The technology behind it is very impressive and they provide the service thanks to thousands of nodes managed by operators like you and me.

The platform has 3 distinct components: Nodes, Satellites and the UpLinks (or Clients)

The node and platform website is: https://storj.io/

While the paying customers use the tardigrade website to purchase the storage: https://tardigrade.io/

StorJ provide some impressive Service Level Agreements (SLA) to their clients, see below:

  • Durability SLA is 9 Nines, that's 99.9999999% (Risk of losing some data) - Current status is 100%
  • Availability SLA is 99,95% (Service is available) - Current status is 99,97%

Upload and Download speeds are also on par to AWS S3 and there are currently over 6,000 active nodes in 85 Countries, sharing over 20 PT (Petabytes).

StorJ now has over 5,000 users using the service via Tardigrade.io and also important partnership with Fluree, CNCTED and FileZilla.

Development for App Integrations to access StorJ via QNAP, Rclone and Restic has been completed and in the roadmap we also have integrations with Western Digital, Synology and FreeNAS.

Top current use cases are:

  • General backup
  • Database Backups
  • Network Attached Storage (NAS)
  • Commercial Storage for Hybrid Cloud and low latency access

One exceptional use case is for people owning a NAS that has lots of spare disk capacity; you can now install StorJ on your QNAP and monetise your spare capacity and with the coins earned you can pay to use the StorJ network to backup the data on your own NAS in the StorJ network!

2020 Roadmap for node operators

Q2

  • NAS Support for QNAP, FreeNAS
  • Linux installer and updater

Q3

  • NAS Support for WD, Synology
  • Multinode Dashboard
  • Multinode deployment

Q4

  • Storage node mobile app
  • Referral program

StorJ have regular Townhalls and project updates on their Youtube channel and on Twitter

UpLinks

The Uplink is the software or service that interact with data on the StorJ network via PUT and GET instructions; it is in broad terms a client trying to read and write data to the storage over the internet.

The UpLinks are also responsible of compressing, encrypting and cutting each file into small blocks (80 in total of which only 29 are needed to rebuild the original file) before they enter the StorJ network. This ensures fast and secure access to the data.

Satellites

Satellites are mediators between the UpLinks and the storage nodes and manage things like storage requests, payments and more. They are essentially the brain of the network.

When a storage node it's created, it registers to satellites available on the network. Satellites can be owned by StorJ or by third-parties.

Every time an UpLink tries to read or write from the network, a Satellite will intercept the request and find which storage node in the network can serve the particular request.

If it's a write request, it will take the data blocks from the UpLinks and create several copies to ensure durability and finally distribute the data blocks in nodes that are geographically distant from each other.

This ensures that in case one or more nodes in the network are not available, they can still reconstruct the data from the other data blocks. It also ensures that all the data blocks are far away from each other to avoid disruptions caused by localised geographical disasters like tornadoes.

Satellites also keep track on which storage node have which data blocks in a distributed database, constantly checking if the nodes are online and tracking data and bandwidth usage to distribute payments to the nodes.

If it's a read request, it will interrogate which nodes have the required blocks to reconstruct the file and retrieve the block from the nodes before returning the data to the UpLinks.

If a node goes offline, the satellites are also responsable of redistributing the blocks originally stored by that node to other nodes to ensure the right amount of data blocks are available on the network.

Nodes

This is where the client's data is stored. Nodes can be a multitude of different systems and the software to manage the node is lightweight and run as a docker image. Anyone with some spare capacity can rent the storage to the network and get paid in StorJ crypto in return.

Nodes are paid by the amount of storage and bandwidth used.

$1.50 per terabyte month

$20 per TB of Egress bandwidth - Data downloaded from your node to be sent back to the UpLinks requesting it

$10 per TB of Repair bandwidth  -Data download from your node used to repair data of other nodes went offline

Note: A percentage of earnings are placed in a holding account. These funds are held until a storage node operator chooses to leave the network. At 15 months, a portion of the balance is returned to the storage node operator, while the remainder is held indefinitely.

More information on the payment system and holding account can be found here: https://documentation.storj.io/resources/faq/held-back-amount

You can also use the profit calculator to estimate how much you could earn: https://storj.io/storage-node-estimator/

 

Keep reading and go to the next article of this series:

Best OS for your StorJ node and which SD card to use

How do you rate this article?


4

0


Tech Projects
Tech Projects

Articles on technology projects

Send a $0.01 microtip in crypto to the author, and earn yourself as you read!

20% to author / 80% to me.
We pay the tips from our rewards pool.