SiaPrime is a decentralized cloud storage project focusing on small and medium sized business (SMB) based on the Sia protocol. Decentralized storage networks eliminate the gatekeeper role played by traditional cloud service providers via cryptography and distributed ledger technology, while exploiting excess storage around the globe. The result is a highly durable and scalable network with cost and performance benefits surpassing traditional cloud storage. Customers always retain final control of data.
Scope / Functional Overview
The SiaPrime network is comprised of a core protocol, a P2P storage network and a marketplace mechanism to match buyers with sellers of storage. On top of this architecture is a robust interface/API to allow expected behavior for a cloud storage product.
The client splits files into chunks, then Reed-Solomon erasure coding expands each chunk into many identical pieces (The number is determined based on desired durability using an M-of-N schema). Pieces are encrypted by the client before upload to a network of disparate storage nodes ensuring data security and durability, requiring the loss of an improbable number of storage nodes client data becomes unrecoverable. With this strategy, we achieve better than 11 Nines of durability.
Smart contracts validated on a public blockchain are negotiated between client and storage nodes in a built-in competitive pricing/performance market. The blockchain uses a Proof of Work algorithm where powerful ASIC computers are required to reach consensus. Anyone may participate in mining to earn rewards in the form of a utility coin from validating blocks. Coins are used as the base currency to purchase and sell storage on the network.
- Full S3 compatibility
- Buckets/Domain addressing
- Lifecycle Rules
- Host segmenting based on key performance metrics
- Web-based and mobile applications
- Security Assertion Markup Language (SAML)
- Single sign-on (SSO)
- Fiat payment rails and currency swaps/transfers
- Customer support, sales and consulting to SMB entities
- Full featured web interface, rich API and SDKs
SiaPrime performs ongoing mapping, auditing and grading of individual storage nodes to allow the creation of ad-hoc cloud storage classes with different performance, durability and pricing characteristics. Host profiling allows incentives to build a larger, more performant network through node revenue-sharing programs in addition to storage revenues earned through basic contract mechanics.
The Relayer is a server-level installation included in customer contracts and may include multiple instances dependent on company size and locations. Relayers act as proxy-clients for end-users and perform file upload/download/delete/replace and metadata attachment. They store Bucket information and attached metadata, which is replicated onto the storage network providing additional disaster recovery and redundancy. Administrative dashboards provide an overview of authorizations, authentication, logging and monitoring. Built-in exchange functionality provides transfer, storage and swaps of cryptocurrency based on the amount of storage clients specify and pay for. The customer is not required to interact with a cryptocurrency at all.
End clients include full consensus, light-versions and mobile applications tethered to the company’s Relayer(s). An easy to use web interface allows file list, delete, upload, download, replace and move capability. Bucket allow users to apply tags, permissions and lifecycle rules to objects created. Modular configuration lets IT admins grant individual access to monitoring and accounting features along with Identity and Access management. Security Assertion Markup Language (SAML) single sign-on and 2 factor authorization provide robust protection wept along with standard whitelist/blacklist capabilities.
– Browser or app-based interface
– REST API for custom interface – customers and 3rd party applications
– Storage tiers; eg Standard, Cold and Frozen
– User authorization/authentication yesterday
– Logging, reporting capability
– Public performance/availability data replication
Storage Network Interaction
– Real-time network mapping, identification and monitoring across sia-based networks
– Ongoing QoS audit of all storage nodes
– Network reporting over specified periods
– Ad-hoc storage node groupings for classes
– Contract creation/renewals with Sia-based network hosts
– Automatically form new contracts upon completion as required
– Upload/Download/Replace files on storage nodes
– Automatic instance backup of customer metadata and upload to network
– Ongoing analytics (csv creation) for class analysis and lifecycle determination
– Ongoing file audits
– Repair operations on failed shard audits
– Node whitelist/blacklist
– Requires no customer access to crypto coin transactions
– Store object keypairs identifying contract info, nodes and sector location
– Store netmap results for a defined period
– Store individual customer metadata, user info, credentials
– Store bucket metadata
– Cache frequently accessed files (if necessary/possible, needs evaluation)
– Incremental backups are uploaded to the most durable node tier for replication
– Key functionality = Create, Delete, List. Every bucket has a unique name/id
– Top level namespace addressing /domain/bucketname
– Bucket properties include owner, date created, date edited
– Allow customer to add, delete, copy, replace, download objects (files)
– User permissions (upload, delete, replace, download, view)
– File Versioning at bucket or object level
– CORS configuration (XML file) for all buckets
– “tags” on buckets and/or objects
– Lifecycle rules at bucket or object level to move from one node tier level to another
– Customer ID account creation (if allowing 1 to 1 relationships)
– Interact with other Relayer exchange clients
– Interact with centralized Exchange APIs
– Atomic swaps of sia-based currencies
– Automatic wallet top ups based on use patters/predicted storage use
– Wallet creation for Relayer customer on all sia-based networks
– Provide transaction reporting, logging and balance information
– Requester pays (via Prepaid download mechanism)
– Provide interface for bucket and file management
– Allow bucket create, rename, delete
– Allow upload, delete, replace, download, view capabilities
– Allow permissions settings per file or per bucket
– Allow Lifecycle settings per file or per bucket
– Provide accounting features
– Provide basic network monitoring features
– Use SAML for a Single Sign-on capability
– Allow 2 Factor authentication
– Host container software installs
– Best practices guidelines
– Storage node incentive program (storage used, capacity offered)
– Node/network performance dashboards
– Minimum configuration/Recommended configuration
Performance - Scalability
Expected storage node/tier response time, data throughput on uploads and downloads as well as utilization statistics highlighting ongoing network performance and bottlenecks. Throughput and latency must meet or exceed current average performance for incumbent providers.
Scale globally with host nodes serving countries as accessible. Product should allow continued node growth to accommodate customer data storage generation with the ability to handle data surges and rapidly increasing needs.
– limited to the number of storage hosts that join the network, theoretically unlimited
Availability - Durability
24/7 with the core protocol delivering always-on functionality. Relayer availability may be tied to customer on-premise equipment.
The network provides unparalleled durability due to erasure coding thru an m of n storage schema. The result is that with nodes maintaining a simple average uptime statistic, an improbable number can fail without losing customer data
Recoverability - Maintenance
Core protocol insures against native data loss, while Relayer software creates metadata for recovery of Bucket information including file directories, links, permissions and historical access information. The Relayer instance and meta are backed up to the network allowing for complete recovery by seed from anywhere.
Provision Relayer via orchestration software making it easy for customers to install and manage. Ongoing updates will be available to download
Decentralized networks enable access to storage nodes in regions under sanction and allow difficult to stop illegal content. We will seek to prevent restricted activity via contract and with software configurations individualized for compliance with regulatory regimes all over the globe. We can also provision compliance specific Relayer configurations for specific industry regulations (ie HIPAA)
Customers own their data on our networks with nobody able to forcibly alter or delete files. Even if a customer becomes adversarial with our company, they can use the open source community edition to access data even without an ongoing contract. They lose access to specific metadata, bucket information and other features but raw file integrity will always remain intact.
Usability - Interoperability
We strive to make our software UX as customer friendly and as easy to use as possible. Providing software for an easy transition from S3, Azure and Google cloud platforms is highest goal. We expect to make ScPrime software usable across a wide array of platforms including Linux, MacOS, Windows as well as mobile device formats. If a client can use a web browser, they should be able to access our standard functionality.