ScPrime Cloud Storage
  • About
    • Team
    • Project Specification
  • Resources
    • Software
    • KPIs
    • Incentives
    • What are SPF?
    • Storage Network Status
    • Coin Info
    • Whitepaper
    • Explorer
    • Consensus Mining (ASIC)
    • Logo and Style Guide
    • Source Code – (Open Only)
  • Documentation
    • Xa-Miner Documentation
    • DIY Provider Documentation
  • ScPrime Articles
    • Two Papa Bravo Event
    • Updating Software for the 222800 Hard Fork
    • Hard Fork Process and Information
    • ScPrime Mining Algorithm Change Proposal
    • What are SPF?
    • SPF-B New Token Format Details
    • The Distributed Storage Competition – Part 2
    • The Distributed Storage Competition – Part 1
  • Xa-Miner
    • Xa-Miner 64TB
    • Xa-Miner 16TB
  • Menu Menu

DIY Getting Started

  • Storage Provider Overview
  • Provider Requirements/Configs
  • Port Forwarding Guide
  • Collateral
  • Licensing FAQ

Storage Provider Setup

  • Windows Command-Line Interface Guide (CLI)
  • Linux Setup
  • Synology/Docker Setup
  • Windows Supervisor-Lite Installation Guide
  • Linux Supervisor-Lite Installation Guide
  • Post-Announce Guide
  • Provider Settings
  • Windows Update Guide
  • Linux Update Guide – Basic
  • Linux Update Guide – Advanced
  • Linux Full License Software Installation Guide
  • Linux Full License Software Fresh Installation Guide
  • Uptime Robot Guide for ScPrime Storage Providers

Software/Wallets

  • Windows WebWallet Guide
  • Linux WebWallet Guide
  • Mac WebWallet Guide
  • Cold Wallet Generator Setup
  • Command-Line Interface (CLI)/Wallet Setup – Windows
  • Command-Line Interface (CLI)/Wallet Setup – Linux
  • Wallet Sweep

Tutorials

  • Linux Folder Provisioning Guide
  • Linux Metadata Guide
  • Linux Multiple Provider Guide
  • Windows to Linux Migration Guide
  • Windows UI to CLI Migration Guide
  • Grafana Provider Status Messages

Information

  • Block 109,000 Fork Technical Issues
  • Storage Provider Earnings
  • ScPrime Community FAQ
  • SCP Supply
  • SCP Supply Precision Change
  • May 15th Fork FAQ
Table of Contents
  • Linux Metadata Guide
    • Starting SPD and your Metadata
    • Using an Incorrect or Non-existent Metadata Folder
    • Where Should I Store My Metadata
    • Backing Up Your Metadata

Linux Metadata Guide

This guide will help DIY Linux users to understand what ‘metadata’ is, why it’s so important to a provider, why you should be backing it up, common mistakes made etc.

When we refer to a provider’s metadata, we are referring to a folder with multiple components in it. Together they are your metadata and are probably the most vital part of a DIY provider.

By default on Linux, metadata is stored in the user’s home folder in a hidden folder called ‘.scprime’. If you’re running as root this would be /root/.scprime or if running as a normal user it would be /home/<USER>/.scprime. In this folder we can see above we have 6 folders plus a file. While you don’t really need to fully understand all the detail as to what all the files in the metadata do, it’s useful to have a high level understanding as that can help in troubleshooting issues relates to your metadata.

apipassword – this is a file with an API password which spc uses when using the spd API. Common errors we see related to this file are caused by performing some actions with a root account and some with a user account. Errors like:

[POST request error; API authentication failed.]

are caused by running spc commands with a different user than is running spd.

consensus – this folder contains the blockchain. When you first start up a provider node you either let spd download the blockchain automatically or you can download the bootstrap file from the software download page, unzip it into the consensus folder and then start up spd and it will sync the last few remaining blocks.

gateway – this folder contains a few files pertaining to other nodes in the network

host – this folder contains various files and folders specific to your provider like the provider settings which you would have setup like pricing, collateral, upload/download pricing etc., information about contracts

siamux – this is a legacy folder no longer used so we can safely ignore this one

transactionpool – this folder contains a database of all the transactions that have taken place for the provider

wallet – this folder contains a database of all your wallet transactions

Starting SPD and your Metadata

The most important concept to take away from this guide is that each time you startup spd, it needs to be told where the metadata folder is located that it’s going to use. If you use a default location then you don’t need to worry about specifying it (and this is the best approach as it has the least chance of going wrong). If you choose to use a non-default location then you must start spd every time with the -d flag and give it the location of your metadata. So for someone using the default location, the following command would work fine:

./spd -M gctwh

However if you’ve chosen to store metadata in the /Metadata folder, then you must always start spd with the following command:

./spd -d /Metadata -M gctwh

Now another important thing to note about Linux is paths are case sensitive so the following 3 paths are all different:

/Metadata

/MetaData

/metadata

So if you specify /MetaData when you were using /Metadata previously, you will create a new provider effectively. The best way to avoid this is to setup the autostart script as documented in the Linux Setup Guide here. Once you have this setup then the easiest way to start up spd correctly every time is to just reboot your Linux PC and let the script start spd in the same way every time.

Using an Incorrect or Non-existent Metadata Folder

If you do happen to start spd with either an incorrect path to your metadata or forget to specify it at all when using a non-default location, spd will startup but will start downloading a new consensus file and so you might see your Consensus not sync’d and a lower block height than expected as below:

You might also find when you try to unlock your wallet, you get an error like below:

Could not unlock wallet: [POST request error; error when calling /wallet/unlock: [wallet has not been encrypted yet; wallet has not been encrypted yet]]

This is another indication you have created a new provider as your correct wallet would be encrypted. Also looking at the output of the following command to see your Provider ID doesn’t match what it was before.

./spc host -v

Or that you see no storage folders at the bottom of the output. These are all signs you have started spd with a new or different set of metadata. At this point you need to stop spc using:

./spc stop

Now you’ll have figure out where you have gone wrong as the situation is recoverable. If you for example add your storage folders back in, then you will lose any data in them and that won’t be recoverable.

Once spd has been stopped and you’ve worked out where your real metadata is, startup spd again pointing at the correct location and you should see your original Provider ID, be able to unlock the wallet with your password or seed phrase and see your original storage folders. At this point it’s worth then deleting the incorrect set of metadata to avoid using it by mistake in the future.

If you are unsure of where your metadata might be or you’ve created a 2nd copy and don’t know where the right one is, use the following command to help find it:

sudo find / -name consensus.db

This command will search your whole file system looking for consensus.db files, if you’ve got multiple sets of metadata then you will find multiple copies of the consensus.db file. For example:

scprime@ScPrime:~/ScPrime/ScPrime-v1.6.0-linux-amd64$ sudo find / -name consensus.db

[sudo] password for scprime:

find: ‘/run/user/1000/gvfs’: Permission denied

/mnt/USB_Backup/metadata_backup/consensus/consensus.db

/home/scprime/.scprime/consensus/consensus.db

Here we have one set of metadata in /mnt/USB_Backup/metadata_backup and another in /home/scprime/.scprime.

Where Should I Store My Metadata

As already mentioned above, the easiest thing to do is just leave it to be in its default location. This assumes that you’ve installed Linux onto an SSD and that the home folders are on that or another SSD. If you choose a non-default location, just ensure that it’s on an SDD drive. We do not recommend storing the metadata folder on a HDD, especially not the HDD used for your storage folder. The metadata is being written to a lot and needs fast read and write speeds for good performance overall of your provider.

Backing Up Your Metadata

It is highly recommended to backup your metadata folder. If anything goes wrong for example running out of disk space on the drive you are storing the metadata on then it can get corrupted and lead to spd not being able to load up and worse case you may have to start over with a new provider. A backup of your metadata can save this scenario from happening which in the future could result in a lot of lost data and therefore earnings.

It is recommended to backup at least once a day and setup an automated backup script to backup the metadata folder off the machine to either a server you have access to or at least a USB drive.

Assuming you have setup the auto startup script to start spd every time your PC reboots, it’s relatively easy to use that script as the basis for a backup script. If you’ve not setup the auto startup script it’s highly recommended to do that, documented in the Linux Setup Guide here.

For the backup script, we’ll create a file called metadata_backup.sh in the folder where spd and spc are located but it can be saved anywhere. Edit the file with Nano or Vi and paste in the following:

#!/bin/bash

SPD_DATA=$HOME/.scprime

SCPRIME=$HOME/ScPrime/ScPrime-v1.6.0-linux-amd64

# first stop spd

$SCPRIME/spc stop

# copy metadata folder to backup location

cp -r $SPD_DATA/* /mnt/USB_Backup/metadata_backup/

# start spd in the same way as normal

nohup $SCPRIME/spd --host-addr :44282 --host-api-addr :44283 -d $SPD_DATA -M gctwh &

# unlock wallet

sleep 5s

SCPRIME_WALLET_PASSWORD=`cat $HOME/ScPrime/.seed`

export SCPRIME_WALLET_PASSWORD

$SCPRIME/spc wallet unlock

Note that your backup location, command to startup spd, metadata location and spd location will very likely be different, you will need to customize this script to your setup.

Save the file and then make it executable with the following command:

chmod 755 metadata_backup.sh

The script basically stops spd from running, copies the metadata folder to our backup location, starts up spd and unlocks the wallet.

It’s important now to test the script so run it manually using the following:

./metadata_backup.sh

Once it’s completed, check the backup folder and ensure you can see a copy of your metadata there and check to ensure spd is started up and your wallet is unlocked. If so, the final step is to create a cronjob to run the script on a regular basis. To do this, enter the following command:

crontab -e

If this brings a prompt to choose an editor, select your favorite one, that will open a file in your editor which is likely just some commented text with #’s at the start of each line. Go to the bottom of the file and enter the following:

@daily /home/scprime/ScPrime/ScPrime-v1.6.0-linux-amd64/metadata_backup.sh

Save the file and exit the editor. This sets the script to run daily. The path to where your script is located will likely differ so ensure you know the full path to your backup script. For the next few days, check your backups to ensure the backups are being taken and that spd is being restarted correctly.

Another option to backup your metadata is the following script written by discord community member Tacom Ogahiehe – https://github.com/Tacombel/scprime_metadata_backup

 

What are your Feelings
Share This Article :
  • Facebook
  • Twitter
  • LinkedIn
  • Pinterest
Table of Contents
  • Linux Metadata Guide
    • Starting SPD and your Metadata
    • Using an Incorrect or Non-existent Metadata Folder
    • Where Should I Store My Metadata
    • Backing Up Your Metadata

Developer

About
Logos
Contact

Resource

Software
Explorer
Docs/Guides
Whitepaper
Open Source

Legal

Privacy Policy
Licensed Provider Terms of Service

© Copyright - SCP, Corp. - Privacy Policy
Scroll to top

This site uses cookies. By continuing to browse the site, you are agreeing to our use of cookies.

OKLearn more

Cookie and Privacy Settings



How we use cookies

We may request cookies to be set on your device. We use cookies to let us know when you visit our websites, how you interact with us, to enrich your user experience, and to customize your relationship with our website.

Click on the different category headings to find out more. You can also change some of your preferences. Note that blocking some types of cookies may impact your experience on our websites and the services we are able to offer.

Essential Website Cookies

These cookies are strictly necessary to provide you with services available through our website and to use some of its features.

Because these cookies are strictly necessary to deliver the website, refusing them will have impact how our site functions. You always can block or delete cookies by changing your browser settings and force blocking all cookies on this website. But this will always prompt you to accept/refuse cookies when revisiting our site.

We fully respect if you want to refuse cookies but to avoid asking you again and again kindly allow us to store a cookie for that. You are free to opt out any time or opt in for other cookies to get a better experience. If you refuse cookies we will remove all set cookies in our domain.

We provide you with a list of stored cookies on your computer in our domain so you can check what we stored. Due to security reasons we are not able to show or modify cookies from other domains. You can check these in your browser security settings.

Google Analytics Cookies

These cookies collect information that is used either in aggregate form to help us understand how our website is being used or how effective our marketing campaigns are, or to help us customize our website and application for you in order to enhance your experience.

If you do not want that we track your visit to our site you can disable tracking in your browser here:

Other external services

We also use different external services like Google Webfonts, Google Maps, and external Video providers. Since these providers may collect personal data like your IP address we allow you to block them here. Please be aware that this might heavily reduce the functionality and appearance of our site. Changes will take effect once you reload the page.

Google Webfont Settings:

Google Map Settings:

Google reCaptcha Settings:

Vimeo and Youtube video embeds:

Other cookies

The following cookies are also needed - You can choose if you want to allow them:

Privacy Policy

You can read about our cookies and privacy settings in detail on our Privacy Policy Page.

Privacy
Accept settingsHide notification only