create account

Best Practice Running Steemd v0.16.0 by abit

View this thread on: hive.blog |  peakd.com |  ecency.com
witness-category · @abit · (edited)
$48.00
Best Practice Running Steemd v0.16.0
This has been mentioned in many discussions. I don't know if it's been posted here by someone.

I think you know what I'm talking about, so no basic tutorial here. I'm running the node in Linux.

Background:
The steemd v0.16.0 release is built on ChainBase library, which primarily brought disk IO issues if running with default settings, even with the kernel parameter tweaks recommended by the dev team in the release blog.

To get around those issues, the best practice (IMHO) is to keep the "shared file" data in memory. The second best is to use the swapping system provided by the OS which has better performance than storing that data on disk directly (by default).

Requirements:
* For low memory nodes, best to have more than 12GB of free RAM, if not, use some swap space, for example 6G RAM and 6G swap
* For full nodes, I have no experience, so try it yourself, or perhaps others will tell you.

How-to:
```
# change the size of /dev/shm
sudo mount -o remount,size=10G /dev/shm
# run steemd
./steemd --shared-file-dir /dev/shm/
```

From my experience, with a good block_log file (pre-downloaded or synced), it takes about 10 minutes to finish replaying on a low_mem node. If sync from scratch, it's expected to take less than a hour.

//Edit 1:
When I was saying "low memory node", actually additionally I mean only the "witness" plugin is enabled, aka in `config.ini` you have:
```
enable-plugin = witness
```
but not (by default):
```
enable-plugin = witness account_history account_by_key
```

//Edit 2:
A low memory node now need around 8GB space for the shared file, so it's safe to set the file size to 9GB so far (in config.ini, listed below). By default the limit is set to 32G, but it's not fully pre-allocated, so shouldn't matter if not change it. Usually I set it to 12G.
```
shared-file-size = 12G
```

//Edit 3:
If you forgot to change the `enable-plugin` setting at the first place, you may run into `Bus error` which will crash the node. In this case, you need to restart `steemd` with `--replay` parameter:
```
./steemd --shared-file-dir /dev/shm/ --replay
```
πŸ‘  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 219 others
properties (23)
authorabit
permlinkbest-practice-running-steemd-v0-16-0
categorywitness-category
json_metadata{"tags":["witness-category","steem","steemd","mining"],"app":"steemit/0.1","format":"markdown"}
created2016-12-09 23:09:57
last_update2016-12-11 17:02:45
depth0
children29
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value44.748 HBD
curator_payout_value3.249 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length2,089
author_reputation141,171,499,037,785
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,965,112
net_rshares74,571,900,003,237
author_curate_reward""
vote details (283)
@bola · 
I was getting this error as well, figured it out two days ago but wish i seen your post earlier.
Thanks for the great info this should help many people out.
properties (22)
authorbola
permlinkre-abit-best-practice-running-steemd-v0-16-0-20161215t025816618z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-15 02:58:18
last_update2016-12-15 02:58:18
depth1
children0
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length156
author_reputation51,245,914,991,562
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id2,007,484
net_rshares0
@clayop · 
Good info!
properties (22)
authorclayop
permlinkre-abit-best-practice-running-steemd-v0-16-0-20161209t231048350z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-09 23:10:51
last_update2016-12-09 23:10:51
depth1
children0
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length10
author_reputation270,845,899,918,618
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,965,121
net_rshares0
@codydeeds · 
Not to bother with a basic question but what are the advantages of running a node? From my basic understanding for a node to profit it must be voted as a witness? I have some serious hardware at my disposal and static public IP's that could be used. Just need some literature on what the situation is if you have any links it is appreciated. I messed around in the github to get an understanding of how to run things locally in case the actual site went down. That is as far as I have gotten. Thanks!
πŸ‘  
properties (23)
authorcodydeeds
permlinkre-abit-best-practice-running-steemd-v0-16-0-20161212t025518482z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-12 02:55:18
last_update2016-12-12 02:55:18
depth1
children2
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length500
author_reputation38,090,771,215,071
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,981,615
net_rshares1,319,812,168
author_curate_reward""
vote details (1)
@abit · 
It's true that not everyone need to run a node. Witnesses and standard miners need to run nodes. And yes you get the point, you can run things locally (although I haven't tried to run a GUI locally, others do). Profitable or not is another topic. In regards to links, I'm not sure what you are looking for exactly, perhaps you can join https://steemit.chat/ and ask in #witness or #dev channel.
πŸ‘  
properties (23)
authorabit
permlinkre-codydeeds-re-abit-best-practice-running-steemd-v0-16-0-20161212t210234687z
categorywitness-category
json_metadata{"tags":["witness","dev","witness-category"],"links":["https://steemit.chat/"]}
created2016-12-12 21:02:33
last_update2016-12-12 21:02:33
depth2
children1
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length394
author_reputation141,171,499,037,785
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,988,095
net_rshares1,411,054,013
author_curate_reward""
vote details (1)
@codydeeds · 
Thanks, will check it out!
πŸ‘  
properties (23)
authorcodydeeds
permlinkre-abit-re-codydeeds-re-abit-best-practice-running-steemd-v0-16-0-20161212t210743986z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-12 21:07:45
last_update2016-12-12 21:07:45
depth3
children0
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length26
author_reputation38,090,771,215,071
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,988,146
net_rshares1,411,054,013
author_curate_reward""
vote details (1)
@felixxx · 
Thanks for posting this.
There are no posts about this at all.

It doesn't work for me, though.
Steemd still tells me  30G memory free (That's on the main partition)
properties (22)
authorfelixxx
permlinkre-abit-best-practice-running-steemd-v0-16-0-20161209t232253576z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-09 23:22:54
last_update2016-12-09 23:22:54
depth1
children9
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length165
author_reputation218,698,764,507,703
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,965,204
net_rshares0
@abit · 
The messages about "30G memory free" doesn't matter. Check whether the files are created in /dev/shm by commands:
```
ls -al /dev/shm/
du -hs /dev/shm/
```
properties (22)
authorabit
permlinkre-felixxx-re-abit-best-practice-running-steemd-v0-16-0-20161209t233548616z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-09 23:36:06
last_update2016-12-09 23:36:06
depth2
children8
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length155
author_reputation141,171,499,037,785
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,965,305
net_rshares0
@felixxx · 
Yeah, I think they were being created.
It also went a lot faster.

After 85% it crashes now.

    Bus error (core dumped)
πŸ‘  
properties (23)
authorfelixxx
permlinkre-abit-re-felixxx-re-abit-best-practice-running-steemd-v0-16-0-20161210t002133171z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-10 00:21:33
last_update2016-12-10 00:21:33
depth3
children7
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length121
author_reputation218,698,764,507,703
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,965,538
net_rshares2,604,332,431
author_curate_reward""
vote details (1)
@fydel · 
But how to use the cli_wallet after disabling account_history and account_by_key?
properties (22)
authorfydel
permlinkre-abit-best-practice-running-steemd-v0-16-0-20161210t191708126z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-10 19:17:09
last_update2016-12-10 19:17:09
depth1
children4
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length81
author_reputation176,564,412,139
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,970,982
net_rshares0
@smooth · (edited)
$0.16
You can probably still use most of the functions but you won't be able to use list_my_accounts or get_account_history

account_by_key doesn't use a huge amount of memory, so you can leave it enabled if you want. account_history does use a lot of memory, but you can eliminate almost all of that usage by only tracking your own account with `--track-account-range`

However, if you are running a witness node (signing blocks) you should not use any extra plugins, even these. Better to run another node for wallet purposes if you need it.
πŸ‘  ,
properties (23)
authorsmooth
permlinkre-fydel-re-abit-best-practice-running-steemd-v0-16-0-20161210t200007800z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-10 20:00:09
last_update2016-12-10 20:06:27
depth2
children3
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.158 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length537
author_reputation260,342,945,372,716
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,971,299
net_rshares3,061,784,535,750
author_curate_reward""
vote details (2)
@fydel · 
Ah. Ok. Thanks for the  info.
properties (22)
authorfydel
permlinkre-smooth-re-fydel-re-abit-best-practice-running-steemd-v0-16-0-20161210t204509111z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-10 20:45:09
last_update2016-12-10 20:45:09
depth3
children0
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length29
author_reputation176,564,412,139
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,971,614
net_rshares0
@throw-away911 · 
http://i.imgur.com/ZYoVy5l.png
http://i.imgur.com/PrhjHuE.png
http://i.imgur.com/KnjsIuS.png
http://i.imgur.com/YjK7SfK.png
# smooth...
properties (22)
authorthrow-away911
permlinkre-smooth-re-fydel-re-abit-best-practice-running-steemd-v0-16-0-20161210t221935828z
categorywitness-category
json_metadata{"tags":["witness-category"],"image":["http://i.imgur.com/ZYoVy5l.png","http://i.imgur.com/PrhjHuE.png","http://i.imgur.com/KnjsIuS.png","http://i.imgur.com/YjK7SfK.png"]}
created2016-12-10 22:19:36
last_update2016-12-10 22:19:36
depth3
children1
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length135
author_reputation175,012,377,966
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,972,209
net_rshares0
@kentchun33333 · 
Is it posible for adapting SSD ?
properties (22)
authorkentchun33333
permlinkre-abit-best-practice-running-steemd-v0-16-0-20170427t071955496z
categorywitness-category
json_metadata{"tags":["witness-category"],"app":"steemit/0.1"}
created2017-04-27 07:19:45
last_update2017-04-27 07:19:45
depth1
children1
last_payout2017-05-04 07:19:45
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length32
author_reputation0
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id3,128,744
net_rshares0
@abit · 
Performance of v0.16.1 and later is much better already, so I guess SSD would be OK, although I'd still prefer RAM.

With v0.16.0, SSD of dedicated server would be OK, but not as good as RAM. Most VPS's aren't with full-powered SSD due to resource shrinking/limiting, so it was not an option.
πŸ‘Ž  , ,
properties (23)
authorabit
permlinkre-kentchun33333-re-abit-best-practice-running-steemd-v0-16-0-20170429t123614573z
categorywitness-category
json_metadata{"tags":["witness-category"],"app":"steemit/0.1"}
created2017-04-29 12:36:12
last_update2017-04-29 12:36:12
depth2
children0
last_payout2017-05-06 12:36:12
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length292
author_reputation141,171,499,037,785
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id3,151,813
net_rshares-140,773,281,847
author_curate_reward""
vote details (3)
@linkback-bot-v0 · 
This post has been linked to from another place on Steem.


  - [Steemit Bergbau #2](https://steemit.com/deutsch/@deutschbot/steemit-bergbau-2) by @deutschbot

  - [Fighting Steem Daemon #5](https://steemit.com/mining/@felixxx/fighting-steem-daemon-5) by @felixxx




Learn more about and upvote to support [**linkback bot v0.5**](https://steemit.com/steemit/@ontofractal/steem-linkback-bot-v0-5-the-reddit-awareness-release). Flag this comment if you don't want the bot to continue posting linkbacks for your posts.

Built by @ontofractal
properties (22)
authorlinkback-bot-v0
permlinkre-abit-best-practice-running-steemd-v0-16-0-linkbacks
categorywitness-category
json_metadata{}
created2016-12-12 22:41:00
last_update2016-12-12 22:41:00
depth1
children0
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length540
author_reputation1,915,954,976,722
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,988,977
net_rshares0
@mondeja · 
Good info. I have a question: Are there any way  for send founds programatically without run your own node? Thanks!
properties (22)
authormondeja
permlinkre-abit-best-practice-running-steemd-v0-16-0-20161211t195226047z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-11 19:52:33
last_update2016-12-11 19:52:33
depth1
children2
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length115
author_reputation9,894,814,339,118
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,978,653
net_rshares0
@abit · 
You can use `cli_wallet` to connect to a public websocket server, for example (https://steemit.com/steemws/@jesta/steem-ws-the-public-steem-api-cluster):
> ./cli_wallet -s wss://node.steem.ws -H 127.0.0.1:8091 --rpc-http-allowip 127.0.0.1

Then  connect to the `cli_wallet` with your own program.

You can also use piston (https://piston.readthedocs.io/en/stable/).
πŸ‘  
properties (23)
authorabit
permlinkre-mondeja-re-abit-best-practice-running-steemd-v0-16-0-20161212t013952342z
categorywitness-category
json_metadata{"tags":["witness-category"],"links":["https://steemit.com/steemws/@jesta/steem-ws-the-public-steem-api-cluster","https://piston.readthedocs.io/en/stable/"]}
created2016-12-12 01:39:51
last_update2016-12-12 01:39:51
depth2
children1
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length365
author_reputation141,171,499,037,785
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,981,192
net_rshares3,578,830,611
author_curate_reward""
vote details (1)
@mondeja · 
Oh, thanks you very much! What noob that I am...
properties (22)
authormondeja
permlinkre-abit-re-mondeja-re-abit-best-practice-running-steemd-v0-16-0-20161212t124538314z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-12 12:45:24
last_update2016-12-12 12:45:24
depth3
children0
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length48
author_reputation9,894,814,339,118
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,984,300
net_rshares0
@virtualgrowth · 
Thanks for sharing.
Shared on [twitter](https://twitter.com/Steem_Land/status/807362798091169797)
properties (22)
authorvirtualgrowth
permlinkre-abit-best-practice-running-steemd-v0-16-0-20161209t231643808z
categorywitness-category
json_metadata{"tags":["witness-category"],"links":["https://twitter.com/Steem_Land/status/807362798091169797"]}
created2016-12-09 23:16:45
last_update2016-12-09 23:16:45
depth1
children2
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length97
author_reputation194,279,551,234,595
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,965,160
net_rshares0
@abit · 
Thanks.
properties (22)
authorabit
permlinkre-virtualgrowth-re-abit-best-practice-running-steemd-v0-16-0-20161209t231916539z
categorywitness-category
json_metadata{"tags":["witness-category"]}
created2016-12-09 23:19:36
last_update2016-12-09 23:19:36
depth2
children0
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length7
author_reputation141,171,499,037,785
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,965,180
net_rshares0
@twitterbot · 
### ![Steem_Land](https://pbs.twimg.com/profile_images/796481381366804481/GZiD2MS0_normal.jpg) **[Steem_Land](https://twitter.com/@Steem_Land/status/807362798091169797)** tweeted @ 09 Dec 2016 - 23:14 UTC

> Best Practice Running Steemd v0.16.0 β€” Steemit [steemit.com/witness-catego…](https://t.co/QZvBZGZB1H)
@SteemUps @SteemitPosts @steemit @steemiobot


###### *Disclaimer: I am just a bot trying to be helpful.*
properties (22)
authortwitterbot
permlinkre-re-abit-best-practice-running-steemd-v0-16-0-20161209t231643808z-20161209t231746
categorywitness-category
json_metadata""
created2016-12-09 23:17:45
last_update2016-12-09 23:17:45
depth2
children0
last_payout2017-01-10 01:45:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length415
author_reputation2,792,128,643,772
root_title"Best Practice Running Steemd v0.16.0"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id1,965,169
net_rshares0
Help/Feedback