create account

Hive Pressure #3: Catching up with the head block. by gtg

View this thread on: hive.blogpeakd.comecency.com
· @gtg ·
$173.55
Hive Pressure #3: Catching up with the head block.
Basic Hive node has a very simple configuration and with minor changes it can serve as a seed node, a witness node, a broadcaster node, a private node for your wallet (that’s what exchanges are using) or even a simple API node for your Hive microservices.

Regardless of its role, as long as a node has unrestricted network access, it will be part of the Hive p2p network, thus supporting Hive reliability and resilience.

Before your node becomes fully functional, it has to reach the head block of the blockchain.

# Get the Hive daemon
- Build it yourself from these sources:
https://github.com/openhive-network/hive (push mirror)
https://gitlab.syncad.com/hive/hive
- Use docker:
`docker pull hiveio/hive`
- Or get the binary from a trusted source:
https://gtg.openhive.network/get/bin/

# Get the blocks
The easy way or the fast way.

- Sync from the p2p network
By default, when a fresh Hive node starts, it connects to the Hive p2p network and retrieves blocks from it.
See: `--resync-blockchain`

- Get blocks yourself
Hive node can use an existing `block_log` either from another instance or from a public source such as https://gtg.openhive.network/get/blockchain
Our goal is to reach the head block as soon as possible so we chose that way.
`block_log` currently takes over 350GB, so depending on your connection and source, downloading it might take less than an hour or even half a day (for 1Gbps and 100Mbps respectively).
By default it’s expected to be located at `~/.hived/blockchain/block_log`.

# Configure your node
Configuration settings are by default in `~/.hived/config.ini`
This should be enough:
```
plugin = witness
plugin = rc

shared-file-dir = "/run/hive"
shared-file-size = 24G

flush-state-interval = 0
```
Please note that I’m using a custom location for `shared_memory.bin` file, keeping it on a `tmpfs` volume for maximum performance, make sure you have enough space there if you are going to use it.

# Process the blocks
Having all the blocks is not enough, your node needs to be aware of the current state of Hive.
Live nodes get blocks from the p2p network and process them updating state one block at a time (every three seconds), but when you start from scratch, you have to catch up.

- Snapshot
Snapshot is the fastest way because most of the job is already done.
That however will work only for compatible configurations.
We will play with snapshots another time.

- Replay
Once you have a `block_log` and `config.ini` files in place, you need to start `hived` with `--replay-blockchain`.
Replay uses the existing `block_log` to build up the shared memory file up to the highest block stored there, and then it continues with sync, up to the head block.
There's very little use of multi-threading here because every block depends on the previous one.
A lot of data is being processed, so your hardware specs really do matter here.
Not long ago Hive crossed the 55 millions block mark.
Let’s see how long does it take to replay that many blocks using different hardware specs.

`hived --force-replay --set-benchmark-interval 100000`
# Test Setups
### Alpha
> A popular workstation setup. Good enough but will run out of storage soon.
```
Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz
64GB RAM (DDR4 4x16GB 2133MHz)
2x256GB SSD in RAID0 (SAMSUNG MZ7LN256HMJP)
```
### Bravo
> Old but not obsolete. CPU released in 2014. New disks after the old ones failed.
```
Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
32GB RAM (DDR3 4x8GB 1600MHz)
2x480GB SSD in RAID0 (KINGSTON SEDC500M480G)
```
### Charlie
> The newest and the most expensive CPU in my list. Also the only AMD.
```
AMD Ryzen 5 3600
64GB RAM (DDR4 4x16GB 2666MHz)
2x512GB NVMe in RAID0 (SAMSUNG MZVLB512HBJQ)
```
### Delta
> My favorite, high quality components for serious tasks.
```
Intel(R) Xeon(R) E-2136 CPU @ 3.30GHz
64GB RAM (DDR4 4x16GB 2666MHz ECC)
2x512GB NVMe in RAID0 (WD CL SN720)
```
# Warning: spoilers ahead
What do you think? Which one will win the race?

<center>https://www.youtube.com/watch?v=vlW9lDE3DuI</center>

# Results

|Server  | [s] |  H:M:S |
|--------|-----|--------|
|Alpha   |28120|7h48m40s|
|Bravo   |26280|7h18m00s|
|Charlie |25032|6h57m12s|
|Delta   |23314|6h28m34s|

What are your `--replay` times?
πŸ‘  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 597 others
properties (23)
authorgtg
permlinkhive-pressure-3-catching-up-with-the-head-block
categoryhive-160391
json_metadata{"tags":["hivepressure","dev","witness-category","hive"],"image":["https://img.youtube.com/vi/vlW9lDE3DuI/0.jpg"],"links":["https://github.com/openhive-network/hive"],"app":"hiveblog/0.1","format":"markdown"}
created2021-07-14 20:15:09
last_update2021-07-14 20:15:09
depth0
children11
last_payout2021-07-21 20:15:09
cashout_time1969-12-31 23:59:59
total_payout_value86.818 HBD
curator_payout_value86.733 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length4,221
author_reputation461,898,288,716,338
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id104,920,352
net_rshares385,820,270,994,017
author_curate_reward""
vote details (661)
@ackza ·
$0.10
run your hive witness inside of @telosnetwork Dstor when it does computations like a3 . its just storage ATM but will soon let you run something like hived bro sdo it do it then we will have hive inside eosio  for 100s of hive chains with mega witness elections witnesses and Bps representing whoel chains going off to the united nations of blockchains lol
πŸ‘  
properties (23)
authorackza
permlinkqwau2p
categoryhive-160391
json_metadata{"users":["telosnetwork"],"app":"hiveblog/0.1"}
created2021-07-15 18:30:24
last_update2021-07-15 18:30:24
depth1
children2
last_payout2021-07-22 18:30:24
cashout_time1969-12-31 23:59:59
total_payout_value0.050 HBD
curator_payout_value0.050 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length356
author_reputation288,294,304,092,017
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id104,942,491
net_rshares226,473,298,186
author_curate_reward""
vote details (1)
@gtg ·
No, sorry to dissapoint you, it's not fast enough for Hive.
πŸ‘  
properties (23)
authorgtg
permlinkqwaw0m
categoryhive-160391
json_metadata{"app":"hiveblog/0.1"}
created2021-07-15 19:12:24
last_update2021-07-15 19:12:24
depth2
children1
last_payout2021-07-22 19:12:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length59
author_reputation461,898,288,716,338
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id104,943,180
net_rshares29,113,500,429
author_curate_reward""
vote details (1)
@ackza ·
it will be
properties (22)
authorackza
permlinkqwayaw
categoryhive-160391
json_metadata{"app":"hiveblog/0.1"}
created2021-07-15 20:01:45
last_update2021-07-15 20:01:45
depth3
children0
last_payout2021-07-22 20:01:45
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length10
author_reputation288,294,304,092,017
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id104,943,958
net_rshares0
@gtg ·
$2.53
### Epsilon
> I hate that one. Destroyer of fun.
```
AMD Ryzen 9 5950X
128GB RAM (DDR4 4x32GB 3600MHz)
4x2TB NVMe in RAID0 (Samsung SSD 980 PRO)
```
### Result
|Server  | [s] |  H:M:S |
|--------|-----|--------|
|Epsilon |18339|5h05m39s|

Courtesy of @blocktrades
πŸ‘  , , , , ,
properties (23)
authorgtg
permlinkqw9x0t
categoryhive-160391
json_metadata{"users":["blocktrades"],"app":"hiveblog/0.1"}
created2021-07-15 06:36:30
last_update2021-07-15 06:36:30
depth1
children0
last_payout2021-07-22 06:36:30
cashout_time1969-12-31 23:59:59
total_payout_value1.266 HBD
curator_payout_value1.265 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length263
author_reputation461,898,288,716,338
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id104,931,259
net_rshares5,683,729,842,705
author_curate_reward""
vote details (6)
@hivebuzz ·
Congratulations @gtg! You have completed the following achievement on the Hive blockchain and have been rewarded with new badge(s) :

<table><tr><td><img src="https://images.hive.blog/60x70/http://hivebuzz.me/@gtg/payout.png?202107200433"></td><td>You received more than 60000 HP as payout for your posts and comments.<br>Your next payout target is 62000 HP.<br><sub>The unit is Hive Power equivalent because your rewards can be split into HP and HBD</sub></td></tr>
</table>

<sub>_You can view your badges on [your board](https://hivebuzz.me/@gtg) and compare yourself to others in the [Ranking](https://hivebuzz.me/ranking)_</sub>
<sub>_If you no longer want to receive notifications, reply to this comment with the word_ `STOP`</sub>

properties (22)
authorhivebuzz
permlinkhivebuzz-notify-gtg-20210720t045832000z
categoryhive-160391
json_metadata{"image":["http://hivebuzz.me/notify.t6.png"]}
created2021-07-20 04:58:30
last_update2021-07-20 04:58:30
depth1
children0
last_payout2021-07-27 04:58:30
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length739
author_reputation369,473,330,110,275
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id105,023,821
net_rshares0
@khammall ·
$0.10
I was all in on Charlie, lol. Nice article. If you ever try this on different machines, plz post it!
πŸ‘  ,
properties (23)
authorkhammall
permlinkqwauzo
categoryhive-160391
json_metadata{"app":"hiveblog/0.1"}
created2021-07-15 18:50:12
last_update2021-07-15 18:50:12
depth1
children0
last_payout2021-07-22 18:50:12
cashout_time1969-12-31 23:59:59
total_payout_value0.052 HBD
curator_payout_value0.051 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length100
author_reputation3,792,629,426
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id104,942,816
net_rshares233,180,239,318
author_curate_reward""
vote details (2)
@meesterboom ·
$0.33
Dammit. My money was on Charlie!
πŸ‘  , , ,
properties (23)
authormeesterboom
permlinkqw966n
categoryhive-160391
json_metadata{"app":"hiveblog/0.1"}
created2021-07-14 20:56:45
last_update2021-07-14 20:56:45
depth1
children4
last_payout2021-07-21 20:56:45
cashout_time1969-12-31 23:59:59
total_payout_value0.164 HBD
curator_payout_value0.164 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length32
author_reputation1,801,848,872,753,377
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id104,920,829
net_rshares734,917,601,406
author_curate_reward""
vote details (4)
@blocktrades ·
$0.32
Epsilon (AMD 5xxx series) is suspiciously missing from this list. My money is still on properly clocked Epsilon!
πŸ‘  , ,
properties (23)
authorblocktrades
permlinkqw9sk4
categoryhive-160391
json_metadata{"app":"hiveblog/0.1"}
created2021-07-15 05:00:03
last_update2021-07-15 05:00:03
depth2
children1
last_payout2021-07-22 05:00:03
cashout_time1969-12-31 23:59:59
total_payout_value0.162 HBD
curator_payout_value0.162 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length112
author_reputation1,294,229,758,643,604
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id104,928,757
net_rshares731,512,824,872
author_curate_reward""
vote details (3)
@gtg ·
$0.21
I just got the results (as a reply to top post).
I was expecting something close to 6h because I saw before what this monster can do with full account history node. But this... I'm impressed!
πŸ‘  ,
properties (23)
authorgtg
permlinkqw9xie
categoryhive-160391
json_metadata{"app":"hiveblog/0.1"}
created2021-07-15 06:47:03
last_update2021-07-15 06:47:03
depth3
children0
last_payout2021-07-22 06:47:03
cashout_time1969-12-31 23:59:59
total_payout_value0.104 HBD
curator_payout_value0.105 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length191
author_reputation461,898,288,716,338
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id104,931,397
net_rshares474,162,069,240
author_curate_reward""
vote details (2)
@khammall ·
I was on this one with you, my friend!
πŸ‘  
properties (23)
authorkhammall
permlinkqwav13
categoryhive-160391
json_metadata{"app":"hiveblog/0.1"}
created2021-07-15 18:51:03
last_update2021-07-15 18:51:03
depth2
children0
last_payout2021-07-22 18:51:03
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length38
author_reputation3,792,629,426
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id104,942,831
net_rshares660,230,651
author_curate_reward""
vote details (1)
@xplosive ·
$0.40
Dammit. I do not even have money. πŸ˜‚
πŸ‘  , , , ,
properties (23)
authorxplosive
permlinkre-meesterboom-2021715t7352558z
categoryhive-160391
json_metadata{"tags":["ecency"],"app":"ecency/3.0.17-mobile","format":"markdown+html"}
created2021-07-15 05:35:03
last_update2021-07-15 05:35:03
depth2
children0
last_payout2021-07-22 05:35:03
cashout_time1969-12-31 23:59:59
total_payout_value0.198 HBD
curator_payout_value0.197 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length35
author_reputation211,471,736,168,502
root_title"Hive Pressure #3: Catching up with the head block."
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id104,930,089
net_rshares891,701,419,139
author_curate_reward""
vote details (5)