create account

RE: Updates on steem-python, SteemData, and the node situation by sneak

View this thread on: hive.blogpeakd.comecency.com

Viewing a response to: @ervin-lemark/re-furion-updates-on-steem-python-steemdata-and-the-node-situation-20170804t083602041z

· @sneak · (edited)
$0.04
> community run

> Steemit Inc. should

Which?

We already pay for and run a high-availability cluster of steemd nodes, at https://steemd.steemit.com.

What **exactly** are you proposing we do?
👍  
properties (23)
authorsneak
permlinkre-ervin-lemark-re-furion-updates-on-steem-python-steemdata-and-the-node-situation-20170805t143341254z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1","links":["https://steemd.steemit.com"]}
created2017-08-05 14:33:42
last_update2017-08-05 14:34:33
depth2
children2
last_payout2017-08-12 14:33:42
cashout_time1969-12-31 23:59:59
total_payout_value0.030 HBD
curator_payout_value0.008 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length193
author_reputation28,694,344,106,492
root_title"Updates on steem-python, SteemData, and the node situation"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id10,855,432
net_rshares10,417,711,065
author_curate_reward""
vote details (1)
@andybets · (edited)
I'm starting to understand the architecture more (I think), and am feeling more confident about the long-term future than I was, but here are some of my thoughts about what you could do:
1) Add some differentiation to the RPC server:
a) Allow it to be run to provide only the last n days worth of vote, comment and transfer data. This is all I would need for running my services on it, and I expect many others could also manage with less historic data. A smaller index surely reduces the RAM requirements significantly.
b) Allow it to run in a 'ledger only' mode, where the index could be smaller (as it would exclude all other operations). This is almost all the exchanges would need I think.
2) Communicate better about how the infrastructure works and the hardware requirements for running each part. I'd love to see some diagrams of the systems architecture.

I realise this will need some work on steemd, but over time, these increasingly demanding running requirements are centralising the network unnecessarily, making it more vulnerable, stifling community innovation and leading to doubts about scalability.

Thanks for your time.
properties (22)
authorandybets
permlinkre-sneak-re-ervin-lemark-re-furion-updates-on-steem-python-steemdata-and-the-node-situation-20170806t062920872z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1"}
created2017-08-06 06:29:21
last_update2017-08-06 06:31:48
depth3
children0
last_payout2017-08-13 06:29:21
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length1,140
author_reputation15,189,090,569,005
root_title"Updates on steem-python, SteemData, and the node situation"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id10,915,784
net_rshares0
@ervin-lemark ·
What I do miss are clear and precise instructions / documentation on how to set up and run a *community* node. Hardware resources needed, recommended, software needed, set up procedure, day to day operations needed. The n the why 's of running a node - benefits, not only financial. 

Perhaps all this is already written and published somewhere? 

All this to distribute the load from your cluster to the community.

Next, looking into the future - one word comes to mind - **sharded**. 

If it really is a requirement to have 128 GB of RAM to run a node then that's way too much. Somehow the load should be lowered to come into a reasonable range. I am not telling you anything new, I guess.

OK, I realize that my suggestions are not very constructive. I can't propose an out of the box solution. I can only voice my concern. I believe that you are working on a long term solution to keep the platform functional and scalable.

Good luck!
👍  
properties (23)
authorervin-lemark
permlinkre-sneak-re-ervin-lemark-re-furion-updates-on-steem-python-steemdata-and-the-node-situation-20170807t075745995z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1"}
created2017-08-07 07:57:51
last_update2017-08-07 07:57:51
depth3
children0
last_payout2017-08-14 07:57:51
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length940
author_reputation473,109,486,518,818
root_title"Updates on steem-python, SteemData, and the node situation"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id11,032,724
net_rshares292,181,389
author_curate_reward""
vote details (1)