create account

RE: Exploring Steem Scalability by talltim

View this thread on: hive.blogpeakd.comecency.com

Viewing a response to: @timcliff/re-talltim-re-steemitblog-exploring-steem-scalability-20180410t185515723z

· @talltim ·
I didn't say that PoW didn't have scaling concerns - I'm just VERY concerned that a RAM allocation tweak is the path that DPoS is going, knowing that Proof-of-Stake has its own unique problems.

But you know what, I don't have to go down that rabbit hole -- if Steemit is still functioning a year from now with a larger user base, then I'll be wrong -- otherwise, I see this as a step down the path to ruin.
properties (22)
authortalltim
permlinkre-timcliff-re-talltim-re-steemitblog-exploring-steem-scalability-20180410t192223728z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1"}
created2018-04-10 19:23:00
last_update2018-04-10 19:23:00
depth3
children12
last_payout2018-04-17 19:23:00
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length407
author_reputation3,763,634,798,681
root_title"Exploring Steem Scalability"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id49,362,046
net_rshares0
@timcliff ·
I wouldn’t characterize this as a RAM allocation tweak. It is more of a change to _not_ use RAM, and instead use something that is more scalable: SSD.
properties (22)
authortimcliff
permlinkre-talltim-re-timcliff-re-talltim-re-steemitblog-exploring-steem-scalability-20180410t193343115z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1"}
created2018-04-10 19:33:42
last_update2018-04-10 19:33:42
depth4
children11
last_payout2018-04-17 19:33:42
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length150
author_reputation272,954,445,077,789
root_title"Exploring Steem Scalability"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id49,363,383
net_rshares0
@justinw ·
Indeed @timcliff you are correct - disk space is relatively inexpensive while RAM is much more so. By using other services to reduce our 'full nodes' to something more like a 'consensus node', we are making it possible to easily and cost effectively scale.
properties (22)
authorjustinw
permlinkre-timcliff-re-talltim-re-timcliff-re-talltim-re-steemitblog-exploring-steem-scalability-20180410t201613302z
categorysteem
json_metadata{"tags":["steem"],"users":["timcliff"],"app":"steemit/0.1"}
created2018-04-10 20:16:15
last_update2018-04-10 20:16:15
depth5
children1
last_payout2018-04-17 20:16:15
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length256
author_reputation15,502,058,309,908
root_title"Exploring Steem Scalability"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id49,368,848
net_rshares0
@smooth ·
This is an excellent point and a more succinct and clear way of expressing most of what the post was trying to say.
properties (22)
authorsmooth
permlinkre-justinw-re-timcliff-re-talltim-re-timcliff-re-talltim-re-steemitblog-exploring-steem-scalability-20180412t110022400z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1"}
created2018-04-12 11:00:24
last_update2018-04-12 11:00:24
depth6
children0
last_payout2018-04-19 11:00:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length115
author_reputation260,342,945,372,716
root_title"Exploring Steem Scalability"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id49,655,028
net_rshares0
@smooth · (edited)
$0.10
Unfortunately that won't take things very far. I slightly disagree with the post about 16 GB witness nodes even with fast SSD (and especially 8 GB). I've tried to do this (and I still have one running) and it is very painful in practice. When the memory usage doubles then 32 GB nodes will become likewise painful. 

But even if you disagree with this, it is still clear that it is going to become a problem at 3x or 4x or so. SSD is really not that much more scalable than RAM given how things work now, rather only a little (using an actual database has the potential to improve that, but that seems somewhat far away).
👍  
properties (23)
authorsmooth
permlinkre-timcliff-re-talltim-re-timcliff-re-talltim-re-steemitblog-exploring-steem-scalability-20180411t040243400z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1"}
created2018-04-11 04:02:48
last_update2018-04-11 05:06:39
depth5
children5
last_payout2018-04-18 04:02:48
cashout_time1969-12-31 23:59:59
total_payout_value0.074 HBD
curator_payout_value0.023 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length621
author_reputation260,342,945,372,716
root_title"Exploring Steem Scalability"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id49,420,822
net_rshares21,840,168,340
author_curate_reward""
vote details (1)
@andrarchy ·
One of the main concerns that the post was trying to address was that the requirements for witness and seed nodes had already reached 64 GB of RAM and were on their way to 128 and continuing to grow.

The belief that the state file should be stored in /dev/shm/, and that it was best to either rely on swapping or having enough RAM to hold the entire state file was a large part of this misconception.

You are correct that at some point in the future, if we continue to grow without making any changes, we would reach a point that 16 GB servers, and even eventually 32 GB would not be enough, but many of the changes we are working on (such as AppBase and RocksDB) are intended to address this long before we reach that point.
properties (22)
authorandrarchy
permlinkre-smooth-re-timcliff-re-talltim-re-timcliff-re-talltim-re-steemitblog-exploring-steem-scalability-20180411t211531861z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1"}
created2018-04-11 21:15:33
last_update2018-04-11 21:15:33
depth6
children2
last_payout2018-04-18 21:15:33
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length727
author_reputation230,168,201,522,782
root_title"Exploring Steem Scalability"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id49,561,325
net_rshares0
@timcliff ·
> But even if you disagree with this, it is still clear that it is going to become a problem at 3x or 4x or so. 

I don’t know enough about the inner workings to know whether this is true or not.

I obviously trust your judgment on these types of things (more than mine) but I am also not 100% convinced that it wouldn’t work to have the shared memory file grow to several hundred GB (or more), and still only require 16-32 GB of RAM.
properties (22)
authortimcliff
permlinkre-smooth-re-timcliff-re-talltim-re-timcliff-re-talltim-re-steemitblog-exploring-steem-scalability-20180411t062822822z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1"}
created2018-04-11 06:28:24
last_update2018-04-11 06:28:24
depth6
children1
last_payout2018-04-18 06:28:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length434
author_reputation272,954,445,077,789
root_title"Exploring Steem Scalability"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id49,438,080
net_rshares0
@talltim · (edited)
Oh no? Then this statement:

>Nodes that are running additional API plugins (especially account history) will require more RAM to support a larger state file.

Certainly isn't congruent at all.

Here's the thing, you're taking one strategy "Put it all in RAM" and substituting "Put it all on SSD and page INTO memory", which doesn't address why you need to do the above in the first place.

The reason you have to split things into "modules" and try to even out the load is that your full nodes using delegated-proof-of-stake will not scale any further without "further optimizations" which right now consists of "offload the stress to cheaper/slower IO device".

Its very similar to the strategies that Ethereum is trying to solve, because their blockchain is bloating way too fast *(lots of blocks filled with all kinds of junk, like cryptocats - the faddish pokemon ripoff)*.

All I'm seeing here is a fire-drill response that will result in short-term relief, but hasn't addressed the fundamental problems that exist.

Its okay, with the retention metrics being consistently crap - I think the only demand you are feeding is that of the bot armies that are diligently sucking the Reward Pool dry.

Its a bit like upgrading your email server because you have a lot of spammers hammering it. It doesn't help anything, because the root problem hasn't been solved.
properties (22)
authortalltim
permlinkre-timcliff-re-talltim-re-timcliff-re-talltim-re-steemitblog-exploring-steem-scalability-20180411t000452612z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1"}
created2018-04-11 00:05:30
last_update2018-04-11 00:06:30
depth5
children2
last_payout2018-04-18 00:05:30
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length1,364
author_reputation3,763,634,798,681
root_title"Exploring Steem Scalability"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id49,393,547
net_rshares0
@timcliff · (edited)
Sorry, but I think you are misunderstanding some of the technical aspects here. There are two different things being discussed.

If you are talking about DPoS - then you are talking about **consensus** nodes. These are the nodes that keep the blockchain state updated, and ensure that all of the new blocks are 'valid'. These nodes are covered in the "witness and seed node" section. Everything that is required for the DPoS portion of Steem to run is contained in these nodes, and the post was very clear that these nodes do **not** require the state file to be stored in RAM.

The part that you are quoting is talking about "full nodes" which get into API calls. API calls are an application layer built **on top** of the blockchain consensus rules. These nodes require more RAM because of the way the code is currently implemented, but eventually (as stated in the post), this logic for all of the non-consensus API methods will be handled separately - through things like HiveMind, SBDS, and RocksDB.

If your concern is rewards pool abuse and spam - those are valid concerns, but they are not going to be resolved in the context of "addressing the scaling requirements of the blockchain". Fixing spam and abuse issues might slow down the growth - but the ever increasing growth of the blockchain problem will still always be there - so scaling is something that still needs to be addressed regardless of what is/isn't done to address spam and abuse.
👍  
properties (23)
authortimcliff
permlinkre-talltim-re-timcliff-re-talltim-re-timcliff-re-talltim-re-steemitblog-exploring-steem-scalability-20180411t004928071z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1"}
created2018-04-11 00:48:57
last_update2018-04-11 00:56:39
depth6
children1
last_payout2018-04-18 00:48:57
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length1,454
author_reputation272,954,445,077,789
root_title"Exploring Steem Scalability"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id49,398,016
net_rshares0
author_curate_reward""
vote details (1)