create account

update for beem: handling Could not find API exceptions and other node failures by holger80

View this thread on: hive.blogpeakd.comecency.com
· @holger80 · (edited)
$16.08
update for beem: handling Could not find API exceptions and other node failures
## Repository
https://github.com/holgern/beem<center>
![beem-logo](https://cdn.steemitimages.com/DQmcRrwLPSywSYMierfP6um6mejeMNGjN9Rxw7audJqTDgb/beem-logo)
</center>
[beem](https://github.com/holgern/beem) is a python library for steem.  The current version is 0.20.23.

There is also a discord channel for beem: https://discord.gg/4HM592V

The newest beem version can be installed by:

```
pip install -U beem
```

## Changelog

* Switch to next node, when current node has the necesary api not enabled
* handle Client returned invalid format. Expected JSON! and switch to next node
* More checks added
* get_estimated_block_num is faster and uses BlockHeader
* exclude_limited=False is default now for get_nodes

## Switching automatically to the next node on `ApiNotSupported` errors
At the moment, the full nodes have some problems and a lot nodes have disabled some apis. beem can now handle this and is automatically switching to the next node in the list.

```
from beem.account import Account
from beem import Steem
stm = Steem("https://steemd.minnowsupportproject.org")
account = Account("steemit", steem_instance=stm)
print(account.get_blog(0, 1))
```
returns the error:
```
beemapi.exceptions.ApiNotSupported: Assert Exception:api_itr != _registered_apis.end(): Could not find API follow_api
```
As there is only one node given, beem cannot switch to then next node.
When using a node list instead:


```
from beem.account import Account
from beem import Steem
from beem.nodelist import NodeList
nodelist = NodeList()
stm = Steem(node=nodelist .get_nodes())
account = Account("steemit", steem_instance=stm)
print(account.get_blog(0, 1))
```
the script returns:
```
Error: Assert Exception:api_itr != _registered_apis.end(): Could not find API follow_api
Lost connection or internal error on node: https://steemd.minnowsupportproject.org (1/100) 

[<Comment @steemit/firstpost>]
```
which means that first failing with https://steemd.minnowsupportproject.org, but then retrying sucessfully with https://rpc.usesteem.com.

## Using update_nodes

The following nodes are currently stored in NodeList:
```
from beem.nodelist import NodeList
nodelist = NodeList()
print(nodelist .get_nodes())
```
the following nodes are returned:
```
['https://steemd.minnowsupportproject.org', 'https://rpc.usesteem.com', 'https://anyx.io', 'https://api.steemit.com', 'https://steemd.privex.io', 'http://anyx.io', 'https://appbasetest.timcliff.com', 'https://api.steem.house']
```

```
from beem.nodelist import NodeList
nodelist = NodeList()
nodelist.update_nodes()
print(nodelist .get_nodes())
```

`nodelist.update_nodes()` fetches the currently working nodelist from @fullnodeupdate. I'm writing a fullnode benchmark every 3 hours to this account, which can be obtained from the json_metadata field.

Currently the following nodes are working:
```
['http://anyx.io', 'https://anyx.io', 'https://steemd.minnowsupportproject.org', 'https://api.steemit.com', 'https://rpc.usesteem.com', 'https://api.steem.house', 'https://steemd.privex.io']

```
πŸ‘  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 230 others
properties (23)
authorholger80
permlinkupdate-for-beem-handling-could-not-find-api-exceptions-and-other-node-failures
categorybeem
json_metadata"{"tags": ["beem", "python", "steemdev", "development", "palnet", "neoxian", "stem"], "users": ["fullnodeupdate"], "image": ["https://cdn.steemitimages.com/DQmcRrwLPSywSYMierfP6um6mejeMNGjN9Rxw7audJqTDgb/beem-logo"], "links": ["https://github.com/holgern/beem", "https://discord.gg/4HM592V", "https://steemd.minnowsupportproject.org", "https://rpc.usesteem.com"], "app": "beempy/0.23.10", "format": "markdown", "canonical_url": "https://hive.blog/beem/@holger80/update-for-beem-handling-could-not-find-api-exceptions-and-other-node-failures"}"
created2019-07-31 14:53:42
last_update2020-05-23 14:25:00
depth0
children9
last_payout2019-08-07 14:53:42
cashout_time1969-12-31 23:59:59
total_payout_value12.656 HBD
curator_payout_value3.421 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length3,038
author_reputation358,857,509,568,825
root_title"update for beem: handling Could not find API exceptions and other node failures"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id88,977,018
net_rshares49,576,306,994,958
author_curate_reward""
vote details (294)
@blockchainstudio ·
> handle Client returned invalid format. Expected JSON! and switch to next node

cool! I was also getting this error many times these days. What's the exact logic? a block is always fetched as a whole block? So no duplicated transactions are possible?
πŸ‘  
properties (23)
authorblockchainstudio
permlinkre-holger80-update-for-beem-handling-could-not-find-api-exceptions-and-other-node-failures-20190731t150152411z
categorybeem
json_metadata{"community":"busy","app":"busy/2.5.6","format":"markdown","tags":["beem"],"users":[],"links":[],"image":[]}
created2019-07-31 15:01:54
last_update2019-07-31 15:01:54
depth1
children2
last_payout2019-08-07 15:01:54
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length251
author_reputation178,988,499,015,921
root_title"update for beem: handling Could not find API exceptions and other node failures"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id88,977,337
net_rshares98,015,461
author_curate_reward""
vote details (1)
@holger80 ·
$0.16
When a node returns an invalid json, I'm switching the next node in list and retry the some call, so no duplicated transaction possible.
πŸ‘  , , ,
properties (23)
authorholger80
permlinkpvihza
categorybeem
json_metadata{"tags":["beem"],"app":"steemit/0.1"}
created2019-07-31 15:06:48
last_update2019-07-31 15:06:48
depth2
children1
last_payout2019-08-07 15:06:48
cashout_time1969-12-31 23:59:59
total_payout_value0.121 HBD
curator_payout_value0.040 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length136
author_reputation358,857,509,568,825
root_title"update for beem: handling Could not find API exceptions and other node failures"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id88,977,524
net_rshares478,557,195,301
author_curate_reward""
vote details (4)
@blockchainstudio ·
Thanks!
properties (22)
authorblockchainstudio
permlinkre-holger80-pvihza-20190731t191139321z
categorybeem
json_metadata{"community":"busy","app":"busy/2.5.6","format":"markdown","tags":["beem"],"users":[],"links":[],"image":[]}
created2019-07-31 19:11:39
last_update2019-07-31 19:11:39
depth3
children0
last_payout2019-08-07 19:11:39
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length7
author_reputation178,988,499,015,921
root_title"update for beem: handling Could not find API exceptions and other node failures"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id88,985,177
net_rshares0
@exhaust ·
Big fan of the beem library. Keep up the good work, brotha.
properties (22)
authorexhaust
permlinkre-holger80-pvihmt
categorybeem
json_metadata{"tags":["beem"],"app":"steempeak/1.14.12"}
created2019-07-31 14:59:18
last_update2019-07-31 14:59:18
depth1
children0
last_payout2019-08-07 14:59:18
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length59
author_reputation31,300,281,602,837
root_title"update for beem: handling Could not find API exceptions and other node failures"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id88,977,209
net_rshares0
@partiko ·
Thank you so much for participating in the Partiko Delegation Plan Round 1! We really appreciate your support! As part of the delegation benefits, we just gave you a 3.00% upvote! Together, let’s change the world!
properties (22)
authorpartiko
permlinkre-update-for-beem-handling-could-not-find-api-exceptions-and-other-node-failures-20190731t153045
categorybeem
json_metadata"{"app": "partiko"}"
created2019-07-31 15:30:48
last_update2019-07-31 15:30:48
depth1
children0
last_payout2019-08-07 15:30:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length213
author_reputation39,207,160,334,751
root_title"update for beem: handling Could not find API exceptions and other node failures"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id88,978,466
net_rshares0
@scorer ·
Thank you!

I finally started to learn Python while debuging your scripts and writing my own for STEEM automation. πŸ˜‚

For those (like me) who forget how to upgrade `pip` packages.

```sh
pip3 install beem --upgrade
```

> Successfully installed beem-0.20.23
properties (22)
authorscorer
permlinkpvij2m
categorybeem
json_metadata{"tags":["beem"],"app":"steemit/0.1"}
created2019-07-31 15:30:24
last_update2019-07-31 15:30:24
depth1
children0
last_payout2019-08-07 15:30:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length257
author_reputation5,082,149,757,589
root_title"update for beem: handling Could not find API exceptions and other node failures"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd0
post_id88,978,444
net_rshares0
@steem-plus ·
SteemPlus upvote
Hi, @holger80!

You just got a **4.3%** upvote from SteemPlus!
To get higher upvotes, earn more SteemPlus Points (SPP). On your Steemit wallet, check your SPP balance and click on "How to earn SPP?" to find out all the ways to earn.
If you're not using SteemPlus yet, please check our last posts in [here](https://steemit.com/@steem-plus) to see the many ways in which SteemPlus can improve your Steem experience on Steemit and Busy.
properties (22)
authorsteem-plus
permlinkupdate-for-beem-handling-could-not-find-api-exceptions-and-other-node-failures---vote-steemplus
categorybeem
json_metadata{}
created2019-08-01 14:23:36
last_update2019-08-01 14:23:36
depth1
children0
last_payout2019-08-08 14:23:36
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length434
author_reputation247,952,188,232,400
root_title"update for beem: handling Could not find API exceptions and other node failures"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id89,015,887
net_rshares0
@steem-ua ·
#### Hi @holger80!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your **UA** account score is currently 7.610 which ranks you at **#48** across all Steem accounts.
Your rank has improved 3 places in the last three days (old rank 51).

In our last Algorithmic Curation Round, consisting of 366 contributions, your post is ranked at **#141**.
##### Evaluation of your UA score:

* Your follower network is great!
* The readers appreciate your great work!
* Try to work on user engagement: the more people that interact with you via the comments, the higher your UA score!


**Feel free to join our [@steem-ua Discord server](https://discord.gg/KpBNYGz)**
properties (22)
authorsteem-ua
permlinkre-update-for-beem-handling-could-not-find-api-exceptions-and-other-node-failures-20190803t002753z
categorybeem
json_metadata"{"app": "beem/0.20.23"}"
created2019-08-03 00:27:54
last_update2019-08-03 00:27:54
depth1
children0
last_payout2019-08-10 00:27:54
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length710
author_reputation23,214,230,978,060
root_title"update for beem: handling Could not find API exceptions and other node failures"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id89,072,887
net_rshares0
@steemprojects ·
This post has been just added as new item to _[timeline of beem on Steem Projects](https://steemprojects.com/projects/p/beem/?utm_source=comment_timeline&utm_medium=steem&utm_campaign=new_event&utm_content=c1)_.

If you want to be notified about new updates from this project, register on Steem Projects and add beem to your favorite projects.
properties (22)
authorsteemprojects
permlinkre-update-for-beem-handling-could-not-find-api-exceptions-and-other-node-failures-20190731t165447
categorybeem
json_metadata""
created2019-07-31 16:54:48
last_update2019-07-31 16:54:48
depth1
children0
last_payout2019-08-07 16:54:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length345
author_reputation29,054,729,340
root_title"update for beem: handling Could not find API exceptions and other node failures"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id88,981,399
net_rshares0