create account

Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled by holger80

View this thread on: hive.blogpeakd.comecency.com
· @holger80 · (edited)
$57.45
Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled
![](https://cdn.steemitimages.com/DQmVhhf7x6ovMeN9qLV7HSPjNea7FM81rMB9Kkt7wu5sT9F/image.png)
## Repository
https://github.com/holgern/steembasicincome

## Comment upvote enabled and sbi_update_post_count.py removed
The old solution with `sbi_update_post_count.py` was checking only one account at a time. This was to slow and the `comment_upvote` flag could not be set fast enough.

The `comment_upvote` flag is set when `last_post` is older than 7 days and comments willl be upvoted until a post is written.

`last_post`, `last_comment` and `comment_upvote` are now updated in `sbi_store_member_hist.py` by:
```
for op in b.stream(start=int(start_block), stop=int(end_block), opNames=["vote", "comment"], threading=False, thread_num=8):
        if op["type"] == "comment":
            if op["author"] not in member_accounts:
                continue
            try:
                c = Comment(op, steem_instance=stm)
                c.refresh()
            except:
                continue
            main_post = c.is_main_post() 
            if main_post:
                member_data[op["author"]]["last_post"] = c["created"]
            else:
                member_data[op["author"]]["last_comment"] = c["created"]
                
            if member_data[op["author"]]["last_post"] is None:
                member_data[op["author"]]["comment_upvote"] = 1
            elif addTzInfo(member_data[op["author"]]["last_post"]) < date_7_before:
                member_data[op["author"]]["comment_upvote"] = 1
            elif member_data[op["author"]]["comment_upvote"] == 1:
                member_data[op["author"]]["comment_upvote"] = 0
            member_data[op["author"]]["updated_at"] = c["created"]
            updated_member_data.append(member_data[op["author"]])
```
where `b` is a Blockchain object from  `beem.blockchain` and `date_7_before` contains the datetime from 7 days before now.
## get_blog and comment_history replaced
As the full-node `api.steemit.com` does not support get_blog anymore, I replaced it with fetching the account history operation itself. I created a new function `get_newest` to read only the newest account history operation from the database:
```
    def get_newest(self, op_types = [], limit=100):
        ops = []
        table = self.db[self.__tablename__]
        for op in table.find(order_by='-op_acc_index'):
            if op["type"] in op_types or len(op_types) == 0:
                ops.append(op)
            if len(ops) >= limit:
                return ops
        return ops
```
This function can be used to replace `get_blog`:
```
if account["name"] == "steembasicincome":
    ops = accountTrx["sbi"].get_newest(op_types=["comment"], limit=500)
else:
    ops = accountTrx[account["name"]].get_newest(op_types=["comment"], limit=500)
blog = []
for op in ops[::-1]:
    comment = (json.loads(op["op_dict"]))    
    if comment["parent_author"] == "" and comment["author"] == account["name"] and formatTimeString(comment["timestamp"]) > addTzInfo(last_paid_post):
        try:
            c = Comment(comment, steem_instance=stm)
            c.refresh()
            blog.append(c)
        except:
            continue
```
`blog` contains now the same posts that would be returned by `get_blog`. `comment_history ` can be replaced with the same code, only 
```
 if comment["parent_author"] == "" and comment["author"] == account["name"] and formatTimeString(comment["timestamp"]) > addTzInfo(last_paid_post):
```
 is replaced by:
```
if comment["parent_author"] != "" and comment["author"] == account["name"] and formatTimeString(comment["timestamp"]) > addTzInfo(last_paid_comment):

``` 
## posts with a nsfw or sbi-skip tag will be skipped
This is hard coded at the moment:
```
skip = False
for tag in c["tags"]:
    if tag.lower() in ["nsfw", "sbi-skip"]:
        skip = True
```
The post and comment database has now a new boolean field which is set by`skip`. Only posts are upvoted with `skip == False`

## Commits
### Replace get_blog and comment_history with already stored account history ops
* [commit 733f0fb](https://github.com/holgern/steembasicincome/commit/733f0fb3009d0d9d40b74143256c2432d9d4cdfc)
* new function get_newest for receiving a limited number of account history ops from the database

### skip posts with specific tags
* [commit 7f453f4](https://github.com/holgern/steembasicincome/commit/7f453f46e290700b0eb4bf29187577216b50f9ad)
* update sql structure
* use accounts and other_accounts from database only
* skip posts with specific tags
* limit upvoted posts/comments to the last 24 hours

### last_comment and last post is took directly from the scanned blocks
* [commit 655122b](https://github.com/holgern/steembasicincome/commit/655122b526aa421f88b023ca8d8e940a99ea934b)
*  sbi_update_post_count.py removed, last_comment and last_post is took directly from the scanned blocks in sbi_store_member_hist.py
* sbi status comments improved, when comments from a member are upvoted
* post will be also upvoted when the comment_upvote flag is set

### Comment upvoting prepared
* [commit 1ab0bd1](https://github.com/holgern/steembasicincome/commit/1ab0bd1459e95b9b8cb64dee8403d8939dafacf5)
* 2x minimum_vote_threshold for comment upvoting
* sbi status adapted for comment upvoting
* bug fix for account history storage

## GitHub Account
https://github.com/holgern
πŸ‘  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 133 others
properties (23)
authorholger80
permlinkupdate-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled
categoryutopian-io
json_metadata"{"community": "busy", "app": "beempy/0.23.10", "format": "markdown", "tags": ["utopian-io", "development", "steembasicincome", "steemdev", "busy"], "users": [], "links": ["https://github.com/holgern/steembasicincome", "https://github.com/holgern/steembasicincome/commit/733f0fb3009d0d9d40b74143256c2432d9d4cdfc", "https://github.com/holgern/steembasicincome/commit/7f453f46e290700b0eb4bf29187577216b50f9ad", "https://github.com/holgern/steembasicincome/commit/655122b526aa421f88b023ca8d8e940a99ea934b", "https://github.com/holgern/steembasicincome/commit/1ab0bd1459e95b9b8cb64dee8403d8939dafacf5", "https://github.com/holgern"], "image": ["https://cdn.steemitimages.com/DQmVhhf7x6ovMeN9qLV7HSPjNea7FM81rMB9Kkt7wu5sT9F/image.png"], "canonical_url": "https://hive.blog/utopian-io/@holger80/update-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled"}"
created2018-12-30 13:43:36
last_update2020-05-23 14:59:57
depth0
children10
last_payout2019-01-06 13:43:36
cashout_time1969-12-31 23:59:59
total_payout_value43.914 HBD
curator_payout_value13.537 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length5,346
author_reputation358,857,509,568,825
root_title"Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd0
post_id77,605,729
net_rshares102,625,318,134,426
author_curate_reward""
vote details (197)
@dera123 ·
$0.03
These updates are awesome, I got surprised by sbi votes on my comments xD

Posted using [Partiko Android](https://steemit.com/@partiko-android)
πŸ‘  , , , , , ,
properties (23)
authordera123
permlinkdera123-re-holger80-update-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled-20181230t193242481z
categoryutopian-io
json_metadata{"app":"partiko"}
created2018-12-30 19:32:42
last_update2018-12-30 19:32:42
depth1
children0
last_payout2019-01-06 19:32:42
cashout_time1969-12-31 23:59:59
total_payout_value0.027 HBD
curator_payout_value0.007 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length143
author_reputation42,142,726,848,567
root_title"Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id77,620,682
net_rshares63,339,339,912
author_curate_reward""
vote details (7)
@josephsavage ·
$0.04
Your code updates do better than my status updates!

Posted using [Partiko Android](https://steemit.com/@partiko-android)
πŸ‘  ,
properties (23)
authorjosephsavage
permlinkjosephsavage-re-holger80-update-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled-20181230t153929136z
categoryutopian-io
json_metadata{"app":"partiko"}
created2018-12-30 15:39:30
last_update2018-12-30 15:39:30
depth1
children0
last_payout2019-01-06 15:39:30
cashout_time1969-12-31 23:59:59
total_payout_value0.028 HBD
curator_payout_value0.009 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length121
author_reputation39,287,695,076,524
root_title"Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id77,611,128
net_rshares67,506,565,621
author_curate_reward""
vote details (2)
@justyy ·
$15.01
Thank you for your contribution.

1. `["nsfw", "sbi-skip"]` could be configured somewhere instead of being hard-coded in the code.
2.  Although must of your code is very clean and straightforward, I find literally no comments - a few comments would be nice.
3. dead code such as `if False:` should be removed. also you have commented out code.
4. constants event it is fixed-account name such as `steembasicincome` should be defined as a constant.

Happy new year!

Your contribution has been evaluated according to [Utopian policies and guidelines](https://join.utopian.io/guidelines), as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, [click here](https://review.utopian.io/result/3/2-3-2-1-2-1-3-).

---- 
Need help? Write a ticket on https://support.utopian.io/. 
Chat with us on [Discord](https://discord.gg/uTyJkNm). 
[[utopian-moderator]](https://join.utopian.io/)
πŸ‘  , , , , , , , , , , , , , , , , , , , , ,
properties (23)
authorjustyy
permlinkre-holger80-update-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled-20181231t094039724z
categoryutopian-io
json_metadata{"tags":["utopian-io"],"links":["https://join.utopian.io/guidelines","https://review.utopian.io/result/3/2-3-2-1-2-1-3-","https://support.utopian.io/","https://discord.gg/uTyJkNm","https://join.utopian.io/"],"app":"steemit/0.1"}
created2018-12-31 09:40:42
last_update2018-12-31 09:40:42
depth1
children1
last_payout2019-01-07 09:40:42
cashout_time1969-12-31 23:59:59
total_payout_value11.331 HBD
curator_payout_value3.675 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length962
author_reputation280,616,224,641,976
root_title"Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id77,651,613
net_rshares26,644,666,638,427
author_curate_reward""
vote details (22)
@utopian-io ·
Thank you for your review, @justyy! Keep up the good work!
properties (22)
authorutopian-io
permlinkre-re-holger80-update-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled-20181231t094039724z-20190102t220529z
categoryutopian-io
json_metadata"{"app": "beem/0.20.9"}"
created2019-01-02 22:05:30
last_update2019-01-02 22:05:30
depth2
children0
last_payout2019-01-09 22:05:30
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length58
author_reputation152,955,367,999,756
root_title"Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id77,776,983
net_rshares0
@malpica1 ·
I still do not understand very well what it consists ... But we will see my respects.
properties (22)
authormalpica1
permlinkre-holger80-update-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled-20181230t140221075z
categoryutopian-io
json_metadata{"tags":["utopian-io"],"app":"steemit/0.1"}
created2018-12-30 14:02:24
last_update2018-12-30 14:02:24
depth1
children0
last_payout2019-01-06 14:02:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length85
author_reputation2,979,703,042,626
root_title"Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id77,606,503
net_rshares0
@partiko ·
Thank you so much for participating the Partiko Delegation Plan Round 1! We really appreciate your support! As part of the delegation benefits, we just gave you a 3.00% upvote! Together, let’s change the world!
properties (22)
authorpartiko
permlinkre-update-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled-20181230t143056
categoryutopian-io
json_metadata""
created2018-12-30 14:30:57
last_update2018-12-30 14:30:57
depth1
children0
last_payout2019-01-06 14:30:57
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length210
author_reputation39,207,160,334,751
root_title"Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id77,607,797
net_rshares0
@pennsif ·
This post has been included in today's  [**SOS Daily News**](https://steemit.com/steem/@pennsif/sosdailynewsnewsaboutthestateofsteem30december2018-gfwcb4fpot) - a digest of all you need to know about the State of Steem.

***

* *Editor of the [**The State of Steem SoS Daily News**](https://steemit.com/steem/@pennsif/sosdailynewsnewsaboutthestateofsteem30december2018-gfwcb4fpot).*

* *Promoter of [**The State of Steem SoS Weekly Forums**](https://steemit.com/dtube/@pennsif/amwntxt3).*

* *Editor of the [**weekly listing of steem radio shows, podcasts & social broadcasts**](https://steemit.com/mspwaves/@pennsif/schedule-of-radio-shows-podcasts-and-social-broadcasts-supported-by-dsound-week-beginning-31-december-2018).*

* *Founder of the [**A Dollar A Day**](https://steemit.com/adollaraday/@adollaraday/a-dollar-a-day-charitable-giving-project-over-usd600-donated-in-november-2018-monthly-summary-of-donations-received-and-given) charitable giving project.*

https://pennsif.com/wp-content/uploads/2018/08/adollaraday-linear-400x98.jpg

***
properties (22)
authorpennsif
permlinkre-holger80-update-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled-20181231t040354692z
categoryutopian-io
json_metadata{"tags":["utopian-io"],"image":["https://pennsif.com/wp-content/uploads/2018/08/adollaraday-linear-400x98.jpg"],"links":["https://steemit.com/steem/@pennsif/sosdailynewsnewsaboutthestateofsteem30december2018-gfwcb4fpot","https://steemit.com/dtube/@pennsif/amwntxt3","https://steemit.com/mspwaves/@pennsif/schedule-of-radio-shows-podcasts-and-social-broadcasts-supported-by-dsound-week-beginning-31-december-2018","https://steemit.com/adollaraday/@adollaraday/a-dollar-a-day-charitable-giving-project-over-usd600-donated-in-november-2018-monthly-summary-of-donations-received-and-given"],"app":"steemit/0.1"}
created2018-12-31 04:03:57
last_update2018-12-31 04:03:57
depth1
children0
last_payout2019-01-07 04:03:57
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length1,049
author_reputation636,410,097,572,565
root_title"Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id77,639,317
net_rshares0
@pryncenas ·
Cool

Posted using [Partiko Android](https://steemit.com/@partiko-android)
properties (22)
authorpryncenas
permlinkpryncenas-re-holger80-update-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled-20181230t154031528z
categoryutopian-io
json_metadata{"app":"partiko"}
created2018-12-30 15:40:30
last_update2018-12-30 15:40:30
depth1
children0
last_payout2019-01-06 15:40:30
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length74
author_reputation-143,183,413,459
root_title"Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id77,611,177
net_rshares0
@steem-ua ·
#### Hi @holger80!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your **UA** account score is currently 7.282 which ranks you at **#65** across all Steem accounts.
Your rank has improved 1 places in the last three days (old rank 66).

In our last Algorithmic Curation Round, consisting of 209 contributions, your post is ranked at **#3**. Congratulations! 
##### Evaluation of your UA score:

* Your follower network is great!
* The readers appreciate your great work!
* Good user engagement!


**Feel free to join our [@steem-ua Discord server](https://discord.gg/KpBNYGz)**
properties (22)
authorsteem-ua
permlinkre-update-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled-20181230t235628z
categoryutopian-io
json_metadata"{"app": "beem/0.20.14"}"
created2018-12-30 23:56:30
last_update2018-12-30 23:56:30
depth1
children0
last_payout2019-01-06 23:56:30
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length633
author_reputation23,214,230,978,060
root_title"Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id77,630,161
net_rshares0
@utopian-io ·
Hey, @holger80!

**Thanks for contributing on Utopian**.
We’re already looking forward to your next contribution!

**Get higher incentives and support Utopian.io!**
 Simply set @utopian.pay as a 5% (or higher) payout beneficiary on your contribution post (via [SteemPlus](https://chrome.google.com/webstore/detail/steemplus/mjbkjgcplmaneajhcbegoffkedeankaj?hl=en) or [Steeditor](https://steeditor.app)).

**Want to chat? Join us on Discord https://discord.gg/h52nFrV.**

<a href='https://steemconnect.com/sign/account-witness-vote?witness=utopian-io&approve=1'>Vote for Utopian Witness!</a>
properties (22)
authorutopian-io
permlinkre-update-for-steem-basic-income-automation-getblog-and-commenthistory-replaced-and-comment-upvoting-enabled-20181231t221437z
categoryutopian-io
json_metadata"{"app": "beem/0.20.9"}"
created2018-12-31 22:14:39
last_update2018-12-31 22:14:39
depth1
children0
last_payout2019-01-07 22:14:39
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length590
author_reputation152,955,367,999,756
root_title"Update for Steem Basic Income Automation - get_blog and comment_history replaced and comment upvoting enabled"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id77,682,899
net_rshares0