create account

Hive As A Black Box - Operations by felixxx

View this thread on: hive.blogpeakd.comecency.com
· @felixxx ·
$21.23
Hive As A Black Box - Operations
## Context

In these posts I try looking at Hive as a black box.
I keep poking at it, to get some better insight on how it works.

### Hive Ops

Heard on the street: Hive has ops. 
Virtual ops on the block.
I investigate. 😎

https://images.hive.blog/0x0/https://files.peakd.com/file/peakd-hive/felixxx/23uEwkJXVQZBWGMuTPbMF8HMbrgmFb8HsRrt8uPYCJVS2xKpwunsw1C7yQ14WRrjmSxa9.png

## block_api

I've [demonstrated](/@felixxx/a-hacky-guide-to-hive-part-223-blocks-engine) how it should be possible to build a whole game engine around [block_ap.get_block_range](https://developers.hive.io/apidefinitions/#block_api.get_block_range), by just looking at all operations in a live stream.

- api.py:

```
import requests

def get_block_range(start, count, url):
    data = '{"jsonrpc":"2.0", "method":"block_api.get_block_range","params":{"starting_block_num":'+str(start)+',"count": '+str(count)+'},"id":1}'
    return requests.post(url, data)
```
Same 'api' as before. Returns the response raw.

- get_ops:

```
def get_ops_with_block_range(start, count, url):
    response = api.get_block_range(start, count, url)
    blocks = response.json()['result']['blocks']
    ops = []
    for block in blocks:
        for transaction in block['transactions']:            
            for operation in transaction['operations']:                
                ops.append(operation)
    return ops
```
Operations are in transactions, transactions are in blocks.
I need to iterate through that nest to collect all ops.

## condenser

[condenser_api.get_ops_in_block](https://developers.hive.io/apidefinitions/#condenser_api.get_ops_in_block) can also return ops. 

- api:

```
def condenser_get_ops_in_block(block, url, only_virtual):
    data = '{"jsonrpc":"2.0", "method":"condenser_api.get_ops_in_block", "params":['+str(block)+','+only_virtual+'], "id":1}'
    return requests.post(url, data)  
```
- get_ops:
```
def get_ops_with_condenser(num, url, only_virtual= 'false'):
    response = api.condenser_get_ops_in_block(num, url, only_virtual)
    ops = response.json()['result']
    return ops
```
That should return the same ops as with block_api get_ops...

### Test

```
import time

url = 'https://api.hive.blog'
print(len(get_ops_with_block_range(89040499, 1, url)))
time.sleep(2)
print(len(get_ops_with_condenser(89040499, url)))
```
returns:
```
91
143
```
## Virtual Operations

There is a difference of **52**.

[You can read more on virtual operations in the documentation](https://developers.hive.io/tutorials-recipes/virtual-operations-when-streaming-blockchain-transactions.html). btw: the examples there don't really work πŸ™„, but:
>Virtual operations (curation rewards, etc) are derived from blockchain activity, but aren’t actually stored as operations themselves. They happen based on consensus from the blockchain based on other user initiated operations. These virtual operations are NOT available on the head block, so a 100% live feed of this information would not be possible. In order then to follow these operations you would have to stream the last_irreversible_block. To get a feed of virtual operations, each of the block transactions needs to be investigated for the type of the operations.

I would explain it more like this:

>'Normal' operations are user operations. They are explicit.
They need to be signed and broadcast by a user: Alice makes a post.
Virtual operations are implicit: _When_ Alice makes a post, determines _when_ the post payout will be.
Post and curation rewards must happen exactly 7 day later and are accounted for as virtual operations.

But this is just a coding adventure, not a guide...

#### Test

```
print(len(get_ops_with_condenser(89040499, url, only_virtual = 'true')))
>> 52
```
### account_history
In more practical terms, the best endpoint I could find to get to virtual operations:
[account_history_api.enum_virtual_ops](https://developers.hive.io/apidefinitions/#account_history_api.enum_virtual_ops)
I **think** account_history plugin (not api) _must_ be activated on all nodes by default...
Information behind all of this is scarce...

- api.py:

```
def account_history_enum_virtual_ops(start, stop, url):
    data = '{"jsonrpc":"2.0", "method":"account_history_api.enum_virtual_ops", "params":{"block_range_begin":'+str(start)+',"block_range_end":'+str(stop)+'}, "id":1}'
    return requests.post(url, data) 
```
- get_ops:
```
def get_ops_with_enum(start, stop, url):
    response = api.account_history_enum_virtual_ops(start, stop, url)
    return response.json()['result']['ops']
```
Once again, the documentation is wrong.
The ['ops'] is important, as the response actually looks like this:
```
next_block_range_begin
next_operation_begin
ops
ops_by_block
```
'next' is used with 'limit' for pagination.
Anyways, after navigating to 'ops':
```
print(len(get_ops_with_enum(89040499, 89040500, url)))
>> 52
```
## Reversibility

If you actually visited the links from above, you probably noticed _reversibility_ being mentioned.
In the documentation there are parts about _head block_ and such.
Blocks used to be reversible for 1 minute. 
That was an issue. The HAF has a huge module 'fork manager' to deal with reversibility.
But the Hive protocol got updated and blocks are kind of insta-irreversible now.
...for this post I am just hacking away at operations. I don't understand it well enough to try explaining things, but I'll investigate further...

## Conclusion

There's more to it, than just looking at blocks.
Virtual operations are not accessible by block_api.
In terms of automating something: If all you ever need is user operations, [streaming](/@felixxx/a-hacky-guide-to-hive-part-222-customyoson) block_api could be reliable enough. Virtual ops are mostly dealing with delayed things like posting rewards - depending on what you want to build, it could matter...
πŸ‘  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 108 others
properties (23)
authorfelixxx
permlinkhive-as-a-black-box-operations
categorydev
json_metadata"{"app":"peakd/2024.8.7","format":"markdown","description":"Dealing with ops","tags":["dev","hive-dev","hivedev","hive","python"],"users":["felixxx"],"image":["https://files.peakd.com/file/peakd-hive/felixxx/23uEwkJXVQZBWGMuTPbMF8HMbrgmFb8HsRrt8uPYCJVS2xKpwunsw1C7yQ14WRrjmSxa9.png"]}"
created2024-09-25 06:13:18
last_update2024-09-25 06:13:18
depth0
children5
last_payout2024-10-02 06:13:18
cashout_time1969-12-31 23:59:59
total_payout_value10.624 HBD
curator_payout_value10.606 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length5,838
author_reputation217,880,960,937,226
root_title"Hive As A Black Box - Operations"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id137,377,478
net_rshares70,744,380,780,604
author_curate_reward""
vote details (172)
@arc7icwolf ·
I was asking myself what those virtual ops were since I first encountered them... thanks for explaining it!

I may have to update the scripts I was working on, because at the time I wasn't aware of this difference between the two APIs - *and I can't remember what API I used!*
properties (22)
authorarc7icwolf
permlinkre-felixxx-skcz7k
categorydev
json_metadata{"tags":["dev"],"app":"peakd/2024.8.7"}
created2024-09-25 07:52:33
last_update2024-09-25 07:52:33
depth1
children4
last_payout2024-10-02 07:52:33
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length276
author_reputation508,055,356,159,370
root_title"Hive As A Black Box - Operations"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id137,378,648
net_rshares0
@felixxx ·
btw: where are you running the scripts, if I may ask?
At home or do you rent server space? Should I write something about server rental on privex?
properties (22)
authorfelixxx
permlinkre-arc7icwolf-skd38m
categorydev
json_metadata{"tags":["dev"],"app":"peakd/2024.8.7"}
created2024-09-25 09:19:36
last_update2024-09-25 09:19:36
depth2
children3
last_payout2024-10-02 09:19:36
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length146
author_reputation217,880,960,937,226
root_title"Hive As A Black Box - Operations"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id137,379,695
net_rshares0
@arc7icwolf ·
Right now I'm only running them locally on my pc when I test them πŸ˜…

If I will ever become decent at coding I may consider to buy something small and cheap to create my own personal server, but I'd be happy to know something more about server rental :)
properties (22)
authorarc7icwolf
permlinkre-felixxx-skd3ty
categorydev
json_metadata{"tags":["dev"],"app":"peakd/2024.8.7"}
created2024-09-25 09:32:24
last_update2024-09-25 09:32:24
depth3
children2
last_payout2024-10-02 09:32:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length252
author_reputation508,055,356,159,370
root_title"Hive As A Black Box - Operations"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id137,379,842
net_rshares0