I have been building a blockchain parser for my [Steem Fundamental Analysis](https://steemit.com/money/@furion/steem-btc-fundamental-analysis-1-why-is-the-price-tanking) project, and I thought it would be a good idea to document my findings. 

**If you're a developer and you'd like to build an app on top of Steem/Steemit, this guide is for you.**

## Setting up your node
First, we need to setup a node. There are plenty of guides and tools (for [Docker](https://steemit.com/steem/@teego/docker-steem-images-are-updated-to-version-0-12-2), [Ubuntu](https://steemit.com/steemit/@kilrathi/project-bullfrog-system), [Windows](https://steemit.com/steem/@bitcube/steem-mining-in-microsoft-windows-a-miner-s-guide-part-2)) on steemit on how to do that, so I won't repeat myself here.

**To get access to all the API's, modify your `config.ini` to have the following values:**
```
enable-plugin = witness account_history tags follow market_history
public-api = database_api login_api network_broadcast_api follow_api market_history_api
```
Also, you should remove `miner` and `witness` fields. We aren't interested in mining, just running a node.

**Now you can run steemd:**
```
./steemd --rpc-endpoint
```
The `--rpc-endpoint` will setup a websocket endpoint at `ws://127.0.0.1:8090`.

You should be able to connect to it from your favorite programming language (Python, JavaScript, etc.), or by using a tool such as [wssh](https://github.com/progrium/wssh).

-------

If you're using Python, then I suggest the awesome [python-steemlib](https://github.com/xeroc/python-steemlib) by @xeroc.

You can connect to the RPC like so:
```
from steemapi.steemnoderpc import SteemNodeRPC

rpc = SteemNodeRPC("ws://127.0.0.1:8090", "", "")
```

-------

## Analyzing the Blockchain
Now that we have a node running, and we are connected to it, we can issue commands.

In this tutorial, we will be using the following RPC commands:
```
get_block()
get_dynamic_global_properties()
```

Again, the RPC is available for us at `ws://localhost:9080`.

----------

The full list of commands is available here (scroll to the bottom):
https://github.com/steemit/steem/blob/master/libraries/app/include/steemit/app/database_api.hpp

----------

In order to get a block from the blockchain, we want to call a RPC command called `get_block`, which takes just one argument: Block ID.

Block ID is just a number, anywhere between 0 and the current block ID. 
Each new block has an ID at an increment of 1.

You can find out what the current block ID is by looking at `steemd` output:
```
651068ms th_a       application.cpp:439           handle_block         ] Got 1 transactions from network on block 3695832
```

Or by asking `steemd` via RPC:
```
 props = rpc.get_dynamic_global_properties()
 last_confirmed_block = props['last_irreversible_block_num']
```


**So to traverse the entire blockchain, from start to finish, we would do something like this:**
```
props = rpc.get_dynamic_global_properties()
current_block = 0
last_confirmed_block = props['last_irreversible_block_num']

while current_block < last_confirmed_block:
    current_block += 1
    print("Processing block %d" % current_block)

    block = rpc.get_block(current_block)
    pprint(block)
```

Now lets look at what `get_block` gives us. If we call `rpc.get_block(3332318)`, we get a block `3332318` in a `json` format, that looks like this:

```
{'extensions': [],
 'previous': '0032d8dd7fef5c0eafd7667805134effb9ef21e9',
 'timestamp': '2016-07-19T13:29:15',
 'transaction_merkle_root': '5be8c3f95a29253153e911fd079528ad8216958a',
 'transactions': [{'expiration': '2016-07-19T13:29:24',
                   'extensions': [],
                   'operations': [['vote',
                                   {'author': 'beowulfoflegend',
                                    'permlink': 'how-steemit-self-polices-content',
                                    'voter': 'jills',
                                    'weight': 10000}]],
                   'ref_block_num': 55512,
                   'ref_block_prefix': 2485993294,
                   'signatures': ['20196c89f5e2ca58da3377a30ee08782df59dbd4b72db40ec3348f3417ff653fa84cf684748853e8ea43363c747c74b4ff015626ef9e2b2e902e0ee98a80195336']},
                  {'expiration': '2016-07-19T13:30:10',
                   'extensions': [],
                   'operations': [['comment',
                                   {'author': 'weenis',
                                    'body': 'This is Amazing! Upvoted! \n',
                                    'json_metadata': '',
                                    'parent_author': 'begstreets',
                                    'parent_permlink': 'abstractions-no-1',
                                    'permlink': 'abstractions-no-1',
                                    'title': ''}]],
                   'ref_block_num': 55516,
                   'ref_block_prefix': 765250389,
                   'signatures': ['1f7711cfc441bbc5e6df16d7b73ea7d4ea7d7a40a3c3a1fd857d67b50b30c139d06f742e5fd15a68f2109469c977a12c50abab81632b8a4ba5e328c5048290153b']},
 ],
 'witness': 'riverhead',
 'witness_signature': '1f4236f6a0ff79ab2ff9a190162aeb25b5e1509ca44df8803c8091a2abcfe2022f733eb77c15ec8265a1b1d0d2226038b3259db9ebe17b871bcbf3be0dee605bc9'}
```

We are mostly interested in `transactions` and `operations`. Transactions contains a list of `operations`, which are basically all the events that happened on the blockchain within the 3 second period included in the block.

## Operation Types
Currently, the Steem blockchain contains the following operations:
- comment, delete_comment, vote
- account_create, account_update, request_account_recovery, recover_account
- limit_order_create, limit_order_cancel
- transfer, transfer_to_vesting, withdraw_vesting, convert
- pow, feed_publish, witness_update, account_witness_vote
- custom, custom_json

### Blog Posts, Comments and Votes

This is a blog post:
```
=====> comment
{'author': 'rozu15',
 'body': 'content omited becuse it breaks my post',
 'json_metadata': '{"tags":["trump"],"links":["https://twitter.com/realDonaldTrump/status/755254384062263296"]}',
 'parent_author': '',
 'parent_permlink': 'trump',
 'permlink': 'us-election-melania-trump-plagiarised-michelle-obama',
 'title': "US election: Melania Trump 'plagiarised' Michelle Obama"}
 ```
 
 This is a comment. It has empty `title`:
 ```
 =====> comment
{'author': 'densmirnov',
 'body': 'nice nickname',
 'json_metadata': '{"tags":["steem"]}',
 'parent_author': 'steemiscrap',
 'parent_permlink': 're-cryptorune-steem-dollar-poloniex-pump-is-a-arbitrage-goldmine-20160719t133438897z',
 'permlink': 're-steemiscrap-re-cryptorune-steem-dollar-poloniex-pump-is-a-arbitrage-goldmine-20160719t133602671z',
 'title': ''}
 ```
 
 The comment has been deleted:
 ```
 =====> delete_comment
{'author': 'densmirnov',
 'permlink': 're-cryptorune-re-densmirnov-re-cryptorune-steem-dollar-poloniex-pump-is-a-arbitrage-goldmine-20160719t133207807z'}
 ```

The comment has received a vote:
```
=====> vote
{'author': 'kain-jc',
 'permlink': 'a-meditation-on-love',
 'voter': 'cantinhodatete',
 'weight': 10000}
```
If we divide the weight by `100`, we get the % of the vote. For instance, a full upvote is `100%`. The full downvote is `-100%`. Half the upvote is `50%`.

### Account Features
Account has been created:
```
=====> account_create
{'active': {'account_auths': [],
            'key_auths': [['STM7huHJRrqHE6FDNHJEsUYNNKuQKBfr4iS3xwgyGzXDurbiCRx2V',
                           1]],
            'weight_threshold': 1},
 'creator': 'steem',
 'fee': '5.000 STEEM',
 'json_metadata': '',
 'memo_key': 'STM7R3iScNyQCytvB8EcRdEYLfJCGp3UkyNm7Mt4c9BE1ZoUTdazw',
 'new_account_name': 'saripdol',
 'owner': {'account_auths': [],
           'key_auths': [['STM8XzPFfu3yCujJiSbgdJvKm69LMcm1ckocuvoSaaWRmPRzfH7jr',
                          1]],
           'weight_threshold': 1},
 'posting': {'account_auths': [],
             'key_auths': [['STM8KKqZufL2PM7V9dHeSW8KpN4wWs8cGaiQsz7FCrPohX1ZkfPvv',
                            1]],
             'weight_threshold': 1}}
```

Account has been updated:
```
=====> account_update
{'account': 'signaltonoise',
 'active': {'account_auths': [],
            'key_auths': [['STM83ihutAefZSHCJRXLHiRnji8VhQHcouMJc6xGtaLW6PCARXiDa',
                           1]],
            'weight_threshold': 1},
 'json_metadata': '',
 'memo_key': 'STM81hFsfm2csFyPYDCdwjuGJX33EB2Dx6rG74AfFgStu9C6PvzMd',
 'owner': {'account_auths': [],
           'key_auths': [['STM8fcgjrpKbVLHyy3RKnQGx3dhYEdY7bKhbv3FaML5ooK8PpdJaR',
                          1]],
           'weight_threshold': 1},
 'posting': {'account_auths': [],
             'key_auths': [['STM75QK5e69SAuMzVtJqGex3HcaqrPcxpXgxb72WxbUJn2apRX7KX',
                            1]],
             'weight_threshold': 1}}
```


### Transfers, Vesting, Conversions

A transfer of `SBD` or `STEEM` from one account to another.
```
=====> transfer
{'amount': '34.428 SBD',
 'from': 'thisischris225',
 'memo': 'ee1d91d4036200c7',
 'to': 'poloniex'}
```

Someone has powered-up:
```
=====> transfer_to_vesting
{'amount': '11.789 STEEM', 'from': 'leksimus', 'to': 'leksimus'}
```

Someone has powered-down:
```
=====> withdraw_vesting
{'account': 'giveandtake1', 'vesting_shares': '1184211.402324 VESTS'}
```

Someone placed a conversion order:
```
=====> convert
{'amount': '0.158 SBD', 'owner': 'jamessmith', 'requestid': 1468935121}
```
This order executes 7 days after its initiated, and takes the 7 day average price as the execution basis. It is useful for placing a big order in the internal market without moving it too much.


### Mining, Witnesses
Finding a pow (mining):
```
=====> pow
{'block_id': '0032d98aaa917775f98859c3235f43286479e1b7',
 'nonce': '6342739805603833088',
 'props': {'account_creation_fee': '0.001 STEEM',
           'maximum_block_size': 131072,
           'sbd_interest_rate': 1000},
 'work': {'input': '507906ad12f0d3bc57f636849dc117c5c970e79f0411e7e07459f78c09b350a0',
          'signature': '209c919b436563ae30171356cb7fcb6677794c9cbf163c695d3ce56841a9c86f1d06ab7f4b28c4c493a5d951664b32b7fb76714b30e49c8035bf12c520b1c6c3df',
          'work': '0000000350c2f2cae73dab00d1c2a1e2fafbf22e1e2fe0bb2a0a4e3a69c3b976',
          'worker': 'STM574Rx1dML1K8Mi9Uz42yWDYAr6ymrxC723HR1YX81Jqk7MSaRT'},
 'worker_account': 'redrockmining'}
 ```
 
 Witness changes its settings:
 ```
 =====> witness_update
{'block_signing_key': 'STM7yFmwPSKUP7FCV7Ut9Aev5cwfDzJZixcreS1U3ha36XG47ZpqT',
 'fee': '0.000 STEEM',
 'owner': 'anyx',
 'props': {'account_creation_fee': '2.600 STEEM',
           'maximum_block_size': 131072,
           'sbd_interest_rate': 1000},
 'url': 'https://steemit.com/witness-category/@anyx/hello-from-anyx'}
 ```
 
 Witness publishes new market rates:
 ```
 =====> feed_publish
{'exchange_rate': {'base': '4.100 SBD', 'quote': '1.000 STEEM'},
 'publisher': 'steemed'}
 ```

### Trading
Place internal market order:
```
=====> limit_order_create
{'amount_to_sell': '94.000 STEEM',
 'expiration': '2016-07-19T16:10:40',
 'fill_or_kill': False,
 'min_to_receive': '260.000 SBD',
 'orderid': 94,
 'owner': 'ledzeppelin'}
```

Cancel internal market order:
```
=====> limit_order_cancel
{'orderid': 1468934907, 'owner': 'fabio'}
```

### Account Sundry
Voting for a witness:
```
=====> account_witness_vote
{'account': 'leontyashka', 'approve': True, 'witness': 'arhag'}
```

Initiating account recovery:
```
=====> request_account_recovery
{'account_to_recover': 'stan',
 'extensions': [],
 'new_owner_authority': {'account_auths': [],
                         'key_auths': [['STM53xgdjGJ26QEF8qUbGsxguRBQU1UMqLKTT3LJ2ysgsLdr3zJk2',
                                        1]],
                         'weight_threshold': 1},
 'recovery_account': 'steem'}
```

Account recovery complete:
```
=====> recover_account
{'account_to_recover': 'val',
 'extensions': [],
 'new_owner_authority': {'account_auths': [],
                         'key_auths': [['STM7igHgvdVmUJFJadc56d3wuk28xPo9NV4qQLhsHBzmpiQvTq2Pe',
                                        1]],
                         'weight_threshold': 1},
 'recent_owner_authority': {'account_auths': [],
                            'key_auths': [['STM4wo7NZSUY86RDmaZrQuhuAybyCNSEfnpN2fHmKX7V4mQtPrsuK',
                                           1]],
                            'weight_threshold': 1}}

```

### Custom
Custom json currently only supports the follow plugin. More features to come in the future.
```
=====> custom_json
{'id': 'follow',
 'json': '{"follower":"manugbr93","following":"natenvos","what":["blog"]}',
 'required_auths': [],
 'required_posting_auths': ['manugbr93']}
```
`what` can be either `"blog","posts"`, `"blog"`, `"posts"` or `""`.
I suspect `""` means unfollow. I do not know what is the difference between `blog` and `posts` however.


An finally, this mysterious `custom` operation:
```
=====> custom
{'data': '046162696408637563697374696d03c6278df998dd7cf0bac77dc3af89debf1f628eebbe9e86daa9762b7590630218023560f4ac629975cda967a694a07ebea5d912fcc31bf732aca9908ec9c04d603850ae00e0fd3705003f8ea76c2073f7cac9ce38f97dd8d32ba5df47adbea29c29758d776467ce16f055a5095563',
 'id': 777,
 'required_auths': ['abid']}
```
@jl777 thinks its an encrypted private message.


## Example: Parsing the Blockchain

Suppose we want to find out how many followers we have, and get the list of people whom are following us.

Here is how we can parse each block, only looking for `custom_json` operations to find out:

```
from steemapi.steemnoderpc import SteemNodeRPC

rpc = SteemNodeRPC("ws://127.0.0.1:8090", "", "")

MY_USERNAME = "furion"

props = rpc.get_dynamic_global_properties()
current_block = 0
last_confirmed_block = props['last_irreversible_block_num']

while current_block < last_confirmed_block:
    current_block += 1
    block = rpc.get_block(current_block)
    
    if "transactions" in block:
        for tx in block["transactions"]:
            for opObj in tx["operations"]:
                #: Each operation is an array of the form
                #:    [type, {data}]
                op_type = opObj[0]
                op = opObj[1]

                if op_type == "custom_json" and op['id'] == "follow":
                    j = json.loads(op['json'])
                    if j['following'] == MY_USERNAME:
                        follower_count += 1
                        print("%s is following my %s, I have %d followers now." % (
                            j["follower"], j['what'], follower_count
                        ))
                        
```

I only have 8 followers, which makes me sad:
```
kilrathi is following my ['blog'], I have 1 followers now.
cloveandcinnamon is following my ['blog'], I have 2 followers now.
dcsignals is following my ['blog'], I have 3 followers now.
jerome-colley is following my ['blog'], I have 4 followers now.
mihserf is following my ['blog'], I have 5 followers now.
shaheer001 is following my ['blog'], I have 6 followers now.
helikopterben is following my ['blog'], I have 7 followers now.
carmasleeper is following my ['blog'], I have 8 followers now.

```

So if you liked this guide, and you'd like to see more, you know what to do.

---------

![](http://i.imgur.com/PPplpRq.jpg)

Thank you for making awesome apps for Steem. I hope you have found my guide useful.
#steem #dev #steemit-dev #development #programming #python