<center>![steem-python.png](https://steemitimages.com/0x0/https://cdn.steemitimages.com/DQmeBnp3UNcivaVgrNiz6EZBQFQsJjnEKDx5mdzCtEABSBD/banner.png)</center>

This tutorial is part of a series where different aspects of programming with `EOS` are explained, the full curriculum can be found below. This part will focus on building an unending stream with fallback `api_endpoints` and goes into more depth about the `head_block` and `irreversible` blocks.

---
#### Repository
https://github.com/EOSIO/eos

#### What will I learn

- Head block & last irreversible block
- Unending stream
- Multiple api_endpoints and switching

#### Requirements

- Python3.6

#### Difficulty

- basic

---

### Tutorial



#### Setup
Download the file from [Github](https://github.com/Juless89/eos-tutorials/tree/master/03). There is 1 file `stream_blocks.py`. Running the file without any arguments will test the `api_endpoints` for their speed to decide what node is best and then start streaming from the current `head_block`. Adding the argument `irreversible` will perform the same test but only stream `irreversible` blocks.

Run scripts as following:
`> python stream_blocks.py`


#### Head block & last irreversible block
On the `EOS` Blockchain new `blocks` are produced by 21 `block producers (BPs)` every halve a second. As this is not enough time for the `blocks` to propagate over the entire network `BPs` are chosen to produce a set amount of `blocks` before another `BP` takes the role as a `producer`. The frequency in which the `BPs` are chosen is based on the amount of votes this `BP` has received.

This means that the current block producing `BP` is slightly ahead of the other `BPs`. For this reason there is a distinction between the `head_block` and the `last_irreversible_block`. Every `block` that has been verified by all` BPs` becomes an `irreversible block`. As it takes time for the `blocks` to propagate over the network and get verified by all `BPs` there is a delay before a `block` becomes `irreversible`.

<center>
![Screenshot 2018-07-15 11.59.17.png](https://cdn.steemitimages.com/DQmWze3XzAdvdtqKn1BGqfCkKJKvZhYpYTJ7EXaAuZPv9n2/Screenshot%202018-07-15%2011.59.17.png)
[eosnetworkmonitor](http://eosnetworkmonitor.io/)
</center>

Depending of the type of application use of the `head_block` or `last_irreversible_block` can be preferred. For example dealing with financial transaction using the `last_irreversible_block` is recommended, while building a chat application speed is more important so the current `head_block` is preferred. Information about the `head_block` and `last_irreversible_block` is found by using the `get_info` API request.


```
def get_head_block(self, irreversible=None):
    api_call = '/v1/chain/get_info'
    address = self.url + api_call

    request_json = self.s.get(url=address).json()

    if not irreversible:
        return request_json['head_block_num']
    else:
        return request_json['last_irreversible_block_num']
```


#### Unending stream
Knowing this a robust unending stream can be made. First the `start_block` has to be decided. If none is given it is taken from the current `head_block` or `last_irreversible_block` which is set in `self.block`. This is to keep track of which blocks have been processed in case the stream crashes and needs to restart, starting at `self.block`.

```
self.block = self.get_head_block(irreversible)

def stream_blocks(self,
                  start_block=None,
                  block_count=None,
                  irreversible=None):

    if not start_block:
        start_block = self.block
    else:
        self.block = start_block
```

When upgrading code for the application where it has to be shut down it can be preferred to start at the last processed block. In which case a `start_block` can be set.
<br>
The next block of codes takes the `start_block` and then ask for the current `head_block` / `last_irreversible_block` to create a range of` blocks`. These `blocks` are requested and after the `start_block` is set to the last requested `block`. Repeating indefinitely.

The upside of this is that it becomes impossible to request `blocks` ahead of the current `head_block` / `last_irreversible_block block`. The downside is that it requires additional API requests to retrieve the information about the `head_block` / `last_irreversible_block`. In the case of using the `last_irreversible_block` this is the only way to make sure the blocks being requested are `irreversible`. As it is possible to requests `blocks` ahead of the `last_irreversible_block`, so no error message would be returned if that were the case.


```
while True:
        head_block = self.get_head_block(irreversible)

        for n in range(start_block, head_block + 1):
            yield self.get_block(n)
            self.block += 1

        start_block = head_block + 1
```

In case something unexpected happens, such as losing internet connection the stream is put into a `while` loop with a `try` and `except` clause. This will keep trying to restart the stream.

```
while True:
    try:
        for block in self.stream_blocks(irreversible):
            print(f'\nBlock {self.block}')
    except Exception as error:
            time.sleep(60)
            print(repr(error))
```

#### Multiple api_endpoints and switching

Problems can also occur at the `api_endpoint` side. Proving additional `api_endpoints` to fall back to solves this problem.

```
self.nodes = ['http://mainnet.eoscanada.com',
              'http://api-mainnet1.starteos.io',
              'http://api.eosnewyork.io']
self.node_index = 0
self.url = self.nodes[self.node_index]
```

In the case the `api_endpoint` returns an unexpected error the `node_index` will be increased and adjusted to stay in the limits of the `nodes` list. The next `api_endpoint` is then selected and the session is renewed.
```
def reset_api_endpoint(self):
    self.node_index = (self.node_index + 1) % len(self.nodes)
    self.url = self.nodes[self.node_index]
    self.s = requests.Session()
    print(f'New api_endpoint: {self.url}')
```

Additionally when starting the stream all the nodes are tested for their response time. For each `api_endpoint` 100 `blocks` are retrieved and it is timed how long this process takes. The results are added to the results list as tuples `(url, time)`. Then the results are ordered by their time `sort(key=operator.itemgetter(1))`. Eventually a sorted nodes list is returned by stripping only the urls from the each tuple in results.

```
def test_api_endpoints(self):
    results = []

    for url in self.nodes:
        self.url = url
        start_time = time.time()
        print(f'\nTesting: {url}')

        for block in self.stream_blocks(start_block=1, block_count=100):
            print(f'{self.block}/100', end='\r')

        end_time = time.time() - start_time
        results.append((url, end_time))
        print(f'Took {end_time:.4} seconds for completion')

    results.sort(key=operator.itemgetter(1))
    self.nodes = [node[0] for node in results]
    print(self.nodes)
```

#### Running the code
The code can be run in two modes, either the current `head_block` mode or the `last_irreversible_block` mode. Regardless of the mode a test of each `api_endpoint` will first be conducted to sort the `api_endpoints` by their speed.

```
python stream_blocks.py irreversible

Testing: http://mainnet.eoscanada.com
Took 27.98 seconds for completion

Testing: http://api-mainnet1.starteos.io
Took 33.16 seconds for completion

Testing: http://api.eosnewyork.io
Took 29.68 seconds for completion
['http://mainnet.eoscanada.com', 'http://api.eosnewyork.io', 'http://api-mainnet1.starteos.io']

Mode: last_irreversible_block
Block 5998244
```


#### Curriculum
- [Part 1: Using Python To Make API Requests To EOS Public API Endpoints](https://steemit.com/utopian-io/@steempytutorials/part-1-using-python-to-make-api-requests-to-eos-public-api-endpoints)
- [Part 2: Streaming EOS Blocks With Python](https://steemit.com/utopian-io/@steempytutorials/part-2-streaming-eos-blocks-with-python)
---

The code for this tutorial can be found on [GitHub](https://github.com/Juless89/eos-tutorials/tree/master/03)!

This tutorial was written by @juliank.