create account

Auto docker image updating in the Fluree Schema Scenario Tool by aioflureedb

View this thread on: hive.blogpeakd.comecency.com
· @aioflureedb · (edited)
$2.14
Auto docker image updating in the Fluree Schema Scenario Tool
The Fluree Schema Scenario Tool ( [fsst](https://github.com/dlosatoes/fluree-schema-scenario-tool) ) is a tool for Test Driven Development with the blockchain based graph database [FlureeDb](https://flur.ee/fluree-products/intelligent-database/). 

The tool has many features that we aren't going to discuss in this post, but its main functionality lies in doing test driven development with the Fluree Database. The tool is a script that uses a docker image that embeds and uses the exact same script inside of the docker container. If you have docker installed, then you mostly don't even need to know that fsst uses docker as the outer fsst script will start docker and within docker that also conatins an instance of FlureeDB, and will then invoke a copy of itself inside of the docker image. 

It is also possible to use the [docker images](https://hub.docker.com/r/dlosatoes/fsst/tags) in a CICD setup in for example in GitLab, and fsst also has features for deployment in such a context, but in this post we are going to look at the typical usage for *local* testing of database schemas and Fluree Smart functions.

If you aren't running fsst yet, or even have never done anything with FlureeDB yet, you can get fsst using pip

```
pip install fsst
```

Or if you are upgrading:

```
pip install fsst -U
```

## The old behaviour and why it's not ideal

In past versions of fsst, the tool would look for the docker image locally, and only if the docker image wasn't found would it look for the version of the docker image online and download it. 

To understand why this isn't always ideal, we need to look at the tag usage that currently get used in docker hub for the fsst+flureedb images.

The docker images on docker hub always come with two products with independent versioning. Sometimes both FlureeDB and fsst get a new version, dometimes only FlureeDB gets a new version and sometimes only fsst gets a new version while the latest version of flureedb stays the same. The fully specified name version of each actual docker image consists of a tag combining both version numbers. For example the latest fully explicit tag on docker hub at time of writing is **v0.8.13-fluree-v2.0.4**. The same image is currently also available using the tags:

* **fluree-v2.0.4** -> v0.8.13-fluree-v2.0.4
* **v0.8.13-stable** -> v0.8.13-fluree-v2.0.4
* **stable** -> v0.8.13-fluree-v2.0.4

When FlureePBC releases a 2.0.5 version of FlureeDB, and we build and push a new docker image to docker hub, assuming there is no new version of fsst available yet, the **stable** tag and the  **v0.8.13-stable** will be set to point to the newer image on github, while the tag **fluree-v2.0.4** will keep pointing to the old current image.

* **fluree-v2.0.4** -> v0.8.13-fluree-v2.0.4
* **v0.8.13-stable** -> **v0.8.13-fluree-v2.0.5**
* **stable** -> **v0.8.13-fluree-v2.0.5**
* **fluree-v2.0.5** -> v0.8.13-fluree-v2.0.5

If instead there is no new version of FlureeDB, but a newer version of fsst is released and we build and push a new image, then almost the oposite happens,  **stable** and **fluree-v2.0.4** end up pointing to the new image while **v0.8.13-stable**  will keep pointing to the old one.

* **fluree-v2.0.4** -> **v0.8.14-fluree-v2.0.4**
* **v0.8.13-stable** -> v0.8.13-fluree-v2.0.4
* **stable** -> **v0.8.14-fluree-v2.0.4**
* **v0.8.14-stable** -> v0.8.14-fluree-v2.0.4

Now we can see how the old behaviour was a problem. What if we were using **stable** as our tag? In fact, **stable** is the default tag for fsst if no tag is specified implicitly. Well, as the old stable is likely available locally, nothing will get fetched from docker hub, and fsst will run with for example an old version of flureedb and not the recently patched and updated version.

## The new behaviour

The latest version of fsst now changes this behaviour, but it goes a step further than checking just in one place. The maintenance of fsst has recently moved from [this](https://hub.docker.com/r/pibara/fsst/tags) repo to [this](https://hub.docker.com/r/dlosatoes/fsst) repo, but old images were not moved. This was done because the maintainer has changed from an individual to a team. So now if fsst starts, it first checks these two repos for the existance of the tag, if for some reasons the same tag (for example **stable**) exists in two repos it remembers the newest and will check if that specific version exists locally. If it does it will still use the local version, but if it doesn't, even if an other image with that specific tag does exist locally, it will download the newer image with that tag from the repo with the newest image for that tag.

Let's demonstrate.

```
$ fsst dockertest --runs 1 --hooks --tag v0.8.11-stable
COMMAND: fsst guesttest --target default --network test47879 --runs 1 --hooks --stages ALL
INFO: Newest image with tag v0.8.11-stable is from pibara
INFO: Looking for pibara/fsst:v0.8.11-stable
NOTICE: Not locally found
NOTICE: Fetching docker image from docker hub
  - Fetched pibara/fsst:v0.8.11-stable from docker hub
INFO: Fetched docker image
# waiting for default-private-key.txt to appear
```  

The tool found the image on the **pibara/fsst** repo, and as it was not available locally, it fetched it first. Lets run the same command again.

```
$ fsst dockertest --runs 1 --hooks --tag v0.8.11-stable
COMMAND: fsst guesttest --target default --network test47885 --runs 1 --hooks --stages ALL
INFO: Newest image with tag v0.8.11-stable is from pibara
INFO: Looking for pibara/fsst:v0.8.11-stable
INFO: No need to fetch from docker hub, newest image available loccally
INFO: Found docker image locally
# waiting for default-private-key.txt to appear
```

Now let's  try a newer one:

```
$ fsst dockertest --runs 1 --hooks --tag v0.8.13-stable
COMMAND: fsst guesttest --target default --network test47898 --runs 1 --hooks --stages ALL
INFO: Newest image with tag v0.8.13-stable is from dlosatoes
INFO: Looking for dlosatoes/fsst:v0.8.13-stable
NOTICE: Not locally found
NOTICE: Fetching docker image from docker hub
  - Fetched dlosatoes/fsst:v0.8.13-stable from docker hub
INFO: Fetched docker image
# waiting for default-private-key.txt to appear
```

We see the image is fetched from the new repo **dlosatoes/fsst**.

## Independent dual-source releases ?: faster docker images releases

The new setup opens some other possibilities that will only be of use if FlureePBC chooses to use it. We changed fsst to be optimistic about this prospect, so next to the two valid repos, fsst also optimistically checks for an fsst repo under the [fluree](https://hub.docker.com/u/fluree) dockerhub account. The idea is that if FlureePBC ever chooses to use the [fsst / fluree docker](https://github.com/dlosatoes/fluree-fsst-docker) files to generate the fsst/flureedb images directly after generating and pushing a parent image, a new image would be available whenever either of the two projects releases a new version.  

If that happens, **stable** will always immediately be up to date (currently it may be many weeks before **stable** gets updated after a new stable flureedb release). For now  this isn't the case, but the code is in fsst so **if** flureePBC ever ends up integrating building the combined image into its releases, it should work instantly.

Not much functionality wa added in this fsst release, but we believe the convenience for the user of not having to worry about manually deleting or fetching images should prove a nice usability feature.

If you haven't checked out FlureeDB or ffst yet, please have a look at other posts from this @aioflureedb account. It's an amazing blockchain powered graph database that should be close to ideal for sidechain projects.
👍  , , , , , , ,
properties (23)
authoraioflureedb
permlinkaudo-docker-image-updating-in-the-fluree-schema-scenario-tool
categoryhive-169321
json_metadata"{"app":"peakd/2024.4.3","description":"How the newest version of this TDD tool for the blockchain based graphDB FlureeDB now automaticly upgrades to the latest version of FlureeDB for its docker hub tag, and why this matters.","format":"markdown","image":[],"tags":["flureedb","testing","tdd","docker","development","database","python"],"users":["aioflureedb"]}"
created2024-04-30 12:33:15
last_update2024-05-01 12:57:54
depth0
children0
last_payout2024-05-07 12:33:15
cashout_time1969-12-31 23:59:59
total_payout_value1.059 HBD
curator_payout_value1.080 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length7,714
author_reputation215,214,186,079
root_title"Auto docker image updating in the Fluree Schema Scenario Tool"
beneficiaries
0.
accounthive-169321
weight200
max_accepted_payout1,000,000.000 HBD
percent_hbd0
post_id133,242,606
net_rshares5,109,830,027,161
author_curate_reward""
vote details (8)