create account

Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem] by ontofractal

View this thread on: hive.blogpeakd.comecency.com
· @ontofractal ·
$494.69
Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]
<center>![](http://i.imgur.com/QKOa4iu.png)</center>
I've been working to improve the developer experience for Steemex and this release does exactly that. I had to introduce some breaking changes but it was worth it. Stage.Blocks,  Stage.RawOps and Statge.MungedOps GenStage producers now emit events with the following shape : %Steem.Event{data: ..., metadata: ...}` where data is a new Steem blockchain operation struct (or a map) and metadata includes timestamp, block height, source blockchain, type and if the operation data was munged.

All event producers follow [GenStage](http://elixir-lang.org/blog/2016/07/14/announcing-genstage/) spec which allows you to easily patch into the transformation pipeline at any step: block producer, ops producer or munged ops producer.


# Repos and docs

* [github](https://github.com/cyberpunk-ventures/steemex)
* [hex.pm](https://hex.pm/packages/steemex)
* [docs](https://hexdocs.pm/steemex)


# Changelog

* breaking: event shape for GenStage pipeline changed from tuple to %Steemex.Event{} struct 
* breaking: all API responses are now structs or atom keyed maps
* all stages now emit %Steemex.Event{data: ..., metadata: ...} 
* added Steemex.Block struct
* added Steemex.Event struct
* Steemex.Stages.Ops.ProducerConsumer renamed to Steemex.Stages.RawOps
* Steemex.Stages.StructuredOps.ProducerConsumer renamed to Steemex.Stages.MungedOps
*  timestamp strings are now parsed into NaiveDateTime
* example consumer was updated to match a new shape of events
* multiple various code improvements

## A GenStage consumer example

```
defmodule Steemex.Stage.ExampleConsumer do
  use GenStage
  alias Steemex.MungedOps
  require Logger

  def start_link(args, options \\ []) do
    GenStage.start_link(__MODULE__, args, options)
  end

  def init(state) do
    Logger.info("Example consumer is initializing...")
    {:consumer, state, subscribe_to: state[:subscribe_to]}
  end

  def handle_events(events, _from, state) do
    for op <- events do
      process_event(op)
    end
    {:noreply, [], state}
  end

  def process_event(%{data: %MungedOps.Reblog{} = data, metadata: %{height: h, timestamp: t} = metadata}) do
      Logger.info """
      New reblog:
      #{inspect data}
      with metadata
      #{inspect metadata}
      """
  end

  def process_event(%{data: data, metadata: %{block_height: h, timestamp: t} = metadata}) do
      Logger.info """
      New operation:
      #{inspect data}
      with metadata
      #{inspect metadata}
      """
  end

end
```


# Roadmap

* Add latest blockchain ops
* Add more structured ops and transformations
* Add transaction broadcast

# About Elixir

Elixir is a functional programming language with superior concurrency primitives, distributed computation and fault tolerance capabilities based on Erlang/OTP.
Personally, I am really happy with my bet on Elixir to build cryptotokens related apps and strongly recommend every developer to try it.

Learning resources:

* [ElixirConf Jose Valim Keynote](http://confreaks.tv/videos/elixirconf2016-keynote)
* [ElixirConf 2016 Chris McCord Keynote](http://confreaks.tv/videos/elixirconf2016-keynote)
* [Elixir School](https://elixirschool.com/)
* [Elixir Crash Course](http://elixir-lang.org/crash-course.html)

<center>(∩^o^)⊃━☆゜.*</center>
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 94 others
properties (23)
authorontofractal
permlinksteemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem
categorysteemdev
json_metadata{"tags":["steemdev","steem","elixir","steemstem","steem-dev"],"image":["http://i.imgur.com/QKOa4iu.png"],"links":["http://elixir-lang.org/blog/2016/07/14/announcing-genstage/","https://github.com/cyberpunk-ventures/steemex","https://hex.pm/packages/steemex","https://hexdocs.pm/steemex","http://confreaks.tv/videos/elixirconf2016-keynote","https://elixirschool.com/","http://elixir-lang.org/crash-course.html"],"app":"steemit/0.1","format":"markdown"}
created2017-06-25 11:11:24
last_update2017-06-25 11:11:24
depth0
children17
last_payout2017-07-02 11:11:24
cashout_time1969-12-31 23:59:59
total_payout_value393.264 HBD
curator_payout_value101.429 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length3,288
author_reputation61,670,850,962,884
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,028,361
net_rshares45,204,888,751,036
author_curate_reward""
vote details (158)
@azlansugihen ·
Wow, thats a good news i think
properties (22)
authorazlansugihen
permlinkre-ontofractal-2017625t19535699z
categorysteemdev
json_metadata{"tags":"steemdev","app":"esteem/1.4.5","format":"markdown+html","community":"esteem"}
created2017-06-25 12:05:39
last_update2017-06-25 12:05:39
depth1
children0
last_payout2017-07-02 12:05:39
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length30
author_reputation471,732,259,874
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries
0.
accountesteemapp
weight500
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,033,251
net_rshares0
@bluethread ·
Thanks to everyone who work to improve the Steemit experience.
properties (22)
authorbluethread
permlinkre-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t114824486z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-25 11:48:24
last_update2017-06-25 11:48:24
depth1
children0
last_payout2017-07-02 11:48:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length62
author_reputation1,489,627,047,120
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,031,693
net_rshares0
@edmonam ·
$0.04
Steemit is getting better and better. That's the only thing I understood from this post :)
👍  , ,
properties (23)
authoredmonam
permlinkre-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t120905126z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-25 12:09:06
last_update2017-06-25 12:09:06
depth1
children3
last_payout2017-07-02 12:09:06
cashout_time1969-12-31 23:59:59
total_payout_value0.038 HBD
curator_payout_value0.003 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length90
author_reputation40,073,203,388
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,033,559
net_rshares3,863,561,003
author_curate_reward""
vote details (3)
@the-ego-is-you · (edited)
Steem, but yes :D
properties (22)
authorthe-ego-is-you
permlinkre-edmonam-re-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170626t175952332z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-26 17:59:51
last_update2017-06-26 18:38:06
depth2
children0
last_payout2017-07-03 17:59:51
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length17
author_reputation7,031,347,556,614
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,212,284
net_rshares0
@watchout2017 ·
$0.41
Jeeeee. At least your hinest.  Yes it is so invest everything into steempower because I dont see it going below 2.00 ever again.

Domi
👍  ,
properties (23)
authorwatchout2017
permlinkre-edmonam-re-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t121857416z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-25 12:19:00
last_update2017-06-25 12:19:00
depth2
children1
last_payout2017-07-02 12:19:00
cashout_time1969-12-31 23:59:59
total_payout_value0.324 HBD
curator_payout_value0.089 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length134
author_reputation237,607,560,257
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,034,453
net_rshares38,105,806,335
author_curate_reward""
vote details (2)
@edmonam ·
True!
properties (22)
authoredmonam
permlinkre-watchout2017-re-edmonam-re-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t122209710z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-25 12:22:09
last_update2017-06-25 12:22:09
depth3
children0
last_payout2017-07-02 12:22:09
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length5
author_reputation40,073,203,388
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,034,746
net_rshares0
@elevator09 ·
great info! thanks for sharing
properties (22)
authorelevator09
permlinkre-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t112951062z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-25 11:29:51
last_update2017-06-25 11:29:51
depth1
children0
last_payout2017-07-02 11:29:51
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length30
author_reputation345,809,116,793,863
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,030,027
net_rshares0
@heelzkinu ·
This one is interesting and have something great information 
Thank You for sharing 
And serving steemit community
👍  
properties (23)
authorheelzkinu
permlinkre-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t111426561z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-25 11:14:30
last_update2017-06-25 11:14:30
depth1
children0
last_payout2017-07-02 11:14:30
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length114
author_reputation4,497,935,121,409
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,028,671
net_rshares539,723,674
author_curate_reward""
vote details (1)
@jessiejesurasa ·
Thank you for sharing
👍  
properties (23)
authorjessiejesurasa
permlinkre-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t123455856z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-25 12:34:54
last_update2017-06-25 12:34:54
depth1
children0
last_payout2017-07-02 12:34:54
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length21
author_reputation35,671,284,053
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,035,997
net_rshares130,601,932
author_curate_reward""
vote details (1)
@jordanlindsey ·
Appreciate your work and helping grow our community.

(∩^o^)⊃━☆゜.*

def process (%{data:awesomoness}

      """
  end
👍  
properties (23)
authorjordanlindsey
permlinkre-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t204624459z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-25 20:46:27
last_update2017-06-25 20:46:27
depth1
children0
last_payout2017-07-02 20:46:27
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length117
author_reputation4,583,874,595,975
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,092,324
net_rshares858,917,637
author_curate_reward""
vote details (1)
@luxman ·
Nice
properties (22)
authorluxman
permlinkre-ontofractal-2017626t142919291z
categorysteemdev
json_metadata{"tags":"steemdev","app":"esteem/1.4.5","format":"markdown+html","community":"esteem"}
created2017-06-26 07:29:24
last_update2017-06-26 07:29:24
depth1
children0
last_payout2017-07-03 07:29:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length4
author_reputation684,988,462,931
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries
0.
accountesteemapp
weight500
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,147,241
net_rshares0
@rizatr ·
"I've been working to improve the developer experience for Steemex and this release does exactly that."
How long did that working last to get the release through? 
Might look an eye on the mobiel app too ? ;)
properties (22)
authorrizatr
permlinkre-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t111423416z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-25 11:14:06
last_update2017-06-25 11:14:06
depth1
children1
last_payout2017-07-02 11:14:06
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length208
author_reputation204,467,685,600
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,028,635
net_rshares0
@ontofractal ·
This is a library for developers, not a user facing app though :)
properties (22)
authorontofractal
permlinkre-rizatr-re-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t111622100z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-25 11:16:21
last_update2017-06-25 11:16:21
depth2
children0
last_payout2017-07-02 11:16:21
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length65
author_reputation61,670,850,962,884
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,028,850
net_rshares0
@ronmamita · (edited)
I apologize for my blockchain-programming ignorance, my 2 basic questions are:
1. will censorship & security issues be addressed?
2. will the endusers like me  see filtering & search features alleviate the congested resteemit mess that clog up flow to original posts on profile pages?

Like I said I do truly apologize if the above is unrelated, but perhaps your programming buddies are looking into these issues and you could discuss it as a seamless fit or transition.

Thanks for sharing and thank you for being here!
properties (22)
authorronmamita
permlinkre-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t124436840z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-25 12:44:36
last_update2017-06-25 12:46:12
depth1
children0
last_payout2017-07-02 12:44:36
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length520
author_reputation770,066,160,449
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,036,954
net_rshares0
@sdilan ·
Guys Elixir is programing language from what i understood. The writter is talking about how it is easy to use and is helpful for creating token apps. right ?
@ontofractal?
👍  
properties (23)
authorsdilan
permlinkre-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170626t012250110z
categorysteemdev
json_metadata{"tags":["steemdev"],"users":["ontofractal"],"app":"steemit/0.1"}
created2017-06-26 01:22:45
last_update2017-06-26 01:22:45
depth1
children0
last_payout2017-07-03 01:22:45
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length171
author_reputation126,588,638,903
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,117,922
net_rshares633,023,718
author_curate_reward""
vote details (1)
@the.dajboz ·
Great blog! Check out my new post @the.dajboz follow me! :)
properties (22)
authorthe.dajboz
permlinkre-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170625t115342024z
categorysteemdev
json_metadata{"tags":["steemdev"],"users":["the.dajboz"],"app":"steemit/0.1"}
created2017-06-25 11:53:42
last_update2017-06-25 11:53:42
depth1
children0
last_payout2017-07-02 11:53:42
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length59
author_reputation69,389,649,798
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,032,126
net_rshares0
@trendo ·
Steemit just keeps improving !
properties (22)
authortrendo
permlinkre-ontofractal-steemex-v0-12-released-now-with-improved-blockchain-event-pipeline-open-source-elixir-library-for-steem-20170626t001136109z
categorysteemdev
json_metadata{"tags":["steemdev"],"app":"steemit/0.1"}
created2017-06-26 00:11:36
last_update2017-06-26 00:11:36
depth1
children0
last_payout2017-07-03 00:11:36
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length30
author_reputation5,793,651,533,029
root_title"Steemex v0.12 released: now with improved blockchain event pipeline [open source Elixir library for Steem]"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id6,112,113
net_rshares0