Viewing a response to: @lemouth/particle-physics-utopian-io-objects-isolation-histogramming-and-a-first-task-request
>Completely unrelated to the rest, there is no dedicated module allowing one to read a histos.saf file and get plots out of it. I would like to get a Python code (potentially relying on matplotlib) allowing to do so. This caught my attention. I spent some of the spring semester writing custom R code to create histograms (actually ended up with probability distro curves, but histos were created along the way) from the raw per-cell data from some fluid dynamics experiments. For example, plotting the volumetric distribution of the kinetic energy dissipation rate throughout the reactor, rather than relying on an average value. edit: Forgot to ask my actual question. Do you know wow utopian deals with multiple people implementing answers to task requests? edit: Forgot to ask my actual question. Do you know wow utopian deals with multiple people implementing answers to task requests?
| author | effofex |
|---|---|
| permlink | re-lemouth-particle-physics-utopian-io-objects-isolation-histogramming-and-a-first-task-request-20180618t190415162z |
| category | utopian-io |
| json_metadata | {"tags":["utopian-io"],"app":"steemit/0.1"} |
| created | 2018-06-18 19:04:15 |
| last_update | 2018-06-18 19:07:30 |
| depth | 1 |
| children | 23 |
| last_payout | 2018-06-25 19:04:15 |
| cashout_time | 1969-12-31 23:59:59 |
| total_payout_value | 0.035 HBD |
| curator_payout_value | 0.012 HBD |
| pending_payout_value | 0.000 HBD |
| promoted | 0.000 HBD |
| body_length | 896 |
| author_reputation | 14,429,105,750,792 |
| root_title | "Particle physics @ utopian-io - Objects isolation, histogramming and a first task request" |
| beneficiaries | [] |
| max_accepted_payout | 1,000,000.000 HBD |
| percent_hbd | 10,000 |
| post_id | 61,263,097 |
| net_rshares | 23,258,854,645 |
| author_curate_reward | "" |
| voter | weight | wgt% | rshares | pct | time |
|---|---|---|---|---|---|
| lemouth | 0 | 23,258,854,645 | 5% |
hiveblocksA few more questions:
* Are the histogram names within the *.saf file unique (will "ptj1" only show up once?)
* There's no info on the x, y, and title in the file, what are the defaults you'd like?
* Are the under/overflow bins shown in the histogram?
* Are you looking for a python module which can be called from code or a command line tool?
* Do you want this to produce an image file or mataplotlib object which can be further edited?
To make sure I understand the data format:
>The total value corresponding to each bin is obtained from the sum of the numbers of the two columns.
For the following data:
```
<Data>
0.000000e+00 0.000000e+00 # underflow
0.000000e+00 1.000000e+00 # bin 1 / 3
3.000000e-04 0.000000e+00 # bin 2 / 3
2.000000e-04 3.000000e-04 # bin 3/ 3
0.000000e+00 0.000000e+00 # overflow
</Data>
```
The value of bins 1, 2, and 3 are 1, 3x10<sup>-4</sup>, and 5x10<sup>-4</sup>, respectively. Further, we don't need to show this split in the histogram. Correct?| author | effofex |
|---|---|
| permlink | re-effofex-re-lemouth-particle-physics-utopian-io-objects-isolation-histogramming-and-a-first-task-request-20180618t200008852z |
| category | utopian-io |
| json_metadata | {"tags":["utopian-io"],"app":"steemit/0.1"} |
| created | 2018-06-18 20:00:09 |
| last_update | 2018-06-18 20:00:09 |
| depth | 2 |
| children | 19 |
| last_payout | 2018-06-25 20:00:09 |
| cashout_time | 1969-12-31 23:59:59 |
| total_payout_value | 0.022 HBD |
| curator_payout_value | 0.005 HBD |
| pending_payout_value | 0.000 HBD |
| promoted | 0.000 HBD |
| body_length | 1,050 |
| author_reputation | 14,429,105,750,792 |
| root_title | "Particle physics @ utopian-io - Objects isolation, histogramming and a first task request" |
| beneficiaries | [] |
| max_accepted_payout | 1,000,000.000 HBD |
| percent_hbd | 10,000 |
| post_id | 61,268,501 |
| net_rshares | 13,786,543,531 |
| author_curate_reward | "" |
| voter | weight | wgt% | rshares | pct | time |
|---|---|---|---|---|---|
| mactro | 0 | 1,887,148,433 | 100% | ||
| dustsweeper | 0 | 11,899,395,098 | 2.6% |
Here are answers to your questions. Do not hesitate to come back to me if necessary. > Are the histogram names within the *.saf file unique (will "ptj1" only show up once?) All histograms are concatenated within a given SAF file. >There's no info on the x, y, and title in the file, what are the defaults you'd like? The title is given between quotes, right after the description. The axis names are not passed. You can leave them blank. However, it may be nice to allow the user to give them. > Are the under/overflow bins shown in the histogram? It would be nice to leave this option to the user. > Are you looking for a python module which can be called from code or a command line tool? Both can be done quite easily. > Do you want this to produce an image file or mataplotlib object which can be further edited? Once again, both. The matplotlib file is important to allow the user to tune it to the layout he/she wants. > The value of bins 1, 2, and 3 are 1, 3x10-4, and 5x10-4, respectively. Further, we don't need to show this split in the histogram. Correct? The total number to put in a bin corresponds to the first column minus the second one.
| author | lemouth |
|---|---|
| permlink | re-effofex-re-effofex-re-lemouth-particle-physics-utopian-io-objects-isolation-histogramming-and-a-first-task-request-20180619t132533827z |
| category | utopian-io |
| json_metadata | {"tags":["utopian-io"],"app":"steemit/0.1"} |
| created | 2018-06-19 13:25:36 |
| last_update | 2018-06-19 13:25:36 |
| depth | 3 |
| children | 18 |
| last_payout | 2018-06-26 13:25:36 |
| cashout_time | 1969-12-31 23:59:59 |
| total_payout_value | 0.060 HBD |
| curator_payout_value | 0.017 HBD |
| pending_payout_value | 0.000 HBD |
| promoted | 0.000 HBD |
| body_length | 1,165 |
| author_reputation | 338,011,164,701,274 |
| root_title | "Particle physics @ utopian-io - Objects isolation, histogramming and a first task request" |
| beneficiaries | [] |
| max_accepted_payout | 1,000,000.000 HBD |
| percent_hbd | 10,000 |
| post_id | 61,359,283 |
| net_rshares | 37,253,181,047 |
| author_curate_reward | "" |
| voter | weight | wgt% | rshares | pct | time |
|---|---|---|---|---|---|
| lordneroo | 0 | 34,190,630,002 | 100% | ||
| effofex | 0 | 3,062,551,045 | 100% |
Thanks for the clarifications. I think I understand it enough to get an implementation coded up.
| author | effofex |
|---|---|
| permlink | re-lemouth-re-effofex-re-effofex-re-lemouth-particle-physics-utopian-io-objects-isolation-histogramming-and-a-first-task-request-20180621t021158643z |
| category | utopian-io |
| json_metadata | {"tags":["utopian-io"],"app":"steemit/0.1"} |
| created | 2018-06-21 02:12:00 |
| last_update | 2018-06-21 02:12:00 |
| depth | 4 |
| children | 1 |
| last_payout | 2018-06-28 02:12:00 |
| cashout_time | 1969-12-31 23:59:59 |
| total_payout_value | 0.000 HBD |
| curator_payout_value | 0.000 HBD |
| pending_payout_value | 0.000 HBD |
| promoted | 0.000 HBD |
| body_length | 96 |
| author_reputation | 14,429,105,750,792 |
| root_title | "Particle physics @ utopian-io - Objects isolation, histogramming and a first task request" |
| beneficiaries | [] |
| max_accepted_payout | 1,000,000.000 HBD |
| percent_hbd | 10,000 |
| post_id | 61,573,274 |
| net_rshares | 0 |
I've got some good news. This is still janky as heck, but I've got a python tool which parses the saf file, converts it into a pandas dataframe, and allows me to spit out some histograms. The janky part is that the histograms are still essentially a proof-of-pipeline, matplotlib has turned out to be very different (not bad, just different) from the plotting language (ggplot2) I am used to.  This image will get better between now and Friday. My plan to is push my local repo to a fork of ma5 on my github account, then send you a pull request on Friday. Based on how that goes and your feedback, then write up a post. The downside is that I probably won't have time for 1c, but I can deal with that. It's been very informative using pandas and matplotlib on non-toy data.
| author | effofex |
|---|---|
| permlink | re-lemouth-re-effofex-re-effofex-re-lemouth-particle-physics-utopian-io-objects-isolation-histogramming-and-a-first-task-request-20180628t021631552z |
| category | utopian-io |
| json_metadata | {"tags":["utopian-io"],"image":["https://cdn.steemitimages.com/DQmVtg8unfSysiAfz7mJLYpq2p5A41KbsgLKcJVPM4xApkf/first_histo.png"],"app":"steemit/0.1"} |
| created | 2018-06-28 02:16:30 |
| last_update | 2018-06-28 02:16:30 |
| depth | 4 |
| children | 11 |
| last_payout | 2018-07-05 02:16:30 |
| cashout_time | 1969-12-31 23:59:59 |
| total_payout_value | 0.000 HBD |
| curator_payout_value | 0.000 HBD |
| pending_payout_value | 0.000 HBD |
| promoted | 0.000 HBD |
| body_length | 897 |
| author_reputation | 14,429,105,750,792 |
| root_title | "Particle physics @ utopian-io - Objects isolation, histogramming and a first task request" |
| beneficiaries | [] |
| max_accepted_payout | 1,000,000.000 HBD |
| percent_hbd | 10,000 |
| post_id | 62,555,997 |
| net_rshares | 0 |
Hi, sorry if this is strange question but I don't understand the resulting bin data values in the SAF file. I thought that in the histogram the bin value should be whole numbers representing the number of particles matching a particular value range. Am I wrong about this?
| author | irelandscape |
|---|---|
| permlink | re-lemouth-re-effofex-re-effofex-re-lemouth-particle-physics-utopian-io-objects-isolation-histogramming-and-a-first-task-request-20180622t155802354z |
| category | utopian-io |
| json_metadata | {"tags":["utopian-io"],"app":"steemit/0.1"} |
| created | 2018-06-22 15:58:03 |
| last_update | 2018-06-22 15:58:03 |
| depth | 4 |
| children | 3 |
| last_payout | 2018-06-29 15:58:03 |
| cashout_time | 1969-12-31 23:59:59 |
| total_payout_value | 0.000 HBD |
| curator_payout_value | 0.000 HBD |
| pending_payout_value | 0.000 HBD |
| promoted | 0.000 HBD |
| body_length | 274 |
| author_reputation | 15,380,678,988,494 |
| root_title | "Particle physics @ utopian-io - Objects isolation, histogramming and a first task request" |
| beneficiaries | [] |
| max_accepted_payout | 1,000,000.000 HBD |
| percent_hbd | 10,000 |
| post_id | 61,790,525 |
| net_rshares | 0 |
> This caught my attention. I spent some of the spring semester writing custom R code to create histograms (actually ended up with probability distro curves, but histos were created along the way) from the raw per-cell data from some fluid dynamics experiments. For example, plotting the volumetric distribution of the kinetic energy dissipation rate throughout the reactor, rather than relying on an average value. Actually, I insist on the python option. The reason is that the code is meant to be used by physicists, and physicists do not use R in general (python, c++ and Fortran are the mostly spread programming languages). > edit: Forgot to ask my actual question. Do you know wow utopian deals with multiple people implementing answers to task requests? I have no idea. I suppose all will be rewarded if they are nicely done. On my side, everything that works well will be advertised on the MadAnalysis 5 website. The best module may even be merged with the main branch of madanalysis 5, provided the author agrees.
| author | lemouth |
|---|---|
| permlink | re-effofex-re-lemouth-particle-physics-utopian-io-objects-isolation-histogramming-and-a-first-task-request-20180619t112625281z |
| category | utopian-io |
| json_metadata | {"tags":["utopian-io"],"app":"steemit/0.1"} |
| created | 2018-06-19 11:26:27 |
| last_update | 2018-06-19 11:26:27 |
| depth | 2 |
| children | 2 |
| last_payout | 2018-06-26 11:26:27 |
| cashout_time | 1969-12-31 23:59:59 |
| total_payout_value | 0.000 HBD |
| curator_payout_value | 0.000 HBD |
| pending_payout_value | 0.000 HBD |
| promoted | 0.000 HBD |
| body_length | 1,026 |
| author_reputation | 338,011,164,701,274 |
| root_title | "Particle physics @ utopian-io - Objects isolation, histogramming and a first task request" |
| beneficiaries | [] |
| max_accepted_payout | 1,000,000.000 HBD |
| percent_hbd | 10,000 |
| post_id | 61,347,510 |
| net_rshares | 1,858,841,207 |
| author_curate_reward | "" |
| voter | weight | wgt% | rshares | pct | time |
|---|---|---|---|---|---|
| mactro | 0 | 1,858,841,207 | 100% |
No worries on the Python requirement, happy to code in that environment.<br/><div class="pull-right"><sub><a href="https://steemit.com/utopian-io/@roxane/fast-reply-v0-1-never-miss-to-answer-a-comment-again-and-do-it-faster-than-ever">Sent with Fast-Reply</a></sub></div>
| author | effofex |
|---|---|
| permlink | re-effofex-re-lemouth-particle-physics-utopian-io-objects-isolation-histogramming-and-a-first-task-request-20180619t112625281z-1529547000061 |
| category | utopian-io |
| json_metadata | {"app":"fast-reply","version":0.2} |
| created | 2018-06-21 02:10:12 |
| last_update | 2018-06-21 02:10:12 |
| depth | 3 |
| children | 1 |
| last_payout | 2018-06-28 02:10:12 |
| cashout_time | 1969-12-31 23:59:59 |
| total_payout_value | 0.000 HBD |
| curator_payout_value | 0.000 HBD |
| pending_payout_value | 0.000 HBD |
| promoted | 0.000 HBD |
| body_length | 271 |
| author_reputation | 14,429,105,750,792 |
| root_title | "Particle physics @ utopian-io - Objects isolation, histogramming and a first task request" |
| beneficiaries | [] |
| max_accepted_payout | 1,000,000.000 HBD |
| percent_hbd | 10,000 |
| post_id | 61,573,124 |
| net_rshares | 0 |
Cool! :)
| author | lemouth |
|---|---|
| permlink | re-effofex-re-effofex-re-lemouth-particle-physics-utopian-io-objects-isolation-histogramming-and-a-first-task-request-1529547000061-20180621t184720755z |
| category | utopian-io |
| json_metadata | {"tags":["utopian-io"],"app":"steemit/0.1"} |
| created | 2018-06-21 18:47:21 |
| last_update | 2018-06-21 18:47:21 |
| depth | 4 |
| children | 0 |
| last_payout | 2018-06-28 18:47:21 |
| cashout_time | 1969-12-31 23:59:59 |
| total_payout_value | 0.000 HBD |
| curator_payout_value | 0.000 HBD |
| pending_payout_value | 0.000 HBD |
| promoted | 0.000 HBD |
| body_length | 8 |
| author_reputation | 338,011,164,701,274 |
| root_title | "Particle physics @ utopian-io - Objects isolation, histogramming and a first task request" |
| beneficiaries | [] |
| max_accepted_payout | 1,000,000.000 HBD |
| percent_hbd | 10,000 |
| post_id | 61,677,817 |
| net_rshares | 0 |