create account

Making the Hive Blog Wallet Financial Report Useable in Cointracking with Python/Pandas/Jupyter by joshman

View this thread on: hive.blogpeakd.comecency.com
· @joshman ·
$64.31
Making the Hive Blog Wallet Financial Report Useable in Cointracking with Python/Pandas/Jupyter
### The Problem
A few posts ago, I complained about the inability to import certain HIVE wallet transactions into Cointracking.  Rather than manually hack at another spreadsheet after January, I decided to try my hand at automating it.  What I needed was a way to translate the CSV file that the Hive.blog wallet spits out to a format that Cointracking recognizes.

![image.png](https://images.hive.blog/DQmSMYqms9nsZQqTL3GhyWzdX47LwmLbbq517suUQXk7pZy/image.png)
![image.png](https://images.hive.blog/DQmbo1N5LB8YqtCCGHyLfPVMNX7e6j3DDBUGS72RBf3CNSZ/image.png)
*Hive.blog export*

![image.png](https://images.hive.blog/DQmXvPEmpViEBcHdFpRoFCLtiq25cbsdEqg7VrHdWZxAHMa/image.png)
![image.png](https://images.hive.blog/DQmWyq8comCmXLzgNDy6QW5jsVLhTY61Rcztu2cdS7qcTwL/image.png)
*Cointracking Import*

As you can see, the CSV formats don't match at all, and unfortunately Cointracking doesn't have native Hive support.
![Untitled design.png](https://images.hive.blog/DQmXq356zS8yBcygigWGQiXkRK6jCyojRq3GGJmnWth7iSw/Untitled%20design.png)

### My Solution
First off, I am not a developer.  At best I'm a coding hack.  If you developers know of a easier way to refactor this, I'm all 'ears'.  For my solution, I decided to use Python with Jupyter notebook and the Pandas library.  Jupyter notebook is great for hacks like me by the way.  You can run all of this yourself by downloading the Anaconda Personal Edition for free ([https://www.anaconda.com/products/individual](https://www.anaconda.com/products/individual)).

The steps I took went basically like this.
* Download a copy of my Hive transaction history and load it into a Pandas data frame.
* Drop any columns from the Hive wallet output I didn't need
* Rename the remaining columns to make them Cointracking compliant
* Remove any rows from the transaction types that require claiming (curation, author, and comment rewards)
* Arrange the rows in the proper order to make them Cointracking compliant
* Insert any columns required by Cointracking (Whether they get populated or not)
* Populate any necessary columns (Exchange, Trade-Group, Comment)
* Split the 'amount' column into two new columns ('Buy Amount' & 'Buy Currency')
* Any required formatting, such as removing whitespace and the 'T' from the Date/timestamp.

![Untitled design.png](https://images.hive.blog/DQmXq356zS8yBcygigWGQiXkRK6jCyojRq3GGJmnWth7iSw/Untitled%20design.png)

### The Code
Here is the code I used.  I got lazy with the variable I used for my data frame, but you should get the idea.  I will probably work to make it more intuitive.  The good news I could  get everything I needed done with just the Pandas libary.

```python
#Import the Pandas Library
import pandas as pd
#read csv into raw var
raw = pd.read_csv('hive-report.csv')
#Drop unneeded columns
raw.drop(columns=[' comment_author',' comment_permlink',' author',' permlink', ' curator', ' payout_must_be_claimed',' reward',' memo',' hbd_payout'], inplace=True)
raw.drop(columns=[' hive_payout',' vesting_payout',' payout',' to',' from' ], inplace=True)
#Rename needed columns
raw.rename(columns={'timestamp': 'Date',' opType': 'Type',' amount': 'Buy Amount' }, inplace=True)
#Remove unneeded transaction types
remove_types = raw[raw["Type"].str.contains(" curation_reward| author_reward| comment_reward")==False]
#Arrange types in proper order
arrange_types = remove_types[['Type','Buy Amount', 'Date']]
#Replace transfer with deposit
arrange_types['Type'] = arrange_types['Type'].replace([' transfer'], 'Income')
#Strip whitespace from Buy Amount
arrange_types['Buy Amount'] = arrange_types['Buy Amount'].str.lstrip()
#Split price from currency
arrange_types[['Buy Amount','Buy Currency']] = arrange_types['Buy Amount'].str.split(' ', 1, expand=True)
#Rearrange columns
arrange_types = arrange_types[['Type','Buy Amount', 'Buy Currency', 'Date']]
#Insert required columns for Cointracking
arrange_types.insert(3,'Sell Amount','')
arrange_types.insert(4,'Sell Currency','')
arrange_types.insert(5,'Fee','')
arrange_types.insert(6,'Fee Currency','')
arrange_types.insert(7,'Exchange','')
arrange_types.insert(8,'Trade-Group','')
arrange_types.insert(9,'Comment','')
#Assign values for exchange, trade-group, and comments
arrange_types.loc[:,'Exchange'] = 'HIVE WALLET'
arrange_types.loc[:,'Trade-Group'] = 'HIVE CSV'
arrange_types.loc[:,'Comment'] = 'Delegation Income'
#Remove T character from date
arrange_types['Date'] = arrange_types['Date'].str.replace('T', ' ')
#Save to a new csv
arrange_types.to_csv('hive-report2.csv', index=False)
```
![Untitled design.png](https://images.hive.blog/DQmXq356zS8yBcygigWGQiXkRK6jCyojRq3GGJmnWth7iSw/Untitled%20design.png)

### Proof of Work

![image.png](https://images.hive.blog/DQmW8DZ9AKAkwe3qTeQfv6Xo7VyDg3AeezatHErwBQw1BN7/image.png)
![image.png](https://images.hive.blog/DQma53bfv57FzBTSjhBDEovUaRmWoKx1TTdQndVnNhkyRAa/image.png)



As you can see above, the CSV I generated was property formatted and successfully imported into cointracking.

![Untitled design.png](https://images.hive.blog/DQmXq356zS8yBcygigWGQiXkRK6jCyojRq3GGJmnWth7iSw/Untitled%20design.png)

### Conclusion

This was a fun exercise for me.  I'm not a developer (obviously).  The good news is I can use similar steps for other Exchanges and wallets that export CSV files that Cointracking doesn't support.  I will not have to hack at spreadsheets every month for various transactions exports. If any Python/Pandas experts out there read this, please do critique my work.  Obviously this is functional, but I would be interested to know if there is a more efficient way of performing the same work.  If you were dying for a way to record HIVE deposits, please feel free to use this and let me know if you have any questions.

Posted Using [LeoFinance <sup>Beta</sup>](https://leofinance.io/@joshman/making-the-hive-blog-wallet-financial-report-useable-in-cointracking-with-python-pandas-jupyter)
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 223 others
properties (23)
authorjoshman
permlinkmaking-the-hive-blog-wallet-financial-report-useable-in-cointracking-with-python-pandas-jupyter
categoryhive-167922
json_metadata{"app":"leofinance/0.2","format":"markdown","tags":["palnet","leofinance","cryptocurrency","hive","python","learntocode","neoxian"],"canonical_url":"https://leofinance.io/@joshman/making-the-hive-blog-wallet-financial-report-useable-in-cointracking-with-python-pandas-jupyter","links":["https://www.anaconda.com/products/individual"],"image":["https://images.hive.blog/DQmSMYqms9nsZQqTL3GhyWzdX47LwmLbbq517suUQXk7pZy/image.png","https://images.hive.blog/DQmbo1N5LB8YqtCCGHyLfPVMNX7e6j3DDBUGS72RBf3CNSZ/image.png","https://images.hive.blog/DQmXvPEmpViEBcHdFpRoFCLtiq25cbsdEqg7VrHdWZxAHMa/image.png","https://images.hive.blog/DQmWyq8comCmXLzgNDy6QW5jsVLhTY61Rcztu2cdS7qcTwL/image.png","https://images.hive.blog/DQmXq356zS8yBcygigWGQiXkRK6jCyojRq3GGJmnWth7iSw/Untitled%20design.png","https://images.hive.blog/DQmW8DZ9AKAkwe3qTeQfv6Xo7VyDg3AeezatHErwBQw1BN7/image.png","https://images.hive.blog/DQma53bfv57FzBTSjhBDEovUaRmWoKx1TTdQndVnNhkyRAa/image.png"]}
created2022-01-27 23:12:33
last_update2022-01-27 23:12:33
depth0
children6
last_payout2022-02-03 23:12:33
cashout_time1969-12-31 23:59:59
total_payout_value32.225 HBD
curator_payout_value32.088 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length5,909
author_reputation277,932,931,100,174
root_title"Making the Hive Blog Wallet Financial Report Useable in Cointracking with Python/Pandas/Jupyter"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id109,911,402
net_rshares51,415,170,537,088
author_curate_reward""
vote details (287)
@carys ·
$0.31
Interesting!  I don't know why you say you are not a developer.  There are developers at all sorts of levels.  Clearly, you have a handle on the fundamental skills you need.  Becoming more senior is a matter of practice, patience, and time.  Ironically, although I never worked as a developer, several developers used to come to my desk to ask for help.  Ironically because, since "developer" was not in my job title or description, I later could not get a job as a developer even though my ability must have been reasonable for the word to have spread enough for people to come for help.  

You are developing software, so... congratulations, you are a developer! My two cents, anyhow.
👍  ,
properties (23)
authorcarys
permlinkre-joshman-2022130t51421707z
categoryhive-167922
json_metadata{"tags":["hive-167922","palnet","leofinance","cryptocurrency","hive","python","learntocode","neoxian"],"app":"ecency/3.0.24-mobile","format":"markdown+html"}
created2022-01-30 12:14:24
last_update2022-01-30 12:14:24
depth1
children1
last_payout2022-02-06 12:14:24
cashout_time1969-12-31 23:59:59
total_payout_value0.155 HBD
curator_payout_value0.154 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length686
author_reputation1,682,171,130,249
root_title"Making the Hive Blog Wallet Financial Report Useable in Cointracking with Python/Pandas/Jupyter"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id109,982,929
net_rshares215,559,390,493
author_curate_reward""
vote details (2)
@joshman ·
$0.02
You're right, technically I developed something.  Thanks for the vote of confidence, I'll take it!  
👍  ,
properties (23)
authorjoshman
permlinkre-carys-r6m4g9
categoryhive-167922
json_metadata{"tags":["hive-167922"],"app":"peakd/2022.01.2"}
created2022-02-01 06:24:12
last_update2022-02-01 06:24:12
depth2
children0
last_payout2022-02-08 06:24:12
cashout_time1969-12-31 23:59:59
total_payout_value0.010 HBD
curator_payout_value0.010 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length100
author_reputation277,932,931,100,174
root_title"Making the Hive Blog Wallet Financial Report Useable in Cointracking with Python/Pandas/Jupyter"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id110,035,147
net_rshares16,235,506,775
author_curate_reward""
vote details (2)
@gangstalking ·
Electronic-terrorism, voice to skull and neuro monitoring on Hive and Steem. You can ignore this, but your going to wish you didnt soon. This is happening whether you believe it or not. https://ecency.com/fyrstikken/@fairandbalanced/i-am-the-only-motherfucker-on-the-internet-pointing-to-a-direct-source-for-voice-to-skull-electronic-terrorism
👎  , , , , , , , , , ,
properties (23)
authorgangstalking
permlinkre-joshman-making-the-hive-blog-wallet-financial-report-useable-in-cointracking-with-python-pandas-jupyter-20220127t231242696z
categoryhive-167922
json_metadata{"app":"hive-bot/0.6.3"}
created2022-01-27 23:12:45
last_update2022-01-27 23:12:45
depth1
children0
last_payout2022-02-03 23:12:45
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length343
author_reputation-67,597,107,868,724
root_title"Making the Hive Blog Wallet Financial Report Useable in Cointracking with Python/Pandas/Jupyter"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id109,911,406
net_rshares-1,914,609,843,936
author_curate_reward""
vote details (11)
@india-leo ·
$0.12
Indiaunited Curation 1643339234492
This post has been manually curated by @bhattg from Indiaunited community. Join us on our [Discord Server](https://discord.gg/bGmS2tE). 

Do you know that you can earn a passive income by delegating your Leo power to @india-leo account? We share 100 % of the curation rewards with the delegators. 

<sub>**Please contribute to the community by upvoting this comment and posts made by @indiaunited.**</sub>
👍  
properties (23)
authorindia-leo
permlinkindiaunited-1643339234492
categoryhive-167922
json_metadata{"app":"hiveblog/0.1","tags":["palnet","leofinance","cryptocurrency","hive","python","learntocode","neoxian"]}
created2022-01-28 03:07:15
last_update2022-01-28 03:07:15
depth1
children0
last_payout2022-02-04 03:07:15
cashout_time1969-12-31 23:59:59
total_payout_value0.061 HBD
curator_payout_value0.061 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length405
author_reputation7,493,899,125,940
root_title"Making the Hive Blog Wallet Financial Report Useable in Cointracking with Python/Pandas/Jupyter"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id109,916,283
net_rshares98,592,154,493
author_curate_reward""
vote details (1)
@logicforce ·
$0.27
Oh this is great but if one wants to use it for tax purposes is there a way to know how much those transactions are in $USD? Thanks
👍  ,
properties (23)
authorlogicforce
permlinkre-joshman-2022128t127100z
categoryhive-167922
json_metadata{"tags":["palnet","leofinance","cryptocurrency","hive","python","learntocode","neoxian"],"app":"ecency/3.0.20-vision","format":"markdown+html"}
created2022-01-28 06:02:06
last_update2022-01-28 06:02:06
depth1
children1
last_payout2022-02-04 06:02:06
cashout_time1969-12-31 23:59:59
total_payout_value0.133 HBD
curator_payout_value0.133 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length131
author_reputation15,626,293,016,011
root_title"Making the Hive Blog Wallet Financial Report Useable in Cointracking with Python/Pandas/Jupyter"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id109,919,527
net_rshares216,679,951,490
author_curate_reward""
vote details (2)
@joshman ·
Good question. Cointracking calculates that for you based upon the date/time.  Most of the tracking software does.  Otherwise you'd need to go look at historical prices and record them for each transaction since the price continually fluctuates.
properties (22)
authorjoshman
permlinkre-logicforce-r6erqs
categoryhive-167922
json_metadata{"tags":["hive-167922"],"app":"peakd/2022.01.2"}
created2022-01-28 07:06:30
last_update2022-01-28 07:06:30
depth2
children0
last_payout2022-02-04 07:06:30
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length245
author_reputation277,932,931,100,174
root_title"Making the Hive Blog Wallet Financial Report Useable in Cointracking with Python/Pandas/Jupyter"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id109,920,507
net_rshares0