<center> https://steemitimages.com/0x0/http://cryptogab.com/wp-content/uploads/2014/09/iiskg2.jpg </center> 

# TL;DR #

I wanted to share with you all how to begin building an open-source platform for analyzing cryptocurrency market data. If you aren't a fan of getting your trading advice from platforms derived from traditional markets and are curious about building your own tools for forecasting and investing, I hope you'll read further and comment.

All feedback is appreciated. 

# Free Datahz? #
There are many APIs to choose from with more detailed options ... often with a price tag.

I chose this API for a few reasons:
* Free (for now?)
* Historical Data for Many Alt-Coins
* Easy of Use!
        (Thanks to http://coinmarketcap.northpole.ro)

# Setup #
<center>https://steemitimages.com/0x0/http://4.bp.blogspot.com/-1zFG4UY4gKg/UpzI685WlKI/AAAAAAAAIco/IHY2I3bmHeU/s1600/to+the+moon+meme+2.jpg </center>
### Get VBox ###
https://www.virtualbox.org/wiki/Downloads

### Get ELK VM ###
https://bitnami.com/redirect/to/151919/bitnami-elk-5.4.1-0-linux-debian-8-x86_64.ova

### Config ELK VM ###
_Change Adapter 1 to Host-Only_
<center>![1.png](https://steemitimages.com/DQmVJFgxN3miT8qVEoTjJZ2wEtFZT8DjL4CRWE6HPD8xLSx/1.png)</center>

_Enable Adapter 2 to NAT_
<center>![2.png](https://steemitimages.com/DQmRJTMCEBYYz8SzpQaPpcfJS9oh62xUz4c5m5NzPMikyj1/2.png)</center>

_For Improved Performance, Adjust RAM/Processors/Etc._
<center>![3.png](https://steemitimages.com/DQmRksixxw8nkhxFMCFhN2WGCpkb5s6Esnd48JooG9x354k/3.png)</center>

_Login with Defaults_
**bitnami/bitnami**
(Change password)

### Enable ssh (this can be skipped but, I preferred ssh for this) ###
    sudo rm -f /etc/ssh/sshd_not_to_be_run
    sudo systemctl enable ssh
    sudo systemctl start ssh

### Install python + pip + elasticsearch.py ###
    sudo apt-get upgrade -y python-dev libssl-dev git-dev
    sudo python < <(curl -s https://bootstrap.pypa.io/get-pip.py)
    sudo pip install elastic search

### Check IP Address ###
    sudo ifconfig
    eth0      Link encap:Ethernet  HWaddr de:ad:be:ef:ca:fe
              inet addr:192.168.100.101  Bcast:192.168.100.255  Mask:255.255.255.0

# Now we are ready to get going ... #
<center>https://steemitimages.com/0x0/http://3.bp.blogspot.com/-xpJ5piAA6vQ/UpzI6GvaoXI/AAAAAAAAIcg/IIbdxoSQt8s/s1600/to+the+moon+meme.jpg</center>
The basic workflow is like so:
1. Fetch JSON data from API
1. Parse JSON data
1. Configure elasticsearch index mapping to use the JSON object timestamp
1. Send result of parsed JSON data to elasticsearch

This script can be saved to to something like chistory.py:

    #Imports
    import sys
    import datetime
    import requests
    import json
    from elasticsearch import Elasticsearch

    #Variables
    server = 'localhost'
    arg1 = sys.argv[1]
    arg2 = sys.argv[2]
    doctype = arg2.lower()
    id = arg1

    #Send data to ELK function
    def sendtoelastic(data):
        try:
            es = Elasticsearch(server)
            es.index(index=id, doc_type=doctype, body=data)
        except Exception as e:
            print 'Error sending to elasticsearch: {0}'.format(str(e))
            pass

    #Send settings to ELK function
    def settingselastic(data):
        try:
            es = Elasticsearch(server)
            es.indices.create(index=id, ignore=400, body=data)
        except Exception as e:
            print 'Error sending to elasticsearch: {0}'.format(str(e))
            pass

    #Build request, download and load JSON data
    payload = {'coin': arg2, 'period': arg1}
    r = requests.get('http://coinmarketcap.northpole.ro/history.json?', params=payload)
    h = r.text
    d = json.loads(h)['history']

    #Send settings
    settings = '''{"mappings": {''' + '"' + arg1 + '"' + ''': {"properties": {"timestamp": {"type": "date","format": "epoch_second"}}}}}'''
    settingselastic(settings)

    #Send data
    for i in d.items():
        for l in i:
            if type(l) is dict:
                sendtoelastic(json.dumps(l))


### **Usage:** ###
    python chistory.py 2017 BTC
    python chistory.py 2017 BTC;python chistory.py 2017 ETH;python chistory.py 2017 DOGE
    python chistory.py 2016 BTC;python chistory.py 2016 ETH;python chistory.py 2016 DOGE

### **Coins:** ###
A current list of coins can be found [here](http://coinmarketcap.northpole.ro/coins.json)

### **Dumping Index:** ###
If for any reason we aren't happy with the results, we can dump elastic like this:

    curl -XDELETE 'http://localhost:9200/_all'
            
# Haz Datahs ... Now What? #
<center> https://steemitimages.com/0x0/https://pbs.twimg.com/media/CEWFP8oWIAAFoaG.jpg </center>

Here's a quick how-to visualize your data in Kibana with a few line graphs which may be of interest ...
* Open your browser and go to http://192.168.100.101/elk (use IP address from above)
(user/bitnami)
* Create Index "201*"

### Create Visualization ###
**__Change.1h x Date__**
1. Click the "Visualize" Button
1. Click "Create New Visualization"
1. Click "Basic Charts > Line"
1. Select Index "201*"
1. Click Drop Down Button for Y-Axis
1. Select Aggregation > "Sum"
1. Select Field > "change1h"
<center> ![5.png](https://steemitimages.com/DQmSnBy9EYyfB6fJxDLsBAsPugPckrveMVU19mtnT9Nhsh4/5.png) </center>
1. Click "Select Buckets Type" > X-Axis
1. Select Aggregation > "Date Histogram"
1. Select Field > "timestamp"
1. Select Interval > "Daily"
<center> ![6.png](https://steemitimages.com/DQmUsaBEpDfHZRmJXU727yx8iAXpAzty2FoDJNGVdi8XwS2/6.png) </center>
1. Click "Add Sub-Buckets" > "Split Series"
1. Sub-Aggregation > "Filters"
1. Filter 1 > "btc"
1. Filter 2 > "eth"
1. Filter 3 > "doge"
<center> ![7.png](https://steemitimages.com/DQmReuqK4DhELJAyaQyhsV4ueFiN9yuQeTRJDd1hxnvAAav/7.png) </center>
1. Save

**__Change.24h x Date__**
1. Change Y-Axis Field > "change24h" 
1. Save
1. Change name Change.24h
1. Check "Save as a New Visualization" Option
1. Save

**__Volume.EUR x Date__**
1. Change Y-Axis Field > "volume24.eur" 
1. Save
1. Change name Volume.EUR
1. Check "Save as a New Visualization" Option
1. Save

### Create a Dashboard ###
1. Select Dashboard Button
1. Create New Dashboard
1. Add Visualizations to Dashboard

![4.png](https://steemitimages.com/DQmcX1wK8BPKWKrHjvzZi6tiNfqocR8aejn9DJdjQrfpTz5/4.png)

# On APIs #
This is just a basic example of leveraging an API for analytics.

The basic mechanics ought be able to be applied to other APIs of interest.

If you'd like to see more API analytics, follow and up vote. I hope this helps someone!

If you made it this far ... Thanks!