create account

HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql? by geekgirl

View this thread on: hive.blogpeakd.comecency.com
· @geekgirl · (edited)
$33.30
HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?
![hivesql.png](https://images.hive.blog/DQmf1s2Twb8MQV8X5UHwRSv4U7d6sKBd78sSY3bry9bJyVV/hivesql.png)

For those who still don't know what `HiveSQL` is, it is a centralized MSSQL database developed and maintained by Arcange. It is one of the oldest services on Hive and has been around for years. It is always updated and provides access to Hive blockchain data with simple or complex SQL queries.

HiveSQL is great for anybody who wants to get data from Hive blockchain fast. If you plan to make an App, or compile some specific stats, or just want to learn and practice your SQL skills with real world database, give HiveSQL a try. It is free for anybody with a Hive account.

When I first started using HiveSQL, I had absolutely zero knowledge of SQL. Using HiveSQL is how learned SQL. 

I prefer to use HiveSQL within python scripts. Because before I was started learning SQL, I was trying to learn python. Knowing a little bit of python gives more options of how to get data from Hive. Using `Beem` module by Holger80, we can get data from Hive blockchain directly. Beem is great. However, getting data directly from the blockchain in some cases may be slow. Sometime even too slow. Perhaps, best way is to use both Beem and HiveSQL.

When I first started using HiveSQL, I used `pypyodbc` module. At first it was working good. But later I realized it wasn't delivering all the data I wanted and some data was getting truncated. Then, Crokkon suggested using `pyodbc` instead and it worked perfectly.

I have used pyodbc module for a while and didn't have any issues until I decided to deploy a simple app to heroku. The app wouldn't build properly due to some problems with pyodbc. I suspect it had to do with having proper driver. Normally on my local computer I would install the microsoft driver. In this case I couldn't do it on Heroku. I tries various solutions offered online. None of them worked. 

Before giving up on deploying the app to heroku, I decided to try one more thing. It was to use a different module - `pmssql`. It uses FreeTDS, which is easy to install on a computer and seems to be available on heroku already.

The best thing is to connect to the an MSSQL database all three of these modules use the similar methods, so it is not difficult to adjust to code from one to another. With pypyodbc and pyodbc we need to pass the driver name as argument, and it is not needed with pymssql. Everything else, was almost same.

Once I changed to pymssql, the app deployed to heroku without any issue. For this reason I would recommend using pymssql. I haven't had any problems using it yet.

I use the following simple python function to connect to HiveSQL and the results back. Afterwards I used the results within python code to do whatever else needed to be come with the data received.

```
def hive_sql(SQLCommand, limit):
    db = os.environ['HIVESQL'].split()
    conn = pymssql.connect(server=db[0], user=db[1], password=db[2], database=db[3])
    cursor = conn.cursor()
    cursor.execute(SQLCommand)
    result = cursor.fetchmany(limit)
    conn.close()
    return result
```

It takes SQL query commands as an argument and returns the results from HiveSQL. Within the python code we can write our SQL query and assign it to SQLCommand variable in a text format like this.

```
SQLCommand = '''
SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'DynamicGlobalProperties'
'''
```

Hive blockchain and HiveSQL are great place to start for those who are interested in learning a little bit of python and SQL.

I would like to continue sharing my experiences using HiveSQL with more task focused code snippets and keep building into something more interesting. Feel free to share you thoughts and experiences with HiveSQL.

Posted Using [LeoFinance <sup>Beta</sup>](https://leofinance.io/@geekgirl/hivesql-in-python-which-module-to-choose-pypyodbc-pyodbc-or-pymssql)
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 410 others
properties (23)
authorgeekgirl
permlinkhivesql-in-python-which-module-to-choose-pypyodbc-pyodbc-or-pymssql
categoryhive-167922
json_metadata{"app":"leofinance/0.2","format":"markdown","tags":["hive-167922","hive","hivesql","sql","python","database","blockchain","neoxian","palnet","posh","leofinance"],"canonical_url":"https://leofinance.io/@geekgirl/hivesql-in-python-which-module-to-choose-pypyodbc-pyodbc-or-pymssql","links":["https://leofinance.io/@geekgirl/hivesql-in-python-which-module-to-choose-pypyodbc-pyodbc-or-pymssql"],"image":["https://images.hive.blog/DQmf1s2Twb8MQV8X5UHwRSv4U7d6sKBd78sSY3bry9bJyVV/hivesql.png"]}
created2021-02-13 02:50:06
last_update2021-02-13 07:09:57
depth0
children17
last_payout2021-02-20 02:50:06
cashout_time1969-12-31 23:59:59
total_payout_value18.119 HBD
curator_payout_value15.177 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length3,911
author_reputation1,586,488,611,824,452
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,849,217
net_rshares69,609,672,649,805
author_curate_reward""
vote details (474)
@ace108 ·
$0.19
I used to use hivesql and used to use with python. I used pypyodbc
👍  
properties (23)
authorace108
permlinkqos8x2
categoryhive-167922
json_metadata{"app":"hiveblog/0.1"}
created2021-02-19 15:41:27
last_update2021-02-19 15:41:27
depth1
children0
last_payout2021-02-26 15:41:27
cashout_time1969-12-31 23:59:59
total_payout_value0.092 HBD
curator_payout_value0.093 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length66
author_reputation1,219,881,571,293,136
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,958,249
net_rshares839,177,579,087
author_curate_reward""
vote details (1)
@amr008 ·
$0.23
Great post :) 
I used to use pyodbc from beginning but unfortunately it didn't work on heroku so I shifted to pymssql . 

> Perhaps, best way is to use both Beem and HiveSQL.

This is true. I sometimes use been API , HiveSQL as well as hiveengine API to make it effective. 

Posted Using [LeoFinance <sup>Beta</sup>](https://leofinance.io/@amr008/re-geekgirl-6ewmnx)
👍  
properties (23)
authoramr008
permlinkre-geekgirl-6ewmnx
categoryhive-167922
json_metadata{"app":"leofinance/0.2","format":"markdown","tags":["hive-167922","leofinance"],"canonical_url":"https://leofinance.io/@amr008/re-geekgirl-6ewmnx"}
created2021-02-13 02:57:27
last_update2021-02-13 02:57:27
depth1
children2
last_payout2021-02-20 02:57:27
cashout_time1969-12-31 23:59:59
total_payout_value0.116 HBD
curator_payout_value0.117 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length366
author_reputation61,403,929,105,681
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,849,279
net_rshares789,394,690,767
author_curate_reward""
vote details (1)
@geekgirl ·
One of these days I will try hive-engine api too. Thanks for reminding of it.

Posted Using [LeoFinance <sup>Beta</sup>](https://leofinance.io/@geekgirl/re-amr008-5enzbf)
properties (22)
authorgeekgirl
permlinkre-amr008-5enzbf
categoryhive-167922
json_metadata{"app":"leofinance/0.2","format":"markdown","tags":["hive-167922","leofinance"],"canonical_url":"https://leofinance.io/@geekgirl/re-amr008-5enzbf"}
created2021-02-13 07:08:54
last_update2021-02-13 07:08:54
depth2
children1
last_payout2021-02-20 07:08:54
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length170
author_reputation1,586,488,611,824,452
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,851,324
net_rshares0
@amr008 ·
Sure. If you need any help please let me know. Since you are good at python, it will be easier for you. 

Posted Using [LeoFinance <sup>Beta</sup>](https://leofinance.io/@amr008/re-geekgirl-26dhkv)
properties (22)
authoramr008
permlinkre-geekgirl-26dhkv
categoryhive-167922
json_metadata{"app":"leofinance/0.2","format":"markdown","tags":["hive-167922","leofinance"],"canonical_url":"https://leofinance.io/@amr008/re-geekgirl-26dhkv"}
created2021-02-13 09:56:54
last_update2021-02-13 09:56:54
depth3
children0
last_payout2021-02-20 09:56:54
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length197
author_reputation61,403,929,105,681
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,853,104
net_rshares0
@crokkon · (edited)
$0.32
.
.
👍  ,
properties (23)
authorcrokkon
permlinkqogf1c
categoryhive-167922
json_metadata"{"app": ""}"
created2021-02-13 06:22:27
last_update2022-08-06 15:43:12
depth1
children1
last_payout2021-02-20 06:22:27
cashout_time1969-12-31 23:59:59
total_payout_value0.159 HBD
curator_payout_value0.159 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length1
author_reputation81,214,366,861,104
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,850,805
net_rshares1,042,696,466,508
author_curate_reward""
vote details (2)
@geekgirl ·
I have been mentioning that in other HiveSQL posts. Forgot this time. I will add that now. Thank you.

Posted Using [LeoFinance <sup>Beta</sup>](https://leofinance.io/@geekgirl/re-crokkon-gcwsv)
properties (22)
authorgeekgirl
permlinkre-crokkon-gcwsv
categoryhive-167922
json_metadata{"app":"leofinance/0.2","format":"markdown","tags":["hive-167922","leofinance"],"canonical_url":"https://leofinance.io/@geekgirl/re-crokkon-gcwsv"}
created2021-02-13 07:07:03
last_update2021-02-13 07:07:03
depth2
children0
last_payout2021-02-20 07:07:03
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length194
author_reputation1,586,488,611,824,452
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,851,303
net_rshares0
@hivecoffee ·
$0.23
This is exactly what I want to learn and like you I have to start from zero. I am following you for more great content.

Posted Using [LeoFinance <sup>Beta</sup>](https://leofinance.io/@hivecoffee/re-geekgirl-6ewmi4)
👍  
properties (23)
authorhivecoffee
permlinkre-geekgirl-6ewmi4
categoryhive-167922
json_metadata{"app":"leofinance/0.2","format":"markdown","tags":["hive-167922","leofinance"],"canonical_url":"https://leofinance.io/@hivecoffee/re-geekgirl-6ewmi4"}
created2021-02-13 04:04:33
last_update2021-02-13 04:04:33
depth1
children1
last_payout2021-02-20 04:04:33
cashout_time1969-12-31 23:59:59
total_payout_value0.116 HBD
curator_payout_value0.117 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length216
author_reputation19,616,190,901,987
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,849,759
net_rshares786,144,390,800
author_curate_reward""
vote details (1)
@geekgirl ·
That is awesome. Let me know how progress goes. Thank you.

Posted Using [LeoFinance <sup>Beta</sup>](https://leofinance.io/@geekgirl/re-hivecoffee-42pswk)
👍  
properties (23)
authorgeekgirl
permlinkre-hivecoffee-42pswk
categoryhive-167922
json_metadata{"app":"leofinance/0.2","format":"markdown","tags":["hive-167922","leofinance"],"canonical_url":"https://leofinance.io/@geekgirl/re-hivecoffee-42pswk"}
created2021-02-13 07:07:42
last_update2021-02-13 07:07:42
depth2
children0
last_payout2021-02-20 07:07:42
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length155
author_reputation1,586,488,611,824,452
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,851,311
net_rshares3,003,059,752
author_curate_reward""
vote details (1)
@khurshid102 ·
$0.18
Wow great information. I'm new and interested in learning. Hope you will help me. I'm following you and also on Twitter now Thanks @geekgirl 
👍  
properties (23)
authorkhurshid102
permlinkre-geekgirl-qorwk3
categoryhive-167922
json_metadata{"tags":["hive-167922"],"app":"peakd/2021.01.3"}
created2021-02-19 11:14:30
last_update2021-02-19 11:14:30
depth1
children0
last_payout2021-02-26 11:14:30
cashout_time1969-12-31 23:59:59
total_payout_value0.092 HBD
curator_payout_value0.092 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length141
author_reputation-760,141,506,833
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,954,377
net_rshares830,849,936,582
author_curate_reward""
vote details (1)
@nickyhavey ·
$0.21
I thought you knew SQL for a long time but that's cool you learned from Hive too! I only had a teeny bit of experience with it before Hive and it works in a similar way to excel in terms of the logic in macros. For me, it's just knowing what all the different terms are like SELECT and UNION etc. 

I can't remember if you shared with me a post that had the different terms used. I have saved the queries I use in the free SQL software I found but I had a lot of help in setting them up.
👍  
properties (23)
authornickyhavey
permlinkre-geekgirl-qogltb
categoryhive-167922
json_metadata{"tags":["hive-167922"],"app":"peakd/2021.01.3"}
created2021-02-13 08:48:48
last_update2021-02-13 08:48:48
depth1
children4
last_payout2021-02-20 08:48:48
cashout_time1969-12-31 23:59:59
total_payout_value0.106 HBD
curator_payout_value0.106 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length487
author_reputation343,022,660,476,683
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,852,411
net_rshares714,130,015,804
author_curate_reward""
vote details (1)
@geekgirl ·
$0.14
I started using HiveSQL when it used to be SteemSQL, so it would be about 3-4 years. I probably should have clarified that. I just generally like to refer to Hive as a rebranding of Steem and consider it as the original chain. :)

See if this cheatsheet can be helpful.

![sql.png](https://images.hive.blog/DQmXYEFZ8EC3AfsFr6piwYEDAkZD5JnCYkr31YUfDpdBCaN/sql.png)

Posted Using [LeoFinance <sup>Beta</sup>](https://leofinance.io/@geekgirl/re-nickyhavey-6tugux)
👍  
properties (23)
authorgeekgirl
permlinkre-nickyhavey-6tugux
categoryhive-167922
json_metadata{"app":"leofinance/0.2","format":"markdown","tags":["hive-167922","leofinance"],"canonical_url":"https://leofinance.io/@geekgirl/re-nickyhavey-6tugux","image":["https://images.hive.blog/DQmXYEFZ8EC3AfsFr6piwYEDAkZD5JnCYkr31YUfDpdBCaN/sql.png"]}
created2021-02-13 09:16:18
last_update2021-02-13 09:16:18
depth2
children3
last_payout2021-02-20 09:16:18
cashout_time1969-12-31 23:59:59
total_payout_value0.080 HBD
curator_payout_value0.063 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length460
author_reputation1,586,488,611,824,452
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,852,735
net_rshares551,717,957,547
author_curate_reward""
vote details (1)
@arcange ·
$0.22
IMHO, your cheatsheet is not the most helpful to share about HiveSQL because it only allows users to issue SELECT statements. 
Any command from the last two columns (except joins) will be rejected.
👍  
properties (23)
authorarcange
permlinkre-geekgirl-qoo99d
categoryhive-167922
json_metadata{"tags":["hive-167922"],"app":"peakd/2021.01.3"}
created2021-02-17 11:58:27
last_update2021-02-17 11:58:27
depth3
children1
last_payout2021-02-24 11:58:27
cashout_time1969-12-31 23:59:59
total_payout_value0.112 HBD
curator_payout_value0.112 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length197
author_reputation1,146,606,601,469,178
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,920,637
net_rshares837,288,663,398
author_curate_reward""
vote details (1)
@nickyhavey ·
$0.20
That's a great cheatsheet! Thank you, I'll be downloading that. I found you can also rename the columns after the SELECT command which is really helpful for uploading your csv files to tax/accountancy calculator websites like Koinly should you need to. They only accept certain tables with certain formats but it's been a lifesaver! 
👍  
properties (23)
authornickyhavey
permlinkre-geekgirl-qogng7
categoryhive-167922
json_metadata{"tags":["hive-167922"],"app":"peakd/2021.01.3"}
created2021-02-13 09:24:06
last_update2021-02-13 09:24:06
depth3
children0
last_payout2021-02-20 09:24:06
cashout_time1969-12-31 23:59:59
total_payout_value0.102 HBD
curator_payout_value0.102 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length333
author_reputation343,022,660,476,683
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,852,830
net_rshares690,429,669,254
author_curate_reward""
vote details (1)
@poshbot ·
https://twitter.com/geekjen/status/1360421942474383361
properties (22)
authorposhbot
permlinkre-hivesql-in-python-which-module-to-choose-pypyodbc-pyodbc-or-pymssql-20210213t025407z
categoryhive-167922
json_metadata"{"app": "beem/0.24.20"}"
created2021-02-13 02:54:06
last_update2021-02-13 02:54:06
depth1
children0
last_payout2021-02-20 02:54:06
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length54
author_reputation5,554,335,374,496
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id101,849,251
net_rshares0
@steevc ·
$0.30
I should have read this earlier. I found pyodbc and managed to install the driver to make it work, but I will bear pymssql in mind for future projects. I will have a look at your other posts around this topic as there is always something to learn.

I am impressed at the speed of HiveSQL queries. I used Beem for other stuff and it takes a while. I may see if SQL is a better option for that.
👍  ,
properties (23)
authorsteevc
permlinkre-geekgirl-qqflnw
categoryhive-167922
json_metadata{"tags":["hive-167922"],"app":"peakd/2021.03.7"}
created2021-03-23 16:55:09
last_update2021-03-23 16:55:09
depth1
children1
last_payout2021-03-30 16:55:09
cashout_time1969-12-31 23:59:59
total_payout_value0.150 HBD
curator_payout_value0.149 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length392
author_reputation1,369,983,255,348,877
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id102,552,404
net_rshares435,363,084,345
author_curate_reward""
vote details (2)
@geekgirl ·
$0.25
Yes HiveSQL is a lot faster. I haven't had any issues with pymssql yet. Implementation of both is similar too, so it wouldn't take much time to switch from pyodbc to pymssql if you ever will need to.
👍  
properties (23)
authorgeekgirl
permlinkqqfo1y
categoryhive-167922
json_metadata{"app":"hiveblog/0.1"}
created2021-03-23 17:46:45
last_update2021-03-23 17:46:45
depth2
children0
last_payout2021-03-30 17:46:45
cashout_time1969-12-31 23:59:59
total_payout_value0.127 HBD
curator_payout_value0.127 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length199
author_reputation1,586,488,611,824,452
root_title"HiveSQL in Python: Which module to choose - pypyodbc, pyodbc, or pymssql?"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id102,553,435
net_rshares371,866,037,890
author_curate_reward""
vote details (1)