Daily tech talk.. GPU based databases by cuttie1979

View this thread on: hive.blog | peakd.com | ecency.com

technology · @cuttie1979 · Dec 12 '17

$327.84

Daily tech talk.. GPU based databases

Today I will write about the GPU based databases.
![DSC_1083.JPG](https://steemitimages.com/DQmY8PKVeMKTsmWT5E4AVkxzrkD4eGjznngNtgNo7DSbjV9/DSC_1083.JPG)

For this topic I selected a picture about one of our server, which has 8 NVIDIA K80 GPU.
The most important factor is not the core numbers which also amazing, more the memory size.
In this machine we have 192GB GDDR5 memory.
This gives total 3840GB/s memory bandwidth, and 39.936 GPU cores
You can imagine, how this works with a GPU based in memory database.

First thing which I found around 10 years ago was an analytical tool which first used the GPU
It was the JEDOX GPU Accelerator
https://knowledgebase.jedox.com/knowledgebase/jedox-gpu-accelerator/

It was a good tool, but not a general use solution.

Later I found an interesting OpenSource project which was Alenka
https://github.com/antonmks/Alenka

What was the most interesting ?
It made far better result on a mid range notebook GPU compared to an Oracle T4-4 server with standard DB benchmark.
This was in 2012!!! Than no one talked about this, and most of my colleagues think I'm crazy.

After a few years I found a commercial product, which made available, and was the first which has regular support.
It was MAPD
https://www.mapd.com

This year this project made opensource, so now anyone able to test it. I can describe it with one word: Amazing!
Originally it is for x86, but immediately after Open Source release, I compiled to our Power8 server.
With one billion record if had the same performance like regular databases with hundreds.
The most powerful feature of this platform, it is not only DB it also has some visualisation, which show immediately the query results. If you put a lots of diagrams, it still has the amazing speed.

For those who interested after this introduction:
https://github.com/mapd/mapd-core

There is now a competitor, who is Kinetica.
https://www.kinetica.com

This DB used at USPS, and OIL and Gas companies. The speed compared to other in-memory databases killer.

I think this will be the future of the databases. What could change this dominance is the FPGA boards, which has far more processing unit, than GPUs. But we need to wait until these will be mainstream at Enterprises.

Anyone has also experience with this toys ? Write comments!

👍 buildawhale, rocky1, upme, appreciator, cuttie1979, pharesim, minnowsupport, dickturpin, dcasia, lpreap, bl0ckchained125, fernowl13, ribalinux, rabeel, ugetfunded, harshallele, jawad09, theb0red1, tuanis, edolascrypto, sircarlo, tpkidkai, funtraveller, isteemithard, alebeta, yassinekhoujjan, innoserv, digi5952, kariofthenorth, keithyoungblood, steemit.lover2, temmy8284, ritchieknopfler, balkantech, thebigkahuna, khalidfarki, steemerturk, jankykang, hectorgomez, rambeesbd, sanga, hotnimmi, cryptopimp, turbo-redwood, syedkashifhussai, quaestor, banjo, sunilbukey, faresellouz, sonofsatoshi, runragged, falseyedols, zikra93, mateakrnic, miranas

`author`	cuttie1979
`permlink`	daily-tech-talk-gpu-based-databases
`category`	technology
`json_metadata`	{"tags":["technology","gpu","inmemory","mapd","analytics"],"image":["https://steemitimages.com/DQmY8PKVeMKTsmWT5E4AVkxzrkD4eGjznngNtgNo7DSbjV9/DSC_1083.JPG"],"links":["https://knowledgebase.jedox.com/knowledgebase/jedox-gpu-accelerator/","https://github.com/antonmks/Alenka","https://www.mapd.com","https://github.com/mapd/mapd-core","https://www.kinetica.com"],"app":"steemit/0.1","format":"markdown"}
`created`	2017-12-12 20:52:27
`last_update`	2017-12-12 20:52:27
`depth`	0
`children`	15
`last_payout`	2017-12-19 20:52:27
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	253.030 HBD
`curator_payout_value`	74.806 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	2,301
`author_reputation`	90,996,652,990,411
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	23,285,745
`net_rshares`	69,320,097,110,003
`author_curate_reward`	""

properties (23)vote details (55)

voter	rshares	pct
pharesim	106,299,513,528	0.02%
theb0red1	4,794,015,163	12%
sircarlo	3,598,308,528	100%
keithyoungblood	1,500,768,236	100%
banjo	461,692,438	1%
dickturpin	26,439,311,830	100%
ribalinux	8,877,763,042	25%
minnowsupport	34,139,365,109	1%
innoserv	2,175,394,697	100%
mateakrnic	131,670,183	100%
appreciator	16,790,876,857,003	20.61%
hotnimmi	571,391,930	100%
balkantech	1,111,626,822	100%
ugetfunded	8,333,414,080	100%
harshallele	7,431,925,907	100%
jawad09	5,671,321,553	100%
cuttie1979	345,270,279,460	100%
bl0ckchained125	14,710,588,379	100%
rabeel	8,575,757,270	100%
steemit.lover2	1,338,565,400	100%
rocky1	17,481,252,726,322	99%
buildawhale	17,548,970,829,677	21.46%
fernowl13	13,182,894,711	100%
turbo-redwood	527,723,100	100%
isteemithard	2,709,173,644	100%
upme	16,814,615,906,851	13.92%
rambeesbd	714,421,309	100%
yassinekhoujjan	2,616,274,181	100%
syedkashifhussai	516,547,170	100%
temmy8284	1,257,044,985	100%
tuanis	4,317,690,894	12.67%
lpreap	24,188,028,783	100%
thebigkahuna	1,041,411,307	100%
dcasia	26,418,133,071	100%
tpkidkai	3,432,719,844	20%
edolascrypto	4,013,644,759	100%
runragged	335,349,400	100%
funtraveller	2,827,181,721	100%
digi5952	2,093,049,072	100%
sanga	623,268,943	100%
quaestor	514,408,810	100%
khalidfarki	929,184,876	100%
miranas	124,600,314	100%
steemerturk	816,436,223	100%
ritchieknopfler	1,144,011,290	100%
sonofsatoshi	350,939,653	100%
kariofthenorth	2,022,770,355	100%
cryptopimp	570,748,491	100%
sunilbukey	424,570,800	100%
falseyedols	279,187,302	100%
alebeta	2,699,928,797	100%
hectorgomez	789,148,372	100%
zikra93	261,115,222	100%
faresellouz	399,958,000	100%
jankykang	806,551,196	100%

@alebeta · Dec 17 '17

Hello @cuttie1979

Do you have experience with Webservers using GPU ?

Thanks for your post, I'm highly interested in this kind of topics, since i have years working building high performance servers for companies I was always thinking about the GPU power in this niche.

All the best and following you

properties (22)

`author`	alebeta
`permlink`	re-cuttie1979-daily-tech-talk-gpu-based-databases-20171217t202959985z
`category`	technology
`json_metadata`	{"tags":["technology"],"users":["cuttie1979"],"app":"steemit/0.1"}
`created`	2017-12-17 20:30:00
`last_update`	2017-12-17 20:30:00
`depth`	1
`children`	2
`last_payout`	2017-12-24 20:30:00
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	302
`author_reputation`	23,514,749,668
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	23,926,795
`net_rshares`	0

@cuttie1979 · Dec 17 '17

$2.40

Yes! This was also one of the topics. But I think the answer is fpga :D

👍 cuttie1979, alebeta

`author`	cuttie1979
`permlink`	re-alebeta-re-cuttie1979-daily-tech-talk-gpu-based-databases-20171217t213835035z
`category`	technology
`json_metadata`	{"tags":["technology"],"app":"steemit/0.1"}
`created`	2017-12-17 21:38:36
`last_update`	2017-12-17 21:38:36
`depth`	2
`children`	1
`last_payout`	2017-12-24 21:38:36
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	1.799 HBD
`curator_payout_value`	0.596 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	71
`author_reputation`	90,996,652,990,411
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	23,933,132
`net_rshares`	385,215,128,831
`author_curate_reward`	""

properties (23)vote details (2)

voter	weight	wgt%	rshares	pct	time
cuttie1979	0 B		382,506,682,252	100%
alebeta	0 B		2,708,446,579	100%

@alebeta · Dec 18 '17

Thank you for sharing such information. I'm already reading about that. I will try to perform at least basic tests, since I'm a benchmark freaky trying to reach out high performance web services.

Also if you are interesting about IT Security you can follow me.

All the best

properties (22)

`author`	alebeta
`permlink`	re-cuttie1979-re-alebeta-re-cuttie1979-daily-tech-talk-gpu-based-databases-20171218t100703661z
`category`	technology
`json_metadata`	{"tags":["technology"],"app":"steemit/0.1"}
`created`	2017-12-18 10:07:03
`last_update`	2017-12-18 10:07:03
`depth`	3
`children`	0
`last_payout`	2017-12-25 10:07:03
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	275
`author_reputation`	23,514,749,668
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	24,000,359
`net_rshares`	0

@appreciator · Dec 18 '17

<p>This post has received gratitude of 20.61 % from @appreciator thanks to: @cuttie1979.</p>

properties (22)

`author`	appreciator
`permlink`	re-cuttie1979-daily-tech-talk-gpu-based-databases-20171218t080055076z
`category`	technology
`json_metadata`	{"tags":["technology"],"app":"drotto/0.0.3rc1"}
`created`	2017-12-18 08:00:54
`last_update`	2017-12-18 08:00:54
`depth`	1
`children`	0
`last_payout`	2017-12-25 08:00:54
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	93
`author_reputation`	55,801,543,512,306
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	23,988,191
`net_rshares`	0

@atherz · Dec 13 '17

nice post. shared :)

properties (22)

@buildawhale · Dec 18 '17

<p>This post has received a 21.46 % upvote from @buildawhale thanks to: @cuttie1979.
Send at least 1 SBD to @buildawhale with a post link in the memo field for a portion of the next vote.</p>

<p>To support our daily curation initiative, please vote on my owner, @themarkymark, as a <a href="https://steemit.com/witness-category/@themarkymark/witness-themarkymark">Steem Witness</a></p>

properties (22)

`author`	buildawhale
`permlink`	re-cuttie1979-daily-tech-talk-gpu-based-databases-20171218t095140264z
`category`	technology
`json_metadata`	{"tags":["technology"],"app":"drotto/0.0.2d"}
`created`	2017-12-18 09:52:45
`last_update`	2017-12-18 09:52:45
`depth`	1
`children`	0
`last_payout`	2017-12-25 09:52:45
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	387
`author_reputation`	25,832,665,864,959,331
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	23,998,745
`net_rshares`	0

@dickturpin · Dec 13 '17

You can download an explanatory eBook <a href="Introduction to GPUs for Data Analytics" target="_blank">Introduction to GPUs for Data Analytics</a> from O'Rielly.

properties (22)

`author`	dickturpin
`permlink`	re-cuttie1979-daily-tech-talk-gpu-based-databases-20171213t100946978z
`category`	technology
`json_metadata`	"{"tags":["technology"],"links":["Introduction to GPUs for Data Analytics"],"app":"steemit/0.1"}"
`created`	2017-12-13 10:09:45
`last_update`	2017-12-13 10:09:45
`depth`	1
`children`	1
`last_payout`	2017-12-20 10:09:45
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	162
`author_reputation`	42,250,387,496,084
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	23,347,243
`net_rshares`	0

@cuttie1979 · Dec 13 '17

Thanks! I really like O'Rielly books :) I didn't know about it!

properties (22)

`author`	cuttie1979
`permlink`	re-dickturpin-re-cuttie1979-daily-tech-talk-gpu-based-databases-20171213t131502123z
`category`	technology
`json_metadata`	{"tags":["technology"],"app":"steemit/0.1"}
`created`	2017-12-13 13:15:03
`last_update`	2017-12-13 13:15:03
`depth`	2
`children`	0
`last_payout`	2017-12-20 13:15:03
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	63
`author_reputation`	90,996,652,990,411
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	23,363,068
`net_rshares`	0

@khalidfarki · Dec 12 '17

nice

👍 khalidfarki

`author`	khalidfarki
`permlink`	re-cuttie1979-daily-tech-talk-gpu-based-databases-20171212t214644689z
`category`	technology
`json_metadata`	{"tags":["technology"],"app":"steemit/0.1"}
`created`	2017-12-12 21:46:42
`last_update`	2017-12-12 21:46:42
`depth`	1
`children`	0
`last_payout`	2017-12-19 21:46:42
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	4
`author_reputation`	67,470,546,880
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	23,290,159
`net_rshares`	891,258,963
`author_curate_reward`	""

properties (23)vote details (1)

voter	weight	wgt%	rshares	pct	time
khalidfarki	0 B		891,258,963	100%

@laique · Dec 19 '17

Nice Post...

👍 laique

`author`	laique
`permlink`	re-cuttie1979-daily-tech-talk-gpu-based-databases-20171219t102551607z
`category`	technology
`json_metadata`	{"tags":["technology"],"app":"steemit/0.1"}
`created`	2017-12-19 10:26:00
`last_update`	2017-12-19 10:26:00
`depth`	1
`children`	0
`last_payout`	2017-12-26 10:26:00
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	12
`author_reputation`	318,104,939,447
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	24,164,316
`net_rshares`	152,149,386
`author_curate_reward`	""

properties (23)vote details (1)

voter	weight	wgt%	rshares	pct	time
laique	0 B		152,149,386	100%

@minnowsupport · Dec 18 '17

<p>Congratulations!  This post has been upvoted from the communal account, @minnowsupport, by cuttie1979 from the Minnow Support Project.  It's a witness project run by aggroed, ausbitbank, teamsteem, theprophet0, someguy123, neoxian, followbtcnews/crimsonclad, and netuoso.  The goal is to help Steemit grow by supporting Minnows and creating a social network.  Please find us in the <a href="https://discord.gg/HYj4yvw">Peace, Abundance, and Liberty Network (PALnet) Discord Channel</a>.  It's a completely public and open space to all members of the Steemit community who voluntarily choose to be there.</p>

<p>If you would like to delegate to the Minnow Support Project you can do so by clicking on the following links: <a href="https://v2.steemconnect.com/sign/delegateVestingShares?delegator=&amp;delegatee=minnowsupport&amp;vesting_shares=102530.639667%20VESTS">50SP</a>, <a href="https://v2.steemconnect.com/sign/delegateVestingShares?delegator=&amp;delegatee=minnowsupport&amp;vesting_shares=205303.639667%20VESTS">100SP</a>, <a href="https://v2.steemconnect.com/sign/delegateVestingShares?delegator=&amp;delegatee=minnowsupport&amp;vesting_shares=514303.639667%20VESTS">250SP</a>, <a href="https://v2.steemconnect.com/sign/delegateVestingShares?delegator=&amp;delegatee=minnowsupport&amp;vesting_shares=1025303.639667%20VESTS">500SP</a>, <a href="https://v2.steemconnect.com/sign/delegateVestingShares?delegator=&amp;delegatee=minnowsupport&amp;vesting_shares=2053030.639667%20VESTS">1000SP</a>, <a href="https://v2.steemconnect.com/sign/delegateVestingShares?delegator=&amp;delegatee=minnowsupport&amp;vesting_shares=10253030.639667%20VESTS">5000SP</a>.  <strong>Be sure to leave at least 50SP undelegated on your account.</strong></p>

properties (22)

`author`	minnowsupport
`permlink`	re-cuttie1979-daily-tech-talk-gpu-based-databases-20171218t213343465z
`category`	technology
`json_metadata`	{"tags":["technology"],"app":"cosgrove/0.0.2"}
`created`	2017-12-18 21:33:42
`last_update`	2017-12-18 21:33:42
`depth`	1
`children`	0
`last_payout`	2017-12-25 21:33:42
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	1,748
`author_reputation`	148,902,805,319,183
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	24,083,968
`net_rshares`	0

@rabeel · Dec 17 '17

wonderful informations

properties (22)

`author`	rabeel
`permlink`	re-cuttie1979-daily-tech-talk-gpu-based-databases-20171217t173309757z
`category`	technology
`json_metadata`	{"tags":["technology"],"app":"steemit/0.1"}
`created`	2017-12-17 17:33:18
`last_update`	2017-12-17 17:33:18
`depth`	1
`children`	0
`last_payout`	2017-12-24 17:33:18
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	22
`author_reputation`	136,994,730,017,550
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	23,907,938
`net_rshares`	0

@rambeesbd · Dec 12 '17

Great technology buddies, thanks for sharing interesting post

properties (22)

`author`	rambeesbd
`permlink`	re-cuttie1979-daily-tech-talk-gpu-based-databases-20171212t205410742z
`category`	technology
`json_metadata`	{"tags":["technology"],"app":"steemit/0.1"}
`created`	2017-12-12 20:54:15
`last_update`	2017-12-12 20:54:15
`depth`	1
`children`	0
`last_payout`	2017-12-19 20:54:15
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	61
`author_reputation`	1,360,732,403,960
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	23,285,916
`net_rshares`	0

@steemerturk · Dec 12 '17

I have low information about computers and features of it. I can start with the learning with your post thanks for sharing . I appreciate it .

properties (22)

`author`	steemerturk
`permlink`	re-cuttie1979-daily-tech-talk-gpu-based-databases-20171212t213717419z
`category`	technology
`json_metadata`	{"tags":["technology"],"app":"steemit/0.1"}
`created`	2017-12-12 21:37:18
`last_update`	2017-12-12 21:37:18
`depth`	1
`children`	0
`last_payout`	2017-12-19 21:37:18
`cashout_time`	1969-12-31 23:59:59
`total_payout_value`	0.000 HBD
`curator_payout_value`	0.000 HBD
`pending_payout_value`	0.000 HBD
`promoted`	0.000 HBD
`body_length`	142
`author_reputation`	1,323,319,025,262
`root_title`	"Daily tech talk.. GPU based databases"
`beneficiaries`	`[]`
`max_accepted_payout`	1,000,000.000 HBD
`percent_hbd`	10,000
`post_id`	23,289,447
`net_rshares`	0

@upme · Dec 17 '17

re-cuttie1979-daily-tech-talk-gpu-based-databases-20171217t175021429z

You got a 13.92% upvote from @upme requested by: @cuttie1979. <br> Send at least 1.5 SBD to @upme with a post link in the memo field to receive upvote next round. <br> To support our activity, please vote for my master @suggeelson, as a [STEEM Witness](https://steemit.com/~witnesses)

properties (22)