create account

Foooking can't grok GroupBy in Postgres by penderis

View this thread on: hive.blogpeakd.comecency.com
· @penderis · (edited)
$5.18
Foooking can't grok GroupBy in Postgres
My kneejerk reaction is that GroupBy in Postgres is just broken. This is obviously not the case, instead I just don't understand the problem it is solving. 

In the particular case I was trying to solve I had in mind that I want to fetch all rows and just group the content on a said key... Very object centric I guess. 

Although I have not found a solution in SQL that particularly matches what I wanted to achieve , and I do think it will just be a subquery where I probably join the same table data after having done the grouping... Not sure really. 

I do know exactly what I want it to look like when I get the results
<code><pre>result = {
&nbsp;&nbsp;&nbsp;&nbsp; 'key1': {
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 'property1': 'value'
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 'property2': 'value'
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 'property3': 'value'
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}
&nbsp;&nbsp;&nbsp;&nbsp; 'key2': {
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 'property1': 'value'
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 'property2': 'value'
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 'property3': 'value'
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}
&nbsp;&nbsp;&nbsp;&nbsp; 'key3': {
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 'property1': 'value'
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 'property2': 'value'
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 'property3': 'value'
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}
}</pre></code>

So after looking at what I supposedly wanted to be returned I realise that it is much more something in the domain of an API or possibly NoSQL. 

Which fair enough if that is the case. 

Actually seeing the result in that form is what made me realise I just don't understand what groupBy is doing and I definitely did not do the appropriate context switch to know that given how Sql will return my data does not align with my case at all. 

To try and explain that, this is what Postgres is going to want to return to me no matter what:

<code>id:key1, property1:value, property2:value, property3:value;
id:key2, property1:value, property2:value, property3:value;
id:key3, property1:value, property2:value, property3:value;
</code>

So given no matter what fancy aggregates I try etc, it will always return them on a per row basis. So yes I literally asked a stupid question because I wanted it to do the "logic" for me. 

For the most part I was hung-up on the fact that I am losing columns when using GroupBy. For example if I want to group on <code>property1</code> then I can't group on it unless I include the other columns also since I am using a <code>SELECT *</code>

And obviously even selecting specific columns those still need to be in the GroupBy... So nothing gets grouped since they are all in the GroupBy and would just return the data as though I did not group anything. 

Obviously y'all are smarter and must be crying in pain at this problem you clearly do not see as a problem, it is just me being stupid. 

Which is what I said each time a tutorial added an aggregate to their GroupBy query and either used that as the GroupBy or a specific column but still leaving behind the rest of the info. 

So I had to ask: "WTF am I trying to do and why am I obsessed with maintaining the extra data?" 

Well, I wanted a tally of all the columns based on property1 , and did the rest matter? 

NO

It did not matter, and so I went and I GroupedBy the property1 and added the total field for the count and boom I got what I wanted. 

Now sure maybe the other data does matter but frankly I can just pull the keys I grouped on and rather do a second call which is simple and straight forward then merge my results accordingly in the code. 

Or I guess some form of a join could be achieved, I am not to sure how heavy a query like that is. 

With varying complexity of course depending on the size. 

So all in all what I learned from that is that GroupBy still sucks but it mainly sucks because I was thinking of my result set in a different context.
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 111 others
properties (23)
authorpenderis
permlinkfoooking-can-t-grok-groupby
categorypostgres
json_metadata"{"app":"ecency/3.1.0-vision","description":"My kneejerk reaction is that GroupBy in Postgres is just broken. This is obviously not the case, instead I just don't understand the problem it is solving. In the particular case I was trying to solve","format":"markdown+html","image":[],"tags":["postgres","coding","database","sql","webdevelopment","proofofbrain","hive-engine"]}"
created2024-03-15 14:18:18
last_update2024-03-15 14:20:45
depth0
children5
last_payout2024-03-22 14:18:18
cashout_time1969-12-31 23:59:59
total_payout_value2.605 HBD
curator_payout_value2.573 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length3,947
author_reputation153,012,169,855,556
root_title"Foooking can't grok GroupBy in Postgres"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id132,073,022
net_rshares10,683,801,239,714
author_curate_reward""
vote details (175)
@pizzabot ·
<center>PIZZA!


$PIZZA slices delivered:
@penderis<sub>(1/10)</sub> tipped @tierra-errante 


</center>
properties (22)
authorpizzabot
permlinkre-foooking-can-t-grok-groupby-20240315t200232z
categorypostgres
json_metadata"{"app": "pizzabot"}"
created2024-03-15 20:02:33
last_update2024-03-15 20:02:33
depth1
children0
last_payout2024-03-22 20:02:33
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length104
author_reputation7,452,540,471,322
root_title"Foooking can't grok GroupBy in Postgres"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id132,079,570
net_rshares0
@tierra-errante ·
$0.02
You don't use any ORM?
👍  ,
properties (23)
authortierra-errante
permlinkre-penderis-2024315t111938715z
categorypostgres
json_metadata{"type":"comment","tags":["postgres","coding","database","sql","webdevelopment","proofofbrain","hive-engine"],"app":"ecency/3.0.46-mobile","format":"markdown+html"}
created2024-03-15 15:19:39
last_update2024-03-15 15:19:39
depth1
children3
last_payout2024-03-22 15:19:39
cashout_time1969-12-31 23:59:59
total_payout_value0.012 HBD
curator_payout_value0.012 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length22
author_reputation8,138,916,505,133
root_title"Foooking can't grok GroupBy in Postgres"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id132,074,171
net_rshares54,176,903,325
author_curate_reward""
vote details (2)
@penderis ·
$0.02
Yup, Eloquent from Laravel which I think is the part that kinda made me insist it can be done the way I want. Since there is the groupBy method for the ORM but also groupBy for result object. The groupBy on the result object does exactly what I showed and keys the results accordingly, where as the groupBy on the ORM or query builder is an interface for the SQL engines methods and in POSTGRES you have to have all columns from the select available in the groupBy + then any aggregates you use can be excluded so counting etc works but if I wanted all the data the best is to use the result and instead groupBy in my code to build the structure I want. 
👍  ,
properties (23)
authorpenderis
permlinkre-tierra-errante-2024315t1739161z
categorypostgres
json_metadata{"tags":["postgres","coding","database","sql","webdevelopment","proofofbrain","hive-engine"],"app":"ecency/3.1.0-vision","format":"markdown+html"}
created2024-03-15 15:39:21
last_update2024-03-15 15:39:21
depth2
children2
last_payout2024-03-22 15:39:21
cashout_time1969-12-31 23:59:59
total_payout_value0.012 HBD
curator_payout_value0.012 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length654
author_reputation153,012,169,855,556
root_title"Foooking can't grok GroupBy in Postgres"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id132,074,546
net_rshares54,166,265,181
author_curate_reward""
vote details (2)
@tierra-errante ·
I think I understand, it just seems ugly to re-execute one query after another to recover the missing data
properties (22)
authortierra-errante
permlinkre-penderis-2024315t153247913z
categorypostgres
json_metadata{"tags":["postgres","coding","database","sql","webdevelopment","proofofbrain","hive-engine"],"app":"ecency/3.1.0-vision","format":"markdown+html"}
created2024-03-15 19:32:48
last_update2024-03-15 19:32:48
depth3
children1
last_payout2024-03-22 19:32:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length106
author_reputation8,138,916,505,133
root_title"Foooking can't grok GroupBy in Postgres"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id132,079,093
net_rshares0