create account

Hivesearcher update, indexing improvements by good-karma

View this thread on: hive.blog |  peakd.com |  ecency.com
hive-139531 · @good-karma ·
$48.27
Hivesearcher update, indexing improvements
Last few weeks, we have made few changes to Hivesearcher to make it compatible with new hardfork and improve indexing.

<center>![hivesearcher-decentralized-search-engine](https://images.ecency.com/p/C3TZR1g81UNbFzApDTmvFYMje8p1iesaAcNjSh7JWYRiPQRX52L6rNRZipeiaBkx9daTxCyJyVvcKo3QMC9tjfRvNzgHxYuYyj6AFmCrd98w27oav6Nh8m4)

As many of you know Hivesearcher is best opensource, decentralized search engine built on Hive. You can find any content regardless of age and author, reward, quickly. Flexibility it offers is huge and of course it is horizontally scalable.

Recent hardfork brought a lot of unexpected changes to Hivemind (layer-2 api), one of those changes were completely removal of `hive_posts_cache` table. 

Hivesearcher is a plugin or extension built on top of Hivemind and `hive_posts_cache` table was used as source of content for Hivesearcher Elastic search engine. After removal, we had to improve code and make some changes so our indexing could continue and search has up to date information. 

Also feeding incorrect information to elastic search instance would disrupt data and thus affects the search results and ranking. That's why we halted syncing after hardfork until Hivemind data was correct and had stable version. Since last week our Hivemind instance seem to be running much better and collecting/giving correct data, so we enabled indexing of newer posts.

#### No changes required from dapps developers
#### [API endpoint](https://ecency.com/hive-139531/@good-karma/hivesearcher-free-integrate-it-into-your-hive-apps) had zero downtime, during those upgrades https://api.hivesearcher.com
#### [API documentation](https://hivesearcher.com/api-docs), integrate fast search engine for your Hive dapps, offer custom feed and content filters to your users.

## Proposal  details

- [Read details of Hivesearcher proposal](https://ecency.com/hive/@good-karma/open-search-engine-development-and-maintenance)  
- [Vote for proposal using Hivesigner](https://hivesigner.com/sign/update-proposal-votes?proposal_ids=%5B114%5D&approve=true)  
- Source: https://github.com/ecency/hive2elastic
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 483 others
properties (23)
authorgood-karma
permlinkhivesearcher-update-indexing-improvements
categoryhive-139531
json_metadata{"links":["https://images.ecency.com/p/C3TZR1g81UNbFzApDTmvFYMje8p1iesaAcNjSh7JWYRiPQRX52L6rNRZipeiaBkx9daTxCyJyVvcKo3QMC9tjfRvNzgHxYuYyj6AFmCrd98w27oav6Nh8m4","https://ecency.com/hive-139531/@good-karma/hivesearcher-free-integrate-it-into-your-hive-apps","https://api.hivesearcher.com","https://hivesearcher.com/api-docs","https://ecency.com/hive/@good-karma/open-search-engine-development-and-maintenance","https://hivesigner.com/sign/update-proposal-votes?proposal_ids=%5B114%5D&approve=true","https://github.com/ecency/hive2elastic"],"tags":["hive-139531","hive","dapps","search","discover","engine","hivemind","layer2"],"app":"ecency/3.0.9-vision","format":"markdown+html"}
created2020-11-09 08:32:15
last_update2020-11-09 08:32:15
depth0
children8
last_payout2020-11-16 08:32:15
cashout_time1969-12-31 23:59:59
total_payout_value26.530 HBD
curator_payout_value21.744 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length2,111
author_reputation656,214,797,217,320
root_title"Hivesearcher update, indexing improvements"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id100,453,842
net_rshares209,458,273,218,461
author_curate_reward""
vote details (547)
@ervin-lemark · 
Good work, as always :)
👍  
properties (23)
authorervin-lemark
permlinkre-good-karma-2020119t95222439z
categoryhive-139531
json_metadata{"tags":["hive-139531","hive","dapps","search","discover","engine","hivemind","layer2"],"app":"ecency/3.0.9-vision","format":"markdown+html"}
created2020-11-09 08:52:24
last_update2020-11-09 08:52:24
depth1
children0
last_payout2020-11-16 08:52:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length23
author_reputation471,380,363,673,617
root_title"Hivesearcher update, indexing improvements"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id100,454,004
net_rshares2,045,784,494
author_curate_reward""
vote details (1)
@fjcalduch · 
Hello, @good-karma. My whole blog, which I started three years ago now with 1450 posts, is about jazz music. However, I write the word "jazz" in the  Hivesearcher and non of my posts appear. Why?
properties (22)
authorfjcalduch
permlinkqjul1r
categoryhive-139531
json_metadata{"users":{},"app":"hiveblog/0.1"}
created2020-11-15 17:10:39
last_update2020-11-15 17:10:39
depth1
children1
last_payout2020-11-22 17:10:39
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length195
author_reputation64,019,430,177,639
root_title"Hivesearcher update, indexing improvements"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id100,542,406
net_rshares0
@ecency · 
You can add `author:` keyword to filter results even further.
https://ecency.com/search/?q=jazz%20author%3Afjcalduch

Using 1 word usually has different weight on different posts, try adding 2 or 3 words to filter results further. With jazz you probably see your posts as well but your content is competing with other content for ranking and how keyword is used. For example I see many posts with jazz in their title...
properties (22)
authorecency
permlinkre-fjcalduch-20201116t121247510z
categoryhive-139531
json_metadata{"tags":["ecency"],"app":"ecency/3.0.9-vision","format":"markdown+html"}
created2020-11-16 10:12:48
last_update2020-11-16 10:12:48
depth2
children0
last_payout2020-11-23 10:12:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length419
author_reputation624,374,916,942,711
root_title"Hivesearcher update, indexing improvements"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id100,553,141
net_rshares0
@khushikhan · 
Reblog it
properties (22)
authorkhushikhan
permlinkre-good-karma-2020119t17757941z
categoryhive-139531
json_metadata{"tags":["ecency"],"app":"ecency/3.0.9-mobile","format":"markdown+html"}
created2020-11-09 12:08:00
last_update2020-11-09 12:08:00
depth1
children0
last_payout2020-11-16 12:08:00
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length9
author_reputation-144,154,038,860
root_title"Hivesearcher update, indexing improvements"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id100,455,796
net_rshares0
@leviet7490 · 
I felt it was fine and improved login problem on different platforms
properties (22)
authorleviet7490
permlinkre-good-karma-20201114t15940348z
categoryhive-139531
json_metadata{"tags":["hive-139531","hive","dapps","search","discover","engine","hivemind","layer2"],"app":"ecency/3.0.9-vision","format":"markdown+html"}
created2020-11-13 18:59:42
last_update2020-11-13 18:59:42
depth1
children0
last_payout2020-11-20 18:59:42
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length68
author_reputation232,859,884,279
root_title"Hivesearcher update, indexing improvements"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id100,515,357
net_rshares0
@pnc · 
great!
properties (22)
authorpnc
permlinkre-good-karma-2020119t174719401z
categoryhive-139531
json_metadata{"tags":["hive-139531","hive","dapps","search","discover","engine","hivemind","layer2"],"app":"ecency/3.0.9-vision","format":"markdown+html"}
created2020-11-09 22:47:21
last_update2020-11-09 22:47:21
depth1
children0
last_payout2020-11-16 22:47:21
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length6
author_reputation31,743,007,487,651
root_title"Hivesearcher update, indexing improvements"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id100,462,967
net_rshares0
@therealwolf · 
I think it would be good to improve what's displayed for `relevance`.

![image.png](https://files.peakd.com/file/peakd-hive/therealwolf/51QqNl9p-image.png)

I'd prefer to see the profile with the newest posts when I enter an account name, not some random comments/posts 😄
👍  
properties (23)
authortherealwolf
permlinkre-good-karma-qjmkgp
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2020.10.9"}
created2020-11-11 09:17:15
last_update2020-11-11 09:17:15
depth1
children1
last_payout2020-11-18 09:17:15
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length272
author_reputation581,693,011,827,252
root_title"Hivesearcher update, indexing improvements"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id100,481,918
net_rshares98,824,002,011
author_curate_reward""
vote details (1)
@good-karma · (edited)
Screen looks like from Ecency.com, we just added search page quickly in last update to make it ready for desktop app, previously it was forwarding user to hivesearcher.com, we will surely add more options and show power of search engine our team built. Right now, you can type `@text` in search field, it will find you users, type `#text` it will find you tags, `$text` will find you communities. So UI side of things will be improved for sure. As for hivesearcher side, we finished migration last week and we will do some improvements in ranking algorithms in coming weeks. Attached few screenshots of quick search:

![find-hive-users-quickly](https://images.ecency.com/DQmPZLFovcUTosd3BmCBsHWkapHYh4YCjPAiWVLcPiLWAsv/screenshot_2020_11_11_at_14.15.19.png)

![trending-topics-on-decentralized-web](https://images.ecency.com/DQmc189BL6tqH4mFC64tLNkYLZ76j9xmQK3WxAN4axz7ETp/screenshot_2020_11_11_at_14.15.53.png)

![find-best-rewarding-communities](https://images.ecency.com/DQmRV327qi2EJPvf1QGsv6s5zQuZP7CgfhAthjY5PHANqLC/image.png)
properties (22)
authorgood-karma
permlinkre-therealwolf-20201111t14186235z
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"ecency/3.0.9-vision","format":"markdown+html"}
created2020-11-11 12:18:06
last_update2020-11-11 12:20:36
depth2
children0
last_payout2020-11-18 12:18:06
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length1,035
author_reputation656,214,797,217,320
root_title"Hivesearcher update, indexing improvements"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id100,483,422
net_rshares0
Help/Feedback