create account

My open source anti-plagiarism steemit bot by djleo

View this thread on: hive.blogpeakd.comecency.com
· @djleo · (edited)
$328.28
My open source anti-plagiarism steemit bot
<html>
<p>In response to @cryptoctopus's request for this, I spent <em>the last full week</em> making an anti-plagiarism bot for steemit. Unfortunately for me, I was too late to claim the 500 odd steem $ bounty.</p>
<p>http://rs304.pbsrc.com/albums/nn180/4chanRus/Awesome%20Face/1213841779552.png~c200</p>
<p>Nevermind though, I had a feeling I'd be beaten as I'm not really a programmer.</p>
<h1><a href="https://github.com/songproducer/SteemitPlagiarismBot">Here is the bot!</a></h1>
<p>http://i.imgur.com/UUmbbNf.jpg</p>
<p><br></p>
<p>Source: <a href="http://www.publicdomainpictures.net/view-image.php?image=183468&amp;picture=robot">publicdomainpictures.net</a></p>
<h2>And what it looks like:</h2>
<p>http://i.imgur.com/NpKOgBX.png?2</p>
<p><br></p>
<p>The famous steem API tool <a href="https://github.com/xeroc/piston">Piston</a>:</p>
<p>http://i.imgur.com/3PkDSYv.png?2</p>
<p><a href="http://www.blendswap.com/blends/view/75521">Piston Bong</a> by XrNiX (changes made) <a href="https://creativecommons.org/licenses/by/3.0/">(CC BY 3.0)</a></p>
<h2>Also a quick and easy way to read through new steemit posts without all the distractions of pictures, reaction-gifs, headers and other styling or waiting for page loading / navigating the website.</h2>
<h2>It was fun learning to code!</h2>
<p>I realise my code is likely unnecessarily unwieldy (it's all on one page) and needs to be refactored for other's sanity. Maybe I could have made better use of Python's inbuilt functions rather than relying on so many regular expressions.</p>
<p><br></p>
<p>I promise in my next project I will make use of functions and classes.</p>
<p>If you want to point out any glaring errors I'd really appreciate the feedback!</p>
<h3>Comments</h3>
<p>I try to explain what's going on in most of the code so check it out!</p>
<h3>What does it do?</h3>
<p>It scans (and displays in the console) newly created posts and searches <a href="https://steemit.com/steemit-ideas/@bunix/a-gplv2-p2p-decentralized-web-search-portal-for-steemit">@bunix's yaCy index</a> for 4 random 'exact phrase' chunks of words. If a search hit comes up, and the author is different, the full content of the posts are compared (minus some formatting). If the match is at least 50% then a comment is made linking to the article and showing the percentage of the match.</p>
<p><br></p>
<h3>What doesn't it do?</h3>
<p><strong>Doesn't search the whole web</strong></p>
<p>I realise that a smarter anti-plagiarism bot might search the entire web using Google, Bing, or <a href="http://www.faroo.com/">Faroo</a> thus finding the original source of the content.</p>
<p><br></p>
<p>But most frustratingly, I just couldn't navigate Google's search API or get any example Python code to work (most written for Python 2.7). Faroo requires manual verification to get access to its API (still waiting for access).</p>
<p><br></p>
<p>In the end I decided it would be an interesting enough experiment to catch out spammers who copy content from other steemit peers.</p>
<h2>Motivations</h2>
<p>Although the bounty was certainly one motivation, learning to code has always been on my todo list. Also, I was getting annoyed at the amount of flagrant plagiarism in the new category, so @cryptoctopus's post resonated with me. This is my first potentially useful Python program!</p>
<h3>I learnt how to use regular expressions (a bit) and debug Python errors</h3>
<p>http://i.imgur.com/QwK4v5R.png</p>
<p><br></p>
<p><em>Image: Copyright © 2013 by </em><a href="http://twiki.org/"><em>TWiki.org</em></a></p>
<h2>Resources I used</h2>
<p>My reliance on <a href="http://pythex.org/">pythex.org</a> was extensive. They make practising regular expressions fun!</p>
<p><a href="https://github.com/xeroc/piston">Piston</a>, obviously :) thanks to @xeroc for all your help!</p>
<p>The <a href="https://docs.python.org/3/">Python documentation</a></p>
<p><a href="http://www.tutorialspoint.com/python/">TutorialsPoint.com</a></p>
<p><a href="http://stackexchange.com/">StackExchange</a></p>
<p><a href="https://freenode.net/">IRC freenode</a> #python channel</p>
<h2>Requirements</h2>
<p>You'll need <a href="https://www.python.org/download/releases/3.0/">Python 3</a>, some pip modules (starting with Python 3.4, it is included by default with the Python binary installers) and most importantly <a href="https://github.com/xeroc/piston">Piston</a>.</p>
<p><em>Note: Starting with Python 3.4, pip is included by default with the Python binary installers.</em></p>
<h2>Bugs</h2>
<p>Some random chunks should come up in the search results but don't. I guess it could be with the way yaCy indexes posts but I'm just guessing.</p>
<p>Another bug (albeit a minor one) is when a match is found it will continuously print '.Percentage difference:' which is pretty annoying - anyone know how to fix it?</p>
<p>Also it doesn't shut down cleanly when you exit (ctrl+c) so you need to mash ctrl+c a few times.</p>
<p>I'm sure there are many more that I haven't discovered yet so please make an issue on the GitHub page if you get one!</p>
<h2>What's the next project?... an intelligent upvote bot!</h2>
<p>I'd like to automate my curation duties using an upvote bot. I really find this AI social media stuff fascinating! If you know of any projects that do this already please shar</p>
</html>
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 22 others
👎  
properties (23)
authordjleo
permlinkmy-open-source-anti-plagiarism-steemit-bot
categorysteemit
json_metadata{"tags":["steemit","steemwork","steemit-ideas","opensource","programming","beyondbitcoin","python"],"users":["cryptoctopus","xeroc"],"image":["http://rs304.pbsrc.com/albums/nn180/4chanRus/Awesome%20Face/1213841779552.png~c200","http://i.imgur.com/UUmbbNf.jpg","http://i.imgur.com/NpKOgBX.png?2","http://i.imgur.com/3PkDSYv.png?2","http://i.imgur.com/QwK4v5R.png"],"links":["https://github.com/songproducer/SteemitPlagiarismBot","http://www.publicdomainpictures.net/view-image.php?image=183468&picture=robot","https://github.com/xeroc/piston","http://www.blendswap.com/blends/view/75521","https://creativecommons.org/licenses/by/3.0/","https://steemit.com/steemit-ideas/@bunix/a-gplv2-p2p-decentralized-web-search-portal-for-steemit","http://www.faroo.com/","http://twiki.org/","http://pythex.org/","https://docs.python.org/3/","http://www.tutorialspoint.com/python/","http://stackexchange.com/","https://freenode.net/","https://www.python.org/download/releases/3.0/"]}
created2016-07-24 23:38:06
last_update2016-07-24 23:42:27
depth0
children19
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value257.830 HBD
curator_payout_value70.451 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length5,320
author_reputation1,834,764,955,389
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,220
net_rshares32,371,560,643,646
author_curate_reward""
vote details (87)
@applecrisp ·
$0.12
Thanks for sharing and going into detail on your programming journey :)

Best wishes on it's continued progress!
👍  
properties (23)
authorapplecrisp
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160725t001949889z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-25 00:19:39
last_update2016-07-25 00:19:39
depth1
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.094 HBD
curator_payout_value0.029 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length112
author_reputation2,828,526,129,552
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,984
net_rshares97,975,592,771
author_curate_reward""
vote details (1)
@calaber24p ·
$0.13
Its good to have multiple. You and cheetah bot together could work nicely.
👍  
properties (23)
authorcalaber24p
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160725t001955873z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-25 00:20:03
last_update2016-07-25 00:20:03
depth1
children1
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.097 HBD
curator_payout_value0.029 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length74
author_reputation313,033,964,963,339
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,991
net_rshares99,935,104,626
author_curate_reward""
vote details (1)
@mattyfu ·
$0.12
Maybe just add something to check for a post by cheetahbot before posting, essentially splitting the workload if cheetahbot did the same? Either way kudos on the effort and maybe the experience will pay off when the next bot bounty gets thrown down.
👍  
properties (23)
authormattyfu
permlinkre-calaber24p-re-djleo-my-open-source-anti-plagiarism-steemit-bot-20160725t004109882z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-25 00:41:15
last_update2016-07-25 00:41:15
depth2
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.090 HBD
curator_payout_value0.029 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length249
author_reputation89,627,656,557
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id335,371
net_rshares96,016,080,915
author_curate_reward""
vote details (1)
@crok ·
$0.12
Thank you for your efforts; I cannot say anything more since i have no clue you approached or achived your programs goals.
Thanks again , good luck on your next learning project ...
👍  
properties (23)
authorcrok
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160724t235213028z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-24 23:53:00
last_update2016-07-24 23:53:00
depth1
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.094 HBD
curator_payout_value0.029 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length181
author_reputation9,979,380,525,768
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,498
net_rshares97,975,592,771
author_curate_reward""
vote details (1)
@cryptoctopus ·
$0.13
sorry! you were a little late. I hope you've learned something useful in the process. :-)
👍  , ,
properties (23)
authorcryptoctopus
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160726t021544868z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-26 02:15:24
last_update2016-07-26 02:15:24
depth1
children1
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.097 HBD
curator_payout_value0.029 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length89
author_reputation365,406,972,531,657
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id364,111
net_rshares113,180,546,890
author_curate_reward""
vote details (3)
@djleo ·
I certainly did! Thanks for your original post that  got me started.
properties (22)
authordjleo
permlinkre-cryptoctopus-re-djleo-my-open-source-anti-plagiarism-steemit-bot-20160726t031850068z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-26 03:18:51
last_update2016-07-26 03:18:51
depth2
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length68
author_reputation1,834,764,955,389
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id365,239
net_rshares0
@cryptohustlin ·
$0.22
I really would like to see how projects like this and cheetah bot play out a few months down the road.
👍  , , ,
properties (23)
authorcryptohustlin
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160724t234352539z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-24 23:43:51
last_update2016-07-24 23:43:51
depth1
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.199 HBD
curator_payout_value0.021 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length102
author_reputation35,183,938,577,926
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,361
net_rshares172,945,349,963
author_curate_reward""
vote details (4)
@darwen ·
Good work.  Thanks to share!
👍  
properties (23)
authordarwen
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160725t011544677z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-25 01:15:48
last_update2016-07-25 01:15:48
depth1
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length28
author_reputation327,384,182,371
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id335,897
net_rshares238,130,116
author_curate_reward""
vote details (1)
@drpepper ·
$0.22
Good job trying coding is way to daunting for me to try
👍  ,
properties (23)
authordrpepper
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160724t234726016z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-24 23:47:27
last_update2016-07-24 23:47:27
depth1
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.170 HBD
curator_payout_value0.050 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length55
author_reputation149,639,650,409
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,416
net_rshares170,380,973,324
author_curate_reward""
vote details (2)
@jholmes91 ·
Interested to hear more, thanks for sharing! :)
properties (22)
authorjholmes91
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160724t235020212z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-24 23:50:15
last_update2016-07-24 23:50:15
depth1
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length47
author_reputation2,806,857,550,600
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,452
net_rshares0
@kenjiby ·
$0.12
Well done, good luck with next project
👍  
properties (23)
authorkenjiby
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160724t234125157z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-24 23:41:27
last_update2016-07-24 23:41:27
depth1
children2
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.122 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length38
author_reputation8,543,158,141,728
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,297
net_rshares97,975,592,771
author_curate_reward""
vote details (1)
@djleo ·
Thanks!
👍  ,
properties (23)
authordjleo
permlinkre-kenjiby-re-djleo-my-open-source-anti-plagiarism-steemit-bot-20160724t234528281z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-24 23:45:27
last_update2016-07-24 23:45:27
depth2
children1
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length7
author_reputation1,834,764,955,389
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,391
net_rshares236,365,798
author_curate_reward""
vote details (2)
@andriana ·
$0.12
Thank you great research have to reread it.
👍  ,
properties (23)
authorandriana
permlinkre-djleo-re-kenjiby-re-djleo-my-open-source-anti-plagiarism-steemit-bot-20160725t060537121z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-25 06:05:36
last_update2016-07-25 06:05:36
depth3
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.097 HBD
curator_payout_value0.025 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length43
author_reputation2,826,053,446
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id339,947
net_rshares98,228,071,156
author_curate_reward""
vote details (2)
@marsresident · (edited)
$0.23
Here are some other Bots
https://steemit.com/steemit/@marsresident/bots-and-the-steemit-ecosystem

If you keep making Bots you will probably get a good amount of STEEM. People upvote Bot Comments not even realizing it sometimes, like Wang, he has over $1,000,000.
👍  , , , , ,
properties (23)
authormarsresident
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160724t234129662z
categorysteemit
json_metadata{"tags":["steemit"],"links":["https://steemit.com/steemit/@marsresident/bots-and-the-steemit-ecosystem"]}
created2016-07-24 23:41:42
last_update2016-07-24 23:42:06
depth1
children2
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.206 HBD
curator_payout_value0.021 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length263
author_reputation10,013,229,877,794
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,303
net_rshares177,392,337,240
author_curate_reward""
vote details (6)
@cryptohustlin ·
$0.12
Can confirm... wang fooled me until i noticed him EVERYWHERE.
👍  ,
properties (23)
authorcryptohustlin
permlinkre-marsresident-re-djleo-my-open-source-anti-plagiarism-steemit-bot-20160724t234500860z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-24 23:45:00
last_update2016-07-24 23:45:00
depth2
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.094 HBD
curator_payout_value0.025 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length61
author_reputation35,183,938,577,926
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,381
net_rshares96,060,702,172
author_curate_reward""
vote details (2)
@djleo ·
Thanks - this is really great!
properties (22)
authordjleo
permlinkre-marsresident-re-djleo-my-open-source-anti-plagiarism-steemit-bot-20160724t234449561z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-24 23:44:48
last_update2016-07-24 23:44:48
depth2
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length30
author_reputation1,834,764,955,389
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,379
net_rshares0
@rafaelleonr ·
bien hecho!
properties (22)
authorrafaelleonr
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160724t234444392z
categorysteemit
json_metadata{"tags":["steemit"]}
created2016-07-24 23:44:45
last_update2016-07-24 23:44:45
depth1
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length11
author_reputation114,732,256,577
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id334,377
net_rshares0
@steve-mcclair ·
Hell yeah! All my content is original. I went to Area 51 to explain my ideas. 

https://steemit.com/area/@steve-mcclair/area-51-steemit-has-arrived
👎  
properties (23)
authorsteve-mcclair
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160726t022805707z
categorysteemit
json_metadata{"tags":["steemit"],"links":["https://steemit.com/area/@steve-mcclair/area-51-steemit-has-arrived"]}
created2016-07-26 02:28:06
last_update2016-07-26 02:28:06
depth1
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length147
author_reputation2,707,974,922,314
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id364,313
net_rshares-97,975,779,235
author_curate_reward""
vote details (1)
@vlad ·
$0.12
Good job!!!
http://svce.acm.org/wp-content/uploads/2013/07/keep-calm-and-try-coding.png
👍  
properties (23)
authorvlad
permlinkre-djleo-my-open-source-anti-plagiarism-steemit-bot-20160725t030710165z
categorysteemit
json_metadata{"tags":["steemit"],"image":["http://svce.acm.org/wp-content/uploads/2013/07/keep-calm-and-try-coding.png"]}
created2016-07-25 03:07:24
last_update2016-07-25 03:07:24
depth1
children0
last_payout2016-08-25 04:28:48
cashout_time1969-12-31 23:59:59
total_payout_value0.094 HBD
curator_payout_value0.029 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length87
author_reputation2,715,371,190,063
root_title"My open source anti-plagiarism steemit bot"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id337,710
net_rshares97,975,592,771
author_curate_reward""
vote details (1)