create account

Scanning the Dump by inertia

View this thread on: hive.blogpeakd.comecency.com
· @inertia · (edited)
$31.08
Scanning the Dump
<div class="pull-right">
  <blockquote>
    <img src="https://steemitimages.com/DQme4CCUe498qexYVhA9NaokJ3rD2QEfMnJarNsMZ9UCiGv/image.png">
  </blockquote>
</div>

So I was minding my own business reading my feed and I noticed that @matt-a posted:

[PSA: NEW HUGE PASSWORD DUMP. Are You One Of The 501,636,842 Earth Dwellers That Have Been Pwn3d?! You Might Want To Find Out. READ ME.](https://steemit.com/infosec/@matt-a/psa-new-huge-password-dump-are-you-one-of-the-501-636-842-earth-dwellers-that-have-been-pwn3d-you-might-want-to-find-out-read-me)

Now I have to write a ruby script to comb this 29 GB file of hashes.  But I have a lot of things to check.  I got one of the files over torrent, and extracted it (`pwned-passwords-2.0.txt`).

Here's what I'm using to check it:

`Gemfile`

```ruby
source 'https://rubygems.org'
gem 'highline'
```
---
`check.rb`

```ruby
require 'rubygems'
require 'bundler/setup'
require 'digest'

Bundler.require

filename = 'pwned-passwords-2.0.txt'
cli = HighLine.new
answers = []

loop do
  answer = cli.ask 'What do you want to check?  Empty line to start check.' do |q|
    q.echo = '*'
  end

  break if answer == ''
  answers << answer
end

shas = answers.map do |answer|
  Digest::SHA1.hexdigest(answer).upcase
end

File.open(filename, 'r') do |f|
  f.each_line do |line|
    exit if shas.empty?

    if line =~ /(#{shas.join('|')})/
      puts line
      sha = line.split(':').first
      shas.delete(sha)
    end
  end
end
```

Enjoy.<div class="chainbb-footer"><hr><em><small><a href="https://chainbb.com/ruby/@inertia/scanning-the-dump">Originally posted</a> in the <a href="https://chainbb.com/f/ruby">/f/ruby</a> forum on <a href="https://chainbb.com">chainBB.com</a> (<a href="https://chainbb.com/chainbb/@jesta/chainbb-frequently-asked-questions-faq">learn more</a>).</small></em></div>
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 3 others
👎  ,
properties (23)
authorinertia
permlinkscanning-the-dump
categoryruby
json_metadata{"app":"chainbb/0.4","namespace":"ruby","format":"markdown+html","tags":["ruby","infosec","security","haveibeenpwned","troy-hunt"]}
created2018-03-01 07:03:00
last_update2018-03-01 21:23:36
depth0
children27
last_payout2018-03-08 07:03:00
cashout_time1969-12-31 23:59:59
total_payout_value24.703 HBD
curator_payout_value6.381 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length1,839
author_reputation346,568,901,399,561
root_title"Scanning the Dump"
beneficiaries
0.
accountchainbb
weight275
1.
accountinertia
weight225
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,312,524
net_rshares6,198,934,207,016
author_curate_reward""
vote details (69)
@abbottsteem ·
Good post my friend
You have provided information to me and other steemit friends thank you @inertia
properties (22)
authorabbottsteem
permlinkre-inertia-scanning-the-dump-20180301t070617874z
categoryruby
json_metadata{"tags":["ruby"],"users":["inertia"],"app":"steemit/0.1"}
created2018-03-01 07:06:30
last_update2018-03-01 07:06:30
depth1
children0
last_payout2018-03-08 07:06:30
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length100
author_reputation37,627,238,412
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,313,132
net_rshares0
@abdurrahim007 ·
Hum Very interesting Coding ! Some this is really gone be awesome Hehe
properties (22)
authorabdurrahim007
permlinkre-inertia-scanning-the-dump-20180301t073403366z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 07:34:09
last_update2018-03-01 07:34:09
depth1
children0
last_payout2018-03-08 07:34:09
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length70
author_reputation85,441,788,682
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,317,632
net_rshares0
@abdurrazzak ·
OMG.... 
that's way,  you  have need long time for the  ready of ruby script.
its a great work....
thanks for the share
👍  
properties (23)
authorabdurrazzak
permlinkre-inertia-scanning-the-dump-20180301t075525433z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 07:55:36
last_update2018-03-01 07:55:36
depth1
children0
last_payout2018-03-08 07:55:36
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length119
author_reputation121,581,509,286
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,321,423
net_rshares362,286,978
author_curate_reward""
vote details (1)
@athena06 ·
What was this all about? I don't understand anything.
properties (22)
authorathena06
permlinkre-inertia-scanning-the-dump-20180302t055301944z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-02 05:53:09
last_update2018-03-02 05:53:09
depth1
children5
last_payout2018-03-09 05:53:09
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length53
author_reputation34,655,287,702
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,559,666
net_rshares0
@inertia ·
$0.38
![](https://steemitimages.com/DQmTAn7CaBfktxV6dPACNa7ahHAdm1LcocneTNUay1xUo5n/image.png)
👍  
properties (23)
authorinertia
permlinkre-athena06-re-inertia-scanning-the-dump-20180302t055547566z
categoryruby
json_metadata{"tags":["ruby"],"image":["https://steemitimages.com/DQmTAn7CaBfktxV6dPACNa7ahHAdm1LcocneTNUay1xUo5n/image.png"],"app":"steemit/0.1"}
created2018-03-02 05:55:48
last_update2018-03-02 05:55:48
depth2
children4
last_payout2018-03-09 05:55:48
cashout_time1969-12-31 23:59:59
total_payout_value0.375 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length88
author_reputation346,568,901,399,561
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,560,030
net_rshares77,195,544,366
author_curate_reward""
vote details (1)
@athena06 ·
![1.jpg](https://steemitimages.com/DQmP2ME7uoZpfhHBb5CeB4kRqmLyanW7Mh5u8NDLoUe6GtN/1.jpg)
properties (22)
authorathena06
permlinkre-inertia-re-athena06-re-inertia-scanning-the-dump-20180302t065823435z
categoryruby
json_metadata{"tags":["ruby"],"image":["https://steemitimages.com/DQmP2ME7uoZpfhHBb5CeB4kRqmLyanW7Mh5u8NDLoUe6GtN/1.jpg"],"app":"steemit/0.1"}
created2018-03-02 06:58:30
last_update2018-03-02 06:58:30
depth3
children3
last_payout2018-03-09 06:58:30
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length89
author_reputation34,655,287,702
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,569,162
net_rshares0
@azizulhassan ·
well its amazing nd new
properties (22)
authorazizulhassan
permlinkre-inertia-scanning-the-dump-20180301t102009946z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 10:20:24
last_update2018-03-01 10:20:24
depth1
children0
last_payout2018-03-08 10:20:24
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length23
author_reputation858,685,273,941
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,346,548
net_rshares0
@bilal-haider ·
I am going to look into it also ..
properties (22)
authorbilal-haider
permlinkre-inertia-scanning-the-dump-20180301t213836318z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 21:38:36
last_update2018-03-01 21:38:36
depth1
children0
last_payout2018-03-08 21:38:36
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length34
author_reputation4,891,139,097,092
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,478,140
net_rshares0
@cryptogee ·
Is this passwords from Bitcoin or just anywhere?

Love the look of Ruby by the way, the syntax is very clean. Do you work with it a lot?

[*Cg*](https://steemit.com/@cryptogee)
properties (22)
authorcryptogee
permlinkre-inertia-scanning-the-dump-20180301t071122240z
categoryruby
json_metadata{"tags":["ruby"],"links":["https://steemit.com/@cryptogee"],"app":"steemit/0.1"}
created2018-03-01 07:11:21
last_update2018-03-01 07:11:21
depth1
children0
last_payout2018-03-08 07:11:21
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length176
author_reputation419,387,439,147,428
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,313,911
net_rshares0
@diya28 ·
so you are going to consume your lot of time
properties (22)
authordiya28
permlinkre-inertia-scanning-the-dump-20180301t161434892z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 16:15:51
last_update2018-03-01 16:15:51
depth1
children0
last_payout2018-03-08 16:15:51
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length44
author_reputation1,661,130,029,399
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,417,313
net_rshares0
@exxodus ·
Hey @inertia, great post! I enjoyed your content. Keep up the good work! It's always nice to see good content here on Steemit! Cheers :)
properties (22)
authorexxodus
permlinkre-inertia-scanning-the-dump-20180301t070348011z
categoryruby
json_metadata{"tags":["ruby"],"users":["inertia"],"app":"steemit/0.1"}
created2018-03-01 07:03:48
last_update2018-03-01 07:03:48
depth1
children0
last_payout2018-03-08 07:03:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length136
author_reputation7,589,375,156,104
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,312,664
net_rshares0
@gaintshoulder ·
This is so advanced for me... are we being hacked?
Because am so worried. Those scripts are not in human language
properties (22)
authorgaintshoulder
permlinkre-inertia-scanning-the-dump-20180301t071307384z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 07:13:12
last_update2018-03-01 07:13:12
depth1
children0
last_payout2018-03-08 07:13:12
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length113
author_reputation709,742,325,817
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,314,193
net_rshares0
@hrishikeshmatre · (edited)
![77.png](https://res.cloudinary.com/hpiynhbhq/image/upload/v1519889124/xsmtrtjo5ttmk0izt9lf.png)

Help me @inertia for setting up of drotto
getting this error after run rake
are you on discord?
Thank you
properties (22)
authorhrishikeshmatre
permlinkre-inertia-scanning-the-dump-20180301t072622990z
categoryruby
json_metadata{"tags":["ruby"],"community":"busy","app":"steemit/0.1","users":["inertia"],"image":["https://res.cloudinary.com/hpiynhbhq/image/upload/v1519889124/xsmtrtjo5ttmk0izt9lf.png"]}
created2018-03-01 07:26:30
last_update2018-03-01 07:29:09
depth1
children2
last_payout2018-03-08 07:26:30
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length204
author_reputation695,961,875,605
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,316,343
net_rshares0
@inertia ·
I recommend rvm to manage ruby if the system version is giving you trouble:

https://rvm.io/
properties (22)
authorinertia
permlinkre-hrishikeshmatre-re-inertia-scanning-the-dump-20180301t073105937z
categoryruby
json_metadata{"tags":["ruby"],"links":["https://rvm.io/"],"app":"steemit/0.1"}
created2018-03-01 07:31:06
last_update2018-03-01 07:31:06
depth2
children1
last_payout2018-03-08 07:31:06
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length92
author_reputation346,568,901,399,561
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,317,121
net_rshares0
@hrishikeshmatre ·
Ok thank you i will do the same
properties (22)
authorhrishikeshmatre
permlinkre-inertia-re-hrishikeshmatre-re-inertia-scanning-the-dump-20180301t073429715z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 07:34:36
last_update2018-03-01 07:34:36
depth3
children0
last_payout2018-03-08 07:34:36
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length31
author_reputation695,961,875,605
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,317,735
net_rshares0
@hrishikeshmatre ·
@inertia 
 Do you have any article on how to do with rvm to manage ruby?
I am new to this so just need help 
Thank you...
properties (22)
authorhrishikeshmatre
permlinkre-inertia-scanning-the-dump-20180301t075607064z
categoryruby
json_metadata{"tags":["ruby"],"users":["inertia"],"app":"steemit/0.1"}
created2018-03-01 07:56:12
last_update2018-03-01 07:56:12
depth1
children3
last_payout2018-03-08 07:56:12
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length121
author_reputation695,961,875,605
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,321,521
net_rshares0
@inertia ·
If you refuse to read the documentation on https://rvm.io/, you can usually just get up and running by blindly entering these commands into your terminal:

```bash
$ gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
$ curl -sSL https://get.rvm.io | bash -s stable
$ source ~/.rvm/scripts/rvm
$ rvm install ruby-head
$ rvm use ruby-head --create
$ gem install bundler
```
👍  
properties (23)
authorinertia
permlinkre-hrishikeshmatre-re-inertia-scanning-the-dump-20180301t081853738z
categoryruby
json_metadata{"tags":["ruby"],"links":["https://rvm.io/"],"app":"steemit/0.1"}
created2018-03-01 08:18:54
last_update2018-03-01 08:18:54
depth2
children2
last_payout2018-03-08 08:18:54
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length410
author_reputation346,568,901,399,561
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,325,397
net_rshares515,499,580
author_curate_reward""
vote details (1)
@hrishikeshmatre ·
ok fine....
properties (22)
authorhrishikeshmatre
permlinkre-inertia-re-hrishikeshmatre-re-inertia-scanning-the-dump-20180301t082158866z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 08:22:06
last_update2018-03-01 08:22:06
depth3
children0
last_payout2018-03-08 08:22:06
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length11
author_reputation695,961,875,605
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,325,921
net_rshares0
@hrishikeshmatre ·
This also  not worked for me...
properties (22)
authorhrishikeshmatre
permlinkre-inertia-re-hrishikeshmatre-re-inertia-scanning-the-dump-20180301t211239239z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 21:12:45
last_update2018-03-01 21:12:45
depth3
children0
last_payout2018-03-08 21:12:45
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length31
author_reputation695,961,875,605
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,473,914
net_rshares0
@matt-a ·
Cool!  Do you plan on attacking this with a SHA-1 dictionary?
properties (22)
authormatt-a
permlinkre-inertia-scanning-the-dump-20180301t214703955z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 21:47:12
last_update2018-03-01 21:47:12
depth1
children1
last_payout2018-03-08 21:47:12
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length61
author_reputation34,621,295,577,150
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,479,507
net_rshares0
@inertia ·
Na, I just wanted a way to dump passwords into it without having to worry about them showing up on the screen.  I also wanted to check efficiently, so it does a regex on all of the passwords at once.  It's almost exactly the same as doing it with grep, except the passwords don't get saved to `.bash_history`.
properties (22)
authorinertia
permlinkre-matt-a-re-inertia-scanning-the-dump-20180301t215212752z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 21:52:12
last_update2018-03-01 21:52:12
depth2
children0
last_payout2018-03-08 21:52:12
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length309
author_reputation346,568,901,399,561
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,480,241
net_rshares0
@moniristi ·
Learn some new from your post. Love to read it.
👍  
properties (23)
authormoniristi
permlinkre-inertia-scanning-the-dump-20180301t075201485z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 07:52:03
last_update2018-03-01 07:52:03
depth1
children0
last_payout2018-03-08 07:52:03
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length47
author_reputation5,625,366,645,317
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,320,818
net_rshares943,992,092
author_curate_reward""
vote details (1)
@shihabieee ·
Thank you for this article. I'll be using this to send to people asking me the exact same questions.
👍  
properties (23)
authorshihabieee
permlinkre-inertia-scanning-the-dump-20180301t075336333z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 07:53:36
last_update2018-03-01 07:53:36
depth1
children0
last_payout2018-03-08 07:53:36
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length100
author_reputation-171,530,316,306
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,321,087
net_rshares964,513,660
author_curate_reward""
vote details (1)
@vaniapriani ·
I like you,,, nice to meet you
properties (22)
authorvaniapriani
permlinkre-inertia-scanning-the-dump-20180301t181339658z
categoryruby
json_metadata{"tags":["ruby"],"app":"steemit/0.1"}
created2018-03-01 18:13:45
last_update2018-03-01 18:13:45
depth1
children0
last_payout2018-03-08 18:13:45
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length30
author_reputation274,853,802,371
root_title"Scanning the Dump"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id41,440,684
net_rshares0