create account

Let's build Bots for STEEM - Part 01: The Server by botsultant

View this thread on: hive.blogpeakd.comecency.com
· @botsultant · (edited)
$0.08
Let's build Bots for STEEM - Part 01: The Server
# Hi Steemians, let's build BOTS!
####
On this blog I want to document my progress in implementing bots for the STEEM blockchain.


https://steemitimages.com/DQmYBEXnPDCks41VWPfQr5HAfE6kfMoH6NYArDbrkzND7pT/robots-764951_1280.png
<center>Image Source: https://pixabay.com/de/roboter-computer-bots-charakter-764951/</center>

# What is a "Bot"?
####
A bot is a piece of code running on a computer and doing some stuff, for instance communicating with the STEEM blockchain, where already a lot of bots are running. 

The first toolkit for building bots was the [PISTON library](https://github.com/xeroc/piston) from @xeroc. I've build some bot's with PISTON, but [since steemit inc has disabled their own wss endpoint](https://steemit.com/steemitdev/@steemitdev/last-chance-to-update-steemd-steemit-com-will-be-retired-on-january-6) it seems to be urgent to migrate to [steem-python](https://github.com/steemit/steem-python), the official successor of piston.

# New Server
####

Since I don't want to kill my running PISTON based bots, I rented a new virtual server with a naked Ubuntu 16 from @privex. 

<center>https://steemitimages.com/DQmbRKYgL51ipgw1pRjaUU9MkYCmMXgMKuRssBV1tZtHwVE/PrivexOrder.png</center>

After one day I got a mail with the login credentials.

<center>https://steemitimages.com/DQmcpTsLdX1WcHusv2oR37pqEzGSrhynknh2parYQEWmuro/LoginData.png</center>

# First Contact
####
Now it's time to open a terminal and to connect to the new server. Just enter 

```
$ ssh ubuntu@178.XXX.XXX.XXX
```
####
into the terminal, answer with **yes** and enter your password (see mail).

```
The authenticity of host '178.XXX.XXX.XXX (178.XXX.XXX.XXX)' can't be established.
ECDSA key fingerprint is SHA256:IjtacL87I6xxxxxxxxxxxxxxxxxxxxxxxxxxxx.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '178.XXX.XXX.XXX' (ECDSA) to the list of known hosts.
ubuntu@178.XXX.XXX.XXX's password:
```
####

After login with the correct password we are asked to change the password.

```
You are required to change your password immediately (root enforced)
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-81-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

89 packages can be updated.
0 updates are security updates.


*** System restart required ***

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

WARNING: Your password has expired.
You must change your password now and login again!
Changing password for ubuntu.
```
####

Enter the old password from the mail and then enter your new password twice. 

```
(current) UNIX password: 
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully
Connection to 178.XXX.XXX.XXX closed.
```
####
After entering the password twice, it is changed and the SSH session to the server is terminated.

# Second Login
####
Now we can login to our server with our new password. Type again 

```
$ ssh ubuntu@178.XXX.XXX.XXX
```
####

and enter your password.

```
Welcome to Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-81-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

89 packages can be updated.
0 updates are security updates.


*** System restart required ***
Last login: Wed Jan 10 06:00:00 2018 from XXX.XXX.XXX.XXX
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@xxxx:~$
```
####
We are now ready to start.

# The Hardening
####
It's important to harden the server to make it as difficult as possible for attackers to gain access to the server. @privex recommends the following steps

https://steemitimages.com/DQmXs7CMWCikaMTGfNN7KAW1sAYZez87Ej8Fvm4nK14xZow/Hardening.png

## fail2ban
####
The installation of **fail2ban** requires root access. We can use the **sudo** command to get temporarily root rights.

```
ubuntu@xxxxxxxx:~$ sudo apt install fail2ban
```
####

```
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libpython-stdlib libpython2.7-minimal libpython2.7-stdlib python python-minimal python2.7 python2.7-minimal python3-pyinotify whois
Suggested packages:
  mailx monit python-doc python-tk python2.7-doc binutils binfmt-support python-pyinotify-doc
The following NEW packages will be installed:
  fail2ban libpython-stdlib libpython2.7-minimal libpython2.7-stdlib python python-minimal python2.7 python2.7-minimal python3-pyinotify whois
0 upgraded, 10 newly installed, 0 to remove and 89 not upgraded.
Need to get 4,194 kB of archives.
After this operation, 18.1 MB of additional disk space will be used.
Do you want to continue? [Y/n]
```
####
Answer **Y** for Yes and the installation will start.
```
Get:1 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython2.7-minimal amd64 2.7.12-1ubuntu0~16.04.2 [338 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python2.7-minimal amd64 2.7.12-1ubuntu0~16.04.2 [1,294 kB]
Get:3 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-minimal amd64 2.7.11-1 [28.2 kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libpython2.7-stdlib amd64 2.7.12-1ubuntu0~16.04.2 [1,880 kB]
Get:5 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 python2.7 amd64 2.7.12-1ubuntu0~16.04.2 [224 kB]
Get:6 http://archive.ubuntu.com/ubuntu xenial/main amd64 libpython-stdlib amd64 2.7.11-1 [7,656 B]
Get:7 http://archive.ubuntu.com/ubuntu xenial/main amd64 python amd64 2.7.11-1 [137 kB]
Get:8 http://archive.ubuntu.com/ubuntu xenial/universe amd64 fail2ban all 0.9.3-1 [227 kB]
Get:9 http://archive.ubuntu.com/ubuntu xenial/main amd64 python3-pyinotify all 0.9.6-0fakesync1 [24.7 kB]
Get:10 http://archive.ubuntu.com/ubuntu xenial/main amd64 whois amd64 5.2.11 [34.0 kB]
Fetched 4,194 kB in 0s (11.3 MB/s)
Selecting previously unselected package libpython2.7-minimal:amd64.
(Reading database ... 81808 files and directories currently installed.)
Preparing to unpack .../libpython2.7-minimal_2.7.12-1ubuntu0~16.04.2_amd64.deb ...
Unpacking libpython2.7-minimal:amd64 (2.7.12-1ubuntu0~16.04.2) ...
Selecting previously unselected package python2.7-minimal.
Preparing to unpack .../python2.7-minimal_2.7.12-1ubuntu0~16.04.2_amd64.deb ...
Unpacking python2.7-minimal (2.7.12-1ubuntu0~16.04.2) ...
Selecting previously unselected package python-minimal.
Preparing to unpack .../python-minimal_2.7.11-1_amd64.deb ...
Unpacking python-minimal (2.7.11-1) ...
Selecting previously unselected package libpython2.7-stdlib:amd64.
Preparing to unpack .../libpython2.7-stdlib_2.7.12-1ubuntu0~16.04.2_amd64.deb ...
Unpacking libpython2.7-stdlib:amd64 (2.7.12-1ubuntu0~16.04.2) ...
Selecting previously unselected package python2.7.
Preparing to unpack .../python2.7_2.7.12-1ubuntu0~16.04.2_amd64.deb ...
Unpacking python2.7 (2.7.12-1ubuntu0~16.04.2) ...
Selecting previously unselected package libpython-stdlib:amd64.
Preparing to unpack .../libpython-stdlib_2.7.11-1_amd64.deb ...
Unpacking libpython-stdlib:amd64 (2.7.11-1) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for mime-support (3.59ubuntu1) ...
Setting up libpython2.7-minimal:amd64 (2.7.12-1ubuntu0~16.04.2) ...
Setting up python2.7-minimal (2.7.12-1ubuntu0~16.04.2) ...
Linking and byte-compiling packages for runtime python2.7...
Setting up python-minimal (2.7.11-1) ...
Selecting previously unselected package python.
(Reading database ... 82554 files and directories currently installed.)
Preparing to unpack .../python_2.7.11-1_amd64.deb ...
Unpacking python (2.7.11-1) ...
Selecting previously unselected package fail2ban.
Preparing to unpack .../fail2ban_0.9.3-1_all.deb ...
Unpacking fail2ban (0.9.3-1) ...
Selecting previously unselected package python3-pyinotify.
Preparing to unpack .../python3-pyinotify_0.9.6-0fakesync1_all.deb ...
Unpacking python3-pyinotify (0.9.6-0fakesync1) ...
Selecting previously unselected package whois.
Preparing to unpack .../whois_5.2.11_amd64.deb ...
Unpacking whois (5.2.11) ...
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for systemd (229-4ubuntu17) ...
Processing triggers for ureadahead (0.100.0-19) ...
Setting up libpython2.7-stdlib:amd64 (2.7.12-1ubuntu0~16.04.2) ...
Setting up python2.7 (2.7.12-1ubuntu0~16.04.2) ...
Setting up libpython-stdlib:amd64 (2.7.11-1) ...
Setting up python (2.7.11-1) ...
Setting up fail2ban (0.9.3-1) ...
Setting up python3-pyinotify (0.9.6-0fakesync1) ...
Setting up whois (5.2.11) ...
Processing triggers for systemd (229-4ubuntu17) ...
Processing triggers for ureadahead (0.100.0-19) ...
ubuntu@xxxxxxxx:~$
```

## Alternative SSH Port
####

In order to change the SSH port we need to change a config file and restart the SSH daemon. Enter 

```
ubuntu@xxxxxxxx:~$ sudo vi /etc/ssh/sshd_config
```
####
to open the vi editor. The fifth row of the config file contains the currently configured SSH port: 22, the default value. In case you have no experience with vi, just click the **i** key to switch to **insert** mode and use the arrow keys to navigate to the port number.

```
# Package generated configuration file
# See the sshd_config(5) manpage for details

# What ports, IPs and protocols we listen for
Port 22
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
#ListenAddress 0.0.0.0
Protocol 2
```
####
Delete the **22** and insert your new port number, for instance 12345.

```
# What ports, IPs and protocols we listen for
Port 12345
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
...
-- INSERT --
```
####
Finally we need to save the changes. First hit the **ESC** key to leave the insert mode and than type 

```
:wq
```
####
to save the changes and close the editor.

Now we need to restart the SSH process (daemon). Enter the following command:
```
ubuntu@xxxxxxxx:~$ sudo /etc/init.d/ssh restart
```
####
```
[ ok ] Restarting ssh (via systemctl): ssh.service.
```
####
Seems that the SSH port change was succesfully.

To test if the port configuration works, open a new terminal and enter

```
$ ssh ubuntu@178.XXX.XXX.XXX -p 12345
ubuntu@178.XXX.XXX.XXX's password: 
```
####
#### Note that we now always need to provided the port number with the **-p** option.
####
After entering your password you are logged in.

## System Updates
####
We need again the **sudo** command to run an **apt update**

```
$ sudo apt update
```
####
```
Hit:1 http://archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Get:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [102 kB]                    
Get:5 http://security.ubuntu.com/ubuntu xenial-security/main Sources [104 kB]
Get:6 http://security.ubuntu.com/ubuntu xenial-security/universe Sources [48.9 kB]
Get:7 http://archive.ubuntu.com/ubuntu xenial-updates/main Sources [290 kB]                   
Get:8 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [419 kB]
Get:9 http://security.ubuntu.com/ubuntu xenial-security/main Translation-en [184 kB]      
Get:10 http://archive.ubuntu.com/ubuntu xenial-updates/universe Sources [186 kB]     
Get:11 http://security.ubuntu.com/ubuntu xenial-security/restricted amd64 Packages [7,224 B]
Get:12 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse Sources [7,968 B]
Get:13 http://security.ubuntu.com/ubuntu xenial-security/restricted Translation-en [2,152 B]
Get:14 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [698 kB]
Get:15 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [194 kB]
Get:16 http://security.ubuntu.com/ubuntu xenial-security/universe Translation-en [100 kB] 
Get:17 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [3,208 B]           
Get:18 http://archive.ubuntu.com/ubuntu xenial-updates/main Translation-en [291 kB]
Get:19 http://archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages [7,588 B]
Get:20 http://archive.ubuntu.com/ubuntu xenial-updates/restricted Translation-en [2,272 B]
Get:21 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [571 kB]
Get:22 http://archive.ubuntu.com/ubuntu xenial-updates/universe Translation-en [230 kB]
Get:23 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [16.2 kB]
Get:24 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse Translation-en [8,052 B]
Fetched 3,678 kB in 0s (4,755 kB/s)                                 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
93 packages can be upgraded. Run 'apt list --upgradable' to see them.
```

We see that 93 packages can be upgraded. Now we run

```
sudo apt upgrade -y
```
####
to update the system. 

## Reboot the system
####
Finally we need to reboot our new server, because we see the **System restart required** message after login. 

```
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-81-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

0 packages can be updated.
0 updates are security updates.


*** System restart required ***
Last login: Wed Jan 10 05:40:08 2018 from XXX.XXX.XXX.XXX
```
####
Just type 

```
$ sudo reboot
```
####
to reboot your server. 

```
Connection to 178.XXX.XXX.XXX closed by remote host.
Connection to 178.XXX.XXX.XXX closed.
```
####
The server shuts down and closes all open connections. After the reboot we need to login again.

```
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-109-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

0 packages can be updated.
0 updates are security updates.
```

# Congratulations
####
Our server is now up and running. 

In my next post I will install steem-python from the scratch. Stay tuned!
👍  , , , , , , ,
properties (23)
authorbotsultant
permlinklet-s-build-bots-for-steem-part-1-the-server
categorysteem
json_metadata{"tags":["steem","steemdev","utopian-io","python","ubuntu"],"users":["xeroc","privex"],"image":["https://steemitimages.com/DQmYBEXnPDCks41VWPfQr5HAfE6kfMoH6NYArDbrkzND7pT/robots-764951_1280.png","https://steemitimages.com/DQmbRKYgL51ipgw1pRjaUU9MkYCmMXgMKuRssBV1tZtHwVE/PrivexOrder.png","https://steemitimages.com/DQmcpTsLdX1WcHusv2oR37pqEzGSrhynknh2parYQEWmuro/LoginData.png","https://steemitimages.com/DQmXs7CMWCikaMTGfNN7KAW1sAYZez87Ej8Fvm4nK14xZow/Hardening.png"],"links":["https://pixabay.com/de/roboter-computer-bots-charakter-764951/","https://github.com/xeroc/piston","https://steemit.com/steemitdev/@steemitdev/last-chance-to-update-steemd-steemit-com-will-be-retired-on-january-6","https://github.com/steemit/steem-python"],"app":"steemit/0.1","format":"markdown"}
created2018-01-10 11:55:36
last_update2018-01-11 10:48:45
depth0
children2
last_payout2018-01-17 11:55:36
cashout_time1969-12-31 23:59:59
total_payout_value0.078 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length14,875
author_reputation113,201,096,904
root_title"Let's build Bots for STEEM - Part 01: The Server"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd0
post_id28,480,785
net_rshares7,711,412,029
author_curate_reward""
vote details (8)
@rondras ·
$0.22
Thank you very much for this initial tutorial. I am really looking forward to the next parts. Especially, I am currently looking for a way to read out my wallet history, so that I can automatically find out who has sent me money and up vote their most recent post. Unfortunately, I have not been able to find the piston documentation for this part yet. If you would cover that, it would be highly appreciated.
👍  ,
properties (23)
authorrondras
permlinkre-botsultant-let-s-build-bots-for-steem-part-1-the-server-20180115t214614331z
categorysteem
json_metadata{"tags":["steem"],"app":"steemit/0.1"}
created2018-01-15 21:46:12
last_update2018-01-15 21:46:12
depth1
children0
last_payout2018-01-22 21:46:12
cashout_time1969-12-31 23:59:59
total_payout_value0.174 HBD
curator_payout_value0.048 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length409
author_reputation18,390,194,709,037
root_title"Let's build Bots for STEEM - Part 01: The Server"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id29,798,008
net_rshares24,046,112,209
author_curate_reward""
vote details (2)
@steem-network ·
<html>
<p>Congratulations <a href="/@botsultant" target="_blank">@botsultant</a>, you have decided to take the next big step with your first post! The Steem Network Team wishes you a great time among this awesome community.</p>
<hr>
<div class="pull-left"><img src="https://steemitimages.com/DQmaAdLUJ3yaSkmcmWECWyPGPWcjfbCoZ8Tu4RM6H4DbjCi/steem-network-thumbs-up.gif" alt="Thumbs up for Steem Network´s strategy" title="I suggest Steem Network´s strategy" width="320" height="222"></div>
<h1>The proven road to boost your personal success in this amazing Steem Network</h1>
<p>Do you already know that awesome content will get great profits by following these <a href="/steem-network/@steem-network/spread-your-posts-through-this-proven-strategy-and-get-great-profits-in-return--for-posts-created-at-2018-01-10" target="_blank" alt="Steem Network" title="Follow Steem Network´s suggestions to boost your success">simple steps</a>, that have been worked out by experts?</p>
</html>
properties (22)
authorsteem-network
permlinkre-let-s-build-bots-for-steem-part-1-the-server-20180110t234347
categorysteem
json_metadata"{"tags": ["steem-network"], "users": ["steem-network", "botsultant"], "image": ["https://steemitimages.com/DQmaAdLUJ3yaSkmcmWECWyPGPWcjfbCoZ8Tu4RM6H4DbjCi/steem-network-thumbs-up.gif"], "links": ["/@botsultant", "/steem-network/@steem-network/spread-your-posts-through-this-proven-strategy-and-get-great-profits-in-return--for-posts-created-at-2018-01-10"], "community": "steem-network", "app": "steem-network/1.0.1", "format": "html"}"
created2018-01-10 23:43:48
last_update2018-01-10 23:43:48
depth1
children0
last_payout2018-01-17 23:43:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length981
author_reputation-788,969,433,247
root_title"Let's build Bots for STEEM - Part 01: The Server"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id28,615,759
net_rshares0