create account

Contribution to dhive : Added memo encryption and decrypt feature by tngflx

View this thread on: hive.blogpeakd.comecency.com
· @tngflx · (edited)
$129.14
Contribution to dhive : Added memo encryption and decrypt feature
Here is the merge request : https://gitlab.syncad.com/hive/dhive/-/merge_requests/12/diffs

Wow I certainly didn't expect developing such a feature would take me up to 2 week to finish it. Embarking on this journey of developing it made me learn about basic of cryptography and how encrypted message can be transfer securely between two parties. Hereby I wish to give a more thorough tutorial on how you can play with encryption on blockchain.

In cryptography there are two major types of encryption schemes are widely used: symmetric encryption (where a single secret key is used to encrypt and decrypt data) and asymmetric encryption (where a public key cryptosystem is used and encryption and decryption is done using a pair of public and corresponding private key).

In this case, hivejs and dhive is using ecies (Elliptic curve integrated encryption scheme). ECIES standard is a hybrid between symmetric and asymmetric encryption.

It consists of :
1) ECC- cryptography (public key cryptosystem)
2) Key-derivation function
3) symmetric encryption algorithm (for example AES encryption)
4) MAC algorithm (Message authentication code)

## So what each of them do?
The main function here is the AES encryption. It outputs ciphertext as its final result by using randomly generated IV + MAC code + input message. 
IV = initial vector (or random salt)
MAC code = Message authentication code exists as an indicator if the encrypted message is being tampered with
input message = plain text message that you wish to encrypt

So what we are doing is just feeding the aes algorithm to do the work.  

Library that we used to implement the above are: secp256k1 or P-521 elliptic curve for the public-key calculations + HKDF for KDF function + AES-CBC for symmetric cipher and authentication tag + HMAC-SHA512 for MAC algorithm (in case of unauthenticated encryption).

https://gblobscdn.gitbook.com/assets%2F-LhlOQMrG9bRiqWpegM0%2F-LhlOTG3w57kUSFndpUZ%2F-LhlPlxMlsLFWd0OlLoQ%2FECIES.png?alt=media
Source from : Nakov cryptobook

Let's look into the code to have a step-by-step understanding of what's really happening.. 

### The encryption process

```js
export function encode(private_key: PrivateKey | string, public_key: PublicKey | string, memo: string, testNonce?: number) {
    if (!/^#/.test(memo)) return memo
    memo = memo.substring(1)

    private_key = toPrivateObj(private_key) as PrivateKey
    public_key = toPublicObj(public_key) as PublicKey

    const { nonce, message, checksum } = Aes.encrypt(private_key, public_key, memo, testNonce);

    let mbuf = new ByteBuffer(ByteBuffer.DEFAULT_CAPACITY, ByteBuffer.LITTLE_ENDIAN)
    Types.EncryptedMemo(mbuf, {
        from: private_key.createPublic(),
        to: public_key,
        nonce,
        check: checksum,
        encrypted: message
    });
    mbuf.flip();
    const data = Buffer.from(mbuf.toBuffer());
    return '#' + bs58.encode(data);
}
```

As you can see encryption function requires sender's privatekey and recepient's publickey, memo as plain text message that you wish to encrypt.

In comparison with hivejs, dhive written in typescript allow type checking straight from the function paramaters. The code is cleaner and much readable.

As you can see we firstly convert the privatekey of sender and publickey of recipient into a workable object. We then use those keys to derive a shared key that can be embedded to our message, this is to indicate if our message is ever tampered with. If the message content is changed, shared key won't be the same and decryption process will be unsuccessful.

Let's have a look how aes work under the hood :
```js
export function encrypt(private_key: PrivateKey, public_key: PublicKey, message: any, nonce) {
    // Change message to varint32 prefixed encoded string
    let mbuf = new ByteBuffer(ByteBuffer.DEFAULT_CAPACITY, ByteBuffer.LITTLE_ENDIAN)
    mbuf.writeVString(message)
    message = Buffer.from(mbuf.flip().toBinary());

    let aesKey = private_key.encapsulate(public_key);
    return crypt(aesKey, nonce = uniqueNonce(), message)
}

export class PrivateKey {
  public secret: Buffer;

  constructor(private key: Buffer) {
    this.secret = key;
    assert(secp256k1.privateKeyVerify(key), "invalid private key");
  }

  /**
   * HMAC based key derivation function
   * @param pub recipient publickey
   */
  public encapsulate(pub: PublicKey): Buffer {
    const master = Buffer.concat([
      pub.uncompressed,
      this.multiply(pub),
    ]);
    return hkdf(master, 64, {
      hash: "SHA-512",
    });
  }

  public multiply(pub: any): Buffer {
    return Buffer.from(secp256k1.publicKeyTweakMul(pub.key, this.secret, false));
  }
}
```

So at the end you will get ciphered data that looks like this :
```
+-------------------------------+----------+----------+-----------------+
| 65 Bytes                      | 16 Bytes | 32 Bytes | == data size    |
+-------------------------------+----------+----------+-----------------+
| Sender Public Key (ephemeral) | Nonce/IV | Tag/MAC  | Encrypted data  |
+-------------------------------+----------+----------+-----------------+
|           Secp256k1           |              AES-256-CBC              |
+-------------------------------+---------------------------------------+
```
### Decryption process
Decryption of encrypted message requires unbundle of this packed Buffer and then using privatekey of recipient to unlock the data.

```js
export function decode(private_key: PrivateKey | string, memo: any) {
    if (!/^#/.test(memo)) return memo
    memo = memo.substring(1)
    // checkEncryption()

    private_key = toPrivateObj(private_key) as PrivateKey

    memo = bs58.decode(memo)
    memo = types.EncryptedMemoD(Buffer.from(memo, 'binary'))

    const { from, to, nonce, check, encrypted } = memo
    const pubkey = private_key.createPublic().toString()
    const otherpub = pubkey === new PublicKey(from.key).toString() ? new PublicKey(to.key) : new PublicKey(from.key)
    memo = Aes.decrypt(private_key, otherpub, nonce, encrypted, check)

    // remove varint length prefix
    const mbuf = ByteBuffer.fromBinary(memo.toString('binary'), ByteBuffer.LITTLE_ENDIAN)
    try {
        mbuf.mark()
        return '#' + mbuf.readVString()
    } catch (e) {
        mbuf.reset()
        // Sender did not length-prefix the memo
        memo = Buffer.from(mbuf.toString('binary'), 'binary').toString('utf-8')
        return '#' + memo
    }
}
```

As dhive itself didn't provide any buffer deserializer. I made one myself as close to serialization function that was written by previous author before.

```js
import * as ByteBuffer from 'bytebuffer'
import { PublicKey } from '../crypto';

export type Deserializer = (buffer: ByteBuffer) => void

const PublicKeyDeserializer = (
        buf: ByteBuffer
) => {
        let c: ByteBuffer = fixed_buf(buf, 33)
        return PublicKey.fromBuffer(c)
}

const UInt64Deserializer = (b: ByteBuffer) => {
        return b.readUint64();
}

const UInt32Deserializer = (b: ByteBuffer) => {
        return b.readUint32();
}

const BinaryDeserializer = (b: ByteBuffer) => {
        let b_copy: ByteBuffer;
        var len = b.readVarint32();
        b_copy = b.copy(b.offset, b.offset + len), b.skip(len);
        return Buffer.from(b_copy.toBinary(), 'binary');
}

const BufferDeserializer = (keyDeserializers: [string, Deserializer][]) => (
        buf: ByteBuffer | Buffer
) => {
        let obj = {};
        for (const [key, deserializer] of keyDeserializers) {
                try {
                        //Decodes a binary encoded string to a ByteBuffer.
                        buf = ByteBuffer.fromBinary(buf.toString("binary"), ByteBuffer.LITTLE_ENDIAN);
                        obj[key] = deserializer(buf)
                } catch (error) {
                        error.message = `${key}: ${error.message}`
                        throw error
                }
        }
        return obj;
}

function fixed_buf(b: ByteBuffer, len: number): Buffer | any {
        if (!b) {
                throw Error('No buffer found on first parameter')
        } else {
                let b_copy = b.copy(b.offset, b.offset + len);
                b.skip(len);
                return Buffer.from(b_copy.toBinary(), 'binary');
        }
}

const EncryptedMemoDeserializer = BufferDeserializer([
        ['from', PublicKeyDeserializer],
        ['to', PublicKeyDeserializer],
        ['nonce', UInt64Deserializer],
        ['check', UInt32Deserializer],
        ['encrypted', BinaryDeserializer]
]);

export const types = {
        EncryptedMemoD: EncryptedMemoDeserializer
}
```

By running the mocha test, you'll get back the string that was encrypted before! Yeay!

![image.png](https://files.peakd.com/file/peakd-hive/tngflx/qWxfnymG-image.png)

If you love to see more features added to hivejs or dhive, do upvote this content as a token of encouragement!

👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 62 others
👎  
properties (23)
authortngflx
permlinkcontribution-to-dhive-added-memo-encryption-and-decrypt-feature
categoryhive-139531
json_metadata"{"app":"peakd/2020.05.5","format":"markdown","description":"Contribution to dhive : Memo encryption and decryption in typescript!","tags":["development","dhive","hive","hivedevs","crypto"],"users":["param"],"links":["https://gitlab.syncad.com/hive/dhive/-/merge_requests/12/diffs"],"image":["https://gblobscdn.gitbook.com/assets%2F-LhlOQMrG9bRiqWpegM0%2F-LhlOTG3w57kUSFndpUZ%2F-LhlPlxMlsLFWd0OlLoQ%2FECIES.png?alt=media","https://files.peakd.com/file/peakd-hive/tngflx/qWxfnymG-image.png"]}"
created2020-06-21 06:59:33
last_update2020-06-21 07:04:15
depth0
children15
last_payout2020-06-28 06:59:33
cashout_time1969-12-31 23:59:59
total_payout_value64.582 HBD
curator_payout_value64.563 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length8,897
author_reputation17,396,455,988,713
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id98,089,749
net_rshares283,425,554,213,780
author_curate_reward""
vote details (127)
@agent14 ·
Appreciate your contribution to our Hive 
properties (22)
authoragent14
permlinkre-tngflx-qcbzfi
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2020.05.5"}
created2020-06-22 14:11:42
last_update2020-06-22 14:11:42
depth1
children1
last_payout2020-06-29 14:11:42
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length41
author_reputation72,504,131,225
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id98,113,777
net_rshares0
@tngflx ·
Appreciate your upvote and comment too!
properties (22)
authortngflx
permlinkre-agent14-qccvvz
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2020.05.5"}
created2020-06-23 01:52:48
last_update2020-06-23 01:52:48
depth2
children0
last_payout2020-06-30 01:52:48
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length39
author_reputation17,396,455,988,713
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id98,124,339
net_rshares0
@borislavzlatanov ·
I thought that the ability to encrypt and decrypt memos already exists. Or did you add a different implementation?
👍  
properties (23)
authorborislavzlatanov
permlinkre-tngflx-qc9pol
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2020.05.5"}
created2020-06-21 08:45:57
last_update2020-06-21 08:45:57
depth1
children1
last_payout2020-06-28 08:45:57
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length114
author_reputation23,539,504,746,700
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id98,091,032
net_rshares17,860,763,255
author_curate_reward""
vote details (1)
@tngflx · (edited)
This is for dhive/dsteem feature. It was not available in dhive before.
properties (22)
authortngflx
permlinkre-borislavzlatanov-qc9q1l
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2020.05.5"}
created2020-06-21 08:53:45
last_update2020-06-21 10:46:30
depth2
children0
last_payout2020-06-28 08:53:45
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length71
author_reputation17,396,455,988,713
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id98,091,107
net_rshares0
@chitty ·
I have picked your post for my daily hive voting initiative, Keep it up and Hive On!!
properties (22)
authorchitty
permlinkre-contribution-to-dhive-added-memo-encryption-and-decrypt-feature-20200627t001014
categoryhive-139531
json_metadata""
created2020-06-27 00:10:18
last_update2020-06-27 00:10:18
depth1
children0
last_payout2020-07-04 00:10:18
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length86
author_reputation86,901,300,608,582
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id98,197,703
net_rshares0
@codingdefined ·
Great job, will sure try it out. Can be a use case for private chatting.
properties (22)
authorcodingdefined
permlinkre-tngflx-2020622t19345256z
categoryhive-139531
json_metadata{"tags":["development","dhive","hive","hivedevs","crypto"],"app":"esteem/2.2.5-mobile","format":"markdown+html","community":"hive-125125"}
created2020-06-22 14:04:57
last_update2020-06-22 14:04:57
depth1
children1
last_payout2020-06-29 14:04:57
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length72
author_reputation536,203,978,055,117
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries
0.
accountesteemapp
weight300
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id98,113,664
net_rshares0
@tngflx ·
Hmm I believe @kingswisdom did such an app and openseed is also working on such a solution. Perhaps you can come out with something with this library :)
properties (22)
authortngflx
permlinkre-codingdefined-qccvvf
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2020.05.5"}
created2020-06-23 01:52:27
last_update2020-06-23 01:52:27
depth2
children0
last_payout2020-06-30 01:52:27
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length152
author_reputation17,396,455,988,713
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id98,124,334
net_rshares0
@fulltimegeek ·
Was this merge request accepted? 

I'm currently looking through dhive's documentation and can't find anything on how to (de|en)crypt messages.
properties (22)
authorfulltimegeek
permlinkqhw92p
categoryhive-139531
json_metadata{"app":"hiveblog/0.1"}
created2020-10-08 17:40:03
last_update2020-10-08 17:40:03
depth1
children1
last_payout2020-10-15 17:40:03
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length143
author_reputation82,542,358,852,913
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id100,021,057
net_rshares0
@tngflx ·
well it wasn't merge yet. but you can download my version of work. You'll see it'll work. They demanded that it will work like the one in hivejs. Well the technicality is quite challenging
properties (22)
authortngflx
permlinkre-fulltimegeek-qhzjhx
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2020.09.5"}
created2020-10-10 12:17:57
last_update2020-10-10 12:17:57
depth2
children0
last_payout2020-10-17 12:17:57
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length188
author_reputation17,396,455,988,713
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id100,046,261
net_rshares0
@lorenzopistolesi ·
you rock
👍  
properties (23)
authorlorenzopistolesi
permlinkqccdw8
categoryhive-139531
json_metadata{"app":"hiveblog/0.1"}
created2020-06-22 19:24:15
last_update2020-06-22 19:24:15
depth1
children4
last_payout2020-06-29 19:24:15
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length8
author_reputation204,869,285,671,935
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id98,119,045
net_rshares17,869,254,355
author_curate_reward""
vote details (1)
@tngflx ·
Thanks! there's so much to develop on hive!
properties (22)
authortngflx
permlinkre-lorenzopistolesi-qccvlq
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2020.05.5"}
created2020-06-23 01:46:39
last_update2020-06-23 01:46:39
depth2
children3
last_payout2020-06-30 01:46:39
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length43
author_reputation17,396,455,988,713
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id98,124,274
net_rshares0
@lorenzopistolesi ·
Are you a geek?
properties (22)
authorlorenzopistolesi
permlinkqcd074
categoryhive-139531
json_metadata{"app":"hiveblog/0.1"}
created2020-06-23 03:25:57
last_update2020-06-23 03:25:57
depth3
children2
last_payout2020-06-30 03:25:57
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length15
author_reputation204,869,285,671,935
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd0
post_id98,125,369
net_rshares0
@quicktrades ·
Nice work! I never tried typescript...
properties (22)
authorquicktrades
permlinkre-tngflx-qcrdm6
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2020.06.2"}
created2020-06-30 21:42:06
last_update2020-06-30 21:42:06
depth1
children0
last_payout2020-07-07 21:42:06
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length38
author_reputation84,707,475,935
root_title"Contribution to dhive : Added memo encryption and decrypt feature"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id98,268,020
net_rshares0