create account

NFT Research - Computation and Storage by disregardfiat

View this thread on: hive.blogpeakd.comecency.com
· @disregardfiat · (edited)
$278.63
NFT Research - Computation and Storage
## Computation Bytes
Welcome to the first in a series of post about NFT building that I hope to get some discussion on; [Continued from Building NFTs on Hive](https://peakd.com/hive-139531/@disregardfiat/building-nfts-on-hive). Non-Fungible Tokens are a tricky problem to find solutions for, especially because several exist. Most of what a solution hopes to find is the most economical way to provide security for these assets. Security in this context is managing permissions and ensuring data maintained.

Economics here are the two aspects of computational efficiency: computation space and computation time. In a distributed ledger that will maintain NFTs and other assets, both are extremely important for scaling. In my view the aspect that should be prioritized is computation space. As long as the computational speed is enough to keep real time, then computational space represents bottlenecks in computer RAM as well as network backups.

Layer 2 networks built on blockchains are Infinite Turing Tape Machines... an infinity anywhere in an equation needs to have an evaluation shortcut. For example, if in 100 or 100000 years somebody needs to audit a transaction/calculation that occurred 99 or 99999 years from now, then replaying the tape/blockchain from its present day genesis is a huge issue in terms of computational complexity. Having backups that would allow replay from a short period before the transaction becomes more economical, and the smaller those backups are the more frequent and easier they are to retrieve and process. 

The Layer 2 software that I've built actually uses these frequent backups to come to a consensus on it's data. Every 5 minutes an entire state backup is performed and it's content is stored on IPFS which provides a hash of that data. If an NFT was an entire photo or video stored on the layer 2 this would get extremely unwieldy. But just as important, if you are storing 10000 1kb NFTs it also get's cumbersome. 

### The First Spec
I'll be going over the basics of three types of NFTs and the first one is by far the most straight forward: IPFS.
This is a fairly common type of NFT in today's economy, an artist uploads a file to IPFS then takes the locator hash and builds an NFT to track the ownership of that particular asset. Given that it's so straight forward it should be relatively easy to track. So let's define what needs to be tracked and any concerns we might have.
|Aspect|Explanation|In Practice|
|-|-|-|
|Type|IPFS|0|
|Item|Location|"QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5"|
|Handling|File Type / Executable|"jpg" or "js"|
|Author|Builder|"disregardfiat"|
|Royalty|10%|1000|
|Owner|Current rights holder|"dlux-io"|
|Created|Block number|50000000|
|Modified|Block number and reason|51000000|
|Collision|Ensure uniqueness|"Qm:WCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5"|

Now let's see this in database records:
``` json
//Set => ./sets/
"QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5":{ //51bytes = 124 bytes average
	"a":"disregardfiat", //upto 27 bytes... average 19
	"c":50000000, //13-14 bytes
	"o":"dlux-io",//upto 27 bytes
	"r":1000, //6-9 bytes
	"t":0, //5 bytes
	"f":"jpg" //8-10 bytes
}
//Item => ./nfts/dlux-io/
"Qm:WCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5":{ //52bytes = 68 bytes with out extras(rent/license/etc)
	"m":51000000, //16-17 bytes
}
```
The first thing most of you will notice is the NFT is split between two records, this is done for a number of reasons, most importantly for search indexing, but also facilitates collision/uniqueness, and much smaller records for other types of NFTs.

By storing the whole record as the item name it prevents the same item from being claimed twice. In the user's directory the file name has a ':' in it which keeps the way larger set records are stored more uniform across the platform. All IPFS locators currently start with 'Qm' which specifies the type of hashing algorithm. It's easy to reassemble this at runtime. Now I can easily find out what somebody owns, or who owns what.

Across multiple layer 2s I can use the created block number to find who signed an item as an NFT first. I can also use the modified field to load specific back ups to compile an NFT history and get the specific transactions that modified the NFT from the layer 1 blocks. The file type let's a client side program load IPFS assets into the web in an easy way such as `<img src="https://ipfs.ip/ipfs/QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5?.jpg">`

All in all this NFT comes out to ~190 bytes... Assuming a Layer 2 has 10M tokens(Like 10M DLUX) and each byte costs ~0.01 tokens to write, the chain would be limited to roughly a gigabyte. Keeping Gigabyte backups would quickly get painful and a change only system like Time Machine by Apple would likely be employed. In actuality these fees would need to go to the nodes that run the layer 2 to pay for their systems to keep up. Really the only way to ensure the data remains. 

![CryptoPunks Image from LarvalLabs](https://www.larvalabs.com/public/images/product/cryptopunks/punk-variety-2x.png)

### The Second Spec
There's a completely different type of NFT going around and this one is based on layering partial images onto a canvas and owning a unique subset of these layers. These would be the Bored Apes and Crypto Punks of the world.
These particular types of sets are a little heavier in general than an IPFS link, but individually can be much smaller. For example defining a few bases and layers only cost that many bytes, but the number of ways they can be put together is much like the alphabet and grows geometrically. 
|Aspect|Explanation|In Practice|
|-|-|-|
|Author|Account defining set|"disregardfiat"|
|Script|Script hash|"QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5"|
|Base|Array of base images|["QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5"]|
|Characteristics|Array of layers|See Below|
|Issued|Counter|1|
|Max|End of counter|4096|
|Royalty|0.01%|1000|
|Name|Name of Set|"punks"|
|Link|Hive Locator for announcement/dApp|"author/permlink"|
|Uniques|Array of UniqueIDs and Owners|["base64_owner"]|
|Type|Layer|1|
|Created|Block number|50000000|
|Modified|Block number|51000000|
|Characteristics|Per item|165|

Now in Code:
``` JSON
//Set ./sets/
"punks":{ //5 plus set name bytes
 	"a":"author", //the account that pays the set fee, --23 bytes
	"s":"QmWCeBSjzeBC8Q9dDhF7ZcHxozgkv9d6XdwSMT7pVwytr5", //build app hash --53bytes
	"i":1, //issued counter for IDs -6bytes
	"m":4096, //max issue -6-10bytes
	"n":"punks", //name of set, 7+ bytes
	"r":100, // 8 bytes
	"t":1, // 5bytes
	"f":"png", //can be put in script
	"l":"author/permlink", //count bytes ~50
	"b":['ipfsNeuterpng', 'ipfsMalePNG', 'ipfsFemalePNG'], //base
	"u":['cFromNFTsBase64_owner'], //unique
	"c":[
		[['ipfsLayer1PNGa', qtyINT, 0, mintedInt, primeInt],['ipfsLayer1PNGb', qtyINT, 1, mintedInt, primeInt]], 
		[['ipfsLayer2PNGa', qtyINT, 0, mintedInt, primeInt],['ipfsLayer2PNGb', qtyINT, 1, mintedInt, primeInt],['ipfsLayer2PNGc', qtyINT, 2, mintedInt, primeInt]]    
	], //characteristics of layers ... count bytes... 
}
// Item ./nfts/dlux-io/
"punks:165":{ // 8-20 bytes
	"m":50000000, //12-13 bytes
	// above"c":165 // ~15 bytes set dependent (165=3*5*11, which is female, layer1a, layer2a)... this will draw out traits : c%primeInt == 0
}
```
Here you can see just how quickly the benefits add up. With a short named set like "punks" and an average account length of 12, each additional item is only adds the base64 bytes of the uniqueness with the owner account twice plus the individual modified time for auditing. Maybe ~50 bytes each. Given a large enough set this smaller size will quickly dominate the space requirement, and maintains our two way index. Where we could have stored 6 million IPFS NFTs we can now approach 20 million Layered NFTs

Reconstruction of the layers via the uniqueness code requires a log n of items per layer, and is about as computationally expensive as an array search.

Minting NFTs like this with a psuedorandom seed from blockchain data will have the most collisions in the middle of minting and return to no collisions at the tail, also trending on a log n of layers. A decent trade off for saving so much space. Several of the items in characteristics can be logically ommited but are named here for clarity, like primeInt. Having a qtyINT also let's the artist define the rarest traits... but a check at the definition stage will need to be performed to ensure the set won't fail uniqueness with the given parameters. For instance having 27 mint tokens for an set of the alphabet(26 items).

### Moving NFTs
These type definitions can easily be built and distributed allowing a layer 2 to compute NFTs of any specific type. 
Once a type definition is available on another layer 2 you should be able to move an NFT from one chain to another. This becomes especially handy if a layer 2 decides that each NFT must pay a rent for the space it's using on the blockchain. Maybe each NFT pays 0.01 rent for every 0.1 mint... every period of time. While a cold storage layer 2 might charge 0.001 rent for the same period... but auctions and account transfers aren't available, and only moving the item to a hot layer 2 will allow item modification.

I personally can't see saving much more than 30 bytes by doing this as the uniqueness locator would just be modified to the account of another layer2... maybe the swap account for hive engine... however... moving an IPFS NFT may save quite a bit more.

I'm inviting discussion of any aspect of NFTs, especially related to moving them around or if there is any part of creation, tracking, and storage that I might be over looking.
👍  , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and 183 others
👎  
properties (23)
authordisregardfiat
permlinknft-research-computation-and-storage
categoryhive-139531
json_metadata"{"app":"peakd/2021.08.2","format":"markdown","description":"First Discussion of NFT standard proposal for Hive","tags":["hive","dlux","nft","defi","dev"],"users":["disregardfiat"],"image":["https://www.larvalabs.com/public/images/product/cryptopunks/punk-variety-2x.png"]}"
created2021-09-12 18:06:09
last_update2021-09-12 21:58:42
depth0
children9
last_payout2021-09-19 18:06:09
cashout_time1969-12-31 23:59:59
total_payout_value139.354 HBD
curator_payout_value139.271 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length9,690
author_reputation348,255,545,717,881
root_title"NFT Research - Computation and Storage"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id106,227,670
net_rshares258,185,193,876,935
author_curate_reward""
vote details (248)
@bobinson ·
Interesting discussion.
👍  
properties (23)
authorbobinson
permlinkqzqesv
categoryhive-139531
json_metadata{"app":"hiveblog/0.1"}
created2021-09-20 12:02:57
last_update2021-09-20 12:02:57
depth1
children0
last_payout2021-09-27 12:02:57
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length23
author_reputation55,343,141,313,811
root_title"NFT Research - Computation and Storage"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id106,429,259
net_rshares0
author_curate_reward""
vote details (1)
@gangstalking ·
Electronic-terrorism, voice to skull and neuro monitoring on Hive and Steem. You can ignore this, but your going to wish you didnt soon. This is happening whether you believe it or not. https://ecency.com/fyrstikken/@fairandbalanced/i-am-the-only-motherfucker-on-the-internet-pointing-to-a-direct-source-for-voice-to-skull-electronic-terrorism
👎  , ,
properties (23)
authorgangstalking
permlinkre-disregardfiat-nft-research-computation-and-storage-20210912t180616958z
categoryhive-139531
json_metadata{"app":"hive-bot/0.6.3"}
created2021-09-12 18:06:18
last_update2021-09-12 18:06:18
depth1
children0
last_payout2021-09-19 18:06:18
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length343
author_reputation-67,597,107,868,724
root_title"NFT Research - Computation and Storage"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id106,227,674
net_rshares-44,131,852,385
author_curate_reward""
vote details (3)
@marki99 ·
$0.04
Reading through your posts, still at the beginning. 

> Every 5 minutes an entire state backup is performed and it's content is stored on IPFS which provides a hash of that data

How can you be sure that the state backup will always remain available on IPFS if it's not incentivized?
👍  
properties (23)
authormarki99
permlinkre-disregardfiat-qzl8m4
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2021.09.1"}
created2021-09-17 17:01:15
last_update2021-09-17 17:01:15
depth1
children4
last_payout2021-09-24 17:01:15
cashout_time1969-12-31 23:59:59
total_payout_value0.018 HBD
curator_payout_value0.018 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length283
author_reputation11,400,723,818,181
root_title"NFT Research - Computation and Storage"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id106,361,552
net_rshares44,895,681,136
author_curate_reward""
vote details (1)
@disregardfiat ·
It actually doesn't, it stay long enough that anybody who need to restart can, but IPFS is nearly built in memory management. 
👍  
properties (23)
authordisregardfiat
permlinkre-marki99-qzl8vm
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2021.09.1"}
created2021-09-17 17:07:03
last_update2021-09-17 17:07:03
depth2
children3
last_payout2021-09-24 17:07:03
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length126
author_reputation348,255,545,717,881
root_title"NFT Research - Computation and Storage"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id106,361,697
net_rshares0
author_curate_reward""
vote details (1)
@marki99 ·
>  it stay long enough that anybody who need to restart can

Any idea how long it's supposed to remain on there?
properties (22)
authormarki99
permlinkre-disregardfiat-qzl93q
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2021.09.1"}
created2021-09-17 17:11:51
last_update2021-09-17 17:11:51
depth3
children2
last_payout2021-09-24 17:11:51
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length112
author_reputation11,400,723,818,181
root_title"NFT Research - Computation and Storage"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id106,361,781
net_rshares0
@nathanielxp ·
NFTs are great technology. It's worth getting interested and investing some money, because you can earn good money. I recommend this post on this topic: https://nftmonk.com/nft-royalties-passive-income-stream-in-a-volatile-market/
properties (22)
authornathanielxp
permlinkre-disregardfiat-s2yxqw
categoryhive-139531
json_metadata{"tags":["hive-139531"],"app":"peakd/2023.10.1"}
created2023-10-23 06:23:21
last_update2023-10-23 06:23:21
depth1
children0
last_payout2023-10-30 06:23:21
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length230
author_reputation3,384,910,590
root_title"NFT Research - Computation and Storage"
beneficiaries[]
max_accepted_payout1,000,000.000 HBD
percent_hbd10,000
post_id128,225,998
net_rshares0
@poshtoken ·
https://twitter.com/Hivebull/status/1437351917773103108
<sub> The rewards earned on this comment will go directly to the person sharing the post on Twitter as long as they are registered with @poshtoken. Sign up at https://hiveposh.com.</sub>
👍  
properties (23)
authorposhtoken
permlinkre-disregardfiat-nft-research-computation-and-storage16700
categoryhive-139531
json_metadata"{"app":"Poshtoken 0.0.1"}"
created2021-09-13 09:47:15
last_update2021-09-13 09:47:15
depth1
children0
last_payout2021-09-20 09:47:15
cashout_time1969-12-31 23:59:59
total_payout_value0.000 HBD
curator_payout_value0.000 HBD
pending_payout_value0.000 HBD
promoted0.000 HBD
body_length243
author_reputation5,752,067,431,016,473
root_title"NFT Research - Computation and Storage"
beneficiaries
0.
accountreward.app
weight10,000
max_accepted_payout1,000,000.000 HBD
percent_hbd0
post_id106,246,318
net_rshares0
author_curate_reward""
vote details (1)