This post is a continuation(part 3) in a new “Understanding IPFS in Depth” series which will help anybody to understand the underlying concepts of IPFS. If you want an overview of what is IPFS and how it works, then you should check out the first part too 😊

Understanding IPFS in Depth(1/6): A Beginner to Advanced Guide

In this part, we will try to understand What IPFS is, Why do we need it and What we can do with it. We will cover all the underlying components of IPFS in brief(which will be explained in depth in further parts) and see how they work together.

SimpleAsWatervasa

In part 2, we discussed the Significance of IPLD(InterPlanetary Linked Data), How it Works and its technical specification. We also went through a tutorial in which we created a medium-like publishing system, solely using IPLD. You can check it out here:

Understanding IPFS in Depth(2/6): What is InterPlanetary Linked Data(IPLD)?

In this part, we will dive into the data model of the content-addressable web. We will explore the details and specs of IPLD and play with it to get more familiar with it.

SimpleAsWatervasa

In this part, we will dive into the Naming System of IPFS, InterPlanetary Naming System(IPNS). We will explore:

• What is the need for IPNS? How is it comparable to today’s DNS(Domain Name System) and how it differs from it?
• We will explore how routing works in IPFS and how IPNS works?
• And lately, we will play with IPNS. We will host and set up routing of my website totally using IPFS stack.

Hope you learn a lot about IPFS from this series. Let’s get started!

Why IPNS?

In order to understand why we need IPNS, let’s see how currently we access our photos, videos, and memes using IPFS.

BTW, if you want to follow along with me, you can download my website like this:

wget --mirror --convert-links --adjust-extension --page-requisites --no-parent https://vaibhavsaini.com

When I add my website to IPFS, I get the following output:

Now, I can access my website at

https://gateway.pinata.cloud/ipfs/QmYVd8qstdXtTd1quwv4nJen6XprykxQRLo67Jy7WyiLMB/

But this link has a few problems:

• Firstly, it’s hard to read, let alone remember.
• Secondly, it’s an immutable link. What I mean by an immutable link is that this link is permanent(due to the nature of content-addressing). If I were to add even a comma anywhere in my website, the CID of the root folder will change, thus changing the link to my website. So, every time I change anything on my website, I have to give the new link to everyone who wants to access my latest website…Not cool.

Here is where IPNS comes in.

By using IPNS you can generate a mutable link, which:

• will be human-readable and easy to remember.
• points to the latest version of your website, profile photo, video, etc.

A name in IPNS(the hash follows /ipns/ in a link) is the hash of a public key. It is associated with a record containing information about the hash it links to that is signed by the corresponding private key. New records can be signed and published at any time.

So, in other words, IPNS is a global namespace based on Public Key Infrastructure (or PKI) which allows us to build trust chains (so you can follow a public key to its route peer), giving us encryption and authentication, and is still actually compatible with other name services. So for example, we can even map things like DNS entries, onion, or bit addresses, etc. to IPNS addresses.

IPNS is not the only way to create mutable addresses on IPFS. You can also use DNSLink (which is currently much faster than IPNS and also uses more readable names. We will learn more about it below). Other community members are exploring ways to use blockchains to store common name records. Here is a great comparison of different projects working on a naming system for the distributed web.

IPNS and DNS share a few similarities. Both solve similar issues in their respective systems, former in a content-addressed system and latter in a location-addressed system.

In a location-addressed system(today’s old school internet), we use IP:PORT combination to access our data. So, in a location-addressed system, my website’s address will be:

104.210.43.77:443

which is not also neither readable nor easy to remember.

But this always points to the latest content hosted on this address.

Using DNS, we associate this IP with a domain name, so you can access the website at vaibhavsaini.com.

Ok, But How it Works?

IPNS can be implemented in many ways, but its current implementation uses Distributed Hash Table (DHT). As a consequence, only the most recent mapping of each URI to its corresponding hash is available for resolution, forgetting any historical mappings. This is not good from the archival perspective as the previous versions of a file might still exist in the IPFS store, but their corresponding URI mappings are lost.

Let’s use ipns node module to understand how an IPNS record is published.

const ipns = require('ipns');
const crypto = require('libp2p-crypto'); //for generating RSA keypair


function generateRsaKeypair(){
    //generating an 2048 bit RSA keypair
    crypto.keys.generateKeyPair('RSA', 2048, async(err, keypair) => {
        if(err){
            console.log('error ', err);
        }
        else{
            console.log("\nGenerated new RSA Keypair\n");
            createIpnsRecord(keypair);
        }
    });
}


/*
Creating an IPNS record with a lifetime
ipns.create(privateKey, value, sequenceNumber, lifetime, [callback])
privateKey (PrivKey RSA Instance): key to be used for cryptographic operations.
value (string): ipfs path of the object to be published.
sequenceNumber (Number): number representing the current version of the record.
lifetime (string): lifetime of the record (in milliseconds).
callback (function): operation result.
*/
function createIpnsRecord(keypair){
    let sequenceNumber = 0;
    let lifetime = 1000000; //1000000 milliseconds
    let value = 'QmYVd8qstdXtTd1quwv4nJen6XprykxQRLo67Jy7WyiLMB'; //hash to my website
    var recordData;
    ipns.create(keypair, value, sequenceNumber, lifetime, (err, entryData) => {
        if(!err){
            //Created new IPNS record
            console.log("\nGenerated new IPNS record\n");
            console.log(entryData);
            validateIpnsRecord(entryData, keypair);
        }
    });
}

/*
Creating an IPNS record with a fixed expiration datetime.
ipns.createWithExpiration(rsa, value, sequenceNumber, expiration, [callback])
privateKey (PrivKey RSA Instance): key to be used for cryptographic operations.
value (string): ipfs path of the object to be published.
sequenceNumber (Number): number representing the current version of the record.
expiration (Date): Date object.
callback (function): operation result.
*/
function createIpnsRecordWithExpiration(keypair){
    ipns.createWithExpiration(keypair, value, sequenceNumber, expiration, (err, entryData)=>{
        if(!err){
            validateIpnsRecord(entryData);
        }
    });
}


/*
Validate an IPNS record previously stored in a protocol buffer.
ipns.validate(publicKey, ipnsEntry, [callback])
publicKey (PubKey RSA Instance): key to be used for cryptographic operations.
ipnsEntry (Object): ipns entry record (obtained using the create function).
callback (function): operation result.
*/
function validateIpnsRecord(entryData, keypair){
    ipns.validate(keypair.public, entryData, (err)=>{
        //if no err then the validation was successful
        if(!err){
            console.log('\nIPNS Record Validation Successful\n');
        }
    }); 
}

generateRsaKeypair();

The above code is commented enough to be self-descriptive…You can also check out the full project here.

If you want to dive deep to know how routing works in IPFS, you can read this thread. I wanted to explain this in the post, but there are too many other interesting things to explore, so I skipped it ;)

Playing with IPNS

Let’s publish our website via IPNS.

ipfs name publish QmYVd8qstdXtTd1quwv4nJen6XprykxQRLo67Jy7WyiLMB

This can take up to a few minutes. You will get an output like this:

Published to Qmb1VVr5xjpXHCTcVm3KF3i88GLFXSetjcxL7PQJRviXSy: /ipfs/QmYVd8qstdXtTd1quwv4nJen6XprykxQRLo67Jy7WyiLMB

Now you can get the latest website here:

https://gateway.pinata.cloud/ipns/Qmb1VVr5xjpXHCTcVm3KF3i88GLFXSetjcxL7PQJRviXSy

Note: IPNS forgets(Time to Live System) published names after about 12 hours. You might want to run a cron job to republish within 12 hours.

If I want to add an updated CID, I will just use the same command:

ipfs name publish <my_new_CID>

You can also check the current CID linked to your peerID:

ipfs name resolve Qmb1VVr5xjpXHCTcVm3KF3i88GLFXSetjcxL7PQJRviXSy

This will return you the latest CID.

For added flexibility, you can also use different keys for different content and/or contexts(like below key name is vasa_blog). For instance, I could publish my website using one key, my blog using another, and my talk videos using yet another.

ipfs key gen --type=rsa --size=2048 vasa_blog
ipfs name publish --key=vasa_blog <cid_to_my_blog>

This solves one of the problems that we stated above(problem with immutable links). But the links are still ugly. To make the links we still need to use DNS. There are other systems which are better suited for content-addressed systems, like CCN/NDN, XIA. But these require upgrading the internet itself, which is really hard to warrant without massive demand. Even with large demand, IPv6 has yet to be fully deployed :( — which does not give me any hope of seeing NDN/CCN massively deployed in the core, without FIRST establishing the use of content-addressed networks. Meaning that end developers (web developers) must be able to use content-addressed networks to move lots of data (video, etc) extremely effectively well before substantial demand to improve the underlying network will materialize. So as we see it, by making IPFS usable to end developers we can create demand for these architectures as well.

Anyways, for now, let’s use DNS to create readable links.

DNSLink

DNSLink uses DNS TXT records to map a domain name(like vaibhavsaini.com) to an IPFS address. Because you can edit your DNS records, you can use them to always point to the latest version of an object in IPFS (remember that an IPFS object’s address changes if you modify the object). But we don’t want to change the TXT records every time we update our website. So we will add an ipns link rather than an ipfs link. Also, because DNSLink uses DNS records, the names it produces are also usually easy to type and read.

A DNSLink address looks like an IPNS address, but it uses a domain name in place of a hashed public key:

/ipns/vaibhavsaini.com

Just like normal IPFS addresses, they can include links to other files:

/ipns/vaibhavsaini.com/assets/images

When an IPFS client or node attempts to resolve that address, it looks for a TXT record for vaibhavsaini.com with content like:

dnslink=/ipfs/<CID for your content here>ORdnslink=/ipns/<hash of public key>

For example, if you lookup vaibhavsaini.com’s DNS records, you’ll see its DNSLink entry:

dig +noall +answer TXT vaibhavsaini.comvaibhavsaini.com. 1 IN TXT "dnslink=/ipns/Qmb1VVr5xjpXHCTcVm3KF3i88GLFXSetjcxL7PQJRviXSy"

Based on that, this address:

/ipns/vaibhavsaini.com/assets/images

Will get you this block:

/ipns/Qmb1VVr5xjpXHCTcVm3KF3i88GLFXSetjcxL7PQJRviXSy/assets/images

Super Cool!

Till now we reduced our address from a complex hash to a readable name.

But, we can do better.

This(the above link) is still pretty messy, and frankly, if we want the average Web2 user of today to access my decentralized Web3 content with minimal effort, we don’t want them to have to deal with gateways and ipns/ipfs prefixes if they don’t have to. A major feeling of the decentralized web community is that the user experience shouldn’t change all that much — the transition should be transparent but easy — that’s how the decentralized web will win. Ideally, we’d like to get to something like this: https://profile.vaibhavsaini.com…

Publishing via a Subdomain

So in order to make our address more readable, we can create an A record pointing our sub-domain to the IP address of an IPFS peer listening on port 80 for HTTP requests (such as any of the public IPFS gateways, or your own if you want).

But wait, we can do even better than this!

Because we don’t want to rely on IP addresses being static, we can use a CNAME record to point at the DNS records of the gateway. That way, if the IP address changes, we’ll still point to the right place. Unfortunately, CNAMErecords don’t allow for other records (like TXT), but the fine folks at IPFS allow us to create a DNS TXT record for _dnslink.your.domain, which IPFS will look for.

This is also useful when you want to improve the security of an automated setup or delegate control over your DNSLink records to a third-party without giving away full control over the original DNS zone.

I am using AWS Route53 for my DNS settings; you can use any provider.

Setting CNAME record

Setting _dnslink TXT record:

Here is how it finally looks:

And Voila! We have our content hosted and resolved using IPFS stack, with an address that can be used by any Web2 user with ease.

You may notice the “Not Secure” warning on the address bar, which is due to the fact that I haven't installed a wildcard certificate ;)

You may notice that the websites take some time to resolve. This is due to the fact that the content for your website is on just one node. If you pin your website on several nodes or other nodes try to access your website(which means your content is popular) it will resolve faster :)

That’s it for this part. In the next part, we will explore Multiformats. You can check it out here.

Understanding IPFS in Depth(4/6): What is MultiFormats?

In this part, we will talk about Why we need Multiformats, How it works and What you as a user/developer can do with it?

SimpleAsWatervasa