[solved] How to use the IPFS self public and private key in node.js?

Ecosystem
1

(posted here as per requested in https://github.com/ipfs/go-ipfs/issues/7715)
Hi,

I’m using IPFS 0.7.0 with the shiny new feature to export keys.

So i do:

ipfs key gen test
ipfs key export test

Which gives me a “test.key” in binary libp2p protobuf format.

What i try to do now is get that key working in nodejs crypto.
This is where i’m hitting some snags.

In node.js i need to import the key as a PEM key to be able to import and use it. This means it needs to be in this exact format:

-----BEGIN PRIVATE KEY-----
<THE KEY>
-----END PRIVATE KEY-----

Begin and end are easy :slight_smile:
The key needs to be in base64.

Now to load this into javascript i did the following:

const fs = require("fs")
const crypto = require('crypto');
const Schema = require("./crypto_pb")

let data = fs.readFileSync("./test.key")
let deserializedData = Schema.PrivateKey.deserializeBinary(data)
let keyBuffer = deserializedData.getData();
console.log(Buffer.from(keyBuffer).toString('base64'))

Note that very last console.log line. There i’m printing the key, which results in:

2qLSIF08WHkcz5f+ca90C+BRe7I8vjNlDarPREXCmkcNUzmzNro7dN0obChI5pQF3aK/owNVgEhw1OMd4dDUiA==

That’s too long.
The keyBuffer object is (i verified that) 64 bytes long. So it does seem to match what the libp2p spec tells.
The base64 result is however 88 characters which doesn’t seem to be right. Importing it as that gives me an error of: header too long.

I’m sure i’m missing a conversion somewhere. But where and from what to what is not clear to me.

Thank a lot!

Cheers,
Mark

Edit 12-10-2020
I changed the title from How to convert an exported key to base64 in nodejs? to [solved] How to use the IPFS self public and private key in node.js? because i essentially want to use the IPFS self key in node.js to add functionality. I want to sign a hash with the node’s private key which anyone else can then verify if they have the node’s peer id (which is also it’s public key as of ipfs 0.7.0).

2

@jacobheun Thank you for your response on that github issue!

You mentioned i should be looking into: https://github.com/libp2p/js-libp2p-crypto#cryptokeysimportencryptedkey-password

2 questions here.

  1. How was i even supposed to know that i need that? Literally nothing in the IPFS documentation gives me any clue about that. ipfs key export --help at the most generous gives me: Exports a named libp2p key to disk. makes me think that the libp2p key is dumped to a readable format on disk, which is totally not the case.
  2. The link you mentioned for crypto.keys.import wants a password as second argument. What is that supposed to be? I never even had the option in IPFS to create a private key with a password. So how do i import it then?

Just to remind you here.
I want to use the IPFS private key to generate a signature of the hash of content it had added.
I then want to let another party receive both that signature and my peer id to verify that i had actually added the content to IPFS.

I do not want to add a new key. The whole point is re-using the existing key to create that signature on one end and verify it on the other. Something that, in my opinion, really should be put into IPFS natively. As not having it natively requires me to either go through these hoops to do it safely or to do my own key management.

3

@markg85 let’s take a step back, can you explain why you’re exporting the key from Go into JS? I’m not clear on what you’re trying to accomplish with this. I understand you want to sign and verify content with your ID, but I’m not clear on why you’re exporting/importing the key for this.

Signature and verification support is possible with your existing ID, and it’s used for things like pubsub message verification and signed peer records. How are you currently trying to generate the signatures?

4

Right, let’s take a step back indeed :slight_smile:

I add content to ipfs, say like: ipfs add somefile
That gives me a Qm… hash

I merely need to have a way for anyone to confirm that I added something to IPFS if that user knows my peer id and the Qm… hash that needs confirmation. I’ll not go into details about why i need this as then this discussion is likely going to derail to discuss that.

What you said with “Signature and verification support is possible with your existing ID” is what i keep reading in places (the release notes of 0.7.0 among them) but nobody anywhere explains how to get that done (and i definitely asked multiple times and in multiple places!). How?

5

I just saw the other post you had at How to verify that <cid> is signed by <peerid>? talking about this, so let me try to clarify.

You cannot currently do signing/verification via the CLI, you need to use IPFS programmatically to achieve this. This will give you access to the sign/verify methods on they key https://github.com/libp2p/go-libp2p-core/blob/v0.7.0/crypto/ed25519.go. The release notes are specifically referring to the ability to programmatically unmarshal the public key from the ed25519 ID, which is not possible with RSA keys, for signature verification.

6

So that means i cannot sign any data with any exported key from ipfs, not even with the (js-)libp2p library, correct?

Is there a plan to add this to IPFS CLI?
Usually this request is immediately shot down with arguments like “but there are dedicated libraries for this, use those”… which is impossible, as I’ve demonstrated.

Having said that, it is possible with other libraries but only if you do all key stuff in there. Not if you want to take the private key from IPFS and do something with it.

7

So that means i cannot sign any data with any exported key from ipfs, not even with the (js-)libp2p library, correct?

You can, you just have to unmarshal the bytes of the exported key. Both Go and JS this would involve reading the file in and then using the unmarshal API for that language, (Go) https://github.com/libp2p/go-libp2p-core/blob/v0.7.0/crypto/key.go#L338, and then you could sign with the returned private key.

Is there a plan to add this to IPFS CLI?

There’s nothing on the immediate roadmap for this (there are also no closed/open issues requesting this). We’re looking at adding better support for using go-ipfs as an API, but in the interim you could look at creating a daemon plugin, https://github.com/ipfs/go-ipfs/blob/master/docs/plugins.md#daemon, to add support for generating signatures for a given CID.

8

Awesome, this brought me further :slight_smile:
This code can sign and verify!

const fs = require("fs");
const crypto = require('libp2p-crypto');

function str2ab(text) {
    return new TextEncoder().encode(text);
}

function ab2str(buf) {
    return new TextDecoder().decode(buf);
}

(async function() {
    try {
        let data = fs.readFileSync("./test.key")
        let key = await crypto.keys.unmarshalPrivateKey(data);
        let sign = await key.sign(str2ab("QmReEZnKLvXvfric19kv6crts7ijK3xPicNEMkHFqBWfEq"));
        let hexSign = Buffer.from(sign).toString("hex");
        
        // At this point you have a readable (hex format) sign buffer.
        console.log("Sign: " + hexSign)

        // This public key should be created from the data you received
        let publicKey = key.public;

        // Verify the hash is signed by the private key
        let verify = await publicKey.verify(str2ab("QmReEZnKLvXvfric19kv6crts7ijK3xPicNEMkHFqBWfEq"), Buffer.from(hexSign, "hex"))

        // Should print true if verified
        console.log(verify)
    } catch (error) {
        console.log(error)
    }
}());

Now i’m stuck with the next issue… It never stops :wink: (well, nearly there)
I want to give “the other party”:

  • the signature
  • my peer id
  • the Qm… it needs to verify with that sign

The way (js-)libp2p seems to be working is by accepting the protobuf key object to unmarshal it. That is fine for the signing side, as there you need to have the private key. It should not be shared so having that in a protobuf as a binary is OK.

However, i would like to send the receiving side a string version of my peer id (which should be a public key). But that won’t work because the peer id is in a different format (not in the protobuf format).

How would i tackle that one?

Also, another issue is that you apparently can’t export the self key. Is there a way to get that key in the protobuf format? In this case it is an option to add a new key, as long as it’s all in ipfs

I’m curious what your thoughts would be for this one and how you’d solve it?

Edit
Got it!

        const multihashes = require('multihashes')
        let peerId = "<your peer id>"
        let multiHashPeerID = multihashes.decode(multihashes.fromB58String(peerId))
        let peerPubKey = await crypto.keys.unmarshalPublicKey(multiHashPeerID.digest)

That gives a proper public key i think :slight_smile:
Took a bit of “reverse logic” fiddling from the go code that extracts a the public key: https://github.com/libp2p/go-libp2p-core/blob/a39b84ea2e340466d57fdb342c7d62f12957d972/peer/peer.go#L92

Last thing i now need to know is how to get the self private key from IPFS?

Edit 2
For whoever ends up here with google. The IPFS private key (the self one that can’t be exported) is in the ipfs config file under the name PrivKey. It’s base64 encoded.

To be complete, loading your private key in (js-)libp2p is done like so:

        let data = Buffer.from("<your private key>", "base64")
        let key = await crypto.keys.unmarshalPrivateKey(data);
        // ... the same as my other example in this post

Got it all working now :smiley:
This took quite a while!
Your help, @jacobheun, was really valuable here! Thank you very much!

1 Like
9

Every application in the world that imports/exports keys uses some kind of industry standard text format like PEM, for example. Am I correct that IPFS cannot export keys to a standard format (based on above discussion)? Protobuf is NOT a standard format. 0.001% of developers have even heard of ProtoBuf and 100% of other apps that import/export keys don’t accept it.

So now after I export a key with (ipfs key export command) I have to run it thru some additional translation utility to convert it from protobuf to a standard format? This can’t possibly be true. I must be missing something.

10

I agree with you @wclayf. It seems to be needlessly (?) complicated.
It’s doable, as i found out, but definitely not trivial.

A PEM or JWT like format somewhere would have been a lot more convenient with importers available for either format in lots of languages.

IPFS does have a tendency in making new innovative stuff that works a bit differently then the conventional ways. Sometimes that’s great! In the case of the keys… They might have overengineered that a bit.

I am curious to know if there is an easier way!

11

@markg85 I think the reason it ended up that way (ProtoBuf format) was likely because the developer who wrote the feature had instructions as specific as “write an import/export feature”, so that’s precisely what he did, however he saw fit.

12

It’s not nearly as well known as something like JSON but I wouldn’t say that ProtoBuf is quite that exotic. If I were to guess, the people that did that didn’t see anyone using the peer keys for anything other than what they’re used for. I’m not sure it’s a great idea to extract them and use them for signing random stuff anyway but I don’t know maybe there is.

13

@zacharywhitley For applications like Social Media where your PrivateKey is your IDENTITY, it’s important for that key to be readable by other apps, imo. Not some proprietary format that no other app can read.

14

I’m just not sure if that public/private key pair should be the same one your use for your peer.

15

The usecase of @wclayf and me might differ a bit here.

I wanted a way to let someone else verify with near certainty that the file i added was in fact added by me. So in that case it made much sense to use what was already there and avoid adding extra bookkeeping.

In the case of a social media kind of app… I’m not so sure if using the IPFS node private key is the best way to go. Specifically because the node that you’re using might not be “your” node. It could be a network node for your whole lan, it could be an in-browser node (like brave has), etc… It might be better to go the wallet route in that usecase. That does probably make it much more complicated as now you have to integrate with a wallet and probably some service tying IPFS and that wallet together (ceramic perhaps?).

16

But even in your case it would only verify the node on which the content entered the IPFS network. I get it. I’ve gone back and forth with just using a node that only you control as a sort of identity. One of the problems is I shouldn’t have to access that one node just to be able to publish something as me. Say I setup a node on my laptop and then I either turn it off, runs out of batteries, leave it at work, whatever. Now I can’t publish anything? I think I might have asked before what happens if you have to nodes on the public IPFS with the same peerId. I don’t think anything bad happens but some of the traffic might randomly go to one or the other. Probably a situation you want to avoid.

17

I just got done creating and testing this script. It creates a key and then uses it to publish an “Identity”. I’m trying to now figure out how to “publish to the world” my PublicKey and have everyone be able to verify that the IPNS data is indeed ‘signed’ (or verified by IPFS publishing) that the PublicKey I claim is mine is the real one that published the IPNS…Sounds similar to your needs @markg85.

EDIT (to clarify):
What I mean is I will be sharing my PublicKey, and I want people to just be able to check if the public key is provably the one that signed the IPNS. Not sure how yet, but the goal is that for Social Media purposes the IPNS is my “index to all content, and all identity”

It happened to all fit in an image so i’m attaching…

image
image1436×1020 146 KB