Reading streams in Javascript

Hi - posting here as its probably a question, though the “issue” would be lack of documentation :slight_smile:

Has anyone tried using the IPFS file streams in Javascript? I’m trying to get something like a Node stream where I can go …
createReadStream(path, opts) => stream
And then can specifiy opts to read a byte range.

Unfortunately in https://github.com/ipfs/interface-ipfs-core/blob/master/SPEC/FILES.md#createaddstream
in files.addReadableStream(options), the options is undocumented and appears to be for working on a single machine, not across IPFS, while
files.cat(path, callback) doesn’t take options.

Am I correct in assuming that I can’t seek on a IPFS stream, meaning that to play the middle of a video will require pulling the entire video, storing in a local file (outside of IPFS) and then creating a local OS seekable from that?

That’s ok - but will have super long delays on big files, so I want to make sure I’m not missreading things.

I am also confused about how streams with IPFS function. To add on to Mitra’s question, would it be possible to share chunks of a file as you are downloading/streaming them with other peers?

I want to post a correction - there is a function to read a stream, there was a documentation issue (now fixed) that meant catReadableStream didn’t show up on https://github.com/ipfs/js-ipfs. I haven’t had time to test it out yet, but it looks like the right way to read a stream in JS.

I’ve just been looking at catReadableStream, looks good for some uses (displaying linearly data as it comes on) but I don’t think I can use it in a media object.

Maybe I’ve got this wrong, but looks to me like … catReadableStream returns a node ReadableStream https://nodejs.org/dist/v8.1.3/docs/api/stream.html#stream_readable_streams which is great, BUT the way this seems to be most often used to play media (e.g. through a Video tag) is through an API that calls back repeatedly to a createReadableStream(path , opts) function with opts[“start”] set to the place the user is viewing from. There is no “seek()” function on node’s ReadableStreams.

The options are defined in https://nodejs.org/api/fs.html#fs_fs_createreadstream_path_options which is somewhat obscure (no link to there from the ReadableStream docs). The important one for media playing being “start”

The opts are crucial as it includes start which is used to seek within a stream. I’m presuming the underlying IPFS could selectively return a byte range but it doesn’t appear to be exposed here.

Anyone know if I’ve got this right, or if there is some other way to access a stream starting at a particular point. (Presuming of course that IPFS will still download most/all of the blocks in the file).

Just stumbled on this link on implementing an io.Seeker in go-ipfs.

Don’t know if this would work for streams and/or your use case.

Probably not … but thanks. There is some code to do this at https://github.com/ipfs/js-ipfs/pull/1231/files which I’m adapting to work with VIDEO tags, currently waiting on a fix to https://github.com/ipfs/js-ipfs/issues/1258 and when that works I think the stream code should too.

In case anyone still finds this … catReadableStream works fine now in JS-IPFS, and allows for passing a start & length option .

1 Like