C as a priority for Linux systems

Ecosystem
,
1

Good evening.

I am of the opinion that IPFS would tremendously be a popular technology for package managers.

I think that a basic IPFS implementation in C which is required for this task of package mangaement would insentivise people from ArchLinux, Debian et al. to make use of IPFS as a standard for retrieving of packages.

Related threads:

2

It is. Itā€™s been tried on Arch by @RubenKelevra See the github repo.

I donā€™t know why itā€™s currently disfunct but i do know that there are just always pesky issues in IPFS popping up in large repos that - sooner or later - just becomes a headache. When that persists people have a tendency to give up.

The programming language is of no concern. It needs to be stable, and well maintained. In fact, iā€™d even argue that Rust would be the language of choice. C (and C++) has many many many guns to shoot yourself in the foot with. And developers often do. Rust is probably more appropriate these days.

3

I find this to be the canary in the coal mine of projects. When a project says it does X and you say to yourself, ā€œWell, that sounds perfect for Yā€ but Y never gets done itā€™s a sign of a problem and a bigger problem when the reasons stopping Y never get addressed.

I find there are way too many Yā€™s with IPFS. Iroh was a good attempt to fix that but Iā€™m afraid it missed the mark.

4

I completely agree!

I too find myself in the position where i want to use IPFS tech but when actually using it i just keep hitting small (sometimes big too) issues that over time just ruin it. Itā€™s a sad realization but it is one that i keep coming back on.

Perhaps IPFS is like google wave. Amazing conceptually, awesome demos, great new foundational work and protocols but just not really usable for the masses. Then years later you get the online office suites and collaborative editing (like what google wave did too) now is a big success! Google wave was ahead of itā€™s time but collaborative editing has a lot of origin in that. Perhaps IPFS is about to fall into that category too? It makes me very curious what the future tech is gonna bethat does the IPFS concepts but just in a better way? :thinking:

5

Well, there was a persistent bug in IPFS regarding performance of large MFS operations, which blocked it for about 3 years, but was recently fixed.

So it will return shortly :slight_smile:

6

3 yearsā€¦ :sob:

That does kinda show the, lets be polite, priority in other places :wink:

Iā€™m glad the IPFS Arch repo is coming back! Till you hit the next issue :stuck_out_tongue:

7

Yeah, three years. To be fair I could probably have worked around it somehow. But I also havenā€™t had that much time to invest in trying stuff out.

I could have also run an older version and, well, as far as I can tell I was one of the few people affected. :slight_smile:

Anyway, itā€™s fixed now, thatā€™s all that matters for me.

8

Yeah, three years. To be fair I could probably have worked around it
somehow. But I also havenā€™t had that much time to invest in trying stuff
out.

My workaround was to write code to build my own unixfs DAG block in one
fell swoop, manually. Itā€™s not as hard as you might think. For a repo
mirror, youā€™re going to have a huge honking directory block. You canā€™t
share that between IPFS nodes, because of block size limits. But you
can force IPFS to shard it for you. Hereā€™s the relevant comment from
one of the scripts I used to maintain my Void mirror a few years back:

;; This is a hideous hack.
;; Large blocks cannot be shared between IPFS nodes. Thereā€™s a 2 MB limit.
;; This is one reason for sharding big directories.
;; However, adding new links to a directory in MFS is terribly slow
;; and inefficient.
;; While large blocks cannot be shared between nodes,
;; they can be added to the local blockstore. What we can do is create
;; a huge directory block, put it to the local blockstore, and then force
;; IPFS to shard it for us. IPFS doesnā€™t give direct control over
;; the sharding of a directory block, so we use a trick. Write a dummy
;; temporary file to the directory. Then remove it. This causes IPFS
;; to shard the block for us. And instead of having the penalty of
;; adding lots of files to an MFS directory, we just have the penalty of
;; adding and removing one file.

So basically, build one huge unixfs directory block. Put that to the
API; youā€™ll need to have the option to ignore large blocks unset.
Then do the equivalent of:
ipfs files cp /ipfs/cid-of-gynormous-directory-block /mydir
ipfs files write /mydir/tempfile.junk ā€˜ā€™
ipfs files rm /mydir/tempfile.junk

And this was extremely efficient compared to the whole ā€œadd files
one-by-one to MFSā€ thing that was horribly slow at the time.

ā€“ Chris

9

markg85 via IPFS Forums notifications@ipfs1.discoursemail.com writes:

  • markg85
    December 27

genghis:

I am of the opinion that IPFS would tremendously be a popular technology
for package managers.

It is. Itā€™s been tried on Arch by @RubenKelevra See the github repo.

I donā€™t know why itā€™s currently disfunct but i do know that there are just
always pesky issues in IPFS popping up in large repos that - sooner or later

  • just becomes a headache. When that persists people have a tendency to give
    up.

And used on Void by me. I even worked around the MFS bug. A side
effect of my experimentation is that my house had a Void mirror that ran
at LAN speed. There are three of us here running Void, and we all miss
that mirror.

Reason I took it down is that when I posted about it on Reddit in
r/voidlinux, someone took down my post as spam, without so much as a
by-your-leave or a message to me about what was wrong with my post. I
complained, and one of the other mods was like, ā€œThis looks fine but it
was taken down as spam. Restored.ā€ And then the person who originally
removed my post did it again with a zero-content message to me saying
that it was spam.
So the reason I took down my Void IPFS mirror was 100% political
and 0% technical, and sadly, I donā€™t even know what the politics are or
whose fee-fees were hurt or whatever. Because thatā€™s how reddit works.
I was just being a nerd doing some experimenting. A nerd who spent 6
months to a year building something that nobody wanted.

I think that a basic IPFS implementation in C which is required for this
task of package mangaement would insentivise people from ArchLinux,
Debian et al. to make use of IPFS as a standard for retrieving of
packages.

The programming language is of no concern. It needs to be stable, and well
maintained. In fact, iā€™d even argue that Rust would be the language of
choice. C (and C++) has many many many guns to shoot yourself in the foot
with. And developers often do. Rust is probably more appropriate these days.

All package managers support fetching over http / https. IPFS nodes
support providing over http / https. Most people arenā€™t gonna want to
run an IPFS mirror for their local LAN, but for the people who do that
(like I did), you have a dedicated machine running an IPFS node and you
point the package managers at that, and the implementation language of the IPFS node
matters not one iota. IPFS in brainfuck would be fine from the
package-manager ā†” IPFS node point of view, as long as that
hypothetical Brainfuck implementation can speak http / https to the
clients (package managers).

ā€“ Chris