Dynamic website with user functionality

Help
1

Greetings! Iā€™ve been following IPFS for over an year now. Iā€™m very captivated by its idea, which ties into an old fascination with the possibility of turning the internet into a worldwide mesh network free of ISP control. Thank you for your wonderful work, and I eagerly await the day that IPFS is finally a mainstream protocol used everywhere :smiley:

Onto my question: Iā€™ve been pondering if and how a website fully hosted within IPFS can be coded to offer user support, allowing people to register and create profiles on which they can make changes and upload content. I know some have argued this is a bad idea, because IPFS is mostly good with static content onlyā€¦ I do however believe that dynamic websites are equally important and IPFS wonā€™t get far without supporting them: What can you do with a site where there are no users capable of posting anything?

Although Iā€™m still new to a lot of things, I can see why this is a tricky question with no clear answer: First of all, all files are subject to immutability, meaning that once created they cannot be edited and must be published as new files with new hashes (URLā€™s). Further more, IPFS sites arenā€™t provided by a web server that can work with specific libraries to modify them in realtime, such as a PHP script reading and writing from / to a MySql databaseā€¦ files are offered as they are, only JavaScript running on browsers can do any kind of processing as far as Iā€™m aware.

Considering those harsh limitations, which as far as Iā€™m aware have no solution, Iā€™ve been trying to think of a workaround. The closest thing that comes to mind so far is this:

The user accesses a HTML page on which a button gives them the option to register. Upon clicking this button, a JavaScript function generates two keys: A public and a private key (like SSH). The user is given both keys, and informed that the public one represents their username while the private one acts like a password. At the same time, the JS function generates a JSon database which contains all the settings of this user: Whenever someone views their profile, this JSon file is fetched and the data transposed by their browser into the HTML file. Whenever changes are made to the profile, this JSon file is replaced with a new one containing new entriesā€¦ Iā€™m unsure how the site itself can be modified to point to the new file as the userā€™s present configuration, but I assume thatā€™s where IPNS comes in. Lastly, the public and private keys are used to check who the user is whenever they make changes to their profile, similarly to checking that they are logged in: Only if the private key provided matches the public key of the channel do modifications get approved, meaning a new user JSon is generated and the site is modified to point to it.

I assume this is a bit like a person jumping off a roof with cardboard wings and actually hoping they will fly; Iā€™ve never heard of a user management system that works this way, and it feels like a very questionable methodā€¦ still it feels borderline possible! I would like to know what you think of this ideaā€¦ and if itā€™s indeed a mistake, what alternative ways are there to allow secure user functionality on a dynamic site hosted exclusively in the IPFS cloud? Thank you.

5 Likes
2

The idea of having the clientside javascript generate keys for new users seems like a workable pattern.
This is similar to how peergos does their authentication: https://peergos.org/ (sorry, i donā€™t have a link directly to how they do that)

If you can restrict your processing entirely to clientside javascript, your app becomes very portable. But you can also write and distribute other apps that interact with the api of a locally running ipfs daemon.

One powerful tool you have available here is pubsub (which will be authenticated in the future, allowing only select peers to participate). This is how peerpad works. You can use pubsub to communicate between users of a given app. Peerpad uses CRDTs to essentially form a fully distributed database between all of its peers.

Also, IPNS is going to get an upgrade sometime (hopefully) soon in the form of IPRS (some WIP stuff is being discussed here ). With IPRS, you would be able to have an IPNS entry that can be published (for example) by anyone whose key is signed by N other users, or N administrators.

Sorry for rambling, I hope that helps.

5 Likes
3

Itā€™s okay, that is very helpful info! Thank you for mentioning this.

I heard about pubsub briefly, the others are new to me. This seems a little more complicated but definitely like the right way to go about this. A proper network wide database is definitely the ideal solution, however itā€™s hard to imagine just how that would work in a torrent cloud which only distributes static files: Whenever you change a file you must upload a new one with a different hash, if you do that you need to edit the site to reference this new file, but if you do that the file of the site itself changes! Is this an external service to IPFS files, which allows data to be modified while being possible to point at using the same address?

4

There is IPNS which is a address based on a key (by default your ā€œselfā€ key, which is the only key that you NEED to have).

# Generate a new key for "my-website"
$ ipfs key gen my-website --type=rsa --size=1024
QmVDE9sVonk5dqn4ox7CqH8xqhkD8ZEvbiRXcdqRomTJ9b

# Add our file
$ ipfs add package.json
added QmReKwDeEo1uLi7sYD2d4aXJeWHmcw9Xwh4GuYLXv1uZMR package.json

# Publish our file under our "my-website" key
$ ipfs name publish --key=my-website QmReKwDeEo1uLi7sYD2d4aXJeWHmcw9Xwh4GuYLXv1uZMR
Published to QmVDE9sVonk5dqn4ox7CqH8xqhkD8ZEvbiRXcdqRomTJ9b: /ipfs/QmReKwDeEo1uLi7sYD2d4aXJeWHmcw9Xwh4GuYLXv1uZMR

# We can now reach our file under "/ipns/QmVDE9sVonk5dqn4ox7CqH8xqhkD8ZEvbiRXcdqRomTJ9b" or under "/ipfs/QmReKwDeEo1uLi7sYD2d4aXJeWHmcw9Xwh4GuYLXv1uZMR"

# Add our updated and modified file
$ ipfs add modified-package.json
added QmY3dYbTM1frLUr9nD6ma4nC11MMPbko7ezf1gxX3zu63k modified-package.json

# Publish it again, under the same key
$ ipfs name publish --key=my-website QmY3dYbTM1frLUr9nD6ma4nC11MMPbko7ezf1gxX3zu63k
Published to QmVDE9sVonk5dqn4ox7CqH8xqhkD8ZEvbiRXcdqRomTJ9b: /ipfs/QmY3dYbTM1frLUr9nD6ma4nC11MMPbko7ezf1gxX3zu63k

# Now we can still reach our old file using the IPFS address, or we can use the IPNS address which now has been updated to point to  QmVDE9sVonk5dqn4ox7CqH8xqhkD8ZEvbiRXcdqRomTJ9b

If you skip the --key argument, IPNS will use the default key.

5

Sounds good, but there is one problem with IPNS here (two actually).

First of all: Is the IPNS only meant to be used as a redirect for the main site? The issue then is that if you have a lot of users, you need to regenerate the IPNS to point to the new site settings file each time. Imagine roughly 100 users modifying the site each secondā€¦ can it even fare and keep track of every change in order, would the performance be decimated?

Then there is another issue: Can browser side JavaScript regenerate the IPNS on its own? Obviously the process has to be done automatically in the cloud! I assume that thanks to file immutability, it should be safe at least, since the rules on how to modify would themselves be in a static file designed by the site administratorā€¦ might need to ponder how site updates would be handled as well then.

6

Just as a tangent, you may be interested in the web cryptography API https://w3c.github.io/webcrypto/Overview.html .

And regarding another tangent, for dynamic sites, one has to take into account ā€œoriginā€: https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy .

If the code will be running on localhost, this opens up the possibility for clashes, e.g., with indexedDB database names, localStorage keys, cookies, and postMessage. Actually this could theoretically be seen as a feature instead of a bug, given its potential to restore to the web the kind of application-agnosticism we have enjoyed on our desktops (no application is any more privileged than others as to viewing or editing the data created by another application), but given that none of the above actually have APIs requiring the user be asked for permission, malicious sites opened under localhost could simply freely overwrite or manipulate databases, etc. also saved on localhost, so at the moment it would not be much of a feature in IPFS. (The only browser attempt I know of to solve the problem of application-agnosticism, globalStorage was nixed in Firefox a long time ago, perhaps because they did not require user permission for that either.)

(I created a Firefox add-on at one point, AsYouWish, which allowed escalation of user privileges upon granularly informed user consent, and it had a means of shared user storage, but the add-on no longer works in modern Firefox (or Chrome) as I havenā€™t updated it to work with WebExtensions, but something like that could be a solution, though it would probably need to be implemented with NativeMessaging, something Iā€™ve been meaning to do with another local-friendly add-on, WebAppFind, which allowed for opening of local files on the desktop into web applications through double-click or ā€œOpen withā€¦ā€ rather than drag-and-drop, but I havenā€™t gotten around to updating it either)

FWIW, some browsers like Chrome restrict the file:// protocol from accessing scripts outside of its directory, but I havenā€™t seen any restrictions on localhost, so one canā€™t rely on paths (e.g., hosting in another non-child directory) for security here either.

2 Likes
7

Besides a browser add-on, one could also get shared storage requiring user consent by hosting a file on trusted domains (ipfs.io?) which explicitly requested permissions of the user before use and which used a service worker to continue working offline after the initial download. Even some browsers at present allow hashes within script tags to insist on a particular version of a file being used so this could be trusted if used as such: https://developer.mozilla.org/en-US/docs/Web/Security/Subresource_Integrity .

One problem here though (besides these scripts not being themselves within IPFS) is that each domain will be given storage limits by the browser, so one could have a script which managed the storage amounts and directed users to other domains when the amounts became unwieldy, though that in turn requires being connected (and a good number of domains mirroring such (identical) code). (This is, from my naive point of view at least, starting to sound a bit like a blockchain itself!)

8

Good to know. The issue here though is that, like you said, it has to do with content origin policies. On IPFS everything is a file that gets torrented by the browser from multiple origins, itā€™s entirely agnostic to source! Thatā€™s precisely the point of why I want to use only IPFS, itā€™s full decentralization where origin doesnā€™t matter in any way. However if you do that, how do you remain sure that you maintain full account security away from hackers?

Anyway Iā€™ve been thinking more about my idea, and remixed it a little in my head: If Iā€™ll ever attempt such an application, I donā€™t think Iā€™m going to make it a real website per say, but instead each user profile will be a site of its own which uses a list of other profiles (ran with the same site code) for features such as watches or search. This would be even more flexible and decentralized, and should be a very interesting concept to see in action :grinning:

9

Okayā€¦ here is an example of what my idea would imply. This is separate from storing user settings, but the answers should apply to that question in a similar fashion.

To make a profile, people would download the site code and use ā€œipfs addā€ and other commands to upload a copy with their username in the IPFS cloud, thus initializing their site-profile. When that happens, the site adds its IPNS address to a cloud list (potentially customizable), which is a simple json array containing the URLā€™s of all profiles created with the site. Oppositely, if a profile removes itself from a cloud list, the entry must be removed from this array too. This list exists so that users have search and notification functions which can access the profiles and posts of other users.

The problem here is this: To do such a thing, there needs to be a file or database in the worldwide IPFS cloud, which can be flexibly edited by JavaScript commands issued by browsers (not manually by people).

  • The file must be possible to reference with the same URL whenever itā€™s edited (IPNS should fix this).
  • Next the file must withstand a tremendous amount of editing, in case a lot of people will add or remove accounts from a cloud list. Depending on how used it would get, even 1000 edits per second.
  • Lastly the changes a site profile can make must always be just removing or adding the entry of the site from the array, nothing else. Such would allow trolls to delete it, or inflate it until it becomes too huge to download! Since JavaScript would do all the processing, trolls may unfortunately be able to do what they want by creating a modified version of the site with the JS functions modified.
10

Looks like https://github.com/ipfs/in-web-browsers/issues/66 may be aiming to provide a solution for the same origin problemā€¦

11

https://blockstack.org is one way to achieve this, as Blockstack supports IPFS as a storage system.

12

I should ask a more specific question, based on my train of thought with what I know at the moment:

Is it possible to have a JavaScript automatically update an IPNS address to point to a new file, after the same script has uploaded a new version of a settings file? This would of course have to be done with a security check built into that IPNS domain: Either username + password, either public-key + private-keyā€¦ so you associate the IPNS with a name or key, and any browser script can then edit this IPNS to point elsewhere but if the given password or key is correct. The point is Iā€™m not sure if JS can on its own edit an IPNS.

Something tells me itā€™s more complicated than that. But Iā€™d rather not have to use external services, which seems to be the main alternative hereā€¦ if I can Iā€™d make such a site work fully within IPFS.

13

Hello @MirceaKitsune! Men, I canā€™t belie I came across this post, for about two days now, I am wondering about the exact same thing! Its freaky how my approach resembles yours: independent app/page for editing content, the use of JSON as a data store, private/public keys for authenticatoin, central page for indexing individual userā€™s pages, etc., for achieving a truly decentralised platform. I already went a bit ahead, sketching out a a renderer function that would update the users ā€œportfolio pageā€, upon changes made to the JSON file. Btw, how is your progress?

14

Greetings. Iā€™m glad that you like my idea, and also that other people are sharing it! So far thereā€™s no progress and only planning: I havenā€™t even touched IPFS as Iā€™m waiting for my Linux distribution to add the IPFS daemon as an official package.

My plan however is to create a fully decentralized and non-moderated alternative to Twitter / Facebook / Youtube / Deviantart / etc: No central authority taking down any content, use weighted tags to both follow content you like and block content you donā€™t like. Iā€™m especially focusing on those plans in retaliation to the European Union wanting to fine social media if they donā€™t police the free speech of users as the government sees fit.

The program itself will be an interface, which does nothing but interpret the profile file you give it (with public / private key pairs) in order to upload or update files: Users register and edit their profiles by sending change requests to any trusted interface, telling it ā€œthis is my profile, here is the private key for its public key so you know itā€™s me, and this is the file I would like you to changeā€ā€¦ the script then generates a new profile json file if the decryption is successful, and makes the profile point to it instead of the old one.

The challenge here is how to point to this new file considering that its hash changes. I worry it requires JavaScript to edit an IPNS address on its own, which doesnā€™t seem very secure and reliable. Remember that I donā€™t wish to use other external services for functionality, only decentralized IPFS where browser-side JavaScript does all of the processing.

15

I absolutely agree on the implementation, and approach. Regarding the issue of keeping updated files ā€œindexedā€, after its hash changed, a simple solution could be to hold all profile files inside a directory, so on the ā€œmain pageā€ you just iterate over the directoryā€™s content (regardless of the hashes) and load the profile files into the view.

16

Hmmm. Iā€™d assume that if you edit a directory, its own hash would changeā€¦ though in either case, the hash of the file itself would still be modified. Iteration canā€™t do much if you have multiple files, as you still need reliable criteria to know which file youā€™re accessing via a constant filename. JavaScript would have to do the editing either way, so the question remains on how to lock changes to a directory behind a password or private key.

The correct solution feels like using JavaScript to update an IPNS address pointing to the profile json file. To make an analogy using the console commands, I believe JS would need to automate this:

ipfs add NEW_PROFILE_SETTINGS.json
ipfs name publish HASH_OF_NEW_PROFILE_SETTINGS.json

But of course if any script would do that, trolls would be able to erase everyoneā€™s profile by messing with the IPNS pointerā€¦ the IPNS would have to be protected by a private key or a password. But of course, I donā€™t even know if browser JavaScript is capable of doing this on its own to begin with!

17

Hmmm, you may right, I havenā€™t had the opportunity to play with this stuff yet. I am about to fire up a Github repo and make some experiments. Would you be interested to join?

18

Sadly Iā€™m both very busy and very new to IPFS. I just got the IPFS daemon working locally (pretty annoyed thereā€™s no RPM repository for openSUSE yet) and messed around with a few basic commands.

I wish to have a good structure and proof-of-concept for the project first, I will likely attempt this locally. If it grows into something big, it will definitely need a team to develop it further, as thereā€™s only so much I can do with my limited experience.

19

Ok, I will be back if I will have any interesting news. :slight_smile:

20

can I do this if I added a directory with 10 files and i want to update only one? or Iā€™ve to re-publish the entire directory?