# Supporting Large IPLD Blocks

**URL:** <https://discuss.ipfs.tech/t/supporting-large-ipld-blocks/15093>\
**Category:** Protocol\
**Tags:** ipld\
**Created:** [August 31, 2022, 9:15pm UTC](https://discuss.ipfs.tech/t/supporting-large-ipld-blocks/15093 "2022-08-31T21:15:27Z")\
**Posts on this page:** 1\
**Showing post:** 20

<div class="post-metadata">

**Author:** ![gkraemer](https://avatars.discourse-cdn.com/v4/letter/g/6de8d8/32.png) [@gkraemer](https://discuss.ipfs.tech/u/gkraemer)\
**Post date:** [February 16, 2024, 2:27pm UTC](https://discuss.ipfs.tech/t/supporting-large-ipld-blocks/15093/20 "2024-02-16T14:27:47Z")

</div>

I am strongly in favor of increasing block size. Currently there is some momentum for serving scientific data using IPFS. IPFS is actually the perfect use-case for scientific data which is archived and distributed through very clunky repositories, DOIs, etc., all things that are solved very elegantly by IPFS.

One major benefit IPFS has over most repositories is that from these repositories you have to download the data before you can use them (I know of only one that offers S3-compatible object store to access the data) but with IPFS you can actually use the data directly without explicitly downloading them. The issue here is performance and because all of the new cloud ready data formats, such as Zarr, already come chunked, any additional chunking just produces unnecessary overhead.

> [@Any plans to increas maximum block size?](https://discuss.ipfs.tech/t/any-plans-to-increas-maximum-block-size/17573/1):
>
> We are storing large array-type data in the zarr format [1]. The zarr format, in theory, is well suited for being distributed through ipfs because it is stored as chunks plus metadata in JSON. There even have been some effort to map the format directly to IPLD [2].
> 
> For good performance, the correct chunking is absolutely important and an empirically good chunk size is in the 1-5MB range, which doesn’t map neatly to the 1MB supported by IPFS. I have just run a small experiment where I benchmarked simple copying via `rsync` vs `ipfs get` and `rsync` was faster by a factor of ~5x. I strongly suspect that the problem was the additional overhead due to the block storage (combined with the slow HDD of the sending machine).
> 
> Are there any efforts to increase the maximum blocksize supported by IPFS?
> 
> [1] [Zarr Storage Specification Version 2 — Zarr specs documentation](https://zarr-specs.readthedocs.io/en/latest/v2/v2.0.html)
> 
> [2] [GitHub - d70-t/ipldstore: zarr backend store using IPLD datastructures](https://github.com/d70-t/ipldstore)

---

_[View the full topic](https://discuss.ipfs.tech/t/supporting-large-ipld-blocks/15093)._