Upload folder of size more than 500 mb to infura ipfs node

Questions
1

I am using the curl API below to upload the content of the folder but getting a 502 gateway error. Is there any way to upload the content of the folder rather than every file one by one.

curl “https://ipfs.infura.io:5001/api/v0/add?wrap-with-directory=true”
-X POST
-F file=@"/home/helium/Downloads/hls-test/master.m3u8"
-F file=@"/home/helium/Downloads/hls-test/master0.ts"
-F file=@"/home/helium/Downloads/hls-test/master1.ts"
-F file=@"/home/helium/Downloads/hls-test/master2.ts"
-F file=@"/home/helium/Downloads/hls-test/master3.ts"
-F file=@"/home/helium/Downloads/hls-test/master4.ts"
-F file=@"/home/helium/Downloads/hls-test/master5.ts"
-F file=@"/home/helium/Downloads/hls-test/master6.ts"
-F file=@"/home/helium/Downloads/hls-test/master7.ts"
-F file=@"/home/helium/Downloads/hls-test/master8.ts"
-F file=@"/home/helium/Downloads/hls-test/master9.ts"
-F file=@"/home/helium/Downloads/hls-test/master10.ts"
-F file=@"/home/helium/Downloads/hls-test/master11.ts"
-F file=@"/home/helium/Downloads/hls-test/master12.ts"
-F file=@"/home/helium/Downloads/hls-test/master13.ts"
-F file=@"/home/helium/Downloads/hls-test/master14.ts"

2

We currently have a 100mb single upload limit, in order to upload a larger file or folder you will have to break it up. We are working on a premium offering that will allow advanced usage such as larger file uploads, we will keep you updated on the release timeline for this feature.

3

If I upload a file on ipfs using infura then till when can I access it? Do you guys delete it after some time ?

4

Hi @Yash_Shukla - welcome to the Infura community! We do delete files that haven’t been touched in 6 months, so as long as it’s being used more frequently than that, it will remain on IPFS.

5

Leyia, how do you delete large files? I don’t understand. Is that possibke to delete files from ipfs?

6

Hi @Andrey_Voronkov - welcome to the Infura community! You can’t delete files from IPFS, as its purpose is to keep every version of your files that you upload. Hope this helps!

7

Hi there, I realize this is an old post but had a question regarding this comment.

Say I have a directory I’d like to upload that contains 2 files, each 80 MB in size. Since this won’t work with a single /add request, how does one accomplish this across several requests so that the files are in the same directory?

Thanks in advance!

8

Hi @pichiste, and welcome to the Infura community! IPFS doesn’t have “directories,” per se, so you can upload both files and be able to access each of them with their unique hash. You can upload both to the same project and that will keep track of the files that go together and both will show up on that project’s dashboard.

9

Hi Leiya, thanks for your response. Unfortunately this won’t work in my case since I need to be able to link arbitrary sets of files to directories (dag nodes).

The solution I found was as follows:

  • Upload each file individually using the /add endpoint
  • Given the CIDs for those files, construct a protobuf dag node object (JSON) that links to them
  • Upload the dag node file using the /object/put endpoint
1 Like
10

Got it - thanks so much for providing your solutions for others who may have the same issue!

11

Hello,

please would you be able to post here, how you did this?

  • Upload each file individually using the /add endpoint
  • Given the CIDs for those files, construct a protobuf dag node object (JSON) that links to them
  • Upload the dag node file using the /object/put endpoint

with best regards
tomas