Size limit on IPFS /get endpoint?

Questions
#1

Hey there! I have been playing around with the IPFS /get Endpoint to get entire directories of NFT Metadata. Using the BAYC CID as an example (https://ipfs.infura.io:5001/api/v0/get?arg=QmeSjSinHpPnmXmspMjwiXyN6zS4E9zccariGR3jxcaWtq), it seems that I am always returned an arbitrary number of characters by your API.

3 Testruns:
Number of characters received: 5242880, 1048576, 4194304
Total Documents: 5120, 1024, 4096 (of which the last one is simply cut-off somewhere in the string).

Is this due to some kind of limitation set by you? I can see all documents using IPFS desktop.

I was planning to use this endpoint to get the directory instead of needing 10k calls to query all tokens. Please let me know if I misunderstood anything or if it should be something on my side!

#2

Hi @Jan and welcome to our Infura community.

You can retrieve IPFS data using the API with a limit of 100 requests/sec via the remaining read-only methods such as:

"/api/v0/cat"
"/api/v0/get"
"/api/v0/dag"

Please check out our documentation here → IPFS | Infura Documentation and let me know if you have any questions.

#3

Hey Flaviu,

I have tried out all your different endpoints (cat, get & dag), and I settled on the get endpoint as it allows me to get the entire directory data without sending hundreds of queries (in theory) and should thus be way faster as it won’t block dozens of threads while waiting on requests.

If I receive random amounts of characters each time I call the endpoint, that should be an issue on your side, no?

#4

I’m also not talking about request limits as in requests/minute, but whether there are limits regarding the size (in bytes/characters) of the response (don’t think there should be)

#5

We don’t have any limits regarding the size (bytes/characters) of the response. @Jan

#6

Thanks for the confirmation - I suspect that there is an issue with your API Endpoint then? If the response was truncated on my side, I should end up with the same amount of characters every time. But as you can see in my initial post, the response arrives arbitrarily truncated (5242880, 1048576, 4194304 Characters respectively) when executing the same request several times. What’s the best approach here? Do I open a ticket or can you check internally?

#7

Hi @Jan please open a ticket and we will take a further look into this issue.

#8

Hey everybody! Wanted to come back and update with the issue in case someone is experiencing the same!

@Flaviu was able to point me in the right direction in a ticket response. When your results are arbitrarily cut-off, make sure you are properly transmitting the auth, as the endpoint will respond even if there’s an issue with your auth, but requests will be limited to 60s of Data-streaming. Just fix your auth and it should work!

1 Like