eth_getLogs returns null results sometimes

Bugs
1

Quite irregularly eth_getLogs is returning null results about 2% of the time.

Given the following request:
curl --location --request POST 'https://mainnet.infura.io/v3/<project-id>' \ --header 'Content-Type: application/json' \ --data-raw '{ "jsonrpc": "2.0", "method": "eth_getLogs", "params": [ { "fromBlock": "0xCA9963", "toBlock": "0xE10F9B", "topics": [ "0x0d3648bd0f6ba80134a33ba9275ac585d9d315f0ad8355cddefde31afa28d0e9" ] } ], "id": 1 }'

I wrote a script that queries 100 times, and am seeing:
... {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"result":null} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"result":null} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} {"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}} ...

With the response being {"jsonrpc":"2.0","id":1,"result":null} for about 1-3 times in those 100 calls.

1 Like
2

Hey :wave: @friedemann and welcome to the Infura community!

There is some limitation on that call:

  • A max of 10,000 results can be returned by a single query

  • Query duration must not exceed 10 seconds

Suggestion for improvement:

  • Limit your query to a smaller number of blocks using fromBlock and toBlock .

  • If querying for commonly used topics , consider limiting to a single Smart Contract address as well.

Alternatively, you can try using JSON-RPC over WS.

1 Like
3

Hi and thanks for your answer!

I’m aware of those limitations and as expected I get an error message in 98% of the cases. I’m only concerned about the times I get no error back as this leads to my application expecting that the call went through and the result is just empty.

{"jsonrpc":"2.0","id":1,"result":null}

Is not the expected response in those cases, as per your docs:

If a query returns too many results or exceeds the max query duration, one of the following errors is returned:

Which is not the case here, even though the request clearly failed.

1 Like
4

I see what you are saying, not the best way to implement error handling.

Let me research it and bring it up to the team, if they will have any suggestions on this.

Could you share with us a script, by any chance?
Have you tried implementing it on ws, do you receive the same results?

1 Like
5

Sorry for the really late reply on this matter. But I’ve just stumbled across the same problem again. Seems like nothing has changed here.

Here’s a very basic script that queries the same (failing) request 100 times and just prints the output:

for i in {1..100}; do curl --request POST --data-raw '{"jsonrpc":"2.0","method":"eth_getLogs","params":[{"fromBlock":"0x1"}],"id":1}' https://mainnet.infura.io/v3/<project-id> ; echo "";  done

It does seem to happen less often but maybe every 200 requests I still see the obscure response:

...
{"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}}
{"jsonrpc":"2.0","id":1,"result":null}
{"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query returned more than 10000 results"}}
...

As a side note:
If using the RPC API directly this isn’t too much of an issue as one could differentiate between a null and [] result. The problem is with the go-ethereum library not exposing this and just returning no logs but no error either. Leading to our application to expect the call to have gone through when actually it should retry with different parameters.

Hope this helps in search of the problem, if there’s anything else I can help with let me know.

1 Like
6

Hi there, I ran your query for 1…300 to see if this error response would pop up but had no luck reproducing it.

I got the following:
{"jsonrpc":"2.0","id":1,"error":{"code":-32005,"data":{"from":"0xCB3D","limit":10000,"to":"0x7B737"},"message":"query returned more than 10000 results. Try with this block range [0xCB3D, 0x7B737]."}}

This is an accurate response since we have a limit in place to protect our infrastructure, as every provider would.
You can query eth_getLogs on any block range with a limit of 10K logs in return.

If fromBlock or/and toBlock are omitted, eth_getLogs returns the entire chain history by default, resulting in overloading the nodes and creating a payload that will be difficult to process and return.

The following advice in this case:

  • Limit your query to a smaller number of blocks using fromBlock and toBlock.
  • If querying for commonly used topics, consider limiting to a single Smart Contract address as well.
3 Likes
7

Thanks for getting back on this! I am aware of those limitations and am actually trying to respect them from within my application. But if a query fails this needs to be reliably propagated as an error, which didn’t seem to always be the case before.

I tried it again just now and wasn’t able to reproduce the error. Seems like there has been some update made - the Try with this block range [0xCB3D, 0x7B737]. part of the response is new. Maybe this also solved the other bug.

I’ll mark this as answered. Have a great day!

3 Likes