Is the Infura endpoint forwarded to different nodes?

I’m testing my client application that interacts with a contract deployed on the mainnet. I get some error that seems an infura endpoint forwarded to different nodes per API call.

Case. 1
I submitted a transaction via eth_sendRawTransaction and get a transaction id. Then, I immediately called eth_getTransactionByHash with the transaction id to the same endpoint for submitting the transaction. But, I cannot retrieve the transaction because the transaction is not found.

Case. 2
I submitted a transaction and checked the transaction is included in the blockchain. Then, I submitted another transaction via the same infura endpoint. I get an error shows replacement transaction underpriced. I’m understanding that the error caused when a transaction is submitted during a node keeps another transaction that has the same nonce in the transaction pool of the node.

In both cases, I can check that transactions are successfully submitted into mainnet via etherscan, after waiting for a little.

Assumption of the cause
I assumed the cause of those errors.

  1. the Infura endpoint is forwarded to different nodes.
    If the infura endpoint is forwarded to different nodes, there is possibly a propagation delay for a transaction. Hence, the error is caused by the propagation delay between a node forwarded for submitting the fist transaction and another node forwarded for the second transaction (The eth_getTransactionByHash/eth_submittRawtransaction is called before the second node received the first transaction/block includes the first transaction).

  2. There is overhead for processing a transaction on a node in the mainnet.
    If there is an overhead for processing a transaction on a node in mainnet and the eth_getTransactionByHash can get only transactions in the transaction pool and blockchain, the case 1 is reasonable (The eth_getTransactionByHash is called before the node collects the submitted transaction into the transaction pool).

Which assumption is reasonable? and is there another cause for the errors?
Additionally, how can I handle those errors?

Hello chike,

For case 1, your assumptions are basically correct. To ensure maximal propagation of transactions to the network, the nodes that service eth_sendRawTransaction do not handle any other RPCs, so they they can focus on broadcasting transactions w/o being impacted by customer load of other RPCs.

However your transaction should end up in the mempool of the nodes servicing eth_getTransactionByHash relatively quickly.

Your case 2 concerns me: you should only see that error if you submit another transaction with the same nonce, the fact that you’ve seen it may mean that you are sending transactions with duplicates nonces, or with “nonce gaps” (which can lead to transactions not being propagated).

Have you verified that when sending transactions you are keeping track of the correct nonce to use? If you send multiple transactions in quick succession in my opinion it’s best to track your nonce locally in your program state rather than relay on, say, eth_getTransactionCount.

Thank you for replying, Ryuan.

I understood the eth_sendRawTransaction locks a node that serves the request. Then, it means that it is required to wait some times (until broadcasting the transaction), correct? and is there a method to verify whether the transaction is broadcasted or not?

Then, My client/contract is an application that manages the state of work.
My client process is as followings;
A1. Check a state of a contract.
A2. Send a transaction that updates the state (It shows that the client is now working).
A3. Check the transaction is included in the blockchain by checking the blockNumber filed in a response to eth_getTransactionByHash is not Null.
A4. Work
A5. Then, Send a transaction that re-update the state.

In case2 I got the error in step A5. In the step for send transactions(A2, A5), the client calls eth_getTransactionCount to set the correct nonce for transactions. However, in the testing for developing the application, I eliminated step 4 because it is difficult to estimate the duration for the work.

Since posted this question, I assumed case 2 is caused by the overhead for updating a world state on a node that serves those requests since the transaction is included in the blockchain. I assume the duration for updating the world state takes some time because the mainnet blockchain is huge so that it takes time to execute state updates on EVM (it requires to loads world state from the huge blockchain).

Combining with the cause of case 1, I assume case 2 caused as followings;
B1. Submit a transaction via the Infura endpoint, forwarded to a node (Node A).
B2. Infura locks Node A for serving eth_sendRawTransaction. Following API calls are forwarded to another node (Node B).
B3. Check the transaction is included in the blockchain, start to update the world state.
B4. Before Node B updated the world state by a block includes the transaction, though Node B receives the transaction (it caused by the propagation delay of the block), the client try to submit a second transaction.
B5. Node B replies an expected nonce according to the world state that has not been updated by the first transaction (It means that the reply is the same nonce as the first transaction).
B6. The client submitted the second transaction that has the same nonce as the first transaction. It causes the error.

I think that I can handle the error by fixing the A3 process so that the client checks the world state instead of checking blockNumber. Do you have better suggestions for fixing this error?

However, it cannot handle the propagation delay between Node A and Node B because the client does not check the state of the contract before submitting the second transaction. The client assumes step A3 ensures that the first transaction has been completed. By referring to the same node for each API call (not considering reorg at this time), I think the fixed A3 steps can ensure it.

As a conclusion, I think it is better to create a new option for calling API in Infura that ensures that API calls are forwarded to the same node. Because the option means a user can lock an infura resource, so I think the option can be not in free plan :stuck_out_tongue:

I would say that in addition to checking if the block number is in the return value from eth_getTransactionByHash I would also check that the value returned by eth_getTransactionCount with the "latest" param is one higher than the nonce you used in step A2.

As a conclusion, I think it is better to create a new option for calling API in Infura that ensures that API calls are forwarded to the same node

We have considered offering such a plan, the main trade-off would be that the rate-limit of requests a user can send must be much stricter than our current plans, since otherwise it’s too easy to saturate a single node. And I agree that if we ended up offering such a plan it would need to be a paid option.

Thank you for your good suggestion! I will try to fix my client.

I agree that the rate limit of API calls is a significant issue for the new option. I understood that developers should implement clients that call Infura API so that clients can handle the switching of forwarding nodes.

Then, I have an additional question. When Infura switches a forward node to another node? You mentioned eth_sendRawTransaction method locks the node and switches. Are there other API methods that cause to switch?