getPastEvents throws errors with websocket provider but not HTTP provider

I’m wondering if I have to use the HTTP provider for such calls? The websocket provider seems to error out. I know there is a limit of 10000 events returned from a call, I start at some past block and request various batches of events. The websocket provider call to getPastEvents returns fine without error but only when 0 events are returned…as soon as it reaches a block with matching events, I get errors (whereas the HTTP provider does not error out with the same call and block range). I have a need for the websocket provider for listening to future events, but I thought I could also use it for fetches like this. Am I wrong? I’m using web3js v1.2.4 in node. It seems to lose its connection once it reaches a block with events? Strange. Am I doing something wrong? Here’s the error I get

Error: CONNECTION ERROR: Couldn’t connect to node on WS.
at Object.InvalidConnection (node_modules\web3-core-helpers\src\errors.js:35:16)
at WebsocketProvider._timeout (node_modules\web3-providers-ws\src\index.js:255:48)
at W3CWebSocket.connection.onclose (node_modules\web3-providers-ws\src\index.js:155:15)
at W3CWebSocket._dispatchEvent [as dispatchEvent] (node_modules\yaeti\lib\EventTarget.js:107:17)
at W3CWebSocket.onClose (node_modules@web3-js\websocket\lib\W3CWebSocket.js:228:10)
at WebSocketConnection. (node_modules@web3-js\websocket\lib\W3CWebSocket.js:201:17)
at WebSocketConnection.emit (events.js:210:5)
at WebSocketConnection.drop (node_modules@web3-js\websocket\lib\WebSocketConnection.js:475:14)
at node_modules@web3-js\websocket\lib\WebSocketConnection.js:303:18
at processTicksAndRejections (internal/process/task_queues.js:75:11)
connection not open on send()
Error: connection not open
at WebsocketProvider.send (node_modules\web3-providers-ws\src\index.js:282:18)
at RequestManager.send (node_modules\web3-core-requestmanager\src\index.js:132:66)
at sendRequest (node_modules\web3-core-method\src\index.js:651:42)
at send (node_modules\web3-core-method\src\index.js:672:13)
at Contract.getPastEvents (node_modules\web3-eth-contract\src\index.js:675:12)

Hi @hexstat welcome to the community! Could you provide the code that is creating your connection and subscription as well as the response you’re getting when 0 events are returning?

To be clear, my websocket subscriptions work fine. It’s the non-subscription queries through the websocket that seem to fail. I’ll show both of those. Here’s my websocket Provider connection, subscription, and query. My query calls are actually inside a loop over chunks of blocks as you guys suggest, to avoid the 10,000 event limit, but I’ve left that code out for simplicity here:

const wsProvider = new Web3.providers.WebsocketProvider(INFURA_WS_ENDPOINT, {
  headers: {
    authorization: MY_AUTH
  }
});
const web3ws = new Web3(wsProvider);
const wsContract = new web3ws.eth.Contract(MY_ABI, MY_CONTRACT_ADDRESS);

// ws subscribe, seems to always work fine
wsContract.events.MyEvent(this.onEvent.bind(this));

// ws query inside a loop over blocks to avoid the 10,000 event limit.
// Fails as soon as it finds results
const events = await wsContract.getPastEvents('MyEvent', {
  fromBlock: fetchFrom, 
  toBlock: fetchTo
}).catch(err => console.error(err));

The value of events from the above getPastEventsCall when called on blocks that have zero results, contains what I’d expect it to:

[]

And then once the loop hits a block range that does have such events, I get the error I originally posted. Thanks for your help!

Hi @hexstat I would recommend using an HTTP provider for these RPC requests and not sending them over the websocket. Would that work for your use case? Use the websocket specifically for the subscription methods to avoid polling for changes.

Yeah that’s what I’m currently doing in production…just hoped it was unnecessary to have have both the WS and HTTP connections. If that’s what it takes to both subscribe and query, that’s fine for now. But is this something that you guys would look at fixing?

Hi @hexstat, this is something we are actively working on, we greatly appreciate the feedback.