Hi,
I have websockets setup on infura, and I keep getting every once in a while an error thrown Internal logs subscription error
… This started happening around 2 weeks ago, and i haven’t changed anything in my setup. Is there some update happening in the background? I’ve raised an issue with the dev on websockets and I think he is pointing me to this problem being from the infura server…
opened 07:32PM - 25 May 21 UTC
closed 04:13PM - 26 May 21 UTC
<!--
Thanks for taking the time to report an issue!
Did you check the FAQ?… Perhaps you'll find the answer you need:
https://websockets.readthedocs.io/en/stable/faq.html
Is your question really about asyncio? Perhaps the dev guide will help:
https://docs.python.org/3/library/asyncio-dev.html
Did you look for similar issues? Please keep the discussion in one place :-)
https://github.com/aaugustin/websockets/issues?q=is%3Aissue
For bugs, providing a reproduction helps a lot. Take an existing example and tweak it!
https://github.com/aaugustin/websockets/tree/main/example
-->
Hi @aaugustin
I really tried looking all over for a solution to this issue i've recently started facing and i'm hoping you could help me out.
I have the below script in a class that connects to a server and process messages from many sockets that are setup. The program needs to kill the sockets and generate new ones every `timerTopics` minutes:
```
async def wssSubscribe(self,payload):
link,data=payload
async with websockets.connect(link) as ws:
#setup subscription
await ws.send(data)
message = await ws.recv()
message = json.loads(message)
subscriptionID = message["result"]
try:
#Get & process messages
async for message in ws:
message = json.loads(message)["params"]["result"]
print(message)
self.dumpToSql(responses=message)
#On cancellation, cancel subscription
except asyncio.CancelledError:
message = self.baseProvider.encode_rpc_request(method='eth_unsubscribe', params=[subscriptionID])
await ws.send(message)
await ws.recv()
await ws.close()
#On other exception
except Exception as e:
#Inform user
print(f"Exception {e} seen while gathering topics wss")
#Triggers task regensis
self.activateTopicsHeal = True
print(traceback.format_exc())
#Unsubscribe
message = self.baseProvider.encode_rpc_request(method='eth_unsubscribe', params=[subscriptionID])
await ws.send(message)
await ws.close()
#Regenerate new tasks automatically
async def taskGenerator(self):
loop = asyncio.get_running_loop()
try:
while True:
await asyncio.sleep(60 * (self.timerTopics - 1 ))
self.tasksToKill = self.wssTasks.copy()
self.wssTasks = list()
for payload in self.payloads:
task = self.wssSubscribe(payload)
task = asyncio.ensure_future(task,loop=loop)
self.wssTasks.append(task)
await self.cancelWSSTasks(self.tasksToKill)
except asyncio.CancelledError:
print("cancellation happened!")
```
Now every now and then since the last 3 weeks I started getting this, showing up under general exception catch...
```
Traceback (most recent call last):
File "C:\kaleb\data_manager\utils\gatherTopicsWSS.py", line 60, in wssSubscribe
async for message in ws:
File "C:\kaleb\miniconda\envs\keeper\lib\site-packages\websockets\protocol.py", line 439, in __aiter__
yield await self.recv()
File "C:\kaleb\miniconda\envs\keeper\lib\site-packages\websockets\protocol.py", line 509, in recv
await self.ensure_open()
File "C:\kaleb\miniconda\envs\keeper\lib\site-packages\websockets\protocol.py", line 812, in ensure_open
raise self.connection_closed_exc()
websockets.exceptions.ConnectionClosedError: code = 1011 (unexpected error), reason = Internal logs subscription error
```
I've looked all over the web and stackoverflow and all i could find is that someone saying the ping/pong is killing my socket, tried setting it to None, the problem persists... do you think i have something deep in my script causing this that i should change...
Forgot to mention, I am on ubuntu with websocket version 8.1 the latest...
Thank you for your help 🙏🏼
sdan
June 2, 2021, 12:14am
2
I have been running into this too! websocket: close 1011 (internal server error): Internal logs subscription error
whenever i leave a websocket connection open with infura for a few hours (that i regularly ping)
Hi @kaleb-keny and welcome to the Infura community, @sdan !
One thing to check is if your library supports automatic/manual Pong, and to make sure your context doesn’t cancel the client by timeout, particularly if you’re using Golang. If you can let us know what language you’re using, we can likely help find a more specific solution, but it sounds like the most likely cause is either the ping/pong isn’t set up correctly, or that something is being blocked by a firewall.
Leiya_Kenney:
One thing to check is if your library supports automatic/manual Pong, and to make sure your context doesn’t cancel the client by timeout, particularly if you’re using Golang. If you can let us know what language you’re using, we can likely help find a more specific solution, but it sounds like the most likely cause is either the ping/pong isn’t set up correctly, or that something is being blocked by a firewall.
Hi Leiya,
yeah i’m using python websockets library and the main snippet of my script is pretty simply:
import json
import websockets
import asyncio
import nest_asyncio
nest_asyncio.apply()
async def wssSubscribe(payload):
link,data=payload
async with websockets.connect(link) as ws:
#setup subscription
await ws.send(data)
message = await ws.recv()
message = json.loads(message)
subscriptionID = message["result"]
async for message in ws:
message = json.loads(message)["params"]["result"]
dumpToSql(responses=message)
where
link = 'wss://mainnet.infura.io/ws/v3/XXXXXXXX'
and
data is the subscription topic
This has happened mostly I think during gas intense times… And it is a fairly recent issue, where it didn’t occur throughout the time I used it in the past. But so far since gas went down, maybe infura infrastructure became less burdoned with these request and the error decreased considerably. I have asked the author of the websockets library and he told me that it is mainly coming from the server: "Your connection handler is throwing an exception. The exception message is: Internal logs subscription error
"
Also getting the same using golang so interested to get a reply from the support team.
In my definitely not a context cancelation.
Using the geth WatchLogs binding.