Print hung this morning, trying to understand log file

What is the problem?
Print hung. I received an update from telegram at 5:58AM Eastern (09:58 UTC). Realized there was a problem after I got to work and texted my wife turn to off the printer. Printer was on, bed and nozzle were hot. She turned it off at 9:12AM Eastern (13:12UTC).

018-04-25 08:32:32,535 - octoprint.util.comm - INFO - Communication timeout while printing, trying to trigger response from printer.

2018-04-25 10:17:01,107 - octoprint.util.comm - INFO - Communication timeout while printing, trying to trigger response from printer.

I have telegram send every 20 minutes, so this corresponds to 20 minutes after the last successful status:
2018-04-25 10:18:15,800 - octoprint.plugins.telegram.listener - ERROR - Setting status: Got an exception while trying to connect to telegram API: ('Connection aborted.', BadStatusLine("''",)). Waiting 2 minutes before trying again.

I have a feeling there's something funky happening to my Pi's network adapter but I don't know where to start looking into that.

Any insight would be much appreciated!

What did you already try to solve it?
Tried canceling print, ended up just rebooting the Pi.

Additional information about your setup
Model 3B with 7" touchscreen attached.
octoprint.log (170.2 KB)

I don't know if it would be useful to read this page and adjust your config.yaml file with different communication timeouts.

Reading your log, it looks like it timed out several times and also had difficulties exercising the telegram, too.

2018-04-25 10:54:17,613 - octoprint.plugins.telegram.listener - ERROR - Setting status: Got an exception while trying to connect to telegram API: HTTPSConnectionPool(host='api.telegram.org', port=443): Max retries exceeded with url: /bot542105178:AAFZ8c8GpLEaKCayqgbM3shjsh_Y-Qe1wl0/getUpdates?timeout=30&offset=1 (Caused by SSLError(SSLEOFError(8, u'EOF occurred in violation of protocol (_ssl.c:581)'),)). Waiting 2 minutes before trying again.
1 Like

Looks like you have three problems here.

One is your instance apparently falling off the network which coincides with the Telegram errors:

2018-04-25 12:44:31,786 - octoprint.util.connectivity_checker - INFO - Connectivity changed from online to offline

There's also a myriad of capture errors from the webcam, indicating something's up there as well - that's probably not network related though since it's failing at capturing from the Pi itself (127.0.0.1) and failing, so network connectivity shouldn't be causing trouble there:

2018-04-25 11:12:24,669 - octoprint.timelapse - ERROR - Could not capture image /home/pi/.octoprint/timelapse/tmp/loubie_adalinda_dragon_20180425011537-3140.jpg from http://127.0.0.1:8080/?action=snapshot
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint/timelapse.py", line 586, in _perform_capture
    verify=self._snapshot_validate_ssl)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/requests-2.18.4-py2.7.egg/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/requests-2.18.4-py2.7.egg/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/requests-2.18.4-py2.7.egg/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/requests-2.18.4-py2.7.egg/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/requests-2.18.4-py2.7.egg/requests/adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', BadStatusLine("''",))

The final issue is that your printer stopped responding to OctoPrint, so after trying to tickle it back into replying for a bit, OctoPrint gave up:

2018-04-25 14:28:03,189 - octoprint.util.comm - INFO - Communication timeout while printing, trying to trigger response from printer.
2018-04-25 14:28:06,468 - octoprint.util.comm - INFO - Communication timeout while printing, trying to trigger response from printer.
2018-04-25 14:28:10,476 - octoprint.util.comm - INFO - Communication timeout while printing, trying to trigger response from printer.
2018-04-25 14:28:14,428 - octoprint.util.comm - INFO - No response from printer after 6 consecutive communication timeouts, considering it dead.
2018-04-25 14:28:15,259 - octoprint.util.comm - INFO - Changing monitoring state from "Printing" to "Offline (Error: Too many consecutive timeouts, printer still connected and alive?)"

To diagnose the last issue further, you'd need to enable serial.log and try to reproduce this problem.

The first is more tricky to figure out. Could be bad reception, could be power issues, could be your network acting up etc. /var/log/syslog might contain something helpful there. The second is something I've never before seen but...

All of those issues could be caused by the Pi browning out, so check that your power supply is a) actually a power supply (and not just a cellphone or tablet charger) and b) is actually putting out stable 5V at 2-3A. You wouldn't believe how many strange errors are caused by inadequate power supplies or power wiring.

1 Like

If there were a screen attached to the Raspi and it's being underpowered (and further assuming that /boot/config.txt doesn't have avoid_warnings=1 in it, then you'll see a lightning bolt periodically in the upper-right corner of the screen to indicate the power situation. But most people run theirs headless...

Running vcgencmd get_throttled should return a variable throttled=0x50005 which ought to be zero if no throttling is occurring (either due to under-voltage or that it's too hot). Following up, a vcgencmd measure_temp (with my own output of temp=41.9'C) will show you if it's looking hot or not which could then rule out temperature.

The low threshold for input power is 4.65VDC, I believe.

1 Like

Thanks for the info. My power supply came with the Pi, it plugs into the USB port and is 5 volts, 2.5 amps.

I think maybe my problem could be heat related. Not sure why I didn't think to mention this but it often shows the thermometer warning and goes over 80C. It's in a case mounted behind the screen. I'm going to take the back off of that and see if it helps some. Pretty sure it has a heat sink, I'll check later.

So, I wonder, if it gets hot enough for long enough could that cause the onboard wifi to stop working? I'll look at syslogs later, too.

You might want one of these and one of these.

If you're a coder, I wrote this JavaScript to tell the temperature of either CPU/GPU of the Raspberry Pi 3: raspi-temp.

But I don't think a throttling condition should affect your networking connectivity, to be honest.

1 Like

No heat sink! I swapped it out with another one I had with heat sinks. I'll keep an eye out for the temp and let you know if this happens again (it wasn't the first time that pi went offline).

My Pi 3B temperature was around 45C with nothing going on. I added a case with a 30mm 5v fan and heatsinks and it went down to around 30C at idle. With a sysbench load, high 70s without the fan, mid 50s with a fan. I'd recommend a fan of some sort.

Definitely will add a fan. With the heat sinks it's down to 50-60. Having chromium running the octoprint page uses 25% cpu and drives the temp to 65-70. Not sure why chromium uses so much cpu. Going to try disabling the temp graph after my current print is over.

Are you running Chromium on the same Pi as OctoPrint and a camera?

I thought the major feature of OctoPrint was to let you control your printer from another machine using the browser on that machine. I'll have to see what running Chromium on my Pi will do to the temperature.

I have it sitting on my print enclosure with the octoprint page running. I usually am in another room when I send to printer from Slic3r, then don't start printing until I'm in the room and can hit the start button on the pi.

It is, and it is also optimized for that (read: lots of stuff offloaded to the client were possible, since that's expected to run on a way beefier machine than a Pi).

I do not recommend running a full fledged browser + OctoPrint's web interface on the same Pi as OctoPrint itself for the very reason that it can cause various issues (overheat, resource consumption that than lack for the actual print etc).

Understood. I didn't realize that when I set this up. With the temp graph disabled the browser doesn't use much cpu. I was hoping I could find a browser that was more stripped down.

Now that I actually have heat sinks installed my temp is staying cool. I'm adding a fan soon, too.

1 Like