Make sure the reverse proxy you are using for SSL termination is setting the
X-Scheme header to
https so OctoPrint knows it is being accessed through HTTPS and not HTTP and knows to generate the correct URLs.
Example: If you've set up your reverse proxy to serve OctoPrint on its address under
https://<your server> you'll need to make it set the header
X-Scheme: https in its requests to OctoPrint.