Hi everyone,
My router went from IPv4 to IPv6 after an update from my ISP back in April, and so I decided to try and get my selfhosted Raspberry Pi server to work with it. It’s been less trivial than I hoped it would be, though. It worked and was reachable when it still used IPv4, but it’s been out of the air since April.
I’m running Arch Linux ARM on the device and use networkd to connect it to the internet. I use https://now-dns.com to get a dynamic DNS and have connected it to my server using their Linux script.
This is my Caddyfile:
{
debug
}
# Jellyfin:
myserver.now-dns.net:26347,
myserver.now-dns.net:443,
[my ipv6]:26347 {
header / {
# Enable cross-site filter (XSS)
# and tell browser to block detected attacks
X-Frame-Options "Deny"
Content-Security-Policy "
default-src 'self' data: blob:;
style-src 'self' 'unsafe-inline' bootstrapcdn.com *.bootstrapcdn.com https://ctalvio.github.io/Monochromic/default_style.css https://ctalvio.github.io/Monochromic/jfblue_style.css https://ctalvio.github.io/Monochromic/jfpurple_style.css https://ctalvio.github.io/Monochromic/bottom-progress_style.css https://ctalvio.github.io/Monochromic/customcolor-advanced_style.css https://ctalvio.github.io/Monochromic/improve-performance_style.css https://fonts.googleapis.com/css2;
script-src 'self' 'unsafe-inline' bootstrapcdn.com *.bootstrapcdn.com googleapis.com *.googleapis.com https://www.gstatic.com/cv/js/sender/v1/cast_sender.js worker-src 'self' blob:;
font-src 'self' bootstrapcdn.com *.bootstrapcdn.com;
img-src data: 'self' imgur.com *.imgur.com;
form-action 'self';
connect-src 'self' pokeapi.co;
frame-ancestors 'self';
report-uri {$CSP_REPORT_URI}
"
}
reverse_proxy 127.0.0.1:8093
#reverse_proxy localhost:8093
}
# Nextcloud:
myserver.now-dns.net:65001 {
root * /usr/share/webapps/nextcloud
file_server
# log {
# output file /var/log/caddy/myserver.now-dns.net.log
# format single_field common_log
# }
#php_fastcgi 127.0.0.1:9000
#php_fastcgi unix//run/php-fpm/php-fpm.sock # veranderd naar correcte adres uit /etc/php/php-fpm.d/www.conf
php_fastcgi unix//run/nextcloud/nextcloud.sock # veranderd naar nieuwe correcte adres uit /etc/php/php-fpm.d/nextcloud.conf
header {
# enable HSTS
Strict-Transport-Security max-age=31536000;
}
redir /.well-known/carddav /remote.php/dav 301
redir /.well-known/caldav /remote.php/dav 301
# .htaccess / data / config / ... shouldn't be accessible from outside
@forbidden {
path /.htaccess
path /data/*
path /config/*
path /db_structure
path /.xml
path /README
path /3rdparty/*
path /lib/*
path /templates/*
path /occ
path /console.php
}
respond @forbidden 404
}
Figuring out how to open the necessary ports took some doing on my router, but now when I test with an IPv6 port scanner (like this one) it shows me that ports 80 and 443 are open, as well as ports 65001 and 26347. It works both when I fill in my public IPv6 address as well as the address I get from now-dns. I still cannot connect to the server with a browser, though.
I have been whittling away at this issue on and off since April and haven’t really made any big breakthroughs. What would be your first steps in troubleshooting this issue?
journalctl -f -u caddy gives the following:
Jul 18 16:28:13 baspi2 caddy[422]: {"level":"debug","ts":1689690493.3595114,"logger":"http.stdlib","msg":"http: TLS handshake error from 198.199.97.61:43266: no certificate available for '192.168.1.96'"}
Jul 18 16:28:16 baspi2 caddy[422]: {"level":"debug","ts":1689690496.401284,"logger":"http.stdlib","msg":"http: TLS handshake error from [2604:a880:400:d0::20e2:c001]:46636: EOF"}
Jul 18 16:28:45 baspi2 caddy[422]: {"level":"debug","ts":1689690525.159631,"logger":"http.stdlib","msg":"http: TLS handshake error from [2607:5300:201:3100::7911]:42978: read tcp [2a02:a465:1b91:1:dea6:32ff:fe54:67fb]:65001->[2607:5300:201:3100::7911]:42978: read: connection reset by peer"}
Jul 18 16:35:44 baspi2 caddy[422]: {"level":"debug","ts":1689690944.3032691,"logger":"http.stdlib","msg":"http: TLS handshake error from [2a01:4f8:1c1c:2d4e::1]:31497: EOF"}
Jul 18 16:41:15 baspi2 caddy[422]: {"level":"debug","ts":1689691275.666184,"logger":"http.stdlib","msg":"http: TLS handshake error from 45.227.254.49:65421: tls: first record does not look like a TLS handshake"}
Jul 18 16:48:14 baspi2 caddy[422]: {"level":"debug","ts":1689691694.1229563,"logger":"events","msg":"event","name":"tls_get_certificate","id":"f6540cc3-dce9-4f75-995a-9d313ad6a9a8","origin":"tls","data":{"client_hello":{"CipherSuites":[49199,49195,49169,49159,49171,49161,49172,49162,5,47,53,49170,10],"ServerName":"","SupportedCurves":[23,24,25],"SupportedPoints":"AA==","SignatureSchemes":[1025,1027,513,515,1025,1281,1537],"SupportedProtos":null,"SupportedVersions":[771,770,769],"Conn":{}}}}
Jul 18 16:48:14 baspi2 caddy[422]: {"level":"debug","ts":1689691694.1232002,"logger":"tls.handshake","msg":"no matching certificates and no custom selection logic","identifier":"192.168.1.96"}
Jul 18 16:48:14 baspi2 caddy[422]: {"level":"debug","ts":1689691694.1232479,"logger":"tls.handshake","msg":"all external certificate managers yielded no certificates and no errors","remote_ip":"192.241.226.31","remote_port":"60480","sni":""}
Jul 18 16:48:14 baspi2 caddy[422]: {"level":"debug","ts":1689691694.1233048,"logger":"tls.handshake","msg":"no certificate matching TLS ClientHello","remote_ip":"192.241.226.31","remote_port":"60480","server_name":"","remote":"192.241.226.31:60480","identifier":"192.168.1.96","cipher_suites":[49199,49195,49169,49159,49171,49161,49172,49162,5,47,53,49170,10],"cert_cache_fill":0.0003,"load_if_necessary":true,"obtain_if_necessary":true,"on_demand":false}
Jul 18 16:48:14 baspi2 caddy[422]: {"level":"debug","ts":1689691694.1235263,"logger":"http.stdlib","msg":"http: TLS handshake error from 192.241.226.31:60480: no certificate available for '192.168.1.96'"}
(Those handshake errors show up when I scan the ports with an online tool.)
A few things may be going on.
The errors seem focused on the tls certificate, which caddy tries to automatically provision.
First, in your caddyfile, “my.server” should reflect the real address used for access. Something like “jellyfin.my-domain.com”. This is important for the tls certificate to be generated correctly.
Once updated, pull out a cell phone, turn off wifi (use LTE/5G), and verify it can connect to your site. This makes sure you can access from outside your home network.
Once confirmed working, try again from your home network. Most likely the page will timeout. This will be due to DNS pointing you back to your own network, which can cause trouble. This can be solved several ways. One is by adding a static DNS entry which points to the IP of your caddy server. You can do this on a per system basis in the hosts file, or at the lan level with you DNS server or router, assuming it allows you to add a custom DNS entry. I do this with my Mikrotik router.
That should get things working internal and external.
Thanks for your reply!
First, in your caddyfile, “my.server” should reflect the real address used for access. Something like “jellyfin.my-domain.com”. This is important for the tls certificate to be generated correctly.
Ah yes, I wasn’t clear in my original post—I had censored the address of my actual web address myself. In actuality it is more like this:
{ debug } # Jellyfin: myserver.now-dns.net:26347, myserver.now-dns.net:443, [my ipv6]:26347 { header / { # Enable cross-site filter (XSS) # and tell browser to block detected attacks X-Frame-Options "Deny" Content-Security-Policy " default-src 'self' data: blob:; style-src 'self' 'unsafe-inline' bootstrapcdn.com *.bootstrapcdn.com https://ctalvio.github.io/Monochromic/default_style.css https://ctalvio.github.io/Monochromic/jfblue_style.css https://ctalvio.github.io/Monochromic/jfpurple_style.css https://ctalvio.github.io/Monochromic/bottom-progress_style.css https://ctalvio.github.io/Monochromic/customcolor-advanced_style.css https://ctalvio.github.io/Monochromic/improve-performance_style.css https://fonts.googleapis.com/css2; script-src 'self' 'unsafe-inline' bootstrapcdn.com *.bootstrapcdn.com googleapis.com *.googleapis.com https://www.gstatic.com/cv/js/sender/v1/cast_sender.js worker-src 'self' blob:; font-src 'self' bootstrapcdn.com *.bootstrapcdn.com; img-src data: 'self' imgur.com *.imgur.com; form-action 'self'; connect-src 'self' pokeapi.co; frame-ancestors 'self'; report-uri {$CSP_REPORT_URI} " } reverse_proxy 127.0.0.1:8093 #reverse_proxy localhost:8093 } # Nextcloud: myserver.now-dns.net:65001 { root * /usr/share/webapps/nextcloud file_server # log { # output file /var/log/caddy/myserver.now-dns.net.log # format single_field common_log # } #php_fastcgi 127.0.0.1:9000 #php_fastcgi unix//run/php-fpm/php-fpm.sock # veranderd naar correcte adres uit /etc/php/php-fpm.d/www.conf php_fastcgi unix//run/nextcloud/nextcloud.sock # veranderd naar nieuwe correcte adres uit /etc/php/php-fpm.d/nextcloud.conf header { # enable HSTS Strict-Transport-Security max-age=31536000; } redir /.well-known/carddav /remote.php/dav 301 redir /.well-known/caldav /remote.php/dav 301 # .htaccess / data / config / ... shouldn't be accessible from outside @forbidden { path /.htaccess path /data/* path /config/* path /db_structure path /.xml path /README path /3rdparty/* path /lib/* path /templates/* path /occ path /console.php } respond @forbidden 404 }Where I replaced myserver by my actual name. I also updated my original post to reflect this just now.
Once updated, pull out a cell phone, turn off wifi (use LTE/5G), and verify it can connect to your site. This makes sure you can access from outside your home network.
Yes, I have done this a lot over the past few months (in the hopes that the problem would’ve somehow fixed itself) but no dice.
Once confirmed working, try again from your home network. Most likely the page will timeout. This will be due to DNS pointing you back to your own network, which can cause trouble. This can be solved several ways. One is by adding a static DNS entry which points to the IP of your caddy server. You can do this on a per system basis in the hosts file, or at the lan level with you DNS server or router, assuming it allows you to add a custom DNS entry. I do this with my Mikrotik router.
Well, from my home network I just used the internal (IPv4) address to connect to Jellyfin, and that still works. Nextcloud did not work. I’ll play around with this once I get the server working again externally.
But yeah, no, I’m afraid I had already taken the steps you outlined here. Any further ideas would be appreciated!
Per Caddy documentation, port 80 is also required, and now I suspect the not serving that port is causing Caddy to fail to issue you a tls certificate.
Try adding a simple text response like this (warning, formatting may not be perfect due to typing on mobile). Also setup a port forward on your router to your caddy host on port 80.
my-domain.com:80 { respond “Buzz off” }
Hopefully this will kick off the tls registration and then get your site on 443 working as well.