Using Rootless Podman - ZONEMD verification failed: verify DNSKEY RRset with trust anchor failed

[TheRettom]

I've tried to faithfully adapt the madnuttah compose.yaml into a Podman Quadlet .pod as a recursive resolver with good success. My lasting problem is this error:
unbound[2:0] warning: auth zone .: ZONEMD verification failed: verify DNSKEY RRset with trust anchor failed

I've tried a few different things based on what I've read, but I don't have details. Every .conf and directory is exactly as listed here with the exception of adding madnuttah's supplied redis.conf and the line root-hints: "/usr/local/unbound/root.hints to trust-anchor.conf, referencing a mounted file directly from this root.hints file.

My Quadlet files are listed here.

[madnuttah]

Hi, have you tried enabling the root.hints and updating them?

#19 (comment)

Will it spin up using the minimal config?

As this is related to podman, I'll convert this into a discussion.

[TheRettom]

Yessir. That's what I listed in the original post.

Even with a minimal configuration, which I'll upload to my Quadlet files so you can see, I still get the error. I'm unsure as to why this would continue to be a problem.

This is the full log output.

Mar 17 14:23:56 homeserver systemd[586]: Starting Unbound Recursive Caching Resolver...
░░ Subject: A start job for unit UNIT has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit UNIT has begun execution.
░░ 
░░ The job identifier is 2702784.
Mar 17 14:23:56 homeserver podman[2843065]: 2025-03-17 14:23:56.653685205 -0600 MDT m=+0.045003289 container create 1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb (image=docker.io/madnuttah/unbound:latest, name=unbound-test, org.opencontainers.image.base.name=madnuttah/unbound, org.opencontainers.image.summary=This distroless Unbound Docker image is based on Alpine Linux with focus on security, privacy, performance and a small image size. And with Pi-hole in mind., org.opencontainers.image.url=https://hub.docker.com/r/madnuttah/unbound, org.opencontainers.image.authors=madnuttah, org.opencontainers.image.licenses=MIT, org.opencontainers.image.created=Tue Feb 11 18:47:21 UTC 2025, org.opencontainers.image.title=madnuttah/unbound, org.opencontainers.image.description=Unbound is a validating, recursive, and caching DNS resolver., org.opencontainers.image.source=https://github.com/madnuttah/unbound-docker, org.opencontainers.image.version=1.22.0-2, PODMAN_SYSTEMD_UNIT=unbound.service)
Mar 17 14:23:56 homeserver podman[2843065]: 2025-03-17 14:23:56.634354288 -0600 MDT m=+0.025672380 image pull 9f4599aa398b231a1122513b0490ad88187a215a96132fa1bf322307be9506a2 docker.io/madnuttah/unbound:latest
Mar 17 14:23:56 homeserver podman[2843065]: 2025-03-17 14:23:56.7596931 -0600 MDT m=+0.151011207 container init 1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb (image=docker.io/madnuttah/unbound:latest, name=unbound-test, org.opencontainers.image.version=1.22.0-2, PODMAN_SYSTEMD_UNIT=unbound.service, org.opencontainers.image.base.name=madnuttah/unbound, org.opencontainers.image.created=Tue Feb 11 18:47:21 UTC 2025, org.opencontainers.image.summary=This distroless Unbound Docker image is based on Alpine Linux with focus on security, privacy, performance and a small image size. And with Pi-hole in mind., org.opencontainers.image.title=madnuttah/unbound, org.opencontainers.image.description=Unbound is a validating, recursive, and caching DNS resolver., org.opencontainers.image.url=https://hub.docker.com/r/madnuttah/unbound, org.opencontainers.image.authors=madnuttah, org.opencontainers.image.licenses=MIT, org.opencontainers.image.source=https://github.com/madnuttah/unbound-docker)
Mar 17 14:23:56 homeserver podman[2843065]: 2025-03-17 14:23:56.763524952 -0600 MDT m=+0.154843031 container start 1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb (image=docker.io/madnuttah/unbound:latest, name=unbound-test, org.opencontainers.image.description=Unbound is a validating, recursive, and caching DNS resolver., org.opencontainers.image.authors=madnuttah, org.opencontainers.image.source=https://github.com/madnuttah/unbound-docker, org.opencontainers.image.created=Tue Feb 11 18:47:21 UTC 2025, org.opencontainers.image.summary=This distroless Unbound Docker image is based on Alpine Linux with focus on security, privacy, performance and a small image size. And with Pi-hole in mind., org.opencontainers.image.title=madnuttah/unbound, org.opencontainers.image.licenses=MIT, org.opencontainers.image.version=1.22.0-2, PODMAN_SYSTEMD_UNIT=unbound.service, org.opencontainers.image.base.name=madnuttah/unbound, org.opencontainers.image.url=https://hub.docker.com/r/madnuttah/unbound)
Mar 17 14:23:56 homeserver systemd[586]: Started Unbound Recursive Caching Resolver.
░░ Subject: A start job for unit UNIT has finished successfully
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit UNIT has finished successfully.
░░ 
░░ The job identifier is 2702784.
Mar 17 14:23:56 homeserver unbound-test[2843113]: ╔═════════════════════════════════════════════════════╗
Mar 17 14:23:56 homeserver unbound-test[2843113]: ║                                                     ║
Mar 17 14:23:56 homeserver unbound[2843065]: 1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb
Mar 17 14:23:56 homeserver unbound-test[2843113]: ║                  MΛDИVTTΛH Unbound                  ║
Mar 17 14:23:56 homeserver unbound-test[2843113]: ║                                                     ║
Mar 17 14:23:56 homeserver unbound-test[2843113]: ║     https://github.com/madnuttah/unbound-docker     ║
Mar 17 14:23:56 homeserver unbound-test[2843113]: ║     https://hub.docker.com/r/madnuttah/unbound      ║
Mar 17 14:23:56 homeserver unbound-test[2843113]: ║                                                     ║
Mar 17 14:23:56 homeserver unbound-test[2843113]: ╚═════════════════════════════════════════════════════╝
Mar 17 14:23:56 homeserver unbound-test[2843113]: 
Mar 17 14:23:56 homeserver unbound-test[2843113]: UNBOUND_UID: 1000
Mar 17 14:23:56 homeserver unbound-test[2843113]: UNBOUND_GID: 1000
Mar 17 14:23:56 homeserver unbound-test[2843113]: 
Mar 17 14:23:56 homeserver unbound-test[2843113]: DISABLE_SET_PERMS: false
Mar 17 14:23:56 homeserver unbound-test[2843113]: 
Mar 17 14:23:56 homeserver unbound-test[2843113]: Mar 17 14:23:56 unbound[2:0] notice: init module 0: validator
Mar 17 14:23:56 homeserver unbound-test[2843113]: Mar 17 14:23:56 unbound[2:0] notice: init module 1: iterator
Mar 17 14:23:57 homeserver unbound-test[2843113]: Mar 17 14:23:57 unbound[2:0] warning: auth zone .: ZONEMD verification failed: verify DNSKEY RRset with trust anchor failed
Mar 17 14:23:57 homeserver unbound-test[2843113]: Mar 17 14:23:57 unbound[2:0] info: start of service (unbound 1.22.0).

This is the full `podman container inspect` result.

[
     {
          "Id": "1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb",
          "Created": "2025-03-17T14:23:56.638536174-06:00",
          "Path": "/sbin/tini",
          "Args": [
               "--",
               "/entrypoint"
          ],
          "State": {
               "OciVersion": "1.2.0",
               "Status": "running",
               "Running": true,
               "Paused": false,
               "Restarting": false,
               "OOMKilled": false,
               "Dead": false,
               "Pid": 2843115,
               "ConmonPid": 2843113,
               "ExitCode": 0,
               "Error": "",
               "StartedAt": "2025-03-17T14:23:56.763474506-06:00",
               "FinishedAt": "0001-01-01T00:00:00Z",
               "CgroupPath": "/user.slice/user-1000.slice/user@1000.service/app.slice/unbound.service/libpod-payload-1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb",
               "CheckpointedAt": "0001-01-01T00:00:00Z",
               "RestoredAt": "0001-01-01T00:00:00Z"
          },
          "Image": "9f4599aa398b231a1122513b0490ad88187a215a96132fa1bf322307be9506a2",
          "ImageDigest": "sha256:08a4f72b57990318e583bf6a6fa650b5c985f46cdd07804b410e895d26150598",
          "ImageName": "docker.io/madnuttah/unbound:latest",
          "Rootfs": "",
          "Pod": "",
          "ResolvConfPath": "/run/user/1000/containers/overlay-containers/1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb/userdata/resolv.conf",
          "HostnamePath": "/run/user/1000/containers/overlay-containers/1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb/userdata/hostname",
          "HostsPath": "/run/user/1000/containers/overlay-containers/1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb/userdata/hosts",
          "StaticDir": "/home/riley/.local/share/containers/storage/overlay-containers/1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb/userdata",
          "OCIConfigPath": "/home/riley/.local/share/containers/storage/overlay-containers/1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb/userdata/config.json",
          "OCIRuntime": "crun",
          "ConmonPidFile": "/run/user/1000/containers/overlay-containers/1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb/userdata/conmon.pid",
          "PidFile": "/run/user/1000/containers/overlay-containers/1d4c2876abbed3bc8528832eac0cb8dd8321c6581cb468e0b260f2f6b2e551fb/userdata/pidfile",
          "Name": "unbound-test",
          "RestartCount": 0,
          "Driver": "overlay",
          "MountLabel": "",
          "ProcessLabel": "",
          "AppArmorProfile": "",
          "EffectiveCaps": [
               "CAP_CHOWN",
               "CAP_DAC_OVERRIDE",
               "CAP_FOWNER",
               "CAP_FSETID",
               "CAP_KILL",
               "CAP_NET_BIND_SERVICE",
               "CAP_SETFCAP",
               "CAP_SETGID",
               "CAP_SETPCAP",
               "CAP_SETUID",
               "CAP_SYS_CHROOT"
          ],
          "BoundingCaps": [
               "CAP_CHOWN",
               "CAP_DAC_OVERRIDE",
               "CAP_FOWNER",
               "CAP_FSETID",
               "CAP_KILL",
               "CAP_NET_BIND_SERVICE",
               "CAP_SETFCAP",
               "CAP_SETGID",
               "CAP_SETPCAP",
               "CAP_SETUID",
               "CAP_SYS_CHROOT"
          ],
          "ExecIDs": [],
          "GraphDriver": {
               "Name": "overlay",
               "Data": {
                    "LowerDir": "/home/riley/.local/share/containers/storage/overlay/f8c5f53fde15ec8f365a3dcf4c38edb03561211a486a2a9b9f4b843d7f789204/diff",
                    "MergedDir": "/home/riley/.local/share/containers/storage/overlay/274d9079a700b36f90c65916cafdaef4cb54e812d6c64f8d205668ae2df8b80b/merged",
                    "UpperDir": "/home/riley/.local/share/containers/storage/overlay/274d9079a700b36f90c65916cafdaef4cb54e812d6c64f8d205668ae2df8b80b/diff",
                    "WorkDir": "/home/riley/.local/share/containers/storage/overlay/274d9079a700b36f90c65916cafdaef4cb54e812d6c64f8d205668ae2df8b80b/work"
               }
          },
          "Mounts": [],
          "Dependencies": [],
          "NetworkSettings": {
               "EndpointID": "",
               "Gateway": "",
               "IPAddress": "",
               "IPPrefixLen": 0,
               "IPv6Gateway": "",
               "GlobalIPv6Address": "",
               "GlobalIPv6PrefixLen": 0,
               "MacAddress": "",
               "Bridge": "",
               "SandboxID": "",
               "HairpinMode": false,
               "LinkLocalIPv6Address": "",
               "LinkLocalIPv6PrefixLen": 0,
               "Ports": {},
               "SandboxKey": "/run/user/1000/netns/netns-6cf64a09-deef-45fb-568a-5aaeec938e4e",
               "Networks": {
                    "dns": {
                         "EndpointID": "",
                         "Gateway": "172.17.0.1",
                         "IPAddress": "172.17.0.4",
                         "IPPrefixLen": 16,
                         "IPv6Gateway": "",
                         "GlobalIPv6Address": "",
                         "GlobalIPv6PrefixLen": 0,
                         "MacAddress": "a6:97:6a:6f:85:90",
                         "NetworkID": "d7048385c72e6370c260dd3dc915f8e3065875e04ae0a9baf05949b864733199",
                         "DriverOpts": null,
                         "IPAMConfig": null,
                         "Links": null,
                         "Aliases": [
                              "1d4c2876abbe",
                              "unbound-test"
                         ]
                    }
               }
          },
          "Namespace": "",
          "IsInfra": false,
          "IsService": false,
          "KubeExitCodePropagation": "invalid",
          "lockNumber": 18,
          "Config": {
               "Hostname": "unbound-test",
               "Domainname": "",
               "User": "",
               "AttachStdin": false,
               "AttachStdout": false,
               "AttachStderr": false,
               "Tty": false,
               "OpenStdin": false,
               "StdinOnce": false,
               "Env": [
                    "IMAGE_BUILD_DATE=Tue Feb 11 18:47:21 UTC 2025",
                    "UNBOUND_DOCKER_IMAGE_VERSION=1.22.0-2",
                    "OPENSSL_BUILDENV_VERSION=3.4.1-0",
                    "PATH=/usr/local/unbound/unbound.d/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                    "container=podman",
                    "UNBOUND_UID=1000",
                    "TZ=America/Boise",
                    "HOSTNAME=unbound-test",
                    "HOME=/root"
               ],
               "Cmd": null,
               "Image": "docker.io/madnuttah/unbound:latest",
               "Volumes": null,
               "WorkingDir": "/",
               "Entrypoint": [
                    "/sbin/tini",
                    "--",
                    "/entrypoint"
               ],
               "OnBuild": null,
               "Labels": {
                    "PODMAN_SYSTEMD_UNIT": "unbound.service",
                    "org.opencontainers.image.authors": "madnuttah",
                    "org.opencontainers.image.base.name": "madnuttah/unbound",
                    "org.opencontainers.image.created": "Tue Feb 11 18:47:21 UTC 2025",
                    "org.opencontainers.image.description": "Unbound is a validating, recursive, and caching DNS resolver.",
                    "org.opencontainers.image.licenses": "MIT",
                    "org.opencontainers.image.source": "https://github.com/madnuttah/unbound-docker",
                    "org.opencontainers.image.summary": "This distroless Unbound Docker image is based on Alpine Linux with focus on security, privacy, performance and a small image size. And with Pi-hole in mind.",
                    "org.opencontainers.image.title": "madnuttah/unbound",
                    "org.opencontainers.image.url": "https://hub.docker.com/r/madnuttah/unbound",
                    "org.opencontainers.image.version": "1.22.0-2"
               },
               "Annotations": {
                    "io.container.manager": "libpod",
                    "io.podman.annotations.autoremove": "TRUE",
                    "io.podman.annotations.cid-file": "/run/user/1000/unbound.cid",
                    "org.opencontainers.image.stopSignal": "15",
                    "org.systemd.property.KillSignal": "15",
                    "org.systemd.property.TimeoutStopUSec": "uint64 10000000"
               },
               "StopSignal": "SIGTERM",
               "HealthcheckOnFailureAction": "none",
               "HealthLogDestination": "local",
               "HealthcheckMaxLogCount": 5,
               "HealthcheckMaxLogSize": 500,
               "CreateCommand": [
                    "/usr/bin/podman",
                    "run",
                    "--name",
                    "unbound-test",
                    "--cidfile=/run/user/1000/unbound.cid",
                    "--replace",
                    "--rm",
                    "--cgroups=split",
                    "--hostname",
                    "unbound-test",
                    "--network",
                    "dns",
                    "--sdnotify=conmon",
                    "-d",
                    "--env",
                    "TZ=America/Boise",
                    "docker.io/madnuttah/unbound:latest"
               ],
               "Umask": "0022",
               "Timeout": 0,
               "StopTimeout": 10,
               "Passwd": true,
               "sdNotifyMode": "conmon",
               "sdNotifySocket": "/run/user/1000/systemd/notify"
          },
          "HostConfig": {
               "Binds": [],
               "CgroupManager": "systemd",
               "CgroupMode": "private",
               "ContainerIDFile": "/run/user/1000/unbound.cid",
               "LogConfig": {
                    "Type": "journald",
                    "Config": null,
                    "Path": "",
                    "Tag": "",
                    "Size": "0B"
               },
               "NetworkMode": "bridge",
               "PortBindings": {},
               "RestartPolicy": {
                    "Name": "no",
                    "MaximumRetryCount": 0
               },
               "AutoRemove": true,
               "AutoRemoveImage": false,
               "Annotations": {
                    "io.container.manager": "libpod",
                    "io.podman.annotations.autoremove": "TRUE",
                    "io.podman.annotations.cid-file": "/run/user/1000/unbound.cid",
                    "org.opencontainers.image.stopSignal": "15",
                    "org.systemd.property.KillSignal": "15",
                    "org.systemd.property.TimeoutStopUSec": "uint64 10000000"
               },
               "VolumeDriver": "",
               "VolumesFrom": null,
               "CapAdd": [],
               "CapDrop": [],
               "Dns": [],
               "DnsOptions": [],
               "DnsSearch": [],
               "ExtraHosts": [],
               "HostsFile": "",
               "GroupAdd": [],
               "IpcMode": "shareable",
               "Cgroup": "",
               "Cgroups": "default",
               "Links": null,
               "OomScoreAdj": 0,
               "PidMode": "private",
               "Privileged": false,
               "PublishAllPorts": false,
               "ReadonlyRootfs": false,
               "SecurityOpt": [],
               "Tmpfs": {},
               "UTSMode": "private",
               "UsernsMode": "",
               "ShmSize": 65536000,
               "Runtime": "oci",
               "ConsoleSize": [
                    0,
                    0
               ],
               "Isolation": "",
               "CpuShares": 0,
               "Memory": 0,
               "NanoCpus": 0,
               "CgroupParent": "",
               "BlkioWeight": 0,
               "BlkioWeightDevice": null,
               "BlkioDeviceReadBps": null,
               "BlkioDeviceWriteBps": null,
               "BlkioDeviceReadIOps": null,
               "BlkioDeviceWriteIOps": null,
               "CpuPeriod": 0,
               "CpuQuota": 0,
               "CpuRealtimePeriod": 0,
               "CpuRealtimeRuntime": 0,
               "CpusetCpus": "",
               "CpusetMems": "",
               "Devices": [],
               "DiskQuota": 0,
               "KernelMemory": 0,
               "MemoryReservation": 0,
               "MemorySwap": 0,
               "MemorySwappiness": 0,
               "OomKillDisable": false,
               "PidsLimit": 2048,
               "Ulimits": [
                    {
                         "Name": "RLIMIT_NOFILE",
                         "Soft": 524288,
                         "Hard": 524288
                    },
                    {
                         "Name": "RLIMIT_NPROC",
                         "Soft": 63228,
                         "Hard": 63228
                    }
               ],
               "CpuCount": 0,
               "CpuPercent": 0,
               "IOMaximumIOps": 0,
               "IOMaximumBandwidth": 0,
               "CgroupConf": null
          },
          "UseImageHosts": false,
          "UseImageHostname": false
     }
]

[madnuttah]

Your unbound configs are nice? docker exec -ti unbound unbound-checkconf

I suppose your date and time is set correctly...

[TheRettom]

unbound-checkconf: no errors in /usr/local/unbound/unbound.conf

[madnuttah]

When issuing docker exec -ti unbound unbound-anchor what's the result?

[TheRettom]

/usr/local/unbound/iana.d/root.key has content

success: the anchor is ok

[madnuttah]

• Might there some issues in your network? What is a leaktest stating?
• What's the outcome when using plain docker?

Other than that I'm currently running out of ideas.

[TheRettom]

I have not had any issues with this specific network. I don't use Docker anymore, but I can attempt to use host networking on Podman, and if that doesn't work, I'll use Docker. I won't be surprised if Docker works fine, but I really don't know what the problem is.

Edit: Host networking did not fix it. Running in regular Docker, there was no error for it. That made me believe that somehow there was a permission problem. I went back to Podman and added CAP_SYS_ADMIN for kicks and giggles. Oddly enough, it didn't correct the issue.

However, after adding DISABLE_SET_PERMS=true, the error is no longer showing up. Since it is running inside the container as root, there is no error for it. Interestingly enough I no longer get:

unbound[2:0] notice: init module 0: validator
unbound[2:0] notice: init module 1: iterator
unbound[2:0] info: start of service (unbound 1.22.0).

Since the container is minimal, I can only do tests through Pihole's container.

These are some of the results of `dig`:

$ podman exec -it pihole dig dnssec.works @172.17.0.10 -p 5335

; <<>> DiG 9.16.48-Debian <<>> dnssec.works @172.17.0.10 -p 5335
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49393
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1472
;; QUESTION SECTION:
;dnssec.works.                  IN      A

;; ANSWER SECTION:
dnssec.works.           3600    IN      A       46.23.92.212

;; Query time: 132 msec
;; SERVER: 172.17.0.10#5335(172.17.0.10)
;; WHEN: Wed Mar 19 22:33:00 MDT 2025
;; MSG SIZE  rcvd: 57

$ podman exec -it pihole dig fail01.dnssec.works @172.17.0.10 -p 5335

; <<>> DiG 9.16.48-Debian <<>> fail01.dnssec.works @172.17.0.10 -p 5335
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 45908
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1472
;; QUESTION SECTION:
;fail01.dnssec.works.           IN      A

;; Query time: 125 msec
;; SERVER: 172.17.0.10#5335(172.17.0.10)
;; WHEN: Wed Mar 19 22:32:17 MDT 2025
;; MSG SIZE  rcvd: 48

$ podman exec -it pihole dig com. SOA +dnssec

; <<>> DiG 9.16.48-Debian <<>> com. SOA +dnssec
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61487
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags: do; udp: 512
;; QUESTION SECTION:
;com.                           IN      SOA

;; ANSWER SECTION:
com.                    51      IN      SOA     a.gtld-servers.net. nstld.verisign-grs.com. 1742444037 1800 900 604800 900
com.                    51      IN      RRSIG   SOA 13 1 900 20250327041357 20250320030357 23202 com. JPHteEfK+Q+kXFI4huY3C30RzlZBf5MGNFLZlDyF
+yPMfQpWPU08Y35h qhnok8+iBvbn/LQi/HGNnD4Cr8DAqA==

;; Query time: 20 msec
;; SERVER: 172.17.0.1#53(172.17.0.1)
;; WHEN: Wed Mar 19 22:28:29 MDT 2025
;; MSG SIZE  rcvd: 204

Any ideas?

[madnuttah]

I don't suspect the docker network mode to be the culprit. You could use the included NLnet Labs drill command.

docker exec -ti unbound drill ...

[TheRettom]

So, after much debugging on my end, I found that I had to manually mount an up-to-date version of root.zone. This removes the error, but using podman exec -ti unbound-server drill -DT badsig.go.dnscheck.tools, I get:

Output

$ podman exec -ti unbound-server drill -DT badsig.go.dnscheck.tools
Warning: No trusted keys were given. Will not be able to verify authenticity!
;; Domain: .
;; Signature ok but no chain to a trusted key or ds record
[S] . 172800 IN DNSKEY 257 3 8 ;{id = 20326 (ksk), size = 2048b}
. 172800 IN DNSKEY 256 3 8 ;{id = 26470 (zsk), size = 2048b}
. 172800 IN DNSKEY 256 3 8 ;{id = 53148 (zsk), size = 2048b}
. 172800 IN DNSKEY 257 3 8 ;{id = 38696 (ksk), size = 2048b}
Checking if signing key is trusted:
New key: .      172800  IN      DNSKEY  256 3 8 AwEAAZ5A7jOztf62cGqhPhutjnyl7KBjIsjbyTb8il+FsgbMUbO2NQHaSbatHdlOlqANncDwSIKZ9ryqd1+Dy1PoGzeTUv95vOJnVVJHlJu7xdavnUmPs+Mh2NV7hDlTTwPn5uXgFxAaxoO9M/YIAC92GryCLjoJEg9JzeevkktEM/sFpmRv4I5jQtlLyRqVbnCzcWpi04XaVLxRKvURkd/Mdb/2RQS3MYvrkEBXuqtnAVBCf6Fx4sgBYOfYvbUuG2diLnGJW/MXvFpctZgQ76+3FwMqAZfR9k5bohL7AF3+jqz4MUiootYoh5koyt7VEnUULxxy6U5PINTGgOC26f3zZuk= ;{id = 26470 (zsk), size = 2048b}
[S] tools. 86400 IN DS 13831 8 2 5ddd852d119830be951afa0d5176443a935eaa350278ef15fbea34156cf5a9ab
;; Domain: tools.
;; Signature ok but no chain to a trusted key or ds record
[S] tools. 3600 IN DNSKEY 256 3 8 ;{id = 7146 (zsk), size = 1024b}
tools. 3600 IN DNSKEY 256 3 8 ;{id = 30999 (zsk), size = 1024b}
tools. 3600 IN DNSKEY 257 3 8 ;{id = 13831 (ksk), size = 2048b}
Checking if signing key is trusted:
New key: tools. 3600    IN      DNSKEY  256 3 8 AwEAAcyiyB+yxjFRaJDjTmEpRSAGIneXKcoUKgHPiK0BbotB9tGipCwFY1LFt0nrP4qu+/g/TXgYq/lJmoWwO+dCozkCz4sB+0XDiKWeZOTnNT+HSIlyR12PSxD72EhX/vb3YzTDW3NQeHX5hNgFnP1i1F+g5Zth4dmzNpSHStcMO/dz ;{id = 7146 (zsk), size = 1024b}
[S] dnscheck.tools. 3600 IN DS 2371 13 2 607d77710a6af5b28007b0302665d57e21194fa8a9e2a0d3d085ca55d320665f
;; Domain: dnscheck.tools.
;; Signature ok but no chain to a trusted key or ds record
[S] dnscheck.tools. 3600 IN DNSKEY 257 3 13 ;{id = 2371 (ksk), size = 256b}
dnscheck.tools. 3600 IN DNSKEY 256 3 13 ;{id = 34505 (zsk), size = 256b}
Checking if signing key is trusted:
New key: dnscheck.tools.        3600    IN      DNSKEY  256 3 13 oJMRESz5E4gYzS/q6XDrvU1qMPYIjCWzJaOau8XNEZeqCYKD5ar0IRd8KqXXFJkqmVfRvMGPmM1x8fGAa2XhSA== ;{id = 34505 (zsk), size = 256b}
[S] go.dnscheck.tools. 300 IN DS 997 13 2 36dd97b0fdad2b01e299f116b74e46f11f19cd80dc6ed5a881bc73ffd32a459f
;; Domain: go.dnscheck.tools.
;; Signature ok but no chain to a trusted key or ds record
[S] go.dnscheck.tools. 300 IN DNSKEY 257 3 13 ;{id = 997 (ksk), size = 256b}
go.dnscheck.tools. 300 IN DNSKEY 256 3 13 ;{id = 35243 (zsk), size = 256b}
[B] Error verifying denial of existence for badsig.go.dnscheck.tools. DS: Bogus DNSSEC signature
;; No ds record for delegation
;; Domain: badsig.go.dnscheck.tools.
;; No DNSKEY record found for badsig.go.dnscheck.tools.
[B] badsig.go.dnscheck.tools.   1       IN      A       116.203.95.251
;; Error: Bogus DNSSEC signature
;;[S] self sig OK; [B] bogus; [T] trusted; [U] unsigned

$ podman exec -ti unbound-server unbound-checkconf
unbound-checkconf: no errors in /usr/local/unbound/unbound.conf

$ podman exec -ti unbound-server unbound-anchor -v
/usr/local/unbound/iana.d/root.key has content
success: the anchor is ok

So it appears that root.key isn't working, either. Why would I need to manually mount an up-to-date root.hints, root.zone, and now, somehow, get or fix the root.key, even though unbound-anchor is finding root.key to be fine?

I'm really confused as to why manually mounting root.key and root.zone is even necessary when it is provided in the image.

Any ideas from here?

Edit: Read the next posts for why this was the solution.

Tl;dr: Both root.zone and root.hints are needed, but root.key works fine.

[madnuttah]

That's strange indeed. The root.zone, yeah fine, but the root.hints shouldn't be needed....

[TheRettom]

I'm trying to recall what would be different about Docker versus rootless Podman other than the daemon and non-root privileges. Nothing really comes to mind, so the issues are very intriguing, especially when 'CAP_SYS_ADMIN' did nothing to help. I'm not sure how much you know about Unbound's general operations from its code and how it works in your image, but it is definitely way more than what I know.

[madnuttah]

Related? https://superuser.com/questions/1701562/dnssec-signature-ok-but-no-chain-to-a-trusted-key-or-ds-record

I suspect everything is working as it should after updating the root.zone...

[TheRettom]

New output.

$ podman exec -ti unbound-server drill -DT -k /usr/local/unbound/iana.d/root.key badsig.go.dnscheck.tools

;; Number of trusted keys: 2
;; Domain: .
[T] . 172800 IN DNSKEY 256 3 8 ;{id = 26470 (zsk), size = 2048b}
. 172800 IN DNSKEY 257 3 8 ;{id = 20326 (ksk), size = 2048b}
. 172800 IN DNSKEY 257 3 8 ;{id = 38696 (ksk), size = 2048b}
. 172800 IN DNSKEY 256 3 8 ;{id = 53148 (zsk), size = 2048b}
Checking if signing key is trusted:
New key: .      172800  IN      DNSKEY  256 3 8 AwEAAZ5A7jOztf62cGqhPhutjnyl7KBjIsjbyTb8il+FsgbMUbO2NQHaSbatHdlOlqANncDwSIKZ9ryqd1+Dy1PoGzeTUv
95vOJnVVJHlJu7xdavnUmPs+Mh2NV7hDlTTwPn5uXgFxAaxoO9M/YIAC92GryCLjoJEg9JzeevkktEM/sFpmRv4I5jQtlLyRqVbnCzcWpi04XaVLxRKvURkd/Mdb/2RQS3MYvrkEBXuqtn
AVBCf6Fx4sgBYOfYvbUuG2diLnGJW/MXvFpctZgQ76+3FwMqAZfR9k5bohL7AF3+jqz4MUiootYoh5koyt7VEnUULxxy6U5PINTGgOC26f3zZuk= ;{id = 26470 (zsk), size = 20
48b}
        Trusted key: .  86400   IN      DNSKEY  257 3 8 AwEAAa96jeuknZlaeSrvyAJj6ZHv28hhOKkx3rLGXVaC6rXTsDc449/cidltpkyGwCJNnOAlFNKF2jBosZBU5e
eHspaQWOmOElZsjICMQMC3aeHbGiShvZsx4wMYSjH8e7Vrhbu6irwCzVBApESjbUdpWWmEnhathWu1jo+siFUiRAAxm9qyJNg/wOZqqzL/dL/q8PkcRU5oUKEpUge71M3ej2/7CPqpdVwu
MoTvoB+ZOT4YeGyxMvHmbrxlFzGOHOijtzN+u1TQNatX2XBuzZNQ1K+s2CXkPIZo7s6JgZyvaBevYtxPvYLw4z9mR7K2vaF18UYH9Z9GNUUeayffKC73PYc= ;{id = 38696 (ksk), s
ize = 2048b}
        Trusted key: .  86400   IN      DNSKEY  257 3 8 AwEAAaz/tAm8yTn4Mfeh5eyI96WSVexTBAvkMgJzkKTOiW1vkIbzxeF3+/4RgWOq7HrxRixHlFlExOLAJr5emL
vN7SWXgnLh4+B5xQlNVz8Og8kvArMtNROxVQuCaSnIDdD5LKyWbRd2n9WGe2R8PzgCmr3EgVLrjyBxWezF0jLHwVN8efS3rCj/EWgvIWgb9tarpVUDK/b58Da+sqqls3eNbuv7pr+eoZG+
SrDK6nWeL3c6H5Apxz7LjVc1uTIdsIXxuOLYA4/ilBmSVIzuDWfdRUfhHdY6+cn8HFRm+2hM8AnXGXws9555KrUB5qihylGa8subX2Nn6UwNR1AkUTV74bU= ;{id = 20326 (ksk), s
ize = 2048b}
        Trusted key: .  172800  IN      DNSKEY  256 3 8 AwEAAZ5A7jOztf62cGqhPhutjnyl7KBjIsjbyTb8il+FsgbMUbO2NQHaSbatHdlOlqANncDwSIKZ9ryqd1+Dy1
PoGzeTUv95vOJnVVJHlJu7xdavnUmPs+Mh2NV7hDlTTwPn5uXgFxAaxoO9M/YIAC92GryCLjoJEg9JzeevkktEM/sFpmRv4I5jQtlLyRqVbnCzcWpi04XaVLxRKvURkd/Mdb/2RQS3MYvr
kEBXuqtnAVBCf6Fx4sgBYOfYvbUuG2diLnGJW/MXvFpctZgQ76+3FwMqAZfR9k5bohL7AF3+jqz4MUiootYoh5koyt7VEnUULxxy6U5PINTGgOC26f3zZuk= ;{id = 26470 (zsk), s
ize = 2048b}
Key is now trusted!

This is truncated.

So I guess this issue is now resolved? How can I confirm that Pihole would be correctly utilizing DNSSEC, or is it mostly on Unbound to do that?

Still, given that it appears to work, I can't come to any real conclusion as to why I have to manually mount those updated root. files.

[madnuttah]

It seems like it has been resolved.

In some cases the root.zone file needs to be manually updated, especially when the iana.d folder got exposed if I recall correctly.

Thanks for taking the time and for using our image. 💚

[TheRettom]

Aye, sir. Thanks for taking time to help resolve a niche use-case, and for creating the image in the first place.