I would compare the two IPs you get. I'm guessing you get a different IP over the wifi.

Mostly likely your local IP for the dedi looks something like: 192.168.1.x
The IP you get on you PC is also something like: 192.168.1.x

When you go wifi, you probably have something like 50.125.x.x

So this shows that you are now on a different subnet than your dedi server and you now need to open the ports for the dedi, same as if you had a friend in another state playing on your dedi. You will also want to use the public IP (the one that looks something like 50.125.x.x) instead of the local IP.

BTW, you can get help with stuff like this on the Valheim discord. Much better to solve this in chat interactively than through a steam forum.