A quick note / command snippet for filtering ICMPv6 packets when troubleshooting IPv6 autoconfiguration.
# tcpdump -i eth0 -v '(icmp6 && ip6[40] == 134) or (icmp6 && ip6[40] == 133) or (icmp6 && ip6[40] == 135) or (icmp6 && ip6[40] == 136)'
The most common ICMPv6 types are:
Posted in Blog
Legacy IPv4 network space is exhausted, and new internet service providers like T-mobile 5G home internet run an IPv6 only network.
Sometimes you can’t use the IPv6 addresses due to your ISP not being mature enough to provide IPv6 prefix delegation in combination with the lack of advanced features on the T-mobile 5G home router like ‘bridge mode’ you may be stuck with a double NAT situation.
In 2021, I would say that implementing IPv6 on your network is almost mandatory. Major service providers like Netflix, Google, Cloudflare and others operate dual-stack networks.
I’ll admit that my home network running OPNsense firewall has had IPv6 disabled to avoid complexities since only 30% of the internet uses IPv6 today. However, I decided that its time to try to fix this since I wanted all my devices on my network to be able to pass IPv6 readiness tests like http://ipv6-test.com and http://test-ipv6.com.
Router: Nokia 5G Fastmile (T-mobile home internet)
Provides all direct-connected devices with a NAT IPv4 addresses in the range of 192.168.12.0/24.
IPv6 addresses are from the public unicast T-mobile segment assigned to your router, however T-mobile IPv6 network does heavy inbound packet filtering. You won’t be able to ping or access your public unicast IPv6 addresses from anywhere (sucks right?)
To configure a dual-stack local area network in OPNSENSE, assume you have a ‘fresh install’ which already has IPv6 enabled on the system.
Your clients on LAN should now get a local IPv6 address from DHCPv6, the outbound NAT rule should allow you to pass the IPv6 tests and communicate with IPv6 internet hosts via the single IPv6 WAN address that the T-mobile home router gives out.
I’m hoping that T-mobile becomes a more mature ISP in the future, while speeds are great there are several shortcomings to relying on 5G Home internet service for home use. These problems don’t exist in Comcast and AT&T Fiber:
– IPv6 Prefix Delegation (PD) so you can subnet all your networks.
– Packet filtering that you can’t disable or manage yourself. On AT&T fiber and comcast you can host your own remotely accessible VPN server. Not on T-mobile so beware if that’s your use case.
Posted in Blog
Tagged 5g home internet, ipv6 prefix delegation, nokia router, packet filtering, tmobile
The following command will allow host 192.168.1.10 to have write access and mount this volume remotely, all while identifying changes as ‘root’ – this is helpful for a container data store when you have docker running on a VM in proxmox but want to piggyback from the resilient ZFS storage in proxmox.
zfs set sharenfs="[email protected]" gdata/containersPosted in Blog
I’m experimenting with TunSafe which is a fork of wireguard that includes traffic obfuscation built-in. Traffic obfuscation is useful to hide VPN traffic from ISP’s network throttling which is done when network packet inspection identifies a tunnel protocol.
You will need a privileged container in proxmox.
Add this to container configuration file
lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=file
Without this setting on the configuration file of the container tunsafe will not start. Error similar to:
~# systemctl status tunsafe.service
● tunsafe.service - TunSafe service
Loaded: loaded (/etc/systemd/system/tunsafe.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2021-01-24 19:22:59 UTC; 26s ago
Process: 6949 ExecStart=/opt/tunsafe/tunsafe start -d /opt/tunsafe/TunSafe.conf (code=exited, status=1/FAILURE)
Main PID: 6949 (code=exited, status=1/FAILURE)
Jan 24 19:22:59 tunsafe-gw systemd[1]: Starting TunSafe service...
Jan 24 19:22:59 tunsafe-gw tunsafe[6949]: Loading file: /opt/tunsafe/TunSafe.conf
Jan 24 19:22:59 tunsafe-gw tunsafe[6949]: Started TCP listening socket on port 443
Jan 24 19:22:59 tunsafe-gw tunsafe[6949]: Error opening tun device
Jan 24 19:22:59 tunsafe-gw systemd[1]: tunsafe.service: Main process exited, code=exited, status=1/FAILURE
Jan 24 19:22:59 tunsafe-gw systemd[1]: tunsafe.service: Failed with result 'exit-code'.
Jan 24 19:22:59 tunsafe-gw systemd[1]: Failed to start TunSafe service.
Posted in Blog
You can speed up resilvering times on ZFS (on linux) by temporarily changing the following settings:
echo 0 > /sys/module/zfs/parameters/zfs_resilver_delay
echo 512 > /sys/module/zfs/parameters/zfs_top_maxinflight
echo 5000 > /sys/module/zfs/parameters/zfs_resilver_min_time_msTo reset back to defaults, do:
echo 2 > /sys/module/zfs/parameters/zfs_resilver_delay
echo 32 > /sys/module/zfs/parameters/zfs_top_maxinflight
echo 3000 > /sys/module/zfs/parameters/zfs_resilver_min_time_msA reboot of the system should also reset these defaults.
Posted in Blog
If you are like me, you wouldn’t want anyone with console access to your server to be able to have root credentials to the database engine if they own your server. The use of “mysql -u root” should not by default grant you root on the database server.
Well apparently in MariaDB 10.1.xx I found out this was the case, because of something they call “plugin” feature of “unix_socket” which is the default.
If you are seeing this on your system and want to confirm that’s why, use the following query in the mysql shell:
MariaDB [(none)]> select Host,User, Password,plugin from mysql.user;
+-----------+------+-------------------------------------------+-------------+
| Host | User | Password | plugin |
+-----------+------+-------------------------------------------+-------------+
| localhost | root | *DEEF4D7D88CD046ECA02A80393B7780A63E7E789 | unix_socket |
+-----------+------+-------------------------------------------+-------------+
So to fix this, reset or update your root password first then execute this inside the mysql shell:
UPDATE mysql.user SET plugin = '' WHERE user = 'root' AND host = 'localhost';Posted in Blog
Due to built-in security of LXC, trying to setup a tunnel interface inside a container is by blocked by default.
ERROR: Cannot open TUN/TAP dev /dev/net/tun
To allow this for a specific container in Proxmox, we need to make a few tweaks to allow this interface to work in a specific container (we don’t want to allow all containers to be able to setup a tunnel – hackers can hide their tracks using it).
How to do this:
ADD these lines to /etc/pve/lxc/<container-id>.conf
lxc.cgroup.devices.allow = c 10:200 rwm
lxc.hook.autodev = sh -c "modprobe tun; cd ${LXC_ROOTFS_MOUNT}/dev; mkdir net; mknod net/tun c 10 200; chmod 0666 net/tun"
Quick post to note how I determined and then fixed the internet access issue I was having when I installed OPNsense on Proxmox.
OPNsense virtual machine is configured with VirtiO network drivers.
Other than the obvious “I can’t access anything on the internet” or can’t reach external IP addresses problem I looked at troubleshooting via nmap – because the devices on the network could ping externally (8.8.8.8) and also resolve DNS requests.
In a broken state you may see ‘tcpwrapper’ when testing a known host serving HTTP, like so:
root@test:~# nmap -p 80 -sV 216.58.194.206
Starting Nmap 7.40 ( https://nmap.org ) at 2018-11-17 17:54 UTC
Nmap scan report for sfo03s01-in-f206.1e100.net (216.58.194.206)
Host is up (0.010s latency).
PORT STATE SERVICE VERSION
80/tcp open tcpwrapped
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 7.91 seconds
To fix this issue, ensure that “Disable hardware checksum offload” is enabled in the OPNsense interface, then reboot the firewall for changes to take effect.
After a reboot, doing another test via nmap will actually respond with HTTP fingerprints, as expected and internet is back.
root@test:~# nmap -p 80 -sV 216.58.194.206
Starting Nmap 7.40 ( https://nmap.org ) at 2018-11-17 18:00 UTC
Nmap scan report for sfo03s01-in-f14.1e100.net (216.58.194.206)
Host is up (0.0096s latency).
PORT STATE SERVICE VERSION
80/tcp open http gws
1 service unrecognized despite returning data. If you know the service/version, please submit the following fingerprint at https://nmap.org/cgi-bin/submit.cgi?new-service :
SF-Port80-TCP:V=7.40%I=7%D=11/17%Time=5BF0574C%P=x86_64-pc-linux-gnu%r(Get
SF:Request,8A7A,"HTTP/1\.0\x20200\x20OK\r\nDate:\x20Sat,\x2017\x20Nov\x202
SF:018\x2018:00:43\x20GMT\r\nExpires:\x20-1\r\nCache-Control:\x20private,\
SF:x20max-age=0\r\nContent-Type:\x20text/html;\x20charset=ISO-8859-1\r\nP3
SF:P:\x20CP=\"This\x20is\x20not\x20a\x20P3P\x20policy!\x20See\x20g\.co/p3p
SF:help\x20for\x20more\x20info\.\"\r\nServer:\x20gws\r\nX-XSS-Protection:\
SF:x201;\x20mode=block\r\nX-Frame-Options:\x20SAMEORIGIN\r\nSet-Cookie:\x2
SF:01P_JAR=2018-11-17-18;\x20expires=Mon,\x2017-Dec-2018\x2018:00:43\x20GM
SF:T;\x20path=/;\x20domain=\.google\.com\r\nSet-Cookie:\x20NID=146=0dp1WLb
SF:UhFIr1MIVwhAglx_4O6x-0eJHrmYFTov9a3oFxE2-lZSUI_9mmKBFXQZjYbjKbSRiirLZ-U
SF:cfybTiNQR_vmHD2MY4RBHP-hj4K7oyQX4lXuCgrSU7ESRXiX2Jn0qwoLWvvEItnC2hgDHEb
SF:oLJffQrfiEazdGDp5XppPU;\x20expires=Sun,\x2019-May-2019\x2018:00:43\x20G
SF:MT;\x20path=/;\x20domain=\.google\.com;\x20HttpOnly\r\nAccept-Ranges:\x
SF:20none\r\nVary:\x20Accept-Encoding\r\n\r\n<!doctype\x20html><html\x20it
SF:emscope=\"\"\x20itemtype=\"http://schema\.org/WebPage\"\x20lang=\"en\">
SF:<head><meta\x20content=\"Search\x20the\x20world's\x20information,\x20in
SF:cluding\x20webpages,\x20images,\x20videos\x20and\x20more\.\x20Google\x2
SF:0has\x20ma")%r(HTTPOptions,71B,"HTTP/1\.0\x20405\x20Method\x20Not\x20Al
SF:lowed\r\nAllow:\x20GET,\x20HEAD\r\nDate:\x20Sat,\x2017\x20Nov\x202018\x
SF:2018:00:44\x20GMT\r\nContent-Type:\x20text/html;\x20charset=UTF-8\r\nSe
SF:rver:\x20gws\r\nContent-Length:\x201592\r\nX-XSS-Protection:\x201;\x20m
SF:ode=block\r\nX-Frame-Options:\x20SAMEORIGIN\r\n\r\n<!DOCTYPE\x20html>\n
SF:<html\x20lang=en>\n\x20\x20<meta\x20charset=utf-8>\n\x20\x20<meta\x20na
SF:me=viewport\x20content=\"initial-scale=1,\x20minimum-scale=1,\x20width=
SF:device-width\">\n\x20\x20<title>Error\x20405\x20\(Method\x20Not\x20Allo
SF:wed\)!!1</title>\n\x20\x20<style>\n\x20\x20\x20\x20\*{margin:0;padding:
SF:0}html,code{font:15px/22px\x20arial,sans-serif}html{background:#fff;col
SF:or:#222;padding:15px}body{margin:7%\x20auto\x200;max-width:390px;min-he
SF:ight:180px;padding:30px\x200\x2015px}\*\x20>\x20body{background:url\(//
SF:www\.google\.com/images/errors/robot\.png\)\x20100%\x205px\x20no-repeat
SF:;padding-right:205px}p{margin:11px\x200\x2022px;overflow:hidden}ins{col
SF:or:#777;text-decoration:none}a\x20img{border:0}@media\x20screen\x20and\
SF:x20\(max-width:772px\){body{background:none;margin-top:0;max-width:none
SF:;padding");
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 52.89 seconds
root@test:~#
Posted in Proxmox, Virtualization
Tagged firewall, networking, opnsense, troubleshooting
If you’re having issues with zfsonlinux and your pool not expanding after replacing your hard drives with larger ones then here is a trick to fix it. Continue reading
One of my Edimax wireless adapters fails to resume network connectivity when restoring the system from hibernation.
So I created a scheduled task that resets the device, after resuming from hibernate open your Event Viewer > System.
Look for event ID 27 – “The boot type was 0x2.” right click “Attach task to this event”
Run program: powershell.exe
Arguments: Restart-NetAdapter -InterfaceDescription ‘Edimax AC1750 Wi-Fi USB Adapter’ -Confirm:$false
This should fix the issue automatically after every reboot. Your interface description may be different, in powershell run “Get-NetAdapter” to get the device’s specific and edit the arguments above as needed.
