Jun 11 12:15:54 gserver xhci_do_command: Command timeout!
Jun 11 12:15:54 gserver xhci_do_command: Controller reset!
Jun 11 12:15:54 gserver xhci0: Resetting controller
Jun 11 12:15:54 gserver usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
Jun 11 12:15:54 gserver xhci_device_state_change:
Jun 11 12:15:54 gserver ugen0.2: <Unknown> at usbus0 (disconnected)
Jun 11 12:15:54 gserver uhub_reattach_port: could not allocate new device
Jun 11 12:15:54 gserver xhci_root_intr: port 2 changed
Jun 11 12:15:55 gserver xhci_root_intr: port 2 changed
Jun 11 12:15:55 gserver xhci_root_intr: port 4 changed
Jun 11 12:15:55 gserver xhci_ep_clear_stall:
Jun 11 12:15:55 gserver xhci_cmd_enable_slot:
Jun 11 12:15:55 gserver xhci_do_command: Command timeout!
Jun 11 12:15:55 gserver xhci_reset_command_queue_locked:
Jun 11 12:15:55 gserver xhci_reset_command_queue_locked: Command ring running
Jun 11 12:15:55 gserver xhci_reset_command_queue_locked: CRCR=0x00000000057cfd80
Jun 11 12:15:55 gserver xhci_do_command: Command timeout!
Jun 11 12:15:55 gserver xhci_do_command: Controller reset!
Jun 11 12:15:55 gserver usb_alloc_device: device init 2 failed (USB_ERR_TIMEOUT, ignored)
Jun 11 12:15:55 gserver xhci_device_state_change:
Jun 11 12:15:55 gserver ugen0.2: <Unknown> at usbus0 (disconnected)
Jun 11 12:15:55 gserver uhub_reattach_port: could not allocate new device
Jun 11 12:15:55 gserver uhub0: at usbus0, port 1, addr 1 (disconnected)
Jun 11 12:15:55 gserver xhci_set_hw_power:
Jun 11 12:15:55 gserver xhci_set_hw_power_sleep: Stopping the XHCI
Jun 11 12:15:55 gserver xhci_halt_controller:
Jun 11 12:15:55 gserver xhci_set_hw_power_sleep: Starting the XHCI
Jun 11 12:15:55 gserver xhci_start_controller:
Jun 11 12:15:56 gserver xhci_start_controller: CONFIG=0x00000000 -> 0x00000020
Jun 11 12:15:56 gserver xhci_start_controller: ERSTSZ=0x00000000 -> 0x00000001
Jun 11 12:15:56 gserver xhci_start_controller: ERDP(0)=0x00000000057cf080
Jun 11 12:15:56 gserver xhci_start_controller: ERSTBA(0)=0x00000000057cf000
Jun 11 12:15:56 gserver xhci_start_controller: CRCR=0x00000000057cfd80
Jun 11 12:15:56 gserver xhci_set_hw_power:
Jun 11 12:15:56 gserver uhub0: <0x1912 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
Jun 11 12:15:56 gserver uhub0: 4 ports with 4 removable, self powered
Jun 11 12:15:56 gserver xhci_set_hw_power:
Jun 11 12:15:56 gserver xhci_root_intr: port 2 changed
Jun 11 12:15:56 gserver xhci_root_intr: port 4 changed
Jun 11 12:15:56 gserver xhci_ep_clear_stall:
Jun 11 12:15:56 gserver xhci_cmd_enable_slot:
Jun 11 12:15:57 gserver xhci_do_command: Command timeout!

Trying the following without rebooting did not work

[root@gserver] ~# sysctl hw.usb.xhci.xhci_port_route=-1

I enabled debugging to get the logs:

[root@gserver] ~# sysctl hw.usb.xhci.debug=1

Apparently we need to add this to: /boot/loader.conf and reboot then try again.

# Workaround desantolo.com USB3 loop
sysctl hw.usb.xhci.xhci_port_route=-1

I have a pair of Samsung 840 Pro 256GB SSDs that I wanted to use for my new homelab that I am currently building (moving from vmware to proxmox). You may be wondering why I want to install the operating system on a partition instead of an entire disk. Several reasons:

1. Proxmox (ZFS-on-Linux) does not yet support SSD TRIM, FreeBSD does support it so migrating from FreeNAS into Proxmox I should be aware of it.
2. Data redundancy for the root filesystem does not need to be large. Even if I do RAID1 with my two SSDs I won’t be storing my critical data or VMs in the rpool – I want a smaller sized root pool that has fault-tolerance (RAID1). A partition of 60GB mirrored in two SSDs should fit the bill here.
3. ZIL Intent Log experimentation, I also want to experiment by using the same two SSDs to speed up my ZFS writes. I want a small partition in a stripe (RAID0) for performance, 45GB total (22.5gb per ssd) is plenty for this.
4. The left over unused space will be left untouched so that the SSD will have more available blocks during the controller’s built-in garbage collection (not the same as TRIM)

I don’t have enough time to go into a lot of details (it’s past 4am), so I will get to how to do it. If you are trying to follow my same steps, you will need at least 3 hard drives.

1. On a hard drive or device you don’t care to use in the final outcome, install Proxmox as you would normally. Wipe the entire partition table and let it install RAID0 on the whole disk.
2. Boot into your new installation, have the two new disks you want to keep attached to the system and ensure linux sees them fdisk should help with this.
3. You will now need to create the partitions on the new disks (not rpool):

You will need to know how to calculate hard disk sectors and multiply by your block size. I don’t have time to go over it but I will do a quick TL;DR example to give you an idea:

We want 25GB slice so that is around 25000000000 bytes / 512 (block size) = 48828125 total sectors to allocate this storage amount.

Take a look at the partition table to make sure you create something similar, fdisk -l /dev/sd$ (your rpool disk). We will leave 8MB disk at the end of the partition, Proxmox by default creates 3 partitions: GRUB_BOOT, ZFS data, Solaris 8MB.

This command creates the partitions for my new array, I’ve described them for you by the -c command. It should be self-explanatory.

# sgdisk -z /dev/sdb
# sgdisk -a1 -n1:34:2047 -t1:EF02 -c1:”BIOS boot” -n2:2048:156252048 -t2:BF01 -c2:”mirror” -n3:156252049:205080174 -t3:BF01 -c3:”stripe” -n4:205080175:205096559 -t4:BF0 /dev/sda

# sgdisk -a1 -n1:34:2047 -t1:EF02 -c1:”BIOS boot” -n2:2048:156252048 -t2:BF01 -c2:”mirror” -n3:156252049:205080174 -t3:BF01 -c3:”stripe” -n4:205080175:205096559 -t4:BF0 /dev/sdc
# zpool create -f stripe -o ashift=13 /dev/sda3 /dev/sdc3
# zpool create -f newroot -o ashift=13 mirror /dev/sda2 /dev/sdc2
# grub-install /dev/disk/by-id/ata-Samsung_SSD_840_PRO_Series_S1ATNSADB46090M
# grub-install /dev/disk/by-id/ata-Samsung_SSD_840_PRO_Series_S12RNEACC59063B

Backup & moving stuff.
# zfs snapshot -r rpool@fullbackup
# zfs list -t snapshot
# zfs send -R rpool@fullbackup | zfs recv -vFd newroot
root@pve:/# zpool get bootfs
NAME PROPERTY VALUE SOURCE
newroot bootfs – default
rpool bootfs rpool/ROOT/pve-1 local
stripe bootfs – default
root@pve:/# zpool set bootfs=newroot/ROOT/pve-1 newroot
zpool export newroot
zpool import -o altroot=/mnt newroot
— rebooted with freenas live cd, enter shell, import newroot with new name rpool. rebooted
— boot into proxmox recovery — once it boots, do recovery
grub-install /dev/sdb
grub-install /dev/sda
update-grub2
update-initramfs -u

#zpool set bootfs=newroot rpool could also work without renaming via FreeNAS but didn’t try.

Out of nowhere there were two additional streams going on in my server onto the internet, I usually share my library with friends and family but what was curious about this traffic was that Plex was claiming that these two streams were “on my local network”

After reviewing firewall logs and traffic reports (pfSense), I saw traffic from some Australia IP addresses as well as Egypt on my plex port. I discovered that when I setup Plex on FreeNAS I had followed someone’s guide and steps and there was a setting that the guide required on file /usr/pbi/plexmediaserver-amd64/plexdata/Plex Media Server/Preferences.xml

disableRemoteSecurity="1"

Somehow for the past 2 years this has gone undetected, mostly because I have never (until now) detected anyone that I did not trust streaming my media library. Some Google searches told part of the story, some websites and facebook pages started sharing links to my public IP address and plex port.

I setup a quick SSH tunnel to one of my servers to get an ‘external’ view from outside my network and sure enough, you could see my library and stream anything, no login required!

After some research, even though my plex settings were set to require a login to be able to stream settings set on the Plex server settings page were being ignored. Finally when I checked the XML file manually I found out that the security was being disabled and so that is why Plex was not applying the settings.

If you have noticed any weird traffic or use the setting above on your FreeNAS + Plex jail – please be wary and you may want to close that loophole by removing that string from the XML Preferences file and restart Plex. You can whitelist your local network so that no login is required (I have 172.16.0.0/20 whitelisted).

It remains a mystery how these people found my public IP address, but I assume someone port scanned me for vulnerabilities and found the web portal wide open, so they started sharing the link.