Home network overhaul: part 1 of n

Motivation

I've been using Google WiFi for a while for my home network, as both router and access points (6 total).  Initially, I went that route for the simplicity - we had 2 young kids and I didn't have time to spend tinkering with configurations, dealing with firmware updates, etc.  And for a while, that worked as intended.  But then problems started to crop up:

• iOS Google Home app had a lot of bugs (though many have since been fixed)
• The app often reports access points as offline, though they seem to be working properly. The only way to get the app to recognize them is to power cycle.
• All APs have wired uplink (no mesh) but throughput started to be less reliable
• Even with 6 WiFi points, I still had annoying "dead zones" around the house 

The kids are now a bit older so I had more time to tinker with new gear.  If I was going to buy something to cover the dead zones, I could look at something new.  We also have a lot more random IoT devices on the network that I would prefer to segregate, so I wanted something VLAN-capable.

The New Gear

Access Points

My research showed that Ubiquiti's UniFi WiFi gear seems to be the consensus winner for access points.  The UniFi6 line was the latest when I started the project, though they were regularly out of stock.  Fortunately, there's a great discord server where you can configure alerts to be notified when specific products are back in stock.

Router

Picking a router was a bit more of a subjective call.  Ubiquiti has a few options, and there definitely seem to be advantages in having an all-Ubiquiti stack, especially as you get into more advanced configurations like VLANs.  But I found a surprising amount of negative opinions about their routers and was also trying to avoid feeling "locked in" to a single vendor like I had been.

pfSense seemed to be a strong option for router software, but you still had to figure out what hardware to run it on.  Like with Ubiquiti, going with the Netgate hardware seemed like a convenient one-stop shop, but again I found a lot of negative feedback.  Protectli got good reviews, so I decided to try that.  Just need to pick among the various models.  I like to pretend that someday I'll add another WAN option besides my Comcast cable modem, so wanted something with at least 4 ports.  There are some super-beefy options if you're planning to do high-bandwidth deep packet inspection but that seemed like overkill, so I went with the FW4B.

Switches

I already had a 16-port Netgear managed PoE switch and 24-port Linksys unmanaged PoE switch, each serving half of the house.  Knowing I wanted a VLAN setup, the Linksys needed to be replaced.  The fan noise in the Linksys drove me nuts, as it was located in a network panel on our bedroom hallway.  The Ubiquiti Switch 24 PoE is fanless and I thought might make for somewhat easier integration with the APs (it sort of does, but not a big deal).  Spoiler: dealing with 2 different switches is kind of a pain, so I'm kind of looking for an excuse to replace the Netgear switch with another Ubiquiti switch.

Installation and migration

Determining how to go about the migration was actually a bit tricky.  First I set up the pfSense router standalone just to make sure it worked.  I thought about just putting the main Google WiFi router behind it, but that wouldn't let me do any of the VLAN configuration.  Instead, I decided to set it up in parallel with the Google WiFi network, so that I could gradually migrate devices over.  

Doing this required setting up the VLAN configuration on the switches, so that the pfSense would be isolated on the LAN from the other devices.  The Netgear UI for this is a little confusing, and a bit tedious, but the simple start was to remove the port from VLAN 1 and add to VLAN 10 (tagged).  Then plug the router into the cable modem and validate by connecting a laptop to the LAN port on the router  All good!

Next up was getting the UniFi APs up and running.  I configured each AP with a few different WiFi networks:

"mynetwork" - same SSID and password as my old Google WiFi network, so that clients would just keep working as APs get swapped out

"mynetwork2" - this will be the primary network once everything is set up

"mynetwork-iot" - separate network for IoT devices

It took me a little while to understand that the Ubiquiti controls software manages VLANs via "Networks" which are separate from, but mapped to, WiFi Networks.  Initially, I left all 3 on "Default" (aka VLAN 1) just to get up and running.  I replaced all but the main Google WiFi point with UniFi APs and all was well.  I moved some of the IoT devices over to the new SSID, though that doesn't provide any isolation yet.

Now back to VLANs.  I knew the router was working well and isolated from the other devices on VLAN 10, but wanted to make sure I could get a new device to connect to it.  I tried using my PC, which was on a separate switch than the router.  So I set up a port on that switch for VLAN 10, connected the PC to it, and configured the ports between the switches as a trunk.  But the PC didn't get a DHCP response, and even manually configuring an IP on the router's LAN network didn't let it connect.  I spent a long time tinkering with the VLAN configuration on the switches, thinking I must have done something wrong there, but it turned out to be something else.  I needed to configure the VLAN interfaces on the router, and separately enable DHCP and firewall rules for each one.

Finally, back to the Wifi networks.  In the UniFi network app, I had to create new Networks for each VLAN segment (VLAN-only), and then specify for each WiFi Network to use the corresponding VLAN Network.

Boom!  Separate subnets for each WiFi Network.  I'm writing this all up after several evenings of tinkering, so it's highly likely I've forgotten to write up some critical step I spent hours figuring out.  But hopefully this helps someone else (or at least me, next time I do this).

Regrets

Funny enough, when we renovated our house, our A/V guy had recommended wiring for ceiling-mounted UniFi APs, but with everything going on with the project, I hadn't wanted to deal with new gear.  Unfortunately, this meant that we didn't run Cat6 in the ceilings to be able to mount APs on the ceiling, instead going for "typical" destinations like desk areas, TV cabinets, etc.  This wasn't a big deal with the compact Google WiFi units, but it's a bit awkward with the larger UniFi APs just laying there.

Next steps

I set up simple firewall rules that allow all traffic from each VLAN network, so they're not truly isolated from each other.  The goal is to configure the IoT network to isolate all devices from each other and disable access to the other networks.  

Nest devices can't be set up with client isolation (they rely on talking to your mobile device during setup, and sometimes talking to each other), so I'm going to set up a separate VLAN segment just for the Nest devices - they'll be able to talk to each other, but no other subnets.

Then the tedious work of migrating all my client devices over to the appropriate new networks.  My current plan is to put most hardwired devices on the "trusted" VLAN.  That should let me get away with the unmanaged local switches I'm using in a few places, but I may want to upgrade those to managed switches eventually.  One thing that threw me off during device migration - somewhere along the way, Apple added a Private WiFi feature that randomizes the MAC address.  This made it much harder to map MAC addresses to useful device names, as it also suppresses the client passing a device name in the DHCP request.  Life is easier disabling that feature on my trusted networks.

There are some nice aspects of the Google WiFi setup I'll need to replicate, such as device groups with scheduled timers and manual toggles to turn on/off internet access.

I also need to check coverage and tune power levels for the new access points.  I'll have to research some tools for that.  I ordered 6 since I needed 6 of the Google WiFi points but I'm hoping I can get away with fewer.

Then the fun new stuff: experimenting with things like PiHole, proper certificate-based VPN access, network quality monitoring, perhaps a DMZ.