Home network overhaul: part 1 of n

Motivation

I've been using Google WiFi for a while for my home network, as both router and access points (6 total).  Initially, I went that route for the simplicity - we had 2 young kids and I didn't have time to spend tinkering with configurations, dealing with firmware updates, etc.  And for a while, that worked as intended.  But then problems started to crop up:

• iOS Google Home app had a lot of bugs (though many have since been fixed)
• The app often reports access points as offline, though they seem to be working properly. The only way to get the app to recognize them is to power cycle.
• All APs have wired uplink (no mesh) but throughput started to be less reliable
• Even with 6 WiFi points, I still had annoying "dead zones" around the house 

The kids are now a bit older so I had more time to tinker with new gear.  If I was going to buy something to cover the dead zones, I could look at something new.  We also have a lot more random IoT devices on the network that I would prefer to segregate, so I wanted something VLAN-capable.

The New Gear

Access Points

My research showed that Ubiquiti's UniFi WiFi gear seems to be the consensus winner for access points.  The UniFi6 line was the latest when I started the project, though they were regularly out of stock.  Fortunately, there's a great discord server where you can configure alerts to be notified when specific products are back in stock.

Router

Picking a router was a bit more of a subjective call.  Ubiquiti has a few options, and there definitely seem to be advantages in having an all-Ubiquiti stack, especially as you get into more advanced configurations like VLANs.  But I found a surprising amount of negative opinions about their routers and was also trying to avoid feeling "locked in" to a single vendor like I had been.

pfSense seemed to be a strong option for router software, but you still had to figure out what hardware to run it on.  Like with Ubiquiti, going with the Netgate hardware seemed like a convenient one-stop shop, but again I found a lot of negative feedback.  Protectli got good reviews, so I decided to try that.  Just need to pick among the various models.  I like to pretend that someday I'll add another WAN option besides my Comcast cable modem, so wanted something with at least 4 ports.  There are some super-beefy options if you're planning to do high-bandwidth deep packet inspection but that seemed like overkill, so I went with the FW4B.

Switches

I already had a 16-port Netgear managed PoE switch and 24-port Linksys unmanaged PoE switch, each serving half of the house.  Knowing I wanted a VLAN setup, the Linksys needed to be replaced.  The fan noise in the Linksys drove me nuts, as it was located in a network panel on our bedroom hallway.  The Ubiquiti Switch 24 PoE is fanless and I thought might make for somewhat easier integration with the APs (it sort of does, but not a big deal).  Spoiler: dealing with 2 different switches is kind of a pain, so I'm kind of looking for an excuse to replace the Netgear switch with another Ubiquiti switch.

Installation and migration

Determining how to go about the migration was actually a bit tricky.  First I set up the pfSense router standalone just to make sure it worked.  I thought about just putting the main Google WiFi router behind it, but that wouldn't let me do any of the VLAN configuration.  Instead, I decided to set it up in parallel with the Google WiFi network, so that I could gradually migrate devices over.  

Doing this required setting up the VLAN configuration on the switches, so that the pfSense would be isolated on the LAN from the other devices.  The Netgear UI for this is a little confusing, and a bit tedious, but the simple start was to remove the port from VLAN 1 and add to VLAN 10 (tagged).  Then plug the router into the cable modem and validate by connecting a laptop to the LAN port on the router  All good!

Next up was getting the UniFi APs up and running.  I configured each AP with a few different WiFi networks:

"mynetwork" - same SSID and password as my old Google WiFi network, so that clients would just keep working as APs get swapped out

"mynetwork2" - this will be the primary network once everything is set up

"mynetwork-iot" - separate network for IoT devices

It took me a little while to understand that the Ubiquiti controls software manages VLANs via "Networks" which are separate from, but mapped to, WiFi Networks.  Initially, I left all 3 on "Default" (aka VLAN 1) just to get up and running.  I replaced all but the main Google WiFi point with UniFi APs and all was well.  I moved some of the IoT devices over to the new SSID, though that doesn't provide any isolation yet.

Now back to VLANs.  I knew the router was working well and isolated from the other devices on VLAN 10, but wanted to make sure I could get a new device to connect to it.  I tried using my PC, which was on a separate switch than the router.  So I set up a port on that switch for VLAN 10, connected the PC to it, and configured the ports between the switches as a trunk.  But the PC didn't get a DHCP response, and even manually configuring an IP on the router's LAN network didn't let it connect.  I spent a long time tinkering with the VLAN configuration on the switches, thinking I must have done something wrong there, but it turned out to be something else.  I needed to configure the VLAN interfaces on the router, and separately enable DHCP and firewall rules for each one.

Finally, back to the Wifi networks.  In the UniFi network app, I had to create new Networks for each VLAN segment (VLAN-only), and then specify for each WiFi Network to use the corresponding VLAN Network.

Boom!  Separate subnets for each WiFi Network.  I'm writing this all up after several evenings of tinkering, so it's highly likely I've forgotten to write up some critical step I spent hours figuring out.  But hopefully this helps someone else (or at least me, next time I do this).

Regrets

Funny enough, when we renovated our house, our A/V guy had recommended wiring for ceiling-mounted UniFi APs, but with everything going on with the project, I hadn't wanted to deal with new gear.  Unfortunately, this meant that we didn't run Cat6 in the ceilings to be able to mount APs on the ceiling, instead going for "typical" destinations like desk areas, TV cabinets, etc.  This wasn't a big deal with the compact Google WiFi units, but it's a bit awkward with the larger UniFi APs just laying there.

Next steps

I set up simple firewall rules that allow all traffic from each VLAN network, so they're not truly isolated from each other.  The goal is to configure the IoT network to isolate all devices from each other and disable access to the other networks.  

Nest devices can't be set up with client isolation (they rely on talking to your mobile device during setup, and sometimes talking to each other), so I'm going to set up a separate VLAN segment just for the Nest devices - they'll be able to talk to each other, but no other subnets.

Then the tedious work of migrating all my client devices over to the appropriate new networks.  My current plan is to put most hardwired devices on the "trusted" VLAN.  That should let me get away with the unmanaged local switches I'm using in a few places, but I may want to upgrade those to managed switches eventually.  One thing that threw me off during device migration - somewhere along the way, Apple added a Private WiFi feature that randomizes the MAC address.  This made it much harder to map MAC addresses to useful device names, as it also suppresses the client passing a device name in the DHCP request.  Life is easier disabling that feature on my trusted networks.

There are some nice aspects of the Google WiFi setup I'll need to replicate, such as device groups with scheduled timers and manual toggles to turn on/off internet access.

I also need to check coverage and tune power levels for the new access points.  I'll have to research some tools for that.  I ordered 6 since I needed 6 of the Google WiFi points but I'm hoping I can get away with fewer.

Then the fun new stuff: experimenting with things like PiHole, proper certificate-based VPN access, network quality monitoring, perhaps a DMZ.

250 Mini Quad Build Log

I like my 3D Robotics Y6 - very stable, good flight time, easily carries the gimbal, and by now I've got it tweaked the way I like it (gimbal and LEDs hard-wired, velcro for easy mounting of components, etc).  I took it on vacation last Summer and got some fun videos but it was a lot to pack - it plus other gear took up half a large suitcase.  So I thought I'd try building a smaller 250 Mini.

Parts

Frame

I was eyeing the QAV250 but couldn't justify spending that kind of money before knowing if I'd actually like/use the smaller size.  So I opted for the HobbyKing FPV250LH ($20).  At first, I thought it would be nice to have the extra cage space for mounting components, but at this point, it doesn't seem worth it.  The electronics can all mount on the top/bottom of the main plates.  I've had trouble with camera mounting - the fatshark camera doesn't work well, there aren't great holes for mounting a camera stand, I tried a board camera but it needs 12V, etc.  I plan to try removing the lower cage.

Motors & Props
For motors, I went for the Sunnysky X2204S 2300kV motors ($22).  A bit pricey at $22/each (looks like $24 now), but I'd been burned by crappy motors on my first build and everyone seems happy with these.  I've been happy, too.  I believe you can now get them in CW/CCW sets.

I got both 5" and 6" gemfans (and even some 5" 3-blades I haven't tried yet).  The 6" props fit, but they're so long they don't leave much clearance for cabling around the APM, GPS, etc.  The first time I tried spinning up the props, I chopped through a bunch of cables.

Electronics
At this point, I've gotten used to APM and ArduCopter, so wanted to stick with that platform.  Poking around on rcgroups, people seemed reasonably happy with the ReadyToFlyQuads MiniReadyToFlyer , so I went for it:

• MiniReadyToFlyer 2.6 V 3.1 ($40) 
• Add-on power module ($13) - note unusual pin layout
• Add-on Mini uBlox 6M GPS w/ Compass ($20)
• Add-on Mini 4-in-1 12A ESC ($33)

If I had to do it again, I'd get the power module from somewhere else, due to the backwards pin layout used on the RTFQ module.  I'm also not sure about the ESC - it's convenient to use with the Mini APM which as a single connector/cable for the motor outputs.  But it was a bit tricky to solder with quite small pads and I'm not great at soldering.

Misc

Battery: 3S 1300mAH ($9)

FatShark 250mW Video Transmitter (re-using from my Y6, which got upgraded to 600mW)

HobbyKing MinimOSD ($21)

Camera: undecided

FrSky D4R-II ($21)

The Build

It's been a while and I didn't take great notes, sadly.

Frame Assembly

The frame assembled pretty easily - no problems with alignment, missing pieces, etc.  I didn't both assembling the lower cage until after I had everything working.  I did have to drill holes for mounting the electronics, using stacked nylon spacers.

When I did eventually put the lower cage on, it holds OK in normal use but comes off pretty easily in a crash.  As I said before, not really sure it's worth it.

With the ESC, APM and GPS all stacked up, the small cage that's meant to mount on top obviously doesn't fit.  I've seen some folks mount it on spacers, but I've just got it open to the air.

Electronics

I had a lot of issues with the electronics.  I was ambitious and decided to solder motors directly to the 4-in-1 ESC.  If you look carefully at the ESC, you can see motor numbers screened onto it.  I goofed up at first and assumed the motor connections were sequential; they're not.  Fortunately, I was able to just move the motors around (rotation direction was correct).  The quad would still flip if I throttled it up, so I did a motor test in mission planner and found they were still wrong.  It turns out the APM->ESC cable that came with everything is wired strangely.  I emailed Paul to ask about it and he said it's intentional.  For this to work, you'd have to ignore the motor numbers screened on the ESC.  My solution was to swap the wires in the connector to make it pass-through (1->1, 2->2, 3->3, 4->4).

The GPS/Compass was a little tricky as well.  There were mixed reports of how to hook it up.  RCGroups suggests using A-port but I'm using B-port with no problem.  With the antenna facing up, you'll want the connectors facing the left side of your quad.  I've also had problems with interference on the cable (I initially tucked the slack under the ESC) so you'll have to keep that tidy.

I also wasn't getting a current reading from the power module - for a while, the RTFQ XT60 power modules had the connectors backwards, which prevents the current sensor from working.  I wrote up a whole post on power modules, but the fix here was to swap out for a standard power module and then re-order the cable to match.

End Result

So far, I'm pretty happy with it.  Just getting started trying to fly FPV, which takes some getting used to.  I also need to find a better camera setup.  I had been velcro-ing in the FatShark camera; the small size naturally angled it up to fit in the camera opening, but it wasn't that stable.  I tried a few more cameras:

SurveilZone CC1333 ($26) - Should fit the camera mount if you trim the border, but requires 12V power supply.  I should probably wire up a tap for this, but haven't gotten around to it.

SurveilZone HS1177 ($35) - I thought the stand would work well but there are no holes to screw it into; would likely need to use mounting tape on the stand.  Unfortunately, I just get a black screen from this camera and need to return it.

Currently trying a Hero3 Black velcro'd onto the front.  We'll see how that works out.

APM Power Modules

I was having a problem with my 250 Mini Quad (need to write up the build log at some point) where I wasn't getting current measurements from the power module.  Most of the parts for my 250 came from http://www.readytoflyquads.com/, including the mini APM and power module.  I had checked and double-checked everything, but no luck.  Some extensive digging on rcgroups turned up this post confirming that the ReadyToFlyQuads have the XT60 connectors on backwards, with the result that it can't measure current properly.  

No problem, I thought - I'll just swap one in from one of my other drones.  I grabbed the HobbyKing power module I use on my 450 and hooked it up.  But when I plugged in the battery, the ESCs didn't give their final chime indicating a signal from the flight controller.  After banging my head for a while, I noticed that the APM wasn't powering up.  This was strange - the mini APM uses a different layout for the PM connector on the board itself, but I assumed the power module layout would be standard (because they're also used with full-size APMs). But no.

Standard power module layout (looking at the receptacle as you'd plug it in with the retaining tabs facing up), a standard power module puts out:
V   V   Is   Vs   G   G 
The ReadyToFlyQuads power module puts out:
G   G   Vs   Is   V   V

[Vs and Is are measured Voltage and Current, respectively]

I re-configured the cable from the mini APM for a standard power module layout and everything now works like a charm.  Hopefully google search will pick this up in case others run into the same problem I did.

Addendum:
Since I needed another power module (figured it was cheaper to buy another one than the hassle of re-connecting the XT60 connectors in order to get current sensor working), I tried going super-cheap and ordered a power module from eBay for $8 shipped.  Standard layout, though the voltage out is slightly lower (5.37v vs 5.40v on the other two).  Will see how it holds up.  On the plus side, it comes with a 6-pin DF13->JST-SH cable for use w/ the mini APMs, including the correct wire order.  I need to put a load on it in order to confirm the current sensor works.

Update 6/6/2015:
I ordered another power module from ReadyToFlyQuads because, why not?  This one came with the XT60 connectors on the right way but still the backwards layout for the power module connector.

Unplanned upgrade of my Y6 frame

I'm going a bit out of order here, as I still haven't written up the build for either my MultiWii Quad or original build of the 3DR Y6.  But this one is more interesting.

I'd flown my Y6 a good number of times, even several in quite windy conditions at Emerald Isle, NC.  Had never had any problems with lack of control or anything.  Last weekend, I finally got around to adding LED strips to the legs to help with orientation, including a 12V regulator (I usually fly with a 4S pack and the LEDs aren't intended for that high voltage).  While I was at it, I hooked up an additional JST connector to my power distribution board to power my gimbal.

After all that, I did a quick test flight in my backyard and everything appeared to be working properly.  So after work, I stopped by the park to try a real flight with some space to maneuver.  I was in a bit of a rush, so didn't connect my tablet or wait for GPS lock, just intending to fly around a bit in Stabilize mode.  

I took off and everything was working fine for about 20 seconds or so.  Then it started flying away.  I tried turning it around to head back towards me, but it kept flying in the same direction.  I pretty quickly ran out of space and crashed into a fence.  The carnage was pretty impressive:

• Broke all 6 props
• Bent both mounting plates for the gimbal as well as the RC connector (though it still works)
• APM and GPS sheared off the mounting plates
• Broke a bunch of the nylon screws for the landing gear and lost a spacer
• Broke all the standoffs for the GPS/accessory plate
• Cracked the carbon fiber base plate and it broke in half

They don't appear to sell replacement parts for the 2013 Y6 frame anymore, so I ordered a 2014 frame kit.  Just swapped out the base and top plates, though - pretty sure the legs are the same and there didn't seem to be much interesting with the new APM or accessory mounting plates.

As to what went wrong, I'm still not clear.  I don't have any experience troubleshooting APM logs so will need to figure that out.  My guess is something to do with the compass - in retrospect, it was behaving as if it was in Simple or Super Simple mode, but I don't even have that configured as an option right now.

Update: Replaced the top and bottom plates with new ones from the 2014 frame kit and it's back in the air.

Digital Music Workflow

I wanted to work through some problems in my music workflow so I figured I'd write up an extended version of a short previous post that had a few notes.  Hopefully this is useful to someone.

Goal

My primary goal is to be able to easily access all my music on a variety of different devices - iPhones, Sonos, cloud-based music services, etc.  Second to this is preserving quality as much as possible.  

Sources

My digital music comes from a few different sources:

CDs

The bulk of my music comes from CD rips - it's the most practical & consistent way to get high-quality source material.  I rip CDs using Exact Audio Copy (EAC) and encode to FLAC - this way, I can get this music into any format I want (mp3, aac, wma) without worrying about loss of quality.  All the CDs end up in a FLAC folder organized as

CD Rips\AlbumArtist\AlbumName\## TrackName.flac

The folder organization is just for convenience - the embedded metadata is what everything keys off of.  There are a few tricky cases with this method of organization:

Various Artists

Compilations get handled one of two ways.  If there is no obvious artist for the album as a whole (e.g. movie soundtracks, Christmas music collections), it goes under "Various Artists" with the FLAC "album artist" field also set to "Various Artists."  When there is an artist for the album (e.g. DJ Mix CDs), it goes under that artist.  In either case, the "artist" field for individual tracks gets set to the artist for that track.

Multi-Disc Albums

Multi-Disc albums get treated as a single album - so all discs go into the same folder and have the same album name (rather than use album names like "The Wall (Disc One)" and "The Wall (Disc Two)."  When doing this, you need to make sure the "discnumber" and "totaldiscs" metadata fields get set accurately.

iTunes Store

Some of our music is purchased from iTunes.  I used to avoid purchasing anything from iTunes when music was encrypted, as it meant it could only be played on Apple devices.  Now that everything is unencrypted, we buy music from iTunes a bit more often - mostly singles.  On disk, I leave this the way iTunes organizes it:

iTunes Media\Music\TrackArtist\AlbumName\## TrackName.m4a

It's a bit annoying that iTunes uses the track artist instead of album artist on the filesystem, but since everything uses the metadata, it doesn't cause problems in practice.

Amazon Music Store

We also have some music from the Amazon music store - mostly singles from before iTunes got rid of DRM.  I manually put this stuff into the same structure as I use for CD rips, since I seem to get a wide range of file naming conventions from Amazon downloads:

Amazon Music\AlbumArtist\AlbumName\## TrackName.mp3

Since I buy so little from Amazon, I haven't bothered to automate any of this.

Other

I've got a few odds and ends from other sources, which go in the same structure as CD rips and Amazon, but with a different root:

Other\AlbumArtist\AlbumName\## TrackName.mp3

Destinations

I stated the primary goal was to get all this music playable on all my devices. Those basically fall into a few categories:

iThings

Since the only way to get music onto iThings is via iTunes, I need to get all this stuff loaded into iTunes.  Music from the iTunes Store, Amazon Music Store, and Other are all pretty easy - just point iTunes at the right folder and you're done.  Since iThings don't support FLAC, though, I had to get the CD rips into a different form for iTunes.  This is the job of the ConvertLibrary tool mentioned in my previous post.  I'll write up a separate post on that, but basically what it does is synchronize a music collection in different formats, resulting in

Converted CDs\AlbumArtist\AlbumName\## TrackName.mp3

I currently sync into mp3 but have been meaning to re-generate my library to AAC, since it's more space-efficient and I've been running out of space on my iThings.

Sonos

I set up a Sonos system in our house a while back, since we have in-wall speakers in a number of different rooms.  The Sonos needs to play music off the network, so I point it at a NAS (Network Attached Storage) on our home network.  A batch job runs every day that ensures the NAS has a current copy of all the music Source folders from above.  I could point the NAS at the individual folders on my home PC, but it's good to have a copy on the NAS anyway.

Google Play Music and Amazon Cloud Player

I don't really use either of these that much but have been tinkering with them a bit.  I set these up basically the same as iTunes - point them at the folders for each Source.  I have Google Play Music set to use the Converted CDs folder instead of the original FLAC rips, though since Google Play Music limits storage by songs instead of size, I suppose I should use the FLAC folder instead (though they'll convert to 320Kbps mp3). 

To Do

A bunch of stuff still doesn't work quite the way I want.  Most notably, there's no synchronization of playlists across devices.  It looks like Sonos has some support for reading iTunes' playlists out of your iTunes library, but since my Sonos uses a different set of files, I think I'll need to do something else.

Many steps are still manual that should be automated:

• When I rip a new CD, I have to manually run the ConvertLibrary tool to generate an iTunes-friendly copy; I need to update ConvertLibrary to monitor the folder and run automatically.
• When any new music is added, I have to manually update the NAS copy for the Sonos (or wait for the batch job to run); similar to the first point, this should monitor the folders for changes.
• I don't have a good system for getting appropriate album art.  Any recommendations?
• My NAS uses FAT, which means it can't handle Unicode characters in filenames - either need to restrict character set for filenames or figure something else out.

Synchronizing music library in multiple formats

Getting a new Sonos player motivated me to clean up some problems in my CD ripping workflow.  First, some background:

I decided a while back to rip all my CDs to FLAC.  The need for specific formats and bitrates are dictated by the device(s) you're trying to use, and with everything in FLAC, I can just re-encode my entire collection to a new format and/or bitrate whenever I want.

I use EAC for ripping.  Works fine.

I wrote my own tool, which I should publish sometime, to handle mirroring a music collection.  Conversion is handled by standard command line tools (e.g. lame) with config scripts that control the rules.  What's been a nagging problem for quite some time is The Compilation Problem.

It's remarkably hard to find good write-ups on the "right" way to handle compilations, so for the sake of future searchers, here you go:

• Set the artist on individual tracks to be the actual track artist.  So this will be a different artist for every track on a Soundtrack or DJ compilation
• Set the album artist ("ALBUM ARTIST" tag in FLAC/Vorbis or "TPE2" in id3v2) to something fixed for the album - "Various Artists" in the case of a soundtrack or the name of the DJ for compilation CDs.
• Ensure you set the "TCMP" tag to 1 in the id3v2 tag - this is what flags the tracks as part of a compilation.  iTunes will then pick these up and merge into one album, using the Album Artist field.

First android app

I frequently worry that I forgot to close my garage door. When this happens, I can either drive back home just to check, or assume that I closed it and try to move on with my life. While there are much better solutions to this problem, I decided to solve it by writing an Android app.

The app periodically takes a picture using the camera in the phone, and uploads it to a site where the picture can be viewed. At first, this was just a single location on my personal server, but for release, I created a simple Google AppEngine application to receive, list, and display the images.

Please post any thoughts/feedback in the comments. Some things I'll probably add:

• Configurable # of recent images to be stored
• Automatic generation of time-lapse video
• Permissions/protection on the posted images

Other weird stuff:

• It asks for permission to read the phone state - this is used to get the device's IMEI as a unique identifier. I'm sure there are better solutions, and you can just manually set an identifier in the settings. Problem will go away when I add permissions.
  
• Dealing with the camera is a pain in the butt. The emulator camera doesn't behave at all like the real hardware. Currently, I re-initialize the camera after every picture, which is probably bad. Focus is flaky. And sometimes it ends up with a blank image, due to race conditions which I need to fix. Blah blah blah.
• It's ugly. I know.