"networking restart" issues, VLANs under Ubuntu

For this post, I'm using Ubuntu Linux 9.04 / "Jaunty Jackalope". This is somewhat a follow-up to my Ubuntu Linux Router Upgrade Project.

Errors during "/etc/init.d/networking restart"

First, assuming a statically-configured LAN for server use, with NetworkManager disabled:

/etc/network/interfaces:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
  address 192.168.1.1
  netmask 255.255.255.0

$ sudo /etc/init.d/networking restart
 * Reconfiguring network interfaces...                                   [ OK ]

The networking simply restarts, without showing any errors or warnings. However, this quickly changes once an additional network adapter is configured. This could be an additional physical adapter, but for my purposes, I had added a virtual LAN (VLAN) to work with my Dell PowerConnect 2716. For Ubuntu, this simply requires installing the "vlan" package, and defining the virtual LAN with an additional entry in /etc/network/interfaces. Note that this is done using a "<base name>.<VLAN ID>" naming scheme. As shown below, I'm adding an interface for VLAN 2:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
  address 192.168.1.1
  netmask 255.255.255.0

auto eth0.2
iface eth0.2 inet static
  address 192.168.2.1
  netmask 255.255.255.0

Please add a comment if you can find any official documentation that documents this functionality of the interfaces file, as I can't. However, this appears to be driven by "/etc/network/if-pre-up.d/vlan" and "/etc/network/if-post-down.d/vlan".

This also requires the "8021q" module, but at least in Jaunty, it is already available by default, as shown by "lsmod | grep 8021q".

This is where I started running into errors:

$ sudo /etc/init.d/networking restart
 * Reconfiguring network interfaces...
RTNETLINK answers: No such process
Removed VLAN -:eth0.2:-
 * if-up.d/mountnfs[eth0]: waiting for interface eth0.2 before doing NFS mounts
Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config
Added VLAN with VID == 2 to IF -:eth0:-
                                                                         [ OK ]

This doesn't actually cause any issues, but it isn't right and should be fixed. I don't see any way to make "networking restart" give more verbose output. Additionally, nothing helpful appears in the logs. However, most of what the networking script does is call "ifdown -a --exclude=lo" and "ifup -a", where "-a" is "affect all interfaces marked auto". Fortunately, repeating this with adding "-v" for verbose mode yields some details:

$ sudo ifdown -av --exclude=lo && sudo ifup -av
Configuring interface eth0=eth0 (inet)
run-parts --verbose /etc/network/if-down.d
run-parts: executing /etc/network/if-down.d/avahi-autoipd
run-parts: executing /etc/network/if-down.d/wpasupplicant

ifconfig eth0 down
run-parts --verbose /etc/network/if-post-down.d
run-parts: executing /etc/network/if-post-down.d/avahi-daemon
run-parts: executing /etc/network/if-post-down.d/bridge
run-parts: executing /etc/network/if-post-down.d/vlan
run-parts: executing /etc/network/if-post-down.d/wireless-tools
run-parts: executing /etc/network/if-post-down.d/wpasupplicant
Configuring interface eth0.2=eth0.2 (inet)
run-parts --verbose /etc/network/if-down.d
run-parts: executing /etc/network/if-down.d/avahi-autoipd
RTNETLINK answers: No such process
run-parts: executing /etc/network/if-down.d/wpasupplicant

ifconfig eth0.2 down
run-parts --verbose /etc/network/if-post-down.d
run-parts: executing /etc/network/if-post-down.d/avahi-daemon
run-parts: executing /etc/network/if-post-down.d/bridge
run-parts: executing /etc/network/if-post-down.d/vlan
Removed VLAN -:eth0.2:-
run-parts: executing /etc/network/if-post-down.d/wireless-tools
run-parts: executing /etc/network/if-post-down.d/wpasupplicant
Configuring interface eth0=eth0 (inet)
run-parts --verbose /etc/network/if-pre-up.d
run-parts: executing /etc/network/if-pre-up.d/bridge
run-parts: executing /etc/network/if-pre-up.d/dhclient3-apparmor
run-parts: executing /etc/network/if-pre-up.d/vlan
run-parts: executing /etc/network/if-pre-up.d/wireless-tools
run-parts: executing /etc/network/if-pre-up.d/wpasupplicant

ifconfig eth0 192.168.1.1 netmask 255.255.255.0       up

run-parts --verbose /etc/network/if-up.d
run-parts: executing /etc/network/if-up.d/avahi-autoipd
run-parts: executing /etc/network/if-up.d/avahi-daemon
run-parts: executing /etc/network/if-up.d/ip
run-parts: executing /etc/network/if-up.d/mountnfs
 * if-up.d/mountnfs[eth0]: waiting for interface eth0.2 before doing NFS mounts
run-parts: executing /etc/network/if-up.d/ntpdate
run-parts: executing /etc/network/if-up.d/wpasupplicant
Configuring interface eth0.2=eth0.2 (inet)
run-parts --verbose /etc/network/if-pre-up.d
run-parts: executing /etc/network/if-pre-up.d/bridge
run-parts: executing /etc/network/if-pre-up.d/dhclient3-apparmor
run-parts: executing /etc/network/if-pre-up.d/vlan
Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config
Added VLAN with VID == 2 to IF -:eth0:-
run-parts: executing /etc/network/if-pre-up.d/wireless-tools
run-parts: executing /etc/network/if-pre-up.d/wpasupplicant

ifconfig eth0.2 192.168.2.1 netmask 255.255.255.0       up

run-parts --verbose /etc/network/if-up.d
run-parts: executing /etc/network/if-up.d/avahi-autoipd
run-parts: executing /etc/network/if-up.d/avahi-daemon
run-parts: executing /etc/network/if-up.d/ip
run-parts: executing /etc/network/if-up.d/mountnfs
run-parts: executing /etc/network/if-up.d/ntpdate
run-parts: executing /etc/network/if-up.d/wpasupplicant

So the error is apparently coming from "/etc/network/if-down.d/avahi-autoipd". (Avahi is a free Zero configuration networking (zeroconf) implementation.) I temporarily edited the "avahi-autoipd" script to add "-x" to enable debugging by changing the first line to "#!/bin/sh -ex". What is happening is that this script first checks for the existence of a route matching "169.254.0.0/16", but doesn't check for what interface it is on. If it finds a matching route, even on another interface, it makes a call to delete the route, but only for the current interface. When this combination doesn't exist, the "RTNETLINK answers: No such process" message is given.

Resolution: The scripts could just be fixed to include the interface in the search. I opened a bug report to Ubuntu on this: #425854. However, a "server"-type system shouldn't be handling these automatic types of network routings, so I just disabled these scripts:

sudo chmod -x /etc/network/if-up.d/avahi-autoipd
sudo chmod -x /etc/network/if-down.d/avahi-autoipd

Shown fixed:

$ sudo /etc/init.d/networking restart
 * Reconfiguring network interfaces...
Removed VLAN -:eth0.2:-
 * if-up.d/mountnfs[eth0]: waiting for interface eth0.2 before doing NFS mounts
Set name-type for VLAN subsystem. Should be visible in /proc/net/vlan/config
Added VLAN with VID == 2 to IF -:eth0:-
                                                                         [ OK ]

DHCP server binding issues

Another issue I noticed with running "/etc/init.d/networking restart" is that the DHCP server ("dhcpd" / "dhcp3-server") quit responding to requests. (Previous posting on dhcp3-server.) Using Wireshark, I noticed that DHCP requests were still being received, but no responses were being sent. Nothing applicable is shown in any of the logs, other than that dhcpd isn't even acknowledging the requests after the interface is brought back up.

Tools such as "lsof" "netstat" show that dhcpd is still properly bound to the interfaces. Other applications such as the SSH server (sshd) don't have this issue, and continue accepting connections even after the interface goes down and comes back up, so I'm not sure why this is an issue for the DHCP server. However, I can't get things to resume working properly short of restarting the DHCP server using "sudo /etc/init.d/dhcp3-server restart" As I'm starting to work with IPv6, I noticed the exact same issue with radvd.

I wasn't exactly sure where the best place was to automate these restarts. I could probably add a script to "/etc/network/if-up.d/", but would then have to include a check that I wasn't unnecessarily restarting the servers for every interface, as they should only be concerned with the interface they are serving requests on, e.g. eth0. For now, I just added these commands to "/etc/network/interfaces":

# …

auto eth0
iface eth0 inet static
  address 192.168.1.1
  netmask 255.255.255.0
  post-up /etc/init.d/dhcp3-server restart
  post-up /etc/init.d/radvd restart

# …