adamflott.com

Network Debugging & Setup

Description: Commands and techniques to debug and setup networking on Linux
Authored: 2022-03-27;
Permalink: https://adamflott.com/networking/debugging/
tags : ipv6; linux; nixos; tcpdump;
categories : networking;


Table of Contents

Debugging IPv6

Get a real-time counter of packets and bytes

$ watch "ip6tables -L -v -n"

Trace ICMPv6 Route Advertisements Packets

Use tcpdump to find all (router advertisements)[https://www.iana.org/assignments/icmpv6-parameters/icmpv6-parameters.xhtml#icmpv6-parameters-codes-12]

$ tcpdump -i any -n 'icmp6 && ip6[40] == 134'                                                                                                                  [11:34:25]
[sudo] password for adam:
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
11:34:30.424922 enp0s4 M   IP6 fe80::1 > ff02::1: ICMP6, router advertisement, length 64
11:34:34.445194 enp0s4 M   IP6 fe80::1 > ff02::1: ICMP6, router advertisement, length 64
11:34:38.454862 enp0s4 M   IP6 fe80::1 > ff02::1: ICMP6, router advertisement, length 64

Dump IPv6 Router Advertisements

Use radvdump from the radvd package

$ nix-shell -p radvd
$ radvdump
#
# radvd configuration generated by radvdump 2.19
# based on Router Advertisement from fe80::1
# received by interface enp0s4
#

interface enp0s4
{
        AdvSendAdvert on;
        # Note: {Min,Max}RtrAdvInterval cannot be obtained with radvdump
        AdvManagedFlag off;
        AdvOtherConfigFlag off;
        AdvReachableTime 0;
        AdvRetransTimer 0;
        AdvCurHopLimit 64;
        AdvDefaultLifetime 60;
        AdvHomeAgentFlag off;
        AdvDefaultPreference medium;
        AdvSourceLLAddress on;
        AdvLinkMTU 1500;

        prefix 2600:3c03::/64
        {
                AdvValidLifetime 90;
                AdvPreferredLifetime 30;
                AdvOnLink off;
                AdvAutonomous on;
                AdvRouterAddr off;
        }; # End of prefix definition

}; # End of interface definition

Trace All ICMPv6 Packets

Warning: this is chatty on most networks!

tcpdump -i any -n icmp6
tcpdump: data link type LINUX_SLL2
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
11:36:06.731780 enp0s4 Out IP6 fe80::91bf:8cd9:dd7d:a7a5 > ff02::1: ICMP6, neighbor advertisement, tgt is 2600:3c03::91bf:8cd9:dd7d:a7a5, length 32
11:36:07.471535 enp0s4 In  IP6 fe80::8678:acff:fe0d:97c1 > fe80::91bf:8cd9:dd7d:a7a5: ICMP6, neighbor solicitation, who has fe80::91bf:8cd9:dd7d:a7a5, length 32
11:36:07.471598 enp0s4 Out IP6 fe80::91bf:8cd9:dd7d:a7a5 > fe80::8678:acff:fe0d:97c1: ICMP6, neighbor advertisement, tgt is fe80::91bf:8cd9:dd7d:a7a5, length 24
11:36:07.733421 enp0s4 Out IP6 fe80::91bf:8cd9:dd7d:a7a5 > ff02::1: ICMP6, neighbor advertisement, tgt is 2600:3c03::91bf:8cd9:dd7d:a7a5, length 32
11:36:08.500401 enp0s4 In  IP6 fe80::8678:acff:fe0d:97c1 > fe80::91bf:8cd9:dd7d:a7a5: ICMP6, neighbor solicitation, who has fe80::91bf:8cd9:dd7d:a7a5, length 32
11:36:08.500467 enp0s4 Out IP6 fe80::91bf:8cd9:dd7d:a7a5 > fe80::8678:acff:fe0d:97c1: ICMP6, neighbor advertisement, tgt is fe80::91bf:8cd9:dd7d:a7a5, length 24
11:36:08.733717 enp0s4 Out IP6 fe80::91bf:8cd9:dd7d:a7a5 > ff02::1: ICMP6, neighbor advertisement, tgt is 2600:3c03::91bf:8cd9:dd7d:a7a5, length 32
11:36:10.461162 enp0s4 M   IP6 fe80::f03c:91ff:febc:daa > ff02::1:ffbf:3451: ICMP6, neighbor solicitation, who has fe80::5430:24ff:febf:3451, length 32
11:36:10.741667 enp0s4 M   IP6 fe80::1 > ff02::1: ICMP6, router advertisement, length 64
11:36:10.742671 enp0s4 Out IP6 fe80::91bf:8cd9:dd7d:a7a5 > ff02::1: ICMP6, neighbor advertisement, tgt is 2600:3c03::91bf:8cd9:dd7d:a7a5, length 32

IPv6 On Linode With NixOS

Linode has great documentation on setting up IPv6. Unfortunately I use NixOS and had to do a little futzing around in order to get it to work. I had to use the techniques in (above)[#Debugging] to figure it out.

If your Linode does not have the correct IPv6 address or any IPv6 address at all, you should verify that you have router advertisements enabled and IPv6 privacy extensions disabled. Your Linode will need to accept router advertisements for SLAAC to function. These settings are properly configured by default in our supported distributions.

First disable IPv6 privacy extensions, in your /etc/nixos/configuration.nix, set

networking.tempAddresses = "disabled";

You can also disable them per interface. The above does for all interfaces.

Next go into your Linode VM's "Network" tab to find the address for type "IPv6 – SLAAC" and add it to the config:

networking.enableIPv6 = true;
networking.interfaces.enp0s4.ipv6.addresses = [
    {
      address = "2600:3c03::f03c:91ff:fe58:846a";
      prefixLength = 64;
    }
];

After a nixos-rebuild switch you should have routes setup

$ ip -6 route show
::1 dev lo proto kernel metric 256 pref medium
2600:3c03::/64 via fe80::1 dev enp0s4 proto ra metric 202 mtu 1500 pref medium
2600:3c03::/64 dev enp0s4 proto kernel metric 256 pref medium
fe80::/64 dev enp0s4 proto kernel metric 100 pref medium
default via fe80::1 dev enp0s4 proto ra metric 202 mtu 1500 pref medium

Test both ingress and egress IPv6 packets:

$ ping -6 -c 1 google.com
PING google.com(lga25s77-in-x0e.1e100.net (2607:f8b0:4006:81d::200e)) 56 data bytes
64 bytes from lga25s77-in-x0e.1e100.net (2607:f8b0:4006:81d::200e): icmp_seq=1 ttl=121 time=1.65 ms

--- google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.652/1.652/1.652/0.000 ms

(from another machine)

$ ping -6 -c 1 adamflott.com
PING adamflott.com(adamflott.com (2600:3c03::f03c:91ff:fe58:846a)) 56 data bytes
64 bytes from adamflott.com (2600:3c03::f03c:91ff:fe58:846a): icmp_seq=1 ttl=53 time=16.6 ms

--- adamflott.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 16.602/16.602/16.602/0.000 ms

Make sure there is a reverse DNS record for that address as well

$ dig -x 2600:3c03::f03c:91ff:fe58:846a


; <<>> DiG 9.16.27 <<>> -x 2600:3c03::f03c:91ff:fe58:846a
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 60537
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;a.6.4.8.8.5.e.f.f.f.1.9.c.3.0.f.0.0.0.0.0.0.0.0.3.0.c.3.0.0.6.2.ip6.arpa. IN PTR

;; ANSWER SECTION:
a.6.4.8.8.5.e.f.f.f.1.9.c.3.0.f.0.0.0.0.0.0.0.0.3.0.c.3.0.0.6.2.ip6.arpa. 86400	IN PTR adamflott.com.