I was investigating traffic in my home lab yesterday, and noticed that about 1% of the traffic was weird. Before I describe the weird, let me show you a normal frame for comparison's sake.


This is a normal frame with Ethernet II encapsulation. It begins with 6 bytes of the destination MAC address, 6 bytes of the source MAC address, and 2 bytes of an Ethertype, which in this case is 0x0800, indicating an IP packet follows the Ethernet header. There is no TCP payload as this is an ACK segment.

You can also see this in Tshark.

$ tshark -Vx -r frame4238.pcap

Frame 1: 66 bytes on wire (528 bits), 66 bytes captured (528 bits)
    Encapsulation type: Ethernet (1)
    Arrival Time: May  7, 2019 18:19:10.071831000 UTC
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1557253150.071831000 seconds
    [Time delta from previous captured frame: 0.000000000 seconds]
    [Time delta from previous displayed frame: 0.000000000 seconds]
    [Time since reference or first frame: 0.000000000 seconds]
    Frame Number: 1
    Frame Length: 66 bytes (528 bits)
    Capture Length: 66 bytes (528 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:ethertype:ip:tcp]
Ethernet II, Src: IntelCor_12:7d:bb (38:ba:f8:12:7d:bb), Dst: Ubiquiti_49:e0:10 (fc:ec:da:49:e0:10)
    Destination: Ubiquiti_49:e0:10 (fc:ec:da:49:e0:10)
        Address: Ubiquiti_49:e0:10 (fc:ec:da:49:e0:10)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Source: IntelCor_12:7d:bb (38:ba:f8:12:7d:bb)
        Address: IntelCor_12:7d:bb (38:ba:f8:12:7d:bb)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 192.168.4.96, Dst: 52.21.18.219
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
        0000 00.. = Differentiated Services Codepoint: Default (0)
        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
    Total Length: 52
    Identification: 0xd98c (55692)
    Flags: 0x4000, Don't fragment
        0... .... .... .... = Reserved bit: Not set
        .1.. .... .... .... = Don't fragment: Set
        ..0. .... .... .... = More fragments: Not set
        ...0 0000 0000 0000 = Fragment offset: 0
    Time to live: 64
    Protocol: TCP (6)
    Header checksum: 0x553f [validation disabled]
    [Header checksum status: Unverified]
    Source: 192.168.4.96
    Destination: 52.21.18.219
Transmission Control Protocol, Src Port: 38828, Dst Port: 443, Seq: 1, Ack: 1, Len: 0
    Source Port: 38828
    Destination Port: 443
    [Stream index: 0]
    [TCP Segment Len: 0]
    Sequence number: 1    (relative sequence number)
    [Next sequence number: 1    (relative sequence number)]
    Acknowledgment number: 1    (relative ack number)
    1000 .... = Header Length: 32 bytes (8)
    Flags: 0x010 (ACK)
        000. .... .... = Reserved: Not set
        ...0 .... .... = Nonce: Not set
        .... 0... .... = Congestion Window Reduced (CWR): Not set
        .... .0.. .... = ECN-Echo: Not set
        .... ..0. .... = Urgent: Not set
        .... ...1 .... = Acknowledgment: Set
        .... .... 0... = Push: Not set
        .... .... .0.. = Reset: Not set
        .... .... ..0. = Syn: Not set
        .... .... ...0 = Fin: Not set
        [TCP Flags: ·······A····]
    Window size value: 296
    [Calculated window size: 296]
    [Window size scaling factor: -1 (unknown)]
    Checksum: 0x08b0 [unverified]
    [Checksum Status: Unverified]
    Urgent pointer: 0
    Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
        TCP Option - No-Operation (NOP)
            Kind: No-Operation (1)
        TCP Option - No-Operation (NOP)
            Kind: No-Operation (1)
        TCP Option - Timestamps: TSval 26210782, TSecr 2652693036
            Kind: Time Stamp Option (8)
            Length: 10
            Timestamp value: 26210782
            Timestamp echo reply: 2652693036
    [Timestamps]
        [Time since first frame in this TCP stream: 0.000000000 seconds]
        [Time since previous frame in this TCP stream: 0.000000000 seconds]

0000  fc ec da 49 e0 10 38 ba f8 12 7d bb 08 00 45 00   ...I..8...}...E.
0010  00 34 d9 8c 40 00 40 06 55 3f c0 a8 04 60 34 15   .4..@.@.U?...`4.
0020  12 db 97 ac 01 bb e3 42 2a 57 83 49 c2 ea 80 10   .......B*W.I....
0030  01 28 08 b0 00 00 01 01 08 0a 01 8f f1 de 9e 1c   .(..............
0040  e2 2c   

You can see Wireshark understands what it is seeing. It decodes the IP header and the TCP header.

So far so good. Here is an example of the weird traffic I was seeing.



Here is what Tshark thinks of it.

$ tshark -Vx -r frame4241.pcap
Frame 1: 66 bytes on wire (528 bits), 66 bytes captured (528 bits)
    Encapsulation type: Ethernet (1)
    Arrival Time: May  7, 2019 18:19:10.073296000 UTC
    [Time shift for this packet: 0.000000000 seconds]
    Epoch Time: 1557253150.073296000 seconds
    [Time delta from previous captured frame: 0.000000000 seconds]
    [Time delta from previous displayed frame: 0.000000000 seconds]
    [Time since reference or first frame: 0.000000000 seconds]
    Frame Number: 1
    Frame Length: 66 bytes (528 bits)
    Capture Length: 66 bytes (528 bits)
    [Frame is marked: False]
    [Frame is ignored: False]
    [Protocols in frame: eth:llc:data]
IEEE 802.3 Ethernet
    Destination: Ubiquiti_49:e0:10 (fc:ec:da:49:e0:10)
        Address: Ubiquiti_49:e0:10 (fc:ec:da:49:e0:10)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Source: IntelCor_12:7d:bb (38:ba:f8:12:7d:bb)
        Address: IntelCor_12:7d:bb (38:ba:f8:12:7d:bb)
        .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
        .... ...0 .... .... .... .... = IG bit: Individual address (unicast)
    Length: 56
        [Expert Info (Error/Malformed): Length field value goes past the end of the payload]
            [Length field value goes past the end of the payload]
            [Severity level: Error]
            [Group: Malformed]
Logical-Link Control
    DSAP: Unknown (0x45)
        0100 010. = SAP: Unknown
        .... ...1 = IG Bit: Group
    SSAP: LLC Sub-Layer Management (0x02)
        0000 001. = SAP: LLC Sub-Layer Management
        .... ...0 = CR Bit: Command
    Control field: U, func=Unknown (0x0B)
        000. 10.. = Command: Unknown (0x02)
        .... ..11 = Frame type: Unnumbered frame (0x3)
Data (49 bytes)
    Data: 84d98d86b5400649eec0a80460341512db97ac0d0be3422a...
    [Length: 49]

0000  fc ec da 49 e0 10 38 ba f8 12 7d bb 00 38 45 02   ...I..8...}..8E.
0010  0b 84 d9 8d 86 b5 40 06 49 ee c0 a8 04 60 34 15   ......@.I....`4.
0020  12 db 97 ac 0d 0b e3 42 2a 57 83 49 c2 ea c8 ec   .......B*W.I....
0030  01 28 17 6f 00 00 01 01 08 0a 01 8f f1 de ed 7f   .(.o............
0040  a5 4a                                             .J

What's the problem? This frame begins with 6 bytes of the destination MAC address and 6 bytes of the source MAC address, as we saw before. However, the next two bytes are 0x0038, which is not the same as the Ethertype of 0x0800 we saw earlier. 0x0038 is decimal 56, which would seem to indicate a frame length (even though the frame here is a total of 66 bytes).

Wireshark decides to treat this frame as not being Ethernet II, but instead as IEEE 802.3 Ethernet. I had to refer to appendix A of my first book to see what this meant.

For comparison, here is the frame format for Ethernet II (page 664):

This was what we saw with frame 4238 earlier -- Dst MAC, Src MAC, Ethertype, then data.

Here is the frame format for IEEE 802.3 Ethernet.


This is much more complicated: Dst MAC, Src MAC, length, and then DSAP, SSAP, Control, and data.

It turns out that this format doesn't seem to fit what is happening in frame 4241, either. While the length field appears to be in the ballpark, Wireshark's assumption that the next bytes are DSAP, SSAP, Control, and data doesn't fit. The clue for me was seeing that 0x45 followed the presumed length field. I recognized 0x45 as the beginning of an IP header, with 4 meaning IPv4 and 5 meaning 5 words (40 bytes) in the IP header.

If we take a manual byte-by-byte comparative approach we can better understand what may be happening with these two frames. (I broke the 0x45 byte into two "nibbles" in one case.)

Note that I have bolded the parts of each frame that are exactly the same.


This analysis shows that these two frames are very similar, especially in places where I would not expect them to be similar. This caused me to hypothesize that frame 4241 was a corrupted version of frame 4238.

I can believe that the frames would share MAC addresses, IP addresses, and certain IP and TCP defaults. However, it is unusual for them to have the same high source ports (38828) but not the same destination ports (443 and 3339).  Very telling is the fact that they have the same TCP sequence and acknowledgement numbers. They also share the same source timestamp.

Notice one field that I did not bold, because they are not identical -- the IP ID value. Frame 4238 has 0xd98c and frame 4241 has 0xd98d. The perfectly incremented IP ID prompted me to believe that frame 4241 is a corrupted retransmission, at the IP layer, of the same TCP segment.

However, I really don't know what to think. These frames were captured in a Linux 16.04 VirtualBox VM by netsniff-ng. Is this a problem with netsniff-ng, or Linux, or VirtualBox, or the Linux host operating system running VirtualBox?

I'd like to thank the folks at ask.wireshark.org for their assistance with my attempts to decode this (and other) frames as 802.3 raw Ethernet. What's that? It's basically a format that Novell used with IPX, where the frame is Dst MAC, Src MAC, length, data.

I wanted to see if I could tell Wireshark to decode the odd frames as 802.3 raw Ethernet, rather than IEEE 802.3 Ethernet with LLC headers.

Sake Blok helpfully suggested I change the pcap's link layer type to User0, and then tell Wireshark how to interpret the frames. I did it this way, per his direction:

$ editcap -T user0 excerpt.pcap excerpt-user0.pcap

Next I opened the trace in Wireshark and saw frame 4241 (here listed as frame 3) as shown below:


DLT 147 corresponds to the link layer type for User0. Wireshark doesn't know how to handle it. We fix that by right-clicking on the yellow field and selecting Protocol Preferences -> Open DLT User preferences:

Next I created an entry fpr User 0 (DLT-147) with Payload protocol "ip" and Header size "14" as shown:

After clicking OK, I returned to Wireshark. Here is how frame 4241 (again listed here as frame 3) appeared:


You can see Wireshark is now making sense of the IP header, but it doesn't know how to handle the TCP header which follows. I tried different values and options to see if I could get Wireshark to understand the TCP header too, but this went far enough for my purposes.

The bottom line is that I believe there is some sort of packet capture problem, either with the softare used or the traffic that is presented to the software by the bridged NIC created by VirtualBox. As this is a lab environment and the traffic is 1% of the overall capture, I am not worried about the results.

I am fairly sure that the weird traffic is not on the wire. I tried capturing on the host OS sniffing NIC and did not see anything resembling this traffic.

Have you seen anything like this? Let me know in a comment here on on Twitter.

PS: I found the frame.number=X Wireshark display filter helpful, along with the frame.len>Y display filter, when researching this activity.