[pfSense] IPv6 & HE.net tunnel - MTU problem confirmed

[Adam Thompson]

I'm having the same problem as a recent reporter (whose email I already can't find).
I've got a tunnel set up to HE.NET, and experience difficulty browsing to (e.g.) redmine.pfsense.org.
Testing shows that the largest ICMP payload I can exchange is 1232 bytes ("ping -l 1232 redmine.pfsense.org" works, 1233 doesn't).
If I stop and reload the page in my browser, everything works fine - I don't know yet if that's because the browser falls back to IPv4 or because the MTU problem suddenly fixes itself.

-Adam Thompson
***@athompso.net
Tel: [phone]
Fax: [phone]

[Adam Hunt]

Thanks for confirming this. I'm glad that I'm not the only one and/or I'm
not completely inept. I'll sit down later today and play with the various
MTU settings (WAN, HEv6 tunnel, and the setting on the "advanced tab" of
Tunnel Broker's site) and see what, if anything, I can get to work
consistently.

I don't know what browser you use but I found a simple Chrome extension
that has been helpful in determining what protocol (v4/v6) is being using
used to connect to any specific site. It's called IPvFoo and is available
in the webstore (http://goo.gl/kxKVhx). It adds a little 4 or 6 icon on the
right of the URI bar that when clicked on shows what portions of the page
were served using what protocol.

Again, thanks for confirming this. At certain points I was beginning to
doubt myself as things would work on second and break for seemingly no
reason the next.

--adam

Post by Adam Thompson
I'm having the same problem as a recent reporter (whose email I already can't find).
I've got a tunnel set up to HE.NET, and experience difficulty browsing to
(e.g.) redmine.pfsense.org.
Testing shows that the largest ICMP payload I can exchange is 1232 bytes
("ping -l 1232 redmine.pfsense.org" works, 1233 doesn't).
If I stop and reload the page in my browser, everything works fine - I
don't know yet if that's because the browser falls back to IPv4 or because
the MTU problem suddenly fixes itself.
-Adam Thompson
Tel: [phone]
Fax: [phone]
_______________________________________________
List mailing list
http://lists.pfsense.org/mailman/listinfo/list

[Jim Thompson]

Thanks for confirming this. I'm glad that I'm not the only one and/or I'm not completely inept. I'll sit down later today and play with the various MTU settings (WAN, HEv6 tunnel, and the setting on the "advanced tab" of Tunnel Broker's site) and see what, if anything, I can get to work consistently.
I don't know what browser you use but I found a simple Chrome extension that has been helpful in determining what protocol (v4/v6) is being using used to connect to any specific site. It's called IPvFoo and is available in the webstore (http://goo.gl/kxKVhx). It adds a little 4 or 6 icon on the right of the URI bar that when clicked on shows what portions of the page were served using what protocol.
Again, thanks for confirming this. At certain points I was beginning to doubt myself as things would work on second and break for seemingly no reason the next.
--adam
I'm having the same problem as a recent reporter (whose email I already can't find).
I've got a tunnel set up to HE.NET, and experience difficulty browsing to (e.g.) redmine.pfsense.org.
Testing shows that the largest ICMP payload I can exchange is 1232 bytes ("ping -l 1232 redmine.pfsense.org" works, 1233 doesn't).
If I stop and reload the page in my browser, everything works fine - I don't know yet if that's because the browser falls back to IPv4 or because the MTU problem suddenly fixes itself.
-Adam Thompson
Tel: [phone]
Fax: [phone]

Hi Adam (and Adam),

Seems easy enough to reproduce, assuming that my substitution of '-s' for '-l' is legit.

jims-mini:~ jim$ ping6 -s 1232 redmine.pfsense.org
PING6(1280=40+8+1232 bytes) 2610:160:11:33:84b5:f958:6545:af1c --> 2610:160:11:3::100
1240 bytes from 2610:160:11:3::100, icmp_seq=0 hlim=62 time=1.625 ms
^C
--- redmine.pfsense.org ping6 statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.625/1.625/1.625/0.000 ms

jims-mini:~ jim$ ping6 -s 1233 redmine.pfsense.org
PING6(1281=40+8+1233 bytes) 2610:160:11:33:84b5:f958:6545:af1c --> 2610:160:11:3::100
^C
--- redmine.pfsense.org ping6 statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss

Note that I'm … "really close".

jims-mini:~ jim$ traceroute6 redmine.pfsense.org
traceroute6 to redmine.pfsense.org (2610:160:11:3::100) from 2610:160:11:33:84b5:f958:6545:af1c, 64 hops max, 12 byte packets
1 2610:160:11:33::2 1.888 ms 1.861 ms 1.461 ms
2 2610:160:11:12::2 1.984 ms 2.107 ms 2.303 ms
3 2610:160:11:3::100 2.172 ms 2.275 ms 2.250 ms
jims-mini:~ jim$

Given same, it almost has to be the pfSense box, since once I'm on redmine, huge packets pass.

***@redmine:/home/jim % traceroute6 -n he.net
traceroute6 to he.net (2001:470:0:76::2) from 2610:160:11:3::100, 64 hops max, 12 byte packets
1 2610:160:11:3::2 0.381 ms 0.336 ms 0.349 ms
2 2610:160:11::1 4.210 ms 1.249 ms 2.435 ms
3 2610:160:0:11::4 2.556 ms 2.611 ms 0.993 ms
4 2610:160:0:53::17 10.253 ms 10.212 ms 10.408 ms
5 2001:504:0:5::6939:1 12.735 ms 10.145 ms 15.192 ms
6 2001:470:0:258::1 32.502 ms 27.384 ms 27.439 ms
7 2001:470:0:24a::2 62.184 ms 43.638 ms 43.681 ms
8 2001:470:0:16a::1 53.841 ms 46.596 ms 53.421 ms
9 2001:470:0:2f::1 59.776 ms
2001:470:0:18d::1 46.394 ms 46.766 ms
10 2001:470:0:2d::1 55.180 ms 49.954 ms 49.308 ms
11 2001:470:0:76::2 50.513 ms 50.814 ms 50.959 ms
***@redmine:/home/jim % sudo ping6 -s 3500 redmine.pfsense.org
PING6(3548=40+8+3500 bytes) 2610:160:11:3::100 --> 2610:160:11:3::100
3508 bytes from 2610:160:11:3::100, icmp_seq=0 hlim=64 time=0.106 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=1 hlim=64 time=0.074 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=2 hlim=64 time=0.076 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=3 hlim=64 time=0.069 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=4 hlim=64 time=0.074 ms
^C
--- redmine.pfsense.org ping6 statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.069/0.080/0.106/0.013 ms
***@redmine:/home/jim %


That said, I'm on redmine with IPvFoo loaded, and it's reporting that I'm hitting the IPv6 site, and I'm not having any issues.

We'll look into it and get back to you.

jim

[Adam Hunt]

What do you have your MTUs and MSS set at on each of the interfaces? From
what I can tell the interfaces that might play a roll in this issue are the
WAN link, the tunnel link, and the MTU on the Tunnel Broker site.

I have to move some furniture for the next couple hours. After that I'll
try to sit down and experiment with various sized packets using ping.

Thanks for the help.

Post by Adam Hunt
Thanks for confirming this. I'm glad that I'm not the only one and/or I'm
not completely inept. I'll sit down later today and play with the various
MTU settings (WAN, HEv6 tunnel, and the setting on the "advanced tab" of
Tunnel Broker's site) and see what, if anything, I can get to work
consistently.
I don't know what browser you use but I found a simple Chrome extension
that has been helpful in determining what protocol (v4/v6) is being using
used to connect to any specific site. It's called IPvFoo and is available
in the webstore (http://goo.gl/kxKVhx). It adds a little 4 or 6 icon on
the right of the URI bar that when clicked on shows what portions of the
page were served using what protocol.
Again, thanks for confirming this. At certain points I was beginning to
doubt myself as things would work on second and break for seemingly no
reason the next.
--adam

Post by Adam Thompson
I'm having the same problem as a recent reporter (whose email I already can't find).
I've got a tunnel set up to HE.NET <http://he.net/>, and experience
difficulty browsing to (e.g.) redmine.pfsense.org.
Testing shows that the largest ICMP payload I can exchange is 1232 bytes
("ping -l 1232 redmine.pfsense.org" works, 1233 doesn't).
If I stop and reload the page in my browser, everything works fine - I
don't know yet if that's because the browser falls back to IPv4 or because
the MTU problem suddenly fixes itself.
-Adam Thompson
Tel: [phone]
Fax: [phone]

Hi Adam (and Adam),
Seems easy enough to reproduce, assuming that my substitution of '-s' for '-l' is legit.
jims-mini:~ jim$ ping6 -s 1232 redmine.pfsense.org
PING6(1280=40+8+1232 bytes) 2610:160:11:33:84b5:f958:6545:af1c --> 2610:160:11:3::100
1240 bytes from 2610:160:11:3::100, icmp_seq=0 hlim=62 time=1.625 ms
^C
--- redmine.pfsense.org ping6 statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.625/1.625/1.625/0.000 ms
jims-mini:~ jim$ ping6 -s 1233 redmine.pfsense.org
PING6(1281=40+8+1233 bytes) 2610:160:11:33:84b5:f958:6545:af1c --> 2610:160:11:3::100
^C
--- redmine.pfsense.org ping6 statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
Note that I'm 
 "really close".
jims-mini:~ jim$ traceroute6 redmine.pfsense.org
traceroute6 to redmine.pfsense.org (2610:160:11:3::100) from
2610:160:11:33:84b5:f958:6545:af1c, 64 hops max, 12 byte packets
1 2610:160:11:33::2 1.888 ms 1.861 ms 1.461 ms
2 2610:160:11:12::2 1.984 ms 2.107 ms 2.303 ms
3 2610:160:11:3::100 2.172 ms 2.275 ms 2.250 ms
jims-mini:~ jim$
Given same, it almost has to be the pfSense box, since once I'm on
redmine, huge packets pass.
traceroute6 to he.net (2001:470:0:76::2) from 2610:160:11:3::100, 64 hops
max, 12 byte packets
1 2610:160:11:3::2 0.381 ms 0.336 ms 0.349 ms
2 2610:160:11::1 4.210 ms 1.249 ms 2.435 ms
3 2610:160:0:11::4 2.556 ms 2.611 ms 0.993 ms
4 2610:160:0:53::17 10.253 ms 10.212 ms 10.408 ms
5 2001:504:0:5::6939:1 12.735 ms 10.145 ms 15.192 ms
6 2001:470:0:258::1 32.502 ms 27.384 ms 27.439 ms
7 2001:470:0:24a::2 62.184 ms 43.638 ms 43.681 ms
8 2001:470:0:16a::1 53.841 ms 46.596 ms 53.421 ms
9 2001:470:0:2f::1 59.776 ms
2001:470:0:18d::1 46.394 ms 46.766 ms
10 2001:470:0:2d::1 55.180 ms 49.954 ms 49.308 ms
11 2001:470:0:76::2 50.513 ms 50.814 ms 50.959 ms
PING6(3548=40+8+3500 bytes) 2610:160:11:3::100 --> 2610:160:11:3::100
3508 bytes from 2610:160:11:3::100, icmp_seq=0 hlim=64 time=0.106 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=1 hlim=64 time=0.074 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=2 hlim=64 time=0.076 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=3 hlim=64 time=0.069 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=4 hlim=64 time=0.074 ms
^C
--- redmine.pfsense.org ping6 statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.069/0.080/0.106/0.013 ms
That said, I'm on redmine with IPvFoo loaded, and it's reporting that I'm
hitting the IPv6 site, and I'm not having any issues.
We'll look into it and get back to you.
jim
_______________________________________________
List mailing list
http://lists.pfsense.org/mailman/listinfo/list

[Adam Hunt]

Have you tried the -m option with ping6? According to the FreeBSD man page
it will suppress fragmentation of the ICMP packets. This might help find
the MTU minimum for the path in question.

Post by Adam Hunt
What do you have your MTUs and MSS set at on each of the interfaces? From
what I can tell the interfaces that might play a roll in this issue are the
WAN link, the tunnel link, and the MTU on the Tunnel Broker site.
I have to move some furniture for the next couple hours. After that I'll
try to sit down and experiment with various sized packets using ping.
Thanks for the help.

Post by Adam Hunt
Thanks for confirming this. I'm glad that I'm not the only one and/or I'm
not completely inept. I'll sit down later today and play with the various
MTU settings (WAN, HEv6 tunnel, and the setting on the "advanced tab" of
Tunnel Broker's site) and see what, if anything, I can get to work
consistently.
I don't know what browser you use but I found a simple Chrome extension
that has been helpful in determining what protocol (v4/v6) is being using
used to connect to any specific site. It's called IPvFoo and is available
in the webstore (http://goo.gl/kxKVhx). It adds a little 4 or 6 icon on
the right of the URI bar that when clicked on shows what portions of the
page were served using what protocol.
Again, thanks for confirming this. At certain points I was beginning to
doubt myself as things would work on second and break for seemingly no
reason the next.
--adam

Post by Adam Thompson
I'm having the same problem as a recent reporter (whose email I already can't find).
I've got a tunnel set up to HE.NET <http://he.net/>, and experience
difficulty browsing to (e.g.) redmine.pfsense.org.
Testing shows that the largest ICMP payload I can exchange is 1232 bytes
("ping -l 1232 redmine.pfsense.org" works, 1233 doesn't).
If I stop and reload the page in my browser, everything works fine - I
don't know yet if that's because the browser falls back to IPv4 or because
the MTU problem suddenly fixes itself.
-Adam Thompson
Tel: [phone]
Fax: [phone]

Hi Adam (and Adam),
Seems easy enough to reproduce, assuming that my substitution of '-s' for '-l' is legit.
jims-mini:~ jim$ ping6 -s 1232 redmine.pfsense.org
PING6(1280=40+8+1232 bytes) 2610:160:11:33:84b5:f958:6545:af1c --> 2610:160:11:3::100
1240 bytes from 2610:160:11:3::100, icmp_seq=0 hlim=62 time=1.625 ms
^C
--- redmine.pfsense.org ping6 statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 1.625/1.625/1.625/0.000 ms
jims-mini:~ jim$ ping6 -s 1233 redmine.pfsense.org
PING6(1281=40+8+1233 bytes) 2610:160:11:33:84b5:f958:6545:af1c --> 2610:160:11:3::100
^C
--- redmine.pfsense.org ping6 statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
Note that I'm 
 "really close".
jims-mini:~ jim$ traceroute6 redmine.pfsense.org
traceroute6 to redmine.pfsense.org (2610:160:11:3::100) from
2610:160:11:33:84b5:f958:6545:af1c, 64 hops max, 12 byte packets
1 2610:160:11:33::2 1.888 ms 1.861 ms 1.461 ms
2 2610:160:11:12::2 1.984 ms 2.107 ms 2.303 ms
3 2610:160:11:3::100 2.172 ms 2.275 ms 2.250 ms
jims-mini:~ jim$
Given same, it almost has to be the pfSense box, since once I'm on
redmine, huge packets pass.
traceroute6 to he.net (2001:470:0:76::2) from 2610:160:11:3::100, 64
hops max, 12 byte packets
1 2610:160:11:3::2 0.381 ms 0.336 ms 0.349 ms
2 2610:160:11::1 4.210 ms 1.249 ms 2.435 ms
3 2610:160:0:11::4 2.556 ms 2.611 ms 0.993 ms
4 2610:160:0:53::17 10.253 ms 10.212 ms 10.408 ms
5 2001:504:0:5::6939:1 12.735 ms 10.145 ms 15.192 ms
6 2001:470:0:258::1 32.502 ms 27.384 ms 27.439 ms
7 2001:470:0:24a::2 62.184 ms 43.638 ms 43.681 ms
8 2001:470:0:16a::1 53.841 ms 46.596 ms 53.421 ms
9 2001:470:0:2f::1 59.776 ms
2001:470:0:18d::1 46.394 ms 46.766 ms
10 2001:470:0:2d::1 55.180 ms 49.954 ms 49.308 ms
11 2001:470:0:76::2 50.513 ms 50.814 ms 50.959 ms
PING6(3548=40+8+3500 bytes) 2610:160:11:3::100 --> 2610:160:11:3::100
3508 bytes from 2610:160:11:3::100, icmp_seq=0 hlim=64 time=0.106 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=1 hlim=64 time=0.074 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=2 hlim=64 time=0.076 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=3 hlim=64 time=0.069 ms
3508 bytes from 2610:160:11:3::100, icmp_seq=4 hlim=64 time=0.074 ms
^C
--- redmine.pfsense.org ping6 statistics ---
5 packets transmitted, 5 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.069/0.080/0.106/0.013 ms
That said, I'm on redmine with IPvFoo loaded, and it's reporting that I'm
hitting the IPv6 site, and I'm not having any issues.
We'll look into it and get back to you.
jim
_______________________________________________
List mailing list
http://lists.pfsense.org/mailman/listinfo/list

[Adam Thompson]

My MTU size was 1280; since the patch applied to fix bug #2674, I can now override this to match the 1480 MTU on the HE side
 I think.

At least, on tunnelbroker.net, the MTU is set to 1480, and on my end, the MTU is set to 1480.

Now I can pass ICMP with a payload of up to 1432 bytes. The root-cause-iest symptom now is that setting the DF bit has *no effect* on whether larger packets make it through or not. This makes me suspect that someone (Myself? HE? Corenap?) is dropping fragmented packets.

I can observe the identical behaviour from the GIF endpoint (pfSense 2.1, build from two days ago IIRC) and from hosts on the LAN behind it.



-Adam Thompson

<mailto:***@athompso.net> ***@athompso.net





From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On Behalf Of Adam Hunt
Sent: Thursday, August 15, 2013 1:47 PM
To: pfSense support and discussion
Subject: Re: [pfSense] IPv6 & HE.net tunnel - MTU problem confirmed



Have you tried the -m option with ping6? According to the FreeBSD man page it will suppress fragmentation of the ICMP packets. This might help find the MTU minimum for the path in question.



On Thu, Aug 15, 2013 at 11:40 AM, Adam Hunt <***@gmail.com <mailto:***@gmail.com> > wrote:

What do you have your MTUs and MSS set at on each of the interfaces?

[Adam Thompson]

Even weirder


Although I can successfully ping at payload sizes up to 1432, I see another more troubling problem: there’s a “hole” where it works with payloads up to 1232, fails with payloads between 1233 and 1255 inclusive, then works again with payloads 1256 bytes and above. WTF????



-Adam Thompson

<mailto:***@athompso.net> ***@athompso.net

Tel: [phone]

Fax: [phone]



From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On Behalf Of Adam Thompson
Sent: Thursday, August 15, 2013 3:14 PM
To: 'pfSense support and discussion'
Subject: Re: [pfSense] IPv6 & HE.net tunnel - MTU problem confirmed



My MTU size was 1280; since the patch applied to fix bug #2674, I can now override this to match the 1480 MTU on the HE side
 I think.

At least, on tunnelbroker.net, the MTU is set to 1480, and on my end, the MTU is set to 1480.

Now I can pass ICMP with a payload of up to 1432 bytes. The root-cause-iest symptom now is that setting the DF bit has *no effect* on whether larger packets make it through or not. This makes me suspect that someone (Myself? HE? Corenap?) is dropping fragmented packets.

I can observe the identical behaviour from the GIF endpoint (pfSense 2.1, build from two days ago IIRC) and from hosts on the LAN behind it.



-Adam Thompson

<mailto:***@athompso.net> ***@athompso.net





From: list-***@lists.pfsense.org <mailto:list-***@lists.pfsense.org> [mailto:list-***@lists.pfsense.org] On Behalf Of Adam Hunt
Sent: Thursday, August 15, 2013 1:47 PM
To: pfSense support and discussion
Subject: Re: [pfSense] IPv6 & HE.net tunnel - MTU problem confirmed



Have you tried the -m option with ping6? According to the FreeBSD man page it will suppress fragmentation of the ICMP packets. This might help find the MTU minimum for the path in question.



On Thu, Aug 15, 2013 at 11:40 AM, Adam Hunt <***@gmail.com <mailto:***@gmail.com> > wrote:

What do you have your MTUs and MSS set at on each of the interfaces?

[Chris Buechler]

Even weirder…
Although I can successfully ping at payload sizes up to 1432, I see another more troubling problem: there’s a “hole” where it works
with payloads up to 1232, fails with payloads between 1233 and 1255 inclusive, then works again with payloads 1256 bytes and above. > WTF????

The original scenario, the diff between 1232 and 1233 is that at 1233,
the echo request no longer fits in the minimum IPv6 size, so it's
fragmented.
20:16:33.241123 IP6 2610:160:11:33::230 > 2610:160:11:3::100: frag
(0|1232) ICMP6, echo request, seq 2, length 1232
20:16:33.241129 IP6 2610:160:11:33::230 > 2610:160:11:3::100: frag (1232|176)

no response to the fragmented request.

20:16:37.260945 IP6 2610:160:11:33::230 > 2610:160:11:3::100: ICMP6,
echo request, seq 0, length 1408
20:16:37.262526 IP6 2610:160:11:3::100 > 2610:160:11:33::230: ICMP6,
echo reply, seq 0, length 1408

bigger request that isn't fragmented is fine.

If you don't specify -m on ping6 (at least with the FreeBSD ping6,
others are likely similar), ping6 asks the kernel to fragment packets
to fit the minimum IPv6 MTU, 1280.

PF has issues with v6 fragmentation that we won't be able to address
until 2.2, which is the root of the problem.

[Adam Hunt]

Thanks for the explanation Chris. I did run across a bug report that seems
to be exactly what we're running into (
http://redmine.pfsense.org/issues/2129).

Are the issues with v6 fragmentation inherent to FreeBSD 8.3 that pfSesne
2.1 is based on? Also, are there any workarounds for those of us running
2.1? I'm not sure when 2.2 will be tagged but it would great if there was
some way, maybe by adjusting the MTU and/or MSS values, that those of us
affected by this bug could use get their v6 tunnels up and running, even if
not at their theoretical peak efficiency.

Thanks for all the help. I realize IPv6 support can be more than a little
tricky. I really appreciate all the work that everyone has done on pfSense,
it's a great tool.

--adam

Post by Adam Thompson

Post by Adam Thompson
Even weirder

Although I can successfully ping at payload sizes up to 1432, I see

another more troubling problem: there’s a “hole” where it works

Post by Adam Thompson
with payloads up to 1232, fails with payloads between 1233 and 1255

inclusive, then works again with payloads 1256 bytes and above. > WTF????
The original scenario, the diff between 1232 and 1233 is that at 1233,
the echo request no longer fits in the minimum IPv6 size, so it's
fragmented.
20:16:33.241123 IP6 2610:160:11:33::230 > 2610:160:11:3::100: frag
(0|1232) ICMP6, echo request, seq 2, length 1232
20:16:33.241129 IP6 2610:160:11:33::230 > 2610:160:11:3::100: frag (1232|176)
no response to the fragmented request.
20:16:37.260945 IP6 2610:160:11:33::230 > 2610:160:11:3::100: ICMP6,
echo request, seq 0, length 1408
20:16:37.262526 IP6 2610:160:11:3::100 > 2610:160:11:33::230: ICMP6,
echo reply, seq 0, length 1408
bigger request that isn't fragmented is fine.
If you don't specify -m on ping6 (at least with the FreeBSD ping6,
others are likely similar), ping6 asks the kernel to fragment packets
to fit the minimum IPv6 MTU, 1280.
PF has issues with v6 fragmentation that we won't be able to address
until 2.2, which is the root of the problem.
_______________________________________________
List mailing list
http://lists.pfsense.org/mailman/listinfo/list

[Adam Thompson]

I'm very glad this email thread has occurred... I was hoping to deploy two pfSense boxes as IPv6 routers.
Now I'm wondering if I should just put in OpenBSD at least for now?
-Adam

Thanks for the explanation Chris. I did run across a bug report that seems to be exactly what we're running into (http://redmine.pfsense.org/issues/2129).
Are the issues with v6 fragmentation inherent to FreeBSD 8.3 that pfSesne 2.1 is based on? Also, are there any workarounds for those of us running 2.1? I'm not sure when 2.2 will be tagged but it would great if there was some way, maybe by adjusting the MTU and/or MSS values, that those of us affected by this bug could use get their v6 tunnels up and running, even if not at their theoretical peak efficiency.
Thanks for all the help. I realize IPv6 support can be more than a little tricky. I really appreciate all the work that everyone has done on pfSense, it's a great tool.
--adam

Post by Adam Thompson
Even weirder

Although I can successfully ping at payload sizes up to 1432, I see another more troubling problem:  there’s a “hole” where it works
with payloads up to 1232, fails with payloads between 1233 and 1255 inclusive, then works again with payloads 1256 bytes and above. > WTF????

The original scenario, the diff between 1232 and 1233 is that at 1233,
the echo request no longer fits in the minimum IPv6 size, so it's
fragmented.
20:16:33.241123 IP6 2610:160:11:33::230 > 2610:160:11:3::100: frag
(0|1232) ICMP6, echo request, seq 2, length 1232
20:16:33.241129 IP6 2610:160:11:33::230 > 2610:160:11:3::100: frag (1232|176)
no response to the fragmented request.
20:16:37.260945 IP6 2610:160:11:33::230 > 2610:160:11:3::100: ICMP6,
echo request, seq 0, length 1408
20:16:37.262526 IP6 2610:160:11:3::100 > 2610:160:11:33::230: ICMP6,
echo reply, seq 0, length 1408
bigger request that isn't fragmented is fine.
If you don't specify -m on ping6 (at least with the FreeBSD ping6,
others are likely similar), ping6 asks the kernel to fragment packets
to fit the minimum IPv6 MTU, 1280.
PF has issues with v6 fragmentation that we won't be able to address
until 2.2, which is the root of the problem.
_______________________________________________
List mailing list
http://lists.pfsense.org/mailman/listinfo/list

[Adam Hunt]

That's an interesting idea. Would there be anything keeping me from using a
my pfSense box as-is for native IPv4 connectivity while using a second box
running OpenBSD or dare I say, Linux as my IPv6 gateway connected to HE via
a 6in4 tunnel? Would I still be able to use pfSense's DHCPv6 server to
create and maintain v6 leases?

Thanks again.

Post by Adam Thompson
I'm very glad this email thread has occurred... I was hoping to deploy two
pfSense boxes as IPv6 routers.
Now I'm wondering if I should just put in OpenBSD at least for now?
-Adam
Thanks for the explanation Chris. I did run across a bug report that seems
to be exactly what we're running into (
http://redmine.pfsense.org/issues/2129).
Are the issues with v6 fragmentation inherent to FreeBSD 8.3 that pfSesne
2.1 is based on? Also, are there any workarounds for those of us running
2.1? I'm not sure when 2.2 will be tagged but it would great if there was
some way, maybe by adjusting the MTU and/or MSS values, that those of us
affected by this bug could use get their v6 tunnels up and running, even if
not at their theoretical peak efficiency.
Thanks for all the help. I realize IPv6 support can be more than a little
tricky. I really appreciate all the work that everyone has done on pfSense,
it's a great tool.
--adam

Post by Adam Thompson

Post by Adam Thompson
Even weirder

Although I can successfully ping at payload sizes up to 1432, I see

another more troubling problem: there’s a “hole” where it works

Post by Adam Thompson
with payloads up to 1232, fails with payloads between 1233 and 1255

inclusive, then works again with payloads 1256 bytes and above. > WTF????
The original scenario, the diff between 1232 and 1233 is that at 1233,
the echo request no longer fits in the minimum IPv6 size, so it's
fragmented.
20:16:33.241123 IP6 2610:160:11:33::230 > 2610:160:11:3::100: frag
(0|1232) ICMP6, echo request, seq 2, length 1232
20:16:33.241129 IP6 2610:160:11:33::230 > 2610:160:11:3::100: frag (1232|176)
no response to the fragmented request.
20:16:37.260945 IP6 2610:160:11:33::230 > 2610:160:11:3::100: ICMP6,
echo request, seq 0, length 1408
20:16:37.262526 IP6 2610:160:11:3::100 > 2610:160:11:33::230: ICMP6,
echo reply, seq 0, length 1408
bigger request that isn't fragmented is fine.
If you don't specify -m on ping6 (at least with the FreeBSD ping6,
others are likely similar), ping6 asks the kernel to fragment packets
to fit the minimum IPv6 MTU, 1280.
PF has issues with v6 fragmentation that we won't be able to address
until 2.2, which is the root of the problem.
_______________________________________________
List mailing list
http://lists.pfsense.org/mailman/listinfo/list