[pfSense] two GWs in WAN, correct static routes to second GW however deault is used and second GW ignored

[Oleg Cherkasov]

Hi,

I am setting up static routes on WAN with two gateways. One gateway is
default ISP and the second is a private network however both are in
public WAN net. I may ping both gateways and of course the default one
works flawlessly. Second GW works ok using other FW GW from other
networks. Both GW are in the same WAN network, the same subnet.

Status shows both gateways are online and I have added static rules to
direct traffic to 4 IPs to the second gateway so I may access resources
in private network via second gateway in WAN network.

All statuses and suggested diagnostics looks good indeed, gateways are
online and static routes are up however whatever I do the default
gateway is used! I am running traceroute/tracepath from clients behind
the firewall and from pfSense WAN itself but it always uses default
gateway and ignores active second gateway and static rules. I have
tried to reboot pfSense of course however the issue remains.

Anyone have any suggestion? How I may verbosely debug static routing?



Cheers,
Oleg

[Chris L]

Oleg -

WAN interfaces (interfaces with a gateway set on them) are treated differently.

The rule set forces all connections out that interface to a specific gateway (the interface gateway) with route-to.

You can add floating pass rules on WAN in the outbound direction to the destinations on the other side of that router (every network with that gateway as a static route) and probably a destination of the gateway address with no gateway set (the default gateway). That will disable route-to for that traffic.

If you want connections from the networks on the other side of the second gateway into pfSense you will need to disable reply-to on those pass rules or reply traffic will be forced to the interface gateway. Disable reply-to is in the advanced section of the rules.

Hi,
I am setting up static routes on WAN with two gateways. One gateway is default ISP and the second is a private network however both are in public WAN net. I may ping both gateways and of course the default one works flawlessly. Second GW works ok using other FW GW from other networks. Both GW are in the same WAN network, the same subnet.
Status shows both gateways are online and I have added static rules to direct traffic to 4 IPs to the second gateway so I may access resources in private network via second gateway in WAN network.
All statuses and suggested diagnostics looks good indeed, gateways are online and static routes are up however whatever I do the default gateway is used! I am running traceroute/tracepath from clients behind the firewall and from pfSense WAN itself but it always uses default gateway and ignores active second gateway and static rules. I have tried to reboot pfSense of course however the issue remains.
Anyone have any suggestion? How I may verbosely debug static routing?
Cheers,
Oleg
_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

[Oleg Cherkasov]

Hi Chris,

Thank you for tip! I have successfully added floating outbound rules
and it works now. Do I need to add static routes and firewall rules or
it would be enough to add just floating rules? I may see static rules
on WAN are redundant than.

Any thoughts about RIP/BGP/OSP routing if my second gateway advertise
routing tables? Do I need to add floating rules as well for advertised
routes via RIP/BGP/OSP? Or with EBFPd daemon it would be more flexible.


Thank you!

Oleg

Post by Chris L
Oleg -
WAN interfaces (interfaces with a gateway set on them) are treated differently.
The rule set forces all connections out that interface to a specific gateway (the interface gateway) with route-to.
You can add floating pass rules on WAN in the outbound direction to the destinations on the other side of that router (every network with that gateway as a static route) and probably a destination of the gateway address with no gateway set (the default gateway). That will disable route-to for that traffic.
If you want connections from the networks on the other side of the second gateway into pfSense you will need to disable reply-to on those pass rules or reply traffic will be forced to the interface gateway. Disable reply-to is in the advanced section of the rules.

Hi,
I am setting up static routes on WAN with two gateways. One gateway is default ISP and the second is a private network however both are in public WAN net. I may ping both gateways and of course the default one works flawlessly. Second GW works ok using other FW GW from other networks. Both GW are in the same WAN network, the same subnet.
Status shows both gateways are online and I have added static rules to direct traffic to 4 IPs to the second gateway so I may access resources in private network via second gateway in WAN network.
All statuses and suggested diagnostics looks good indeed, gateways are online and static routes are up however whatever I do the default gateway is used! I am running traceroute/tracepath from clients behind the firewall and from pfSense WAN itself but it always uses default gateway and ignores active second gateway and static rules. I have tried to reboot pfSense of course however the issue remains.
Anyone have any suggestion? How I may verbosely debug static routing?
Cheers,
Oleg
_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

[Chris L]

Oleg -

Glad that helped.

You need the static routes to get the proper traffic sent to the correct gateway. That floating rule essentially just removes the route-to for traffic already routed that way.

If you want to run routing protocols, etc, out on the WAN subnet it might be best to just eliminate the gateway from the WAN interface configuration and manually set a default gateway + the static routes or the routing protocol to the other router.

That will disable all of the reply-to and route-to functionality reverting to the routing table as being authoritative.

It will also make things like automatic outbound NAT not know it is a WAN interface so those rules will have to be added manually. (If you set manual and save before deleting the gateway rules for what interfaces are already there will be created for you.)

That configuration might be incompatible with Multi-WAN to another ISP on another interface if it is ever added. Especially if the system ever thought the other WAN was the default gateway. Things would break.

Another option might be moving that second router off of the WAN subnet and onto it’s own transit network to pfSense.

Post by Oleg Cherkasov
Hi Chris,
Thank you for tip! I have successfully added floating outbound rules and it works now. Do I need to add static routes and firewall rules or it would be enough to add just floating rules? I may see static rules on WAN are redundant than.
Any thoughts about RIP/BGP/OSP routing if my second gateway advertise routing tables? Do I need to add floating rules as well for advertised routes via RIP/BGP/OSP? Or with EBFPd daemon it would be more flexible.
Thank you!
Oleg

Post by Chris L
Oleg -
WAN interfaces (interfaces with a gateway set on them) are treated differently.
The rule set forces all connections out that interface to a specific gateway (the interface gateway) with route-to.
You can add floating pass rules on WAN in the outbound direction to the destinations on the other side of that router (every network with that gateway as a static route) and probably a destination of the gateway address with no gateway set (the default gateway). That will disable route-to for that traffic.
If you want connections from the networks on the other side of the second gateway into pfSense you will need to disable reply-to on those pass rules or reply traffic will be forced to the interface gateway. Disable reply-to is in the advanced section of the rules.

Hi,
I am setting up static routes on WAN with two gateways. One gateway is default ISP and the second is a private network however both are in public WAN net. I may ping both gateways and of course the default one works flawlessly. Second GW works ok using other FW GW from other networks. Both GW are in the same WAN network, the same subnet.
Status shows both gateways are online and I have added static rules to direct traffic to 4 IPs to the second gateway so I may access resources in private network via second gateway in WAN network.
All statuses and suggested diagnostics looks good indeed, gateways are online and static routes are up however whatever I do the default gateway is used! I am running traceroute/tracepath from clients behind the firewall and from pfSense WAN itself but it always uses default gateway and ignores active second gateway and static rules. I have tried to reboot pfSense of course however the issue remains.
Anyone have any suggestion? How I may verbosely debug static routing?
Cheers,
Oleg
_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold