Discussion:
[pfSense] Packet capture
Jason Pyeron
2013-04-28 12:47:10 UTC
Permalink
I am looking to capture all the packets that are traversing and attempting to
traverse the firewall.

If I use tcpdump -i WAN I get all the packets, if I use tcpdump -i LAN then I
only get the packets that made it past the firewall plus the recursive traffic
of my pcap data leaving the firewall too.

This is telling me I should be using another port, but still does not help me
separate the pcap data into 2 buckets:

1: blocked
2: not blocked

Any suggestions?

--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- -
- Jason Pyeron PD Inc. http://www.pdinc.us -
- Principal Consultant 10 West 24th Street #100 -
- +1 (443) 269-1555 x333 Baltimore, Maryland 21218 -
- -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
Trevor Benson
2013-04-28 14:14:23 UTC
Permalink
Have you tried using the built in packet capture under diagnostics? This
will clean up your ssh traffic, which is what I assume you mean by tcpdump
recursice traffic. Plus you can download a pcap to examine more closely in
wireshark.

As for traffic denied by the firewall have you tried looking at the
firewall logs?

Trevor
Post by Jason Pyeron
I am looking to capture all the packets that are traversing and attempting to
traverse the firewall.
If I use tcpdump -i WAN I get all the packets, if I use tcpdump -i LAN then I
only get the packets that made it past the firewall plus the recursive traffic
of my pcap data leaving the firewall too.
This is telling me I should be using another port, but still does not help me
1: blocked
2: not blocked
Any suggestions?
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- -
- Jason Pyeron PD Inc. http://www.pdinc.us -
- Principal Consultant 10 West 24th Street #100 -
- +1 (443) 269-1555 x333 Baltimore, Maryland 21218 -
- -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
_______________________________________________
List mailing list
http://lists.pfsense.org/mailman/listinfo/list
Jason Pyeron
2013-04-28 16:46:38 UTC
Permalink
Yes the interface for packet capture is nice for a interactive quick look, but
it is not a solution for an automated ingest system for 24x7 capture.

regarding the logs:

{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 00:00:00.001738 rule 23/0(match):
block in on em0: (tos 0x0, ttl 116, id 4687, offset 0, flags [DF], proto UDP
(17), length 66)

{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 31.222.133.87.53 > 67.90.184.35.53:
952+ [1au] ANY? ripe.net. (38)

the detail is insufficient. I tried Show raw filter logs, but there does not
seem to be any apprciable difference. I have a backend system (IDS type of
thing) which ingests pcap data as well as syslog, in this case the syslog from
the pfSesne is to light weight.

can I sniff the bridge [BRIDGE0]?

-Jason




_____

From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of Trevor Benson
Sent: Sunday, April 28, 2013 10:14
To: pfSense support and discussion
Subject: Re: [pfSense] Packet capture



Have you tried using the built in packet capture under diagnostics? This will
clean up your ssh traffic, which is what I assume you mean by tcpdump recursice
traffic. Plus you can download a pcap to examine more closely in wireshark.

As for traffic denied by the firewall have you tried looking at the firewall
logs?

Trevor


On Apr 28, 2013 5:47 AM, "Jason Pyeron" <***@pdinc.us> wrote:


I am looking to capture all the packets that are traversing and attempting to
traverse the firewall.

If I use tcpdump -i WAN I get all the packets, if I use tcpdump -i LAN then I
only get the packets that made it past the firewall plus the recursive traffic
of my pcap data leaving the firewall too.

This is telling me I should be using another port, but still does not help me
separate the pcap data into 2 buckets:

1: blocked
2: not blocked

Any suggestions?



--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- -
- Jason Pyeron PD Inc. http://www.pdinc.us
<http://www.pdinc.us/> -
- Principal Consultant 10 West 24th Street #100 -
- +1 (443) 269-1555 x333 Baltimore, Maryland 21218 -
- -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
Jason Pyeron
2013-04-28 16:55:57 UTC
Permalink
To answer my own question:

***@black /tmp
$ ssh ***@firewall 'tcpdump -i bridge0 -s0 -w -' > /tmp/fw35.pcap
tcpdump: WARNING: bridge0: no IPv4 address assigned
tcpdump: listening on bridge0, link-type EN10MB (Ethernet), capture size 65535
bytes

now to subtract the bridge0 from the em0 pcap data.



_____

From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of Jason Pyeron
Sent: Sunday, April 28, 2013 12:47
To: 'pfSense support and discussion'
Subject: Re: [pfSense] Packet capture


Yes the interface for packet capture is nice for a interactive quick look, but
it is not a solution for an automated ingest system for 24x7 capture.

regarding the logs:

{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 00:00:00.001738 rule 23/0(match):
block in on em0: (tos 0x0, ttl 116, id 4687, offset 0, flags [DF], proto UDP
(17), length 66)

{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 31.222.133.87.53 > 67.90.184.35.53:
952+ [1au] ANY? ripe.net. (38)

the detail is insufficient. I tried Show raw filter logs, but there does not
seem to be any apprciable difference. I have a backend system (IDS type of
thing) which ingests pcap data as well as syslog, in this case the syslog from
the pfSesne is to light weight.

can I sniff the bridge [BRIDGE0]?

-Jason




_____

From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of Trevor Benson
Sent: Sunday, April 28, 2013 10:14
To: pfSense support and discussion
Subject: Re: [pfSense] Packet capture



Have you tried using the built in packet capture under diagnostics? This will
clean up your ssh traffic, which is what I assume you mean by tcpdump recursice
traffic. Plus you can download a pcap to examine more closely in wireshark.

As for traffic denied by the firewall have you tried looking at the firewall
logs?

Trevor


On Apr 28, 2013 5:47 AM, "Jason Pyeron" <***@pdinc.us> wrote:


I am looking to capture all the packets that are traversing and attempting to
traverse the firewall.

If I use tcpdump -i WAN I get all the packets, if I use tcpdump -i LAN then I
only get the packets that made it past the firewall plus the recursive traffic
of my pcap data leaving the firewall too.

This is telling me I should be using another port, but still does not help me
separate the pcap data into 2 buckets:

1: blocked
2: not blocked

Any suggestions?





--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- -
- Jason Pyeron PD Inc. http://www.pdinc.us
<http://www.pdinc.us/> -
- Principal Consultant 10 West 24th Street #100 -
- +1 (443) 269-1555 x333 Baltimore, Maryland 21218 -
- -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
James Records
2013-04-28 16:57:47 UTC
Permalink
Jason,

I think what you want is the pflog0 interface.
--
James Records | Principle Network Engineer

M 425.984.4349 E ***@northshoresoftware.com

W www.northshoresoftware.com
Post by Jason Pyeron
**
Yes the interface for packet capture is nice for a interactive quick look,
but it is not a solution for an automated ingest system for 24x7 capture.
{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 00:00:00.001738 rule
23/0(match): block in on em0: (tos 0x0, ttl 116, id 4687, offset 0, flags
[DF], proto UDP (17), length 66)
{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 31.222.133.87.53 >
67.90.184.35.53: 952+ [1au] ANY? ripe.net. (38)
the detail is insufficient. I tried *Show raw filter logs, but there
does not seem to be any apprciable difference. I have a backend system (IDS
type of thing) which ingests pcap data as well as syslog, in this case the
syslog from the pfSesne is to light weight.*
*can I sniff the bridge [*BRIDGE0*]?*
*-Jason*
------------------------------
*Sent:* Sunday, April 28, 2013 10:14
*To:* pfSense support and discussion
*Subject:* Re: [pfSense] Packet capture
Have you tried using the built in packet capture under diagnostics? This
will clean up your ssh traffic, which is what I assume you mean by tcpdump
recursice traffic. Plus you can download a pcap to examine more closely in
wireshark.
As for traffic denied by the firewall have you tried looking at the
firewall logs?
Trevor
Post by Jason Pyeron
I am looking to capture all the packets that are traversing and attempting to
traverse the firewall.
If I use tcpdump -i WAN I get all the packets, if I use tcpdump -i LAN then I
only get the packets that made it past the firewall plus the recursive traffic
of my pcap data leaving the firewall too.
This is telling me I should be using another port, but still does not help me
1: blocked
2: not blocked
Any suggestions?
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- -
- Jason Pyeron PD Inc. http://www.pdinc.us -
- Principal Consultant 10 West 24th Street #100 -
- +1 (443) 269-1555 x333 Baltimore, Maryland 21218 -
- -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
_______________________________________________
List mailing list
http://lists.pfsense.org/mailman/listinfo/list
Jason Pyeron
2013-04-28 20:21:34 UTC
Permalink
Nice. I did not now about that.

"When a packet is logged by PF, a copy of the packet header is sent to a
pflog(4)
<http://www.openbsd.org/cgi-bin/man.cgi?query=pflog&sektion=4&manpath=OpenBSD+5.
2> interface along with some additional data such as the interface the packet
was transiting, the action that PF took (pass or block), etc. "

I will now look for a way to get it to pass the full packet, as I need to do
deep packet inspections.

Thanks!

-Jason



_____

From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of James Records
Sent: Sunday, April 28, 2013 12:58
To: pfSense support and discussion
Subject: Re: [pfSense] Packet capture


Jason,

I think what you want is the pflog0 interface.
--
James Records | Principle Network Engineer

M 425.984.4349 E ***@northshoresoftware.com

W www.northshoresoftware.com <http://www.northshoresoftware.com/>


<https://mail.google.com/mail/u/0/?ui=2&ik=3456340655&view=att&th=13ab8f806fccb0
7e&attid=0.2&disp=inline&realattid=f_h8z0yrka2&safe=1&zw&saduie=AG9B_P_0HvEbIe6v
cnhsenP3ZJiz&sadet=1352854635474&sads=QIpOFwfaK2xnZX61g1WsD4mNl08>

On Sun, Apr 28, 2013 at 9:46 AM, Jason Pyeron <***@pdinc.us> wrote:



Yes the interface for packet capture is nice for a interactive quick look, but
it is not a solution for an automated ingest system for 24x7 capture.

regarding the logs:

{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 00:00:00.001738 rule 23/0(match):
block in on em0: (tos 0x0, ttl 116, id 4687, offset 0, flags [DF], proto UDP
(17), length 66)

{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 31.222.133.87.53 > 67.90.184.35.53:
952+ [1au] ANY? ripe.net. (38)

the detail is insufficient. I tried Show raw filter logs, but there does not
seem to be any apprciable difference. I have a backend system (IDS type of
thing) which ingests pcap data as well as syslog, in this case the syslog from
the pfSesne is to light weight.

can I sniff the bridge [BRIDGE0]?

-Jason




_____

From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of Trevor Benson
Sent: Sunday, April 28, 2013 10:14
To: pfSense support and discussion
Subject: Re: [pfSense] Packet capture



Have you tried using the built in packet capture under diagnostics? This will
clean up your ssh traffic, which is what I assume you mean by tcpdump recursice
traffic. Plus you can download a pcap to examine more closely in wireshark.

As for traffic denied by the firewall have you tried looking at the firewall
logs?

Trevor


On Apr 28, 2013 5:47 AM, "Jason Pyeron" <***@pdinc.us> wrote:


I am looking to capture all the packets that are traversing and attempting to
traverse the firewall.

If I use tcpdump -i WAN I get all the packets, if I use tcpdump -i LAN then I
only get the packets that made it past the firewall plus the recursive traffic
of my pcap data leaving the firewall too.

This is telling me I should be using another port, but still does not help me
separate the pcap data into 2 buckets:

1: blocked
2: not blocked

Any suggestions?


--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- -
- Jason Pyeron PD Inc. http://www.pdinc.us
<http://www.pdinc.us/> -
- Principal Consultant 10 West 24th Street #100 -
- +1 (443) 269-1555 x333 Baltimore, Maryland 21218 -
- -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
James Records
2013-04-28 20:28:58 UTC
Permalink
Jason,

Take a look at this:

http://www.openbsd.org/faq/pf/logging.html

Should help you out a bit.
--
James Records | Principle Network Engineer

M 425.984.4349 E ***@northshoresoftware.com

W www.northshoresoftware.com
Post by Jason Pyeron
**
Nice. I did not now about that.
"When a packet is logged by PF, a copy of the packet header is sent to a
pflog(4)<http://www.openbsd.org/cgi-bin/man.cgi?query=pflog&sektion=4&manpath=OpenBSD+5.2>interface along with some additional data such as the interface the packet
was transiting, the action that PF took (pass or block), etc. "
I will now look for a way to get it to pass the full packet, as I need to
do deep packet inspections.
Thanks!
-Jason
------------------------------
*Sent:* Sunday, April 28, 2013 12:58
*To:* pfSense support and discussion
*Subject:* Re: [pfSense] Packet capture
Jason,
I think what you want is the pflog0 interface.
--
James Records | Principle Network Engineer
W www.northshoresoftware.com
Post by Jason Pyeron
**
Yes the interface for packet capture is nice for a interactive quick
look, but it is not a solution for an automated ingest system for 24x7
capture.
{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 00:00:00.001738 rule
23/0(match): block in on em0: (tos 0x0, ttl 116, id 4687, offset 0, flags
[DF], proto UDP (17), length 66)
{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 31.222.133.87.53 >
67.90.184.35.53: 952+ [1au] ANY? ripe.net. (38)
the detail is insufficient. I tried *Show raw filter logs, but there
does not seem to be any apprciable difference. I have a backend system (IDS
type of thing) which ingests pcap data as well as syslog, in this case the
syslog from the pfSesne is to light weight.*
*can I sniff the bridge [*BRIDGE0*]?*
*-Jason*
------------------------------
*Sent:* Sunday, April 28, 2013 10:14
*To:* pfSense support and discussion
*Subject:* Re: [pfSense] Packet capture
Have you tried using the built in packet capture under diagnostics?
This will clean up your ssh traffic, which is what I assume you mean by
tcpdump recursice traffic. Plus you can download a pcap to examine more
closely in wireshark.
As for traffic denied by the firewall have you tried looking at the
firewall logs?
Trevor
Post by Jason Pyeron
I am looking to capture all the packets that are traversing and attempting to
traverse the firewall.
If I use tcpdump -i WAN I get all the packets, if I use tcpdump -i LAN then I
only get the packets that made it past the firewall plus the recursive traffic
of my pcap data leaving the firewall too.
This is telling me I should be using another port, but still does not help me
1: blocked
2: not blocked
Any suggestions?
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- -
- Jason Pyeron PD Inc. http://www.pdinc.us -
- Principal Consultant 10 West 24th Street #100 -
- +1 (443) 269-1555 x333 Baltimore, Maryland 21218 -
- -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
_______________________________________________
List mailing list
http://lists.pfsense.org/mailman/listinfo/list
Jason Pyeron
2013-04-28 23:16:42 UTC
Permalink
Yeah, that is what I quoted. Once you told me about the pflog0 I googled it. It
seems that it is not just a copy of the headers that get sent to that virtual
interface, but it is really pflogd that truncates the packets when putting them
in /var/log/pflog. The page lied :)

So now I have pflog0 (updated all the rules to log) and the bridge0 feeding in
to the IPS/IDS. I don't think the jitter in the sequence between the two pcap
streams will matter.

As a side, do you think I should stream the pcap data by ssh or some other
means? Would there be a more efficient means from the firewall performance point
of view?

-Jason


_____

From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of James Records
Sent: Sunday, April 28, 2013 16:29
To: pfSense support and discussion
Subject: Re: [pfSense] Packet capture


Jason,


Take a look at this:


http://www.openbsd.org/faq/pf/logging.html



Should help you out a bit.
--
James Records | Principle Network Engineer


M 425.984.4349 E ***@northshoresoftware.com


W www.northshoresoftware.com <http://www.northshoresoftware.com/>



<https://mail.google.com/mail/u/0/?ui=2&ik=3456340655&view=att&th=13ab8f806fccb0
7e&attid=0.2&disp=inline&realattid=f_h8z0yrka2&safe=1&zw&saduie=AG9B_P_0HvEbIe6v
cnhsenP3ZJiz&sadet=1352854635474&sads=QIpOFwfaK2xnZX61g1WsD4mNl08>





On Sun, Apr 28, 2013 at 1:21 PM, Jason Pyeron <***@pdinc.us> wrote:



Nice. I did not now about that.

"When a packet is logged by PF, a copy of the packet header is sent to a
pflog(4)
<http://www.openbsd.org/cgi-bin/man.cgi?query=pflog&sektion=4&manpath=OpenBSD+5.
2> interface along with some additional data such as the interface the packet
was transiting, the action that PF took (pass or block), etc. "

I will now look for a way to get it to pass the full packet, as I need to do
deep packet inspections.

Thanks!

-Jason




_____

From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of James Records
Sent: Sunday, April 28, 2013 12:58

To: pfSense support and discussion
Subject: Re: [pfSense] Packet capture


Jason,

I think what you want is the pflog0 interface.
--
James Records | Principle Network Engineer

M 425.984.4349 E ***@northshoresoftware.com

W www.northshoresoftware.com <http://www.northshoresoftware.com/>



On Sun, Apr 28, 2013 at 9:46 AM, Jason Pyeron <***@pdinc.us> wrote:



Yes the interface for packet capture is nice for a interactive quick look, but
it is not a solution for an automated ingest system for 24x7 capture.

regarding the logs:

{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 00:00:00.001738 rule 23/0(match):
block in on em0: (tos 0x0, ttl 116, id 4687, offset 0, flags [DF], proto UDP
(17), length 66)

{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 31.222.133.87.53 > 67.90.184.35.53:
952+ [1au] ANY? ripe.net. (38)

the detail is insufficient. I tried Show raw filter logs, but there does not
seem to be any apprciable difference. I have a backend system (IDS type of
thing) which ingests pcap data as well as syslog, in this case the syslog from
the pfSesne is to light weight.

can I sniff the bridge [BRIDGE0]?

-Jason




_____

From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of Trevor Benson
Sent: Sunday, April 28, 2013 10:14
To: pfSense support and discussion
Subject: Re: [pfSense] Packet capture



Have you tried using the built in packet capture under diagnostics? This will
clean up your ssh traffic, which is what I assume you mean by tcpdump recursice
traffic. Plus you can download a pcap to examine more closely in wireshark.

As for traffic denied by the firewall have you tried looking at the firewall
logs?

Trevor


On Apr 28, 2013 5:47 AM, "Jason Pyeron" <***@pdinc.us> wrote:


I am looking to capture all the packets that are traversing and attempting to
traverse the firewall.

If I use tcpdump -i WAN I get all the packets, if I use tcpdump -i LAN then I
only get the packets that made it past the firewall plus the recursive traffic
of my pcap data leaving the firewall too.

This is telling me I should be using another port, but still does not help me
separate the pcap data into 2 buckets:

1: blocked
2: not blocked

Any suggestions?


--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- -
- Jason Pyeron PD Inc. http://www.pdinc.us
<http://www.pdinc.us/> -
- Principal Consultant 10 West 24th Street #100 -
- +1 (443) 269-1555 x333 Baltimore, Maryland 21218 -
- -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
James Records
2013-05-01 18:40:18 UTC
Permalink
Jason,

Sorry it took me a bit to get back to you. Many years ago (and on OpenBSD)
I did something like this to get these logs off the box:

echo -n 'Starting PF Logging...'
ifconfig pflog0 up
( /usr/sbin/tcpdump -l -e -n -t -v -i pflog0 2>&1 | /usr/bin/logger -p
local0.info -t pf) &
echo 'done'


You'll want to modify your tcpdump statement to what you want to collect
and maybe send these to a new (separate) facility, but at that point you
can just point your logs to a remote server and you should be good to go.

I think there is a way to do a rc.local on Pfsense, though I've never done
this, but with some tweaking, you can probably get this to do what you want
without the need for remote ssh access.
--
James Records | Principle Network Engineer

M 425.984.4349 E ***@northshoresoftware.com

W www.northshoresoftware.com
Post by Jason Pyeron
**
Yeah, that is what I quoted. Once you told me about the pflog0 I googled
it. It seems that it is not just a copy of the headers that get sent to
that virtual interface, but it is really pflogd that truncates the packets
when putting them in /var/log/pflog. The page lied :)
So now I have pflog0 (updated all the rules to log) and the bridge0
feeding in to the IPS/IDS. I don't think the jitter in the sequence between
the two pcap streams will matter.
As a side, do you think I should stream the pcap data by ssh or some other
means? Would there be a more efficient means from the firewall performance
point of view?
-Jason
------------------------------
*Sent:* Sunday, April 28, 2013 16:29
*To:* pfSense support and discussion
*Subject:* Re: [pfSense] Packet capture
Jason,
http://www.openbsd.org/faq/pf/logging.html
Should help you out a bit.
--
James Records | Principle Network Engineer
W www.northshoresoftware.com
Post by Jason Pyeron
**
Nice. I did not now about that.
"When a packet is logged by PF, a copy of the packet header is sent to a
pflog(4)<http://www.openbsd.org/cgi-bin/man.cgi?query=pflog&sektion=4&manpath=OpenBSD+5.2>interface along with some additional data such as the interface the packet
was transiting, the action that PF took (pass or block), etc. "
I will now look for a way to get it to pass the full packet, as I need to
do deep packet inspections.
Thanks!
-Jason
------------------------------
*Sent:* Sunday, April 28, 2013 12:58
*To:* pfSense support and discussion
*Subject:* Re: [pfSense] Packet capture
Jason,
I think what you want is the pflog0 interface.
--
James Records | Principle Network Engineer
W www.northshoresoftware.com
Post by Jason Pyeron
**
Yes the interface for packet capture is nice for a interactive quick
look, but it is not a solution for an automated ingest system for 24x7
capture.
{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 00:00:00.001738 rule
23/0(match): block in on em0: (tos 0x0, ttl 116, id 4687, offset 0, flags
[DF], proto UDP (17), length 66)
{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 31.222.133.87.53 >
67.90.184.35.53: 952+ [1au] ANY? ripe.net. (38)
the detail is insufficient. I tried *Show raw filter logs, but there
does not seem to be any apprciable difference. I have a backend system (IDS
type of thing) which ingests pcap data as well as syslog, in this case the
syslog from the pfSesne is to light weight.*
*can I sniff the bridge [*BRIDGE0*]?*
*-Jason*
------------------------------
*Sent:* Sunday, April 28, 2013 10:14
*To:* pfSense support and discussion
*Subject:* Re: [pfSense] Packet capture
Have you tried using the built in packet capture under diagnostics?
This will clean up your ssh traffic, which is what I assume you mean by
tcpdump recursice traffic. Plus you can download a pcap to examine more
closely in wireshark.
As for traffic denied by the firewall have you tried looking at the
firewall logs?
Trevor
Post by Jason Pyeron
I am looking to capture all the packets that are traversing and attempting to
traverse the firewall.
If I use tcpdump -i WAN I get all the packets, if I use tcpdump -i LAN then I
only get the packets that made it past the firewall plus the recursive traffic
of my pcap data leaving the firewall too.
This is telling me I should be using another port, but still does not help me
1: blocked
2: not blocked
Any suggestions?
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- -
- Jason Pyeron PD Inc. http://www.pdinc.us -
- Principal Consultant 10 West 24th Street #100 -
- +1 (443) 269-1555 x333 Baltimore, Maryland 21218 -
- -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
_______________________________________________
List mailing list
http://lists.pfsense.org/mailman/listinfo/list
Jason Pyeron
2013-05-02 00:13:43 UTC
Permalink
Nice script. I am going to have to keep it in pcap format, and since that email
I have been using 2 remotely intiated ssh connections to ingest pflog0 and
bridge0 using '-s0 -w -'. My firewall load has not exceeded 0.06 sofar. Since
using ssh encrypts the data, I have no IA issues either.

Putting it in the rc.local could ensure that connection is under the firewall's
control to reduce the number of accounts with access.


_____

From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of James Records
Sent: Wednesday, May 01, 2013 14:40
To: pfSense support and discussion
Subject: Re: [pfSense] Packet capture


Jason,

Sorry it took me a bit to get back to you. Many years ago (and on OpenBSD) I
did something like this to get these logs off the box:



echo -n 'Starting PF Logging...'
ifconfig pflog0 up

( /usr/sbin/tcpdump -l -e -n -t -v -i pflog0 2>&1 | /usr/bin/logger -p
local0.info -t pf) &

echo 'done'


You'll want to modify your tcpdump statement to what you want to collect and
maybe send these to a new (separate) facility, but at that point you can just
point your logs to a remote server and you should be good to go.

I think there is a way to do a rc.local on Pfsense, though I've never done this,
but with some tweaking, you can probably get this to do what you want without
the need for remote ssh access.
--
James Records | Principle Network Engineer

M 425.984.4349 E ***@northshoresoftware.com

W www.northshoresoftware.com <http://www.northshoresoftware.com/>


<https://mail.google.com/mail/u/0/?ui=2&ik=3456340655&view=att&th=13ab8f806fccb0
7e&attid=0.2&disp=inline&realattid=f_h8z0yrka2&safe=1&zw&saduie=AG9B_P_0HvEbIe6v
cnhsenP3ZJiz&sadet=1352854635474&sads=QIpOFwfaK2xnZX61g1WsD4mNl08>


On Sun, Apr 28, 2013 at 4:16 PM, Jason Pyeron <***@pdinc.us> wrote:



Yeah, that is what I quoted. Once you told me about the pflog0 I googled it. It
seems that it is not just a copy of the headers that get sent to that virtual
interface, but it is really pflogd that truncates the packets when putting them
in /var/log/pflog. The page lied :)

So now I have pflog0 (updated all the rules to log) and the bridge0 feeding in
to the IPS/IDS. I don't think the jitter in the sequence between the two pcap
streams will matter.

As a side, do you think I should stream the pcap data by ssh or some other
means? Would there be a more efficient means from the firewall performance point
of view?

-Jason


_____


From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of James Records

Sent: Sunday, April 28, 2013 16:29

To: pfSense support and discussion
Subject: Re: [pfSense] Packet capture


Jason,


Take a look at this:


http://www.openbsd.org/faq/pf/logging.html



Should help you out a bit.
--
James Records | Principle Network Engineer


M 425.984.4349 E ***@northshoresoftware.com


W www.northshoresoftware.com <http://www.northshoresoftware.com/>








On Sun, Apr 28, 2013 at 1:21 PM, Jason Pyeron <***@pdinc.us> wrote:



Nice. I did not now about that.

"When a packet is logged by PF, a copy of the packet header is sent to a
pflog(4)
<http://www.openbsd.org/cgi-bin/man.cgi?query=pflog&sektion=4&manpath=OpenBSD+5.
2> interface along with some additional data such as the interface the packet
was transiting, the action that PF took (pass or block), etc. "

I will now look for a way to get it to pass the full packet, as I need to do
deep packet inspections.

Thanks!

-Jason




_____

From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of James Records
Sent: Sunday, April 28, 2013 12:58

To: pfSense support and discussion
Subject: Re: [pfSense] Packet capture


Jason,

I think what you want is the pflog0 interface.
--
James Records | Principle Network Engineer

M 425.984.4349 E ***@northshoresoftware.com

W www.northshoresoftware.com <http://www.northshoresoftware.com/>



On Sun, Apr 28, 2013 at 9:46 AM, Jason Pyeron <***@pdinc.us> wrote:



Yes the interface for packet capture is nice for a interactive quick look, but
it is not a solution for an automated ingest system for 24x7 capture.

regarding the logs:

{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 00:00:00.001738 rule 23/0(match):
block in on em0: (tos 0x0, ttl 116, id 4687, offset 0, flags [DF], proto UDP
(17), length 66)

{mail} Sun Apr 28 11:07:58 EDT 2013 INFO pf: 31.222.133.87.53 > 67.90.184.35.53:
952+ [1au] ANY? ripe.net. (38)

the detail is insufficient. I tried Show raw filter logs, but there does not
seem to be any apprciable difference. I have a backend system (IDS type of
thing) which ingests pcap data as well as syslog, in this case the syslog from
the pfSesne is to light weight.

can I sniff the bridge [BRIDGE0]?

-Jason




_____

From: list-***@lists.pfsense.org [mailto:list-***@lists.pfsense.org] On
Behalf Of Trevor Benson
Sent: Sunday, April 28, 2013 10:14
To: pfSense support and discussion
Subject: Re: [pfSense] Packet capture



Have you tried using the built in packet capture under diagnostics? This will
clean up your ssh traffic, which is what I assume you mean by tcpdump recursice
traffic. Plus you can download a pcap to examine more closely in wireshark.

As for traffic denied by the firewall have you tried looking at the firewall
logs?

Trevor


On Apr 28, 2013 5:47 AM, "Jason Pyeron" <***@pdinc.us> wrote:


I am looking to capture all the packets that are traversing and attempting to
traverse the firewall.

If I use tcpdump -i WAN I get all the packets, if I use tcpdump -i LAN then I
only get the packets that made it past the firewall plus the recursive traffic
of my pcap data leaving the firewall too.

This is telling me I should be using another port, but still does not help me
separate the pcap data into 2 buckets:

1: blocked
2: not blocked

Any suggestions?




--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
- -
- Jason Pyeron PD Inc. http://www.pdinc.us
<http://www.pdinc.us/> -
- Principal Consultant 10 West 24th Street #100 -
- +1 (443) 269-1555 x333 Baltimore, Maryland 21218 -
- -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
This message is copyright PD Inc, subject to license 20080407P00.
Loading...