Introduction
The IPFIX (IP Flow Information Export) protocol was standardized by the Internet Engineering Task Force (IETF) in 2013 to be a multi-vendor universal metadata protocol for exporting IP flow information from network devices, such as switches, routers, firewalls to network monitoring and analysis applications or “collector” systems. The IETF IPFIX standard defines how flow information is exported, formatted, and transferred from the IPFIX “agent” devices to collector systems for further segmentation, analysis and logging.
Flow data utilized by IPFIX consists of all IP data traffic that belongs to the same “connection” or “conversation” between two devices on a particular protocol. Flow information is periodically transmitted to the collector devices without any interaction by the receiver and can be customized to include a range of pre-defined or user-defined information/data types. This flexibility is one of the protocol’s strong suits, as vendors can create custom templates with custom information they wish to collect and analyze.
Why IPFIX
- Packet Storage – Full packet capture and pervasive monitoring in multiple data centers can be costly for most deployments. A network link of 10 megabits per second (Mbps) produces 1TB of data per day. Therefore, a multi-gigabit network requiring data to be stored for a few weeks translates to immense hardware requirements.
- CPU Power – Network analysis based on direct data capture is very processor intensive. Extracting large data sets from of a network session requires an enormous amount of processing power (i.e. cost). Additionally, if the data is encrypted – assuming the keys are available to decrypt.
- Time Sensitivity – Analysis of network conversations to distinguish between a threat to the business requires a significant amount of knowledge about network forensics. Gathering all stages of an attack can be a wearying process requiring manual correlation of activities from across the network.
- Security & Privacy – There are cases where full packet capture requires the IT department make special considerations and practices to address data that may be considered private. Metadata allows for traffic management while reducing or eliminating these privacy concerns.
IPFIX captures all IP flow information flowing through an interface and reports data traffic across the entire network without missing any transaction. While flow sampling is a valid method for network management use cases, flow sampling is ineffective for security use cases because it can omit certain flows that could potentially be a security breach that should have been captured. IPFIX can provide un-sampled accounting of all network activity on an IP flow-enabled interface and is useful in event correlation and data analytics for network security purposes.
Capturing and exporting IPFIX flow data, however, can increase overhead on constrained routers and switches. The possibility of overburdening network infrastructure often inhibits network engineers from enabling IPFIX flow reporting on their network for fear of reducing capacity or affecting quality. The underlying concern may be regarding the introduction of increased jitter and delay, which may impact network services and applications utilizing these devices.
Best Practices and Use Cases
IPFIX presents a wide-ranging use in network monitoring for network security particularly and is used for a range of applications from internal/external threat detection to network capacity planning.
Below are some of the main use cases for IPFIX data:
- Allowing the export of IPFIX flow records to IPFIX collectors helps enhance visibility into network traffic and behavior, improves collection of network utilization, and assists in network capacity planning.
- IPFIX provides visibility to establish a reference point for network traffic behavior, examine which internal devices a host is communicating with, and apply the behavior and communication to a set of rules and policies to determine if a security threat such as malware may be spreading.
- IPFIX provides exposure of security susceptibility through improved understanding of network traffic flows which help in discovery of new IP applications and security vulnerabilities.
- IPFIX can be used in uncovering network reconnaissance through detection of various forms of scans including TCP and UDP scans and Internet Control Message Protocol (ICMP) scans.
- Using IPFIX, network segmentation policies can be monitored for compliance and any unexpected transactions taking place between the segments of the network can be detected using analysis of flow record data.
- With IPFIX, granular traffic flow visibility can be help prevent security incidents against a business’ financial data, intellectual property, customer data, or trade secrets.
Conclusion
IPFIX defines how flow data should be laid out and transmitted from exporters to one or more collector devices. IPFIX network monitoring utilizes flow export. Network packets are aggregated into flows and exported for storage and analysis. A flow consists of IP packets having a set of common properties such as packet header fields, such as source and destination IP addresses and port numbers, and information derived by IP packet forwarding.
The architecture of a typical IPFIX deployment comprises Packet Observation in which packets are observed and processed, Flow Metering and Export, which consists of a Metering Process and an Exporting Process in which packets are aggregated into flows and exported and a Collecting Process by collector devices for reception, aggregation, filtering, data compression, storage, and summary generation of flow data.
