Fluent Bit

Fluent Bit integration with PacketAI
Input
The following section defines the fluent-bit input section. for more details see fluent-bit input here​
​
[INPUT]
    Name        tail
    # setting the path for the file to tail
    Path        /var/log/**/*.log
    Tag   mac_logs
[INPUT]
    Name        cpu
    # setting the path for the file to tail
    Tag   my_cpu_metrics
    Interval_Sec 10
​
Filters
We use filters to add some PacketAI metadata to easily process. It's important to have packetai.cluster_name and packetai.app_name. These 2 fields help us to quickly add logpatterns and more easy to manage opensearch indexes.
// Some code
[FILTER]
    Name record_modifier
    Match *_logs //here it's matches tag: mac_logs, it's impoortant that you tag your fluent-bit inputs
    Record hostname ${HOSTNAME}
    Record packetai.cluster_name macos
    Record packetai.app_name varlogs # For metrics this is optional.
Output
The output section is used to send the logs/metrics to PacketAI.
Logs
For logs the below output configuration is being used. We need to configure the Host, Header, and Match sections.
The Host varies depending on your packetai URL. Please check with PacketAI team if you are not sure which URL to use.
Headers X-PAI-TOKEN and X-PAI-IID needs to get it from packetai URL. After login, you can navigate to Deploy PacketAI/Agent/logstash click on credentials It will show X-PAI-TOKEN and X-PAI-IID, copy these values and replace YOUR_PAI_TOKEN and YOUR_PAI_IID.
​
[OUTPUT]
    Name  http
    Port 443
    Tls On
    Host vector-ingester-logpatterns.packetai.co # Check your URL, it might be different from this
    URI /fluent/log
    Header X-PAI-TOKEN YOUR_PAI_TOKEN # This token needs to get it from packetai UI section
    Header X-PAI-IID YOUR_PAI_IID
    Format json
    json_date_format iso8601
    json_date_key timestamp
    Match *_logs
Metrics
Similarly for Metrics we need configure the Host, Header, and Match sections.
The Host varies depending on your packetai URL. Please check with PacketAI team if you are not sure which URL to use.
Headers X-PAI-TOKEN and X-PAI-IID needs to get it from packetai URL. After login, you can navigate to Deploy PacketAI/Agent/logstash click on credentials It will show X-PAI-TOKEN and X-PAI-IID, copy these values and replace YOUR_PAI_TOKEN and YOUR_PAI_IID.
// Some code
​
[OUTPUT]
    Name  http
    Port 443
    Tls On
    Host vector-ingester-logpatterns.packetai.co # Check your URL, it might be different from this
    URI /fluent/metric
    Header X-PAI-TOKEN YOUR_PAI_TOKEN # This token needs to get it from packetai UI section
    Header X-PAI-IID YOUR_PAI_IID # This token needs to get it from packetai UI section
    Format json
    json_date_format iso8601
    json_date_key timestamp
    Match *_metrics
Example configuration
​
[INPUT]
    name            cpu
    tag             my_cpu_metrics
    Interval_Sec 10
​
[FILTER]
    Name record_modifier
    Match *
    Record hostname ${HOSTNAME}
    Record packetai.cluster_name macos
    Record packetai.app_name macos
​
[OUTPUT]
    Name  http
    Port 443
    Tls On
    Host vector-ingester-logpatterns.packetai.co
    URI /fluent/metric
    Header X-PAI-TOKEN YOUR_PAI_TOKEN
    Header X-PAI-IID YOUR_PAI_ID
    Format json
    json_date_format iso8601
    json_date_key timestamp
    Match *_metrics
Set Up - Previous
Winlogbeat
Last modified 4mo ago