Metricbeat
Using metricbeat to send the metrics to PacketAI
Download metricbeat using below bash script:
https://www.elastic.co/downloads/past-releases/metricbeat-8-4-3curl https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.4.3-linux-x86_64.tar.gz -o metricbeat.tar.gz
tar -xf metricbeat.tar.gz
mv metricbeat-8.4.3-linux-x86_64 metricbeat
rm metricbeat.tar.gz
Download metricbeat using Powershell script:
$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest -Uri https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.4.3-windows-x86_64.zip -OutFile mbeat.zip
Expand-Archive .\mbeat.zip
mv ./mbeat/metricbeat-8.4.3-windows-x86_64/ ./metricbeat
rm -r ./mbeat
rm ./mbeat.zip
cd ./metricbeat
Edit metricbeat.yml, use the below metricbeat configuration, we need to customise this according to your requirements, explained in details about each section below. The complete metricbeat configuration can be found here
metricbeat.config.modules:
# Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml
# Set to true to enable config reloading
reload.enabled: true
# Period on which files under path should be checked for changes
#reload.period: 10s
setup.template.enabled: false
setup.dashboards.enabled: false
setup.ilm.enabled: false
setup.kibana:
# ================================== Outputs ===================================
# Configure what output to use when sending the data collected by the beat.
#output.console:
# pretty: true
# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
# Array of hosts to connect to.
allow_older_versions: true
hosts: ["beats-ingester-gcpdev.packetai.co:443"]
protocol: https
path: /elasticsearch/mb
compression_level: 9
index: "es"
headers:
X-PAI-IID: YOUR_PAI_IID
X-PAI-TOKEN: YOUR_PAI_TOKEN
# ================================= Processors =================================
# Configure processors to enhance or manipulate events generated by the beat.
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
- add_fields:
fields:
clusterName: YOUR_CLUSTER_NAME
target: fields
metricbeat modules: - To enable the windows module, please run the below command.
metricbeat.exe modules enable windows- There are multiple modules available on metricbeat, they can be enabled to similar to the above command, the list of available modules are listed here.
- The list of enabled modules are available in
modules.ddirectory - To disable a module we need to run this command:
metricbeat.exe modules disable windows
output.elasticsearch:hosts: Make sure that your hosts entries are correct.hosts: ["beats-ingester-logpatterns.packetai.co:443"]change URL if your PacketAI URL's are different- compression_level is between 0 to 9, 0 being no compression at all, and 9 being best compression, we would suggest to use the value 6. higher compression values means higher cpu usage and lower network usage.
headerssub section needs to update according to youPAI_IIDandPAI_TOKEN. you can get them on packetai after login, and under the section ofDeploy PacketAI / Agent
processors:- We need to modify the YOUR_CLUSTER_NAME with appropriate cluster name. clusterName can be used to manage the retention period on PacketAI managed. Please note that clusterName cannot contain character "-". Currently this field is mandatory.
- add_cloud_metadata: ~ is optional, this adds the metadata of the cloud. i.e. region, zone, machine_id etc...
- add_docker_metadata is optional, this also docker metadata, docker container name, image name, docker labels, etc.... This extra metadata will increase the index size at PacketAI.
Linux:
We could run the metricbeat with the below command to start the metricbeat, we need to install a service, if we want to automatically start the metricbeat service on system startup.
./metricbeat -c metricbeat.yml
systemd service file (
metricbeat.service): Here we are assuming that metricbeat is located at /opt/metricbeat, if the metricbeat is located somewhere please update the metricbeat.service file to reflect the same.# metricbeat.service
[Unit]
Description=metricbeat
After=syslog.target network.target remote-fs.target nss-lookup.target
[Service]
Type=simple
ExecStart=/opt/metricbeat/metricbeat -c metricbeat.yml
Restart=on-failure
WorkingDirectory=/opt/metricbeat
[Install]
WantedBy=multi-user.target
copy the file to
/etc/systemd/system directory, and run the below commands. In// Some code
systemctl enable metricbeat
systemctl start metricbeat
Windows:
In order to install the metricbeat on Windows, we need to run the below command, in metricbeat folder.
./install-service-metricbeat.ps1
Start-Service metricbeat
Last modified 1mo ago