pmgoriya journal
Monitoring different components of server with Grafana

Monitoring different components of server with Grafana

Pankaj's photo
Pankaj
·

3 min read

Table of contents

In the last article, we have learned How to set up Grafana in our local environment.

Now, let's proceed with creating a Grafana monitoring by integrating Loki and Promtail for efficient log aggregation and analysis. Our goal is to connect a Linux/ Windows EC2 instance with Grafana to monitor various components of the servers in real time.

A Brief Introduction to Loki and Promtail:

Loki: A horizontally scalable log aggregation system that efficiently stores and indexes log data, serving as the central repository for fast, distributed log processing.

Promtail: A lightweight log collector operating on individual servers or containers, collecting log data from various sources. It tails log files or streams and sends the data to Loki for storage and analysis.

Things will get clearer as we do we hands-on.

Task-01: Monitoring different components of server with Grafana

After installing Grafana on our server(For more details refer my last blog here:

    • Install Docker:

         sudo apt-get update
   sudo apt install docker.io
   sudo usermod -aG docker $USER
   sudo reboot

      • Install Loki and Promtail using Docker.

  2. Download Configuration Files:

    • Loki Config:

        mkdir grafana_configs
  cd grafana_configs
  wget https://raw.githubusercontent.com/grafana/loki/v2.8.0/cmd/loki/loki-local-config.yaml -O loki-config.yaml

    • Promtail Config:

        wget https://raw.githubusercontent.com/grafana/loki/v2.8.0/clients/cmd/promtail/promtail-docker-config.yaml -O promtail-config.yaml

    • Run Loki Docker Container:

    sudo docker run -d --name loki -v $(pwd):/mnt/config -p 3100:3100 grafana/loki:2.8.0 --config.file=/mnt/config/loki-config.yaml

This command initiates a Docker container named "loki" from the Grafana Loki image version 2.8.0. It runs the container in detached mode (-d), maps the current directory as a volume inside the container (-v $(pwd):/mnt/config), and exposes port 3100 on the host to port 3100 on the container (-p 3100:3100). Additionally, it specifies the Loki configuration file as /mnt/config/loki-config.yaml using the --config.file option. This configuration file holds settings for Loki, such as storage configurations and retention policies.

Configure Security Group:

  • Edit the inbound rule to allow port 3100 in the security group of your EC2 instance.

Check Loki Readiness:

  • Open a browser and visit https://<public-ip>:3100/ready to confirm Loki's readiness.

To discover the specific logs or metrics present in Loki, you can obtain them using the command /metrics.

Run Promtail Docker Container:

sudo docker run -d --name promtail -v $(pwd):/mnt/config -v /var/log:/var/log --link loki grafana/promtail:2.8.0 --config.file=/mnt/config/promtail-config.yaml

This command launches a Docker container named "promtail" using the Grafana Promtail image version 2.8.0. The container runs in detached mode (-d), mounts the current directory as a volume inside the container (-v $(pwd):/mnt/config), and also mounts the host's /var/log directory to the container's /var/log directory (-v /var/log:/var/log). It establishes a network link with the previously created "loki" container using --link loki. Additionally, it specifies the Promtail configuration file as /mnt/config/promtail-config.yaml using the --config.file option. This configuration file contains settings for Promtail, such as log file paths and target Loki endpoints.

Setting Up Grafana Data Source:

  1. Log in to your Grafana web interface.

  2. Navigate to "Configuration" > "Data Sources" > "Add data source."

  3. Enter the Loki server URL, typically in the format http://:9090.

  4. Choose "Loki" as the type of data source.

Creating Grafana Dashboards:

Develop Grafana dashboards to visualize the metrics gathered from both Linux and Windows instances.

Include panels for CPU usage, memory usage, disk space, network activity, and any other metrics of interest.

Monitoring and Analysis:

Now that Loki and Promtail are configured, you can effectively monitor and analyze various components of your Linux and Windows EC2 instances.

Thanks for reading until here. We will deep dive more into dashboards and stuff in the next one.

Grafana#promtailloki#90daysofdevopsmonitoring