Tutorial – Build a Kibana dashboard to monitor your LoRa devices
You have a fleet of LoRa devices and want to monitor them ? You can keep track of your devices' KPI on the powerful tool Kibana provided by Live Objects. Learn how with our tutorial.
Messages from your LoRa devices come not only with the device-generated payload, but also with a lot of information from the network. The following dashboard makes use of the network data to monitor LoRa devices. This step-by-step tutorial shows how to build this Kibana dashboard in a few minutes.
Kibana is a very powerful dashboard tool but it can be difficult to configure it at first. In this tutorial, we will explain how to create each panel separately, then we will show how to assemble them on a dashboard. Finally we will tell you how to use the full potential of your dashboard.
What you need
Your LoRa devices registered on Live Objects and … that’s all because Kibana is provided by Live Objects and your devices’ data can be accessed through Kibana.
Step 1: Open Kibana
On the Live Objects portal, go to “Data/Kibana” section and click on the “Kibana” button.
Kibana is an option of LoRa Premium offer. If you cannot access Kibana, please contact us so we can add this option to your account.
Everything in Kibana is filtered by the time window in the top right corner. You can set the time window to see only recent data (last x hours, last x days) or data received in a specific period.
There are 3 main menus in Kibana:
- Discover: This page lists all data received in the selected time window. It includes not only LoRa data messages, but also events from the Simple Event Processing. If you want to see only LoRa data message, then make sure to add this filter in the search bar: metadata.connector:lora
- Visualize: Here you can create and modify panels such as bar charts, online charts, point clouds, pie charts, and maps using large volumes of data.
- Dashboard: In this page you can access your dashboards.
To find more about Kibana, read the user guide.
Step 2: Create a filter for LoRa messages
Enter the “Discover” menu.
By default it shows all messages received in the selected time window, however we only need messages from LoRa device.
So in the search bar, type “metadata.connector:lora” to filter messages.
Then click on the “Save” button in the top right corner, type a name such as “LoRa messages” and click on Save.
In the next step, we will use this saved search in every panel.
Step 3: Create your panels
Enter the “Visualize” menu and click on the “+” button to create your first panel as shown in the following image.
Number of communicating devices
We build a simple metric panel showing the number of devices that have sent at least one data message in the selected time window.
Select the “Metric” visualization.
Then you will be prompted to select a search source, click on your saved search “LoRa messages”.
The resulting metric appears on the right and can be configured in the left column.
Configure the metric with a “Unique Count” aggregation on the “metadata.source” field, as a result, the number of different LoRa devices appears on the right.
In addition, we add a custom label that appears above the metric.
In the “Options” tab, you can change the visual aspect of the panel, for example we set the Font Size to 37pt.
Click on the “Play” icon to apply the configuration.
Lastly, we need to save our panel:
- Click on “Save” on the top right corner
- Choose a name, it will help you find your panel later
- Click on “Save”
Great! You just build your first panel! We will put it aside and come back to it later.
Number of uplinks
For the second panel, we build a timeline of the number of messages received from your LoRa device.
In the “Visualize” menu, click on the “+” button and choose “Area”.
Again, when you are prompted to select a search source, click on “LoRa messages”.
To create a timeline:
- Select “X-Axis” under “buckets”
- Choose a “Date histogram” aggregation
- Select the “timestamp” field and the “Auto” interval
Then you can add a custom label for your axis or change the visual aspect in the “Metrics & Axis” and “Panel Settings” tabs.
Click on the “play” icon to apply your configuration.
Lastly, save your panel to use it later.
Messages types distribution
We build a pie graph showing the distribution of messages types. It can be useful if your devices send different types of messages (confirmed and unconfirmed uplinks).
Enter the “Visualization” menu, click on “+” and chose “Pie”.
Under “metrics”, select a “Count” aggregation.
Under “buckets”, click on “Split Slices” and select the following configuration:
- Aggregation: “Terms”
- Field: metadata.network.lora.messageType
Then click on the “play” icon and save.
Number of received and missed uplinks
Now we build a panel with two metrics:
- the number of LoRa messages received
- the number of LoRa messages that were sent by your devices but never reached the network.
In the “Visualization” menu, click on the “+” button and select “Metric”.
The first metric is the number of received messages: choose a “Count” aggregation and type a custom label.
The second metric is the number of missed messages: click on “Add metrics”, then choose a “Sum” aggregation on the “metadata.network.lora.missingFcnt” field, finally type a custom label.
In the “Options” tab, we decrease the Font Size to 29pt, it may look small now but it should fit nicely in the dashboard afterward.
Click on the “Play” icon and save your panel.
Uplink success rate
This panel is a timeline showing the success rate of your devices’ messages.
Before building the timeline, we will note down the index, as we will need it to configure this panel.
To find your index, go to the “Visualization” menu, click on the “+” button and select “Metric”. Your index pattern appears on the left, it should look like lom-5756e9f80cf2c64c81b6d60f-*.
Now, go back and select the “Visual Builder” visualization type.
This “Visual Builder” is a bit different than the previous types of visualizations: the graph type is at the top, the result is in the middle and the configuration is below it.
By default, the result is a red panel with no data so we will fix that.
Firstly, go to “Panel Options” tab:
- enter your index pattern
- in “Time Field”, select “timestamp”
- in “Panel Filter”, type “metadata.connector:lora”
Click on “Apply Changes”, as a result you should see some data instead of the red panel.
Secondly, go to the “Data” tab:
- let the “Count” aggregation as is, because this is the number of messages received from your devices
- on the right of the first metric, click on the “+” button to add a second metric
- for the second metric, choose a “Sum” aggregation on the “metadata.network.lora.missingFcnt” field, this is the number of messages that did not reach the network
- again click on the “+” button to add a third metric
- for the third aggregation, select a “Calculation” aggregation, set two variables “uplinks” and “missing” corresponding to the 2 previous metrics and copy the following formula:
params.uplinks / ( params.missing + params.uplinks )
The configuration is shown in the following image.
Thirdly, in the “Options” tab, select “Percent” as the Data Formatter.
Finally, click on “Apply Changes” and save your panel.
Success rate for devices with the most packet loss
In this panel we focus on the devices sending messages that the network does not receive. It can be useful to identify at a glance devices with poor signal reception.
We will not create this panel from scratch, instead we will duplicate the previous panel. To do so:
- enter the “Visualization” menu
- click on the panel you created earlier, you should find it with its name
- click on the “Save” button in the top right corner
- type a new name
- check the box to “Save as a new visualization”
- then click on “Save”
You have a copy of the previous panel. We will now modify it:
- select the “Top N” type of panel
- add a “Cumulative Sum” aggregation on the metric “Count”
- add a “Cumulative Sum” aggregation on the metric “Sum of metadata.network.lora.missingFcnt”
- reorder the aggregation so that the “Calculation” aggregation is the last one
- in the “Calculation” aggregation, set the variable “uplink” to “Cumulative Sum of Count” and “missing” to “Cumulative Sum of Sum of metadata.network.lora.missingFcnt”
- in the “Group By” line, select “Terms”, the “metadata.network.lora.devEUI” field, top “20” and order by “Sum of metadata.network.lora.missingFcnt”
Also, change the formula to the following:
params.missing / ( params.missing + params.uplinks )
Finally, click on “Apply changes”, it generates the list of the devices with the more messages lost. The percentage on the right is the packet loss. Note that the resulting list only contains the 20 devices that lost the most messages regardless of the packet loss rate.
Number of gateways per device
Now we build a table that lists the devices and shows the average number of gateways that receive their messages. As a reminder, a high number of gateways often ensures a high success rate.
Go to the “Visualization” menu, click on “+” and select “Data table”.
The configuration under “metrics” is:
- Aggregation: “Average”
- Field: “metadata.network.lora.gatewayCnt”
- Custom Label: Number of gateways
Under “buckets”, click on “Split Rows” and select the following configuration:
- Aggregation: “Terms”
- Field: metadata.network.lora.devEUI”
- Order By: “metric:Number of gateways”
- Order: Ascending
- Size: 20
- Custom Label: Device
This configuration generates the list of the 20 devices with the lowest number of gateways. However, if you are more interested in the devices with the highest number of gateways, select the “Descending” order.
Uplinks on a map
This panel shows where your messages come from.
In the “Visualization” menu, click on “+” and select “Coordinate Map”.
For this panel, the metric is a “Count” aggregation, the bucket type is “Geo Coordinates” with a “Geohash” aggregation on the field “location.@geopoint”.
Note that in the case your data goes through a transformation pipeline (link), the field “location.@geopoint” corresponds to the last location set by the pipeline. If you want to use the original LoRa network location instead, select “metadata.network.lora.location.@geopoint”.
Evolution of LoRa KPI
In this section, the idea is to reproduce the 6 graphs shown in all LoRa devices’ details page. These graphs are a timeline of the device’s KPI:
- Space redundancy (number of gateways)
- LoRa signal quality
- Signal Noise Ratio (SNR)
- Received Signal Strength Indicator (RSSI)
- Spreading Factor (SF)
- Estimated Signal Power (ESP)
Although these graphs are already present on Live Objects’ portal, building them in Kibana will allow you to supervise the KPIs for the whole fleet or for a selected part of the fleet (see step 5 below).
Building the SNR graph
This example will show you how to build the SNR graph. The other graphs are very similar, so you can build them by changing only the targeted field.
Create a new visualization and select the “Line” type.
First we setup the bucket by clicking on “X-Axis”. The aggregation is a “Date Histogram” and the field is “timestamp”.
Then we setup 3 metrics:
Note that the “average” metric is computed from the SNR in all LoRa messages. If we receive more messages from one device than the others, this device will be overrepresented. Instead, if you wish all your devices to be equally represented, you can add the following metric.
Be aware that this metric can slow down the generation of the panel, so we do not recommend it for fleets with more than 500 devices.
Add a “Y-Axis” metric with an “Average Bucket” aggregation. For this type of aggregation we need to configure the bucket and the metric.
For the bucket:
- SubAggregation: Terms
- Field: metadata.source
- Order By: TermQuery
- Order: Descending
- Size: 500 (the size should be higher than the number of LoRa devices in your fleet)
For the metric:
- Aggregation: Average
- Field: metadata.network.lora.snr
Finally for the custom label we suggest “Average per device”.
How to build the other KPI graphs
We built a timeline of the Signal Noise Ratio. Similarly, you can build this timeline for the other KPI. To do this, use the corresponding field in each metric:
|Space redundancy (number of gateways)||metadata.network.lora.gatewayCnt|
|LoRa signal quality||metadata.network.lora.signalLevel|
|Signal Noise Ratio (SNR)||metadata.network.lora.snr|
|Received Signal Strength Indicator (RSSI)||metadata.network.lora.rssi|
|Spreading Factor (SF)||metadata.network.lora.sf|
|Estimated Signal Power (ESP)||metadata.network.lora.esp|
To avoid creating each graph from scratch, you can save a copy as we did for the “Success rate for devices with the most packet loss” panel.
Distribution of LoRa KPI
As in the previous section, we aim to build 6 panels, one for each LoRa KPI. Each graph will represent the distribution of the values for the corresponding KPI.
In this example, we will detail how to build the distribution of the Received Signal Strength Indicator (RSSI).
Create a new visualization with the type “Vertical Bar”.
The metric is a “Count” aggregation” and the Custom Label is “Number of uplinks”.
The bucket configuration is:
- Aggregation: Histogram
- Field: metadata.network.lora.rssi
- Interval: 5
- Custom Label: RSSI
To build the same graph for the other KPI, set the field and interval according to the following table:
|Space redundancy (number of gateways)||metadata.network.lora.gatewayCnt||1|
|LoRa signal quality||metadata.network.lora.signalLevel||1|
|Signal Noise Ratio (SNR)||metadata.network.lora.snr||5|
|Received Signal Strength Indicator (RSSI)||metadata.network.lora.rssi||5|
|Spreading Factor (SF)||metadata.network.lora.sf||1|
|Estimated Signal Power (ESP)||metadata.network.lora.esp||5|
Remember to modify the custom labels before saving your panels.
Step 4: Assemble your dashboard
Now that you have all your panels ready, it is time to create your dashboard.
Enter the “Dashboard” menu and click on the “+” button. It will create an empty dashboard that we will fill with the panel we created earlier in this tutorial.
Click on “Add” in the top right corner and select a panel. It will appear in the dashboard. Repeat this until all panels are in the dashboard.
Each panel can be resized, moved or removed. You can choose to show or hide the legend. You can also modify your panels from here by clicking on the pen icon. Now that your panels are in the dashboard you might want to adjust their visual aspects, such as the font size or the labels.
Do not forget to save your dashboard.
Step 5: Use your dashboard to monitor LoRa devices
Customize your dashboard on the go
Now let’s talk about the powerful dashboard you just created.
This dashboard shows various information on your LoRa devices, consequently it allows you to monitor them and notice unexpected behaviour of your fleet at a glance.
You can change the time window whenever you need, the time window at the top right corner is applied to all panels in the dashboard.
By default the dashboard contains data from all of your LoRa devices that communicated inside the time window.
However you and the other users of this dashboard can easily make it show data from a section of your fleet, without modifying the dashboard.
For instance, the dashboard can target devices in one specific group or one specific tag, this way you can monitor exactly the part of your fleet that you need.
To do this, simply type the corresponding filter in the search bar above the dashboard.
Add 2 more panels to help users target a part of the fleet
To help you and the other users of the dashboard, you can add the following panels as tooltips:
Here is the configuration for the first panel:
To target a part of your fleet in this dashboard, please enter one of the following filters in the search bar : | field | : search example | |-------|-------------------------| | devEUI | : metadata.network.lora.devEUI:"0123456789ABCDEF" | | group | : metadata.group.path:"/mygroup" | | tag | : tag:"my tag" | | properties | : extra.myPropertyKey:"my property value" |
The second panel’s purpose is to indicate the values you can use in the filter. In our example we show tags but you can choose any other field. Here is the configuration:
Type: Tag Cloud (or you can use Data table for a more compact view)
Metrics: “Count” aggregation
Bucket: “Terms” aggregation on “tags” field
We suggest to place these 2 tooltips panels at the top of the dashboard.
In this tutorial, we detailed how to build 12 different panels in Kibana. Then we showed how to create and use a dashboard with these panels.
Here we focused on monitoring LoRa devices, in a next article we will build a dashboard to supervise LoRa gateways.