Common issues 5/5: Understand indexing issues

Many applications from dataviz to analytics use the Live Objects search API to build business processes. In case of bad indexation you can encounter various problems:

• Data not displayed in your Live Objects widgets or in your dataviz app
• False statistics
• “missing data”
• …

To avoid such issue, you just have to keep in mind the basics of Live Objects Data model. Let’s take a real data message stored in Live Object:

In the example above you can see two parts in the message:

• Information added by Live Objects during the enrichment process. All these data are part of Live Objects data model; their structure and type are known and don’t vary, so they are automatically indexed.
• Information issued by the device which are stored in the “value”: {…} section. Because this content depends on the device, Live Objects cannot index it by default. This is done thanks to the field “model” which is set by a decoder, in case of LoRa as above, or by the device itself for MQTT devices who send directly structured json data. The data contained in “value”: {…} section will be indexed as @model.value.thedata.

In our example the temperature and pressure reported by the device will be indexed as @myDecoderDataModel.value.temperature and @myDecoderDataModel.value.pressure

This is crucial to keep this in mind, especially in development phases, because it is the source of common issues:

• The field model is missing: all data in “value”: {…} are not indexed
• With a same model you use two different types for temperature (one numeric, and one string for example). This leads to an indexing conflict, the new data will not be indexed and an error is logged in your Live Objects Activity Logs

Please note that maximum number of different fields you can index is 1,000. This limit is quite big and should not limit you in normal cases.

To know more about Live Objects data model and indexing service, you can refer to the developer guide here.