This document explains how to push data from City Pilots to IoT Platform.
Basically it covers 3 things:
How we can use Device data model for IoT devices in pilots.
How to push real-time Device data to IoT Platform.
How to push locally collected (e.g. scada or data loggers) Device data to IoT Platform.
How to push all locally collected as a batch operation
Before starting to describe Device data model, you should ask yourself:
Should I use Device data model or a more specific one related to the domain of use:
This is a very important question to ask yourself before choosing to use Device data mode.
DCA can choose to use a more specific data model for pushing the data they get from IoT devices, e.g. weather environmental stations IoT device can/should push data using WeatherObserved data model.
This is probably a more convenient option regarding the posterior data usage (consumers of data).
In other words consumers of data will appreciate you using domain specif data models (such as WeatherOberved) rather than using more generic models (such as Device).
Device data models should be used when handling large heterogeneous data sets, which dont fit well any of the well-known data models.
1. Using Device Data model
Some pilots data collectors (DCA) have expressed the need of pushing measurements coming from IoT devices or from SCADA systems.
The data is contained in an array, using the attribute called value.
The attribute controlledPropoerty, which is also an array describes the measurands of value.
The attribute location describes the location of the sensor using gps coordinates.
This model is simple, and flexible, as we can include any sensed data we want in the device entity.
On the other side, there is an added complexity for the consumers, and they dont know which attributes are included in an entity until they got fetch the data. This is not the case for example for a WeatherObserved entity, as we always know that these kind of entityes always have an attribute describing e.g. the temperature.
To overcome this added complexity this document describes how to use the NGSIv2 powerful querying mechanisms and getting the data you want efficiently, without having to do too much filtering.
Creating the entity in the IoT platform
As an example, you can create an entity using bash:
If you know specifically which device you want data from:
curl --location--request GET 'http://18.104.22.168:1026/v2/entities/urn:ngsi-ld:Device:Device-test'\--header'Fiware-Service: carouge'
When trying to get all devices in carouge which sense soil_moisture, you can:
curl --location--request GET 'http://22.214.171.124:1026/v2/entities?type=Device&q=controlledProperty==soil_moisture&limit=100'\--header'Fiware-Service: carouge'
in this case q=controlledProperty==soil_moisture provides us with the filtering features we want. Other attributes could be used for extending the filtering features.
If you want to know how to get the historical values of a certain value please keep reading.
2. How to push real-time Device data to IoT Platform.
This described how to push data using Device model into the CONTEXT MANAGER (real-time data).
If you read section 1. you know there is an entity called urn:ngsi-ld:Device:Device-test in the IoT Platform.
Now suppose we have collected a new value for temperature and soil_moisture, and hence we want to update those values.
We may want to update a certain attribute temperature, or soil_moisture, or both.
For this we could simply update the Device.value like this:
Et voilà! Your data is updated inside the context manager of the platform
3. How to push locally collected (e.g. scada or data loggers) Device data to IoT Platform.
This described how to push data using Device model into the HISTORIC DATA component (time series data).
You data is not real-time data, as you have collected it locally, and some time passed between that and now.
Also, you probably have more than one set of data, e.g. you may have collected data during the last 24 hours or more, taking samples every 15 mins.
NOTE WELL: All fields of the data model need to include this metadata, else, the current time will be used.
4. How to push all locally collected as a batch operation
If you have plenty of records and not just a single one you can add those to the data of the HTTP message, e.g. image you have locally collected data each hour.
In the following example we show how to push two registers, one generated at "2020-10-21T16:16:16.00Z" and the other generates at "2020-10-21T17:16:16.00Z". The same thing can be done for any amount of registers, although the amount of data that can be received at the same time is limited by a couple of mega bytes.
We recommend not pushing more than 1000 registers at a time.