INTER-MW Architecture
The core functionality of INTERMW is facilitation of interoperability among IoT Middleware platforms, as well as the provision of a common abstraction layer to provide access to platform's features and information. While the features are described in the previous section, here the architecture of INTERMW is provided and elaborated.
The communication among components is asynchronous and implemented, at logical level, through the introduction of "conversations" and consequently of data flow control. A group of messages belongs to the same conversation, if they share the same conversation identifier (ConversationID). For example, in a single conversation we would typically have first a message, that subscribes to a particular group of sensors, and then messages with sensor readings, going upstream from the sensors to the application. Subscriptions in INTERMW are also tracked by ConversationID. Technically, data flows are implemented through a message broker.
The integration with IPSM is achieved through the IPSM Request Manager component that orchestrates the communication between IPSM and INTERMW components (Bridges, Platform Request Manager).
A triplestore database provides persistence and advanced querying mechanisms for the Services subsystem. All registry-related requirements that need persistence or querying support are indirectly implemented through this triplestore. Currently, Platform Registry, Resource Registry and Subscriptions Registry use functionalities provided by this component. The Parliament triplestore database is used.
Syntactic conversion and semantic translation of platform-specific messages have been de-coupled. Implementation of a Bridge to provide interoperability for a new platform means, for a bridge developer, implementation of a communication protocol with the platform and syntactic translation of the message between platform-specific format and INTER-IoT JSON-LD. Definition of rules for semantic translation (alignments) is still necessary, but not at the Bridge level. That part of the process if fully implemented in IPSM.
Components
REST Interface
INTERMW provides REST API interface that can be used by client applications. Swagger (OpenAPI) REST API definition language has been selected for definition of all REST API interfaces throughout INTER-Layer component. To facilitate the development of INTERMW and keep the definitions up-to-date, the Swagger annotation library for Java has been used to document the INTERMW REST API.
Data Flow Management
The data flow management concept has been elaborated through the definition of a "conversation" concept and of an abstract broker client interface. On a conceptual level, INTERMW components communicate through channels (queues) that connect two components. The full list of those channels is provided in the table below.
Conversations get initiated at the API Request Manager (ARM) level. This means that, for each new group of messages, ARM creates a new Conversation Id and returns it to the caller. The caller than uses this identifier in two scenarios: when sending requests for change of the original request in that conversation (for example, unsubscribe to an original subscribe request) or when matching responses from the middleware to an original request (for example, receiving observations - sensor readings- for a subscribe request). As consequence, components such as Bridges or Services must keep track of active conversations an match responses to a correct conversation. We could also think of conversations as permanent "sessions" or "channels" between API clients and middleware components.
The actual messaging has to be instantiated through implementation of message broker clients. Basically, any message broker that provides the basic messaging functionality can be used. At the moment of writing of this deliverable, clients for the following message brokers have been implemented: Apache ActiveMQ, Apache kafka, generic MQTT client, Vortex OpenSplice and RabbitMQ.
| Topic | Description |
|---|---|
| arm_prm | API Request Manager messages to Platform request manager. ARM relates API requests with client identifiers and forwards messages to PRM. ARM communicates only with the PRM. |
| prm_arm | Platform request manager messages to the API Request Manager. API request manager relates message responses, through conversation identifiers, with API client identifiers and invokes client callback messages. |
| prm_srm | Platform request manager to Services. PRM sends all requests related to registry and discovery messages to SRM. The invocation of Services depends on message types. |
| srm_prm | Services response to PRM. In principle, the response contains a set of attributes from the registry. It may also contain a discovery request for platforms, which means that PRM will re-route it southbound, towards the bridges. |
| prm_ipsmrm | All southbound flow towards platforms passes through the IPSM Request Manager (IPSMRM), which takes care of message translation (by invocation of IPSM) and forwarding to bridges. |
| ipsmrm_prm | IPSM Request Manager to Platform request manager messages are already translated responses from platforms. |
| mw-ipsm-interiot-format-{Platform Type} | Messages in InterIotMsg format sent for translation (see note 1). These are messages to be sent to platforms after they have been translated by IPSM. |
| ipsm-mw-platform-format-{Platform Type} | Messages in IoT platform-specific formant returned from IPSM (see note 1). These are messages ready to be sent to platforms. |
| mw-ipsm-platform-format-{Platform Type} | Messages in IoT platform-specific format sent for translation (see note 1). These are messages sent by platforms to INTERMW. |
| ipsm-mw-interiot-format-{Platform Type} | Messages in InterIotMsg format translated by IPSM (see note 1). These are messages sent by platforms to INTERMW and already translated by IPSM. |
| ipsmrm_bridge_{Platform Id} | Message flow from IPSRM to a bridge (see note 2). |
| bridge_ipsmrm_{Platform Id} | Message flow from bridge to IPSRM (see note 2). |
| rest_api_{client Id} | Topic for client callback messages (REST). There is one queue per REST API client. Clients use pull method to get messages form the response queue. |
| error | Global INTERMW error topic. In case of any errors, messages get posted to this topic. |
INTERMW inter-component communication topics
Notes to table:
- The coding convention of topic names between INTERMW and IPSM is as follows.
([mw-ipsm | ipsm-mw])is the direction of the flow (source to destination), which means from INTERMW (MW) to IPSM or the opposite. The second part,([interiot-format | platform-format]), designates the format that is posted to the topic: InterIotMsg format or platform specific. The last part is the platform type (FIWARE, universAAL, etc.). - These are messages sent between bridges and IPSRM. The suffix is the platform identifier. Messages flowing through these channels use platform-specific semantics. Each bridge instance is subscribed to its own topic and published data to a specific topic as well. Bridges do never share topics.
Messages
The core of INTER-MW are messages expressed in the common ontology. The same message structure is used by IPSM, whose role is to semantically translate messages to and from the common INTER-IoT ontology. The details of the INTER-MW message are described in a dedicated section.
IPSM Request Manager
IPSM Request Manager acts as a mediator between the Platform Request Manager, Bridges and the IPSM. It redirects messages for translation towards IPSM and receives translated messages from the IPSM. There is no direct connection between the bridges and IPSM, but instead all communication with Bridges for IoT platforms is now handled by the IPSM Request Manager. It also implements the IPSM-specific broker mechanism, which in principle can be different that the one used throughout INTERMW. The current implementation used RabbitMQ for communication among INTERMW components, while Kafka is used for communication (INTERMW →IPSM).
Services
Implementation of services has been realized by connecting these to the Parliament triplestore-based database and introducing support for the SPARQL RDF query language in the QUERY and DISCOVERY message types. SPARQL is custom-tailored for complex queries, and the Parliament database allows for fast (optimised) execution of those. An example of such a complex query would be returning a list of all sensors, connected to a specific platform and which are located in one location, as well as owned by one person. Wrapping such queries in QUERY and DISCOVERY messages enables the applications from the outside world and the Platform Request Manager within INTERMW to obtain lists of resources (connected platforms, things, etc.) dynamically. QUERY and DISCOVERY messages also abstract the need for INTERMW users/developers to understand the specifics of different platforms or learn discovery mechanisms they may (or may not) support.
Bridges
Bridges in INTERMW act as a middleman between INTERMW and IoT platforms. For messages that come downstream from INTERMW, a bridge processes the message and acts upon it if necessary. When a platform wants to send a message upstream, the bridge creates a message using information provided by the IoT platform and sends it upstream towards INTERMW. In addition to communication with platforms bridges also handle syntactic translation of payload between syntax used in the data model and the IoT platform format.
Semantic translation (alignments) performed by IPSM complements the syntactic conversion performed by bridges. Data models of platforms participating in communication through INTERMW thus use a two-step approach to map between format and semantics of the INTER-IoT data model. The need for platform's compliance with the internal INTER-IoT data model is in this way totally nullified, thus facilitating interoperability among heterogeneous middleware platforms, as long as someone has developed bridges for these platforms. Addition of a new platform does not require any changes in implementation of already existing bridges. This allows effective decoupling at both conceptual and implementation middleware levels.
API (and Extensibility)
There are several areas where INTERMW provides API and extensibility mechanisms:
- REST API. REST API is provided for application-layer usage. Applications can make use of the REST API in order to implement high level, application-specific functionality. This is also the interface exposed to INTER FW.
- *IoT Platform Bridges. For each IoT platform to be added to INTERMW a new bridge has to be developed. In order to allow developers add new bridges with minimal effort, this mechanism will be refined and provided as part of the SDK being developed in INTER FW.
- Services. INTERMW has the capability to provide various MW2MW services. Currently, only registry and discovery services are provided, as they represent the minimum set of functionalities needed for MW2MW interoperability. However, more complex services may be needed in some use case scenarios, as for example Routing and Roaming Services. INTERMW provides an abstract interface that can be implemented for provision of new services. Documentation, libraries and possibly an SDK for services development will be provided in the second phase of INTERMW implementation.
- Broker abstraction. INTERMW defines an abstract broker interface that is being used for all INTERMW communication between components. This interface can be implemented for any message broker that supports a basic set of functionalities. As a need to extent the system to a wide range of message broker implementations has not been identified, there will be no dedicated SDK solutions for this type of extensions. Extensible documentation and examples will be provided in order to facilitate implementation.