Rechercher

Article
· Sept 30 9m de lecture

Plateforme de Données InterSystems IRIS pour les Applications IoT

La plateforme de données InterSystems IRIS est une solution complète, multi-modèle et multi-charge de travail, idéale pour répondre aux exigences complexes des applications de l’Internet des Objets (IoT). Il s’agit d’une plateforme unifiée, cohérente et complète pour le développement, l’exécution et la maintenance des applications IoT.

Elle repose sur une architecture distribuée, capable de gérer des volumes massifs de données et des taux d’ingestion très élevés, tout en offrant la flexibilité et la robustesse d’une base de données transactionnelle multi-modèle de niveau entreprise. Cela permet d’ingérer, de traiter et de stocker des données provenant d’une grande variété de dispositifs et de formats.

La plateforme propose un ensemble complet de fonctionnalités d’intégration, de traitement d’événements et d’analytique intégrée, incluant une prise en charge complète du SQL, du traitement de texte, de l’orchestration des processus métier, ainsi qu’un environnement de développement basé sur les standards.

Se connecter à, ingérer et stocker une grande variété de types et formats de données issus de dispositifs disparates

Les types de données associés aux applications IoT sont souvent hétérogènes, car ils proviennent de dispositifs aux fonctions variées, fabriqués par différents fournisseurs. La plateforme de données sous-jacente doit être capable d’ingérer et de traiter un large éventail de données brutes dans leurs formats d’origine.

De nombreuses applications exigent également que la plateforme conserve toutes ces données sources disparates afin de détecter les écarts par rapport aux plages normales, de permettre des analyses ad hoc en aval, de respecter les obligations réglementaires ou de répondre à d’autres besoins spécifiques.

InterSystems IRIS simplifie la connexion à tout type de dispositif, via n’importe quel protocole, que ce soit pour consommer des données issues des dispositifs ou pour leur envoyer des données ou des instructions. La plateforme offre un ensemble complet de fonctionnalités garantissant la connectivité entrante et sortante avec n’importe quel dispositif ou protocole.

Elle intègre une bibliothèque d’adaptateurs prêts à l’emploi, permettant la connectivité et la transformation des données selon les standards, protocoles et technologies traditionnels de l’industrie, tels que REST, SOAP, HTTPS et JMS, ainsi que des protocoles plus récents et spécifiques à l’IoT, comme MQTT.

InterSystems IRIS permet également aux développeurs d’applications de créer rapidement des adaptateurs personnalisés (ainsi que la logique métier associée) pour prendre en charge pratiquement n’importe quel dispositif ou environnement.

InterSystems IRIS intègre une base de données transactionnelle multi-modèle éprouvée, de niveau entreprise, conçue pour traiter les données à grande échelle. Elle offre la flexibilité nécessaire pour stocker les données entrantes dans le format le plus adapté, notamment :

  • Modèles de données documentaires sans schéma, idéaux pour stocker des données brutes issues des dispositifs (température, vitesse, etc.) ainsi que les métadonnées associées (horodatage, identifiant du dispositif, etc.), afin d’offrir une flexibilité maximale pour les analyses ad hoc en aval.
  • Tableaux multidimensionnels, pouvant être stockés avec un nombre quelconque de sous-scripts.
  • Structures relationnelles, adaptées aux types de données bien structurés.
  • Modèles orientés objet, conçus pour les types de données complexes. Les données sont décrites une seule fois dans un dictionnaire de données intégré et sont accessibles via des accès objets, du SQL haute performance, et un accès multidimensionnel – tous pouvant interagir simultanément avec les mêmes données.

Ingérer, Traiter et Stocker les Données Entrantes des Dispositifs à des Taux d’Ingestion Élevés

Les applications IoT doivent pouvoir gérer des volumes massifs de données générées en continu par les dispositifs — parfois à raison de centaines de milliers, voire de millions de messages ou transactions par seconde. Les bases de données traditionnelles n’ont tout simplement pas été conçues pour supporter de tels taux d’ingestion de données. À titre de comparaison, moins de 10 millions de transactions sont exécutées chaque jour en moyenne sur le marché boursier du Nasdaq. En revanche, une application type de compteur intelligent dans une ville de petite ou moyenne taille doit ingérer et traiter plus d’un milliard de transactions par jour.

InterSystems IRIS est conçue pour traiter les données entrantes à des taux d’ingestion extrêmement élevés, comme l’exigent les environnements IoT, de manière efficace et rentable. InterSystems a passé plusieurs décennies à optimiser la performance et la scalabilité de sa technologie afin de répondre aux exigences strictes et aux accords de niveau de service (SLA) de ses clients.

Par exemple, l’Agence Spatiale Européenne utilise la technologie InterSystems pour traiter de très grands volumes de données satellites à des taux d’ingestion très élevés. Son application, fonctionnant sur un seul processeur Intel 64 bits à 8 cœurs, ingère et stocke cinq milliards d’objets Java discrets d’environ 600 octets chacun en 12 heures et 18 minutes, à un taux moyen d’insertion de 112 000 objets par seconde.

InterSystems IRIS prend en charge des niveaux élevés d’accès concurrent et des volumes de données très importants. Le scalabilité horizontale est disponible aussi bien pour les déploiements sur site que dans le cloud, offrant aux clients des options de déploiement flexibles. Cette scalabilité, qui permet le partage des données entre les nœuds, est rendue possible grâce à un protocole de cache hautement optimisé, garantissant le partage de données tout en préservant la fonctionnalité transactionnelle et l’intégrité des données.

Intégrer des Données Disparates, Réaliser des Analyses Avancées et Exécuter des Processus Automatisés en Temps Réel

La plateforme technologique sous-jacente doit être capable de traiter différents types d’analyses sur l’ensemble des données historiques non agrégées, et permettre aux analystes et aux data scientists d’identifier des corrélations entre les données des dispositifs et d’autres ensembles de données externes. Cela permet d’intégrer les informations issues de ces analyses dans des workflows programmatiques en temps réel, afin de réaliser des processus métier et des actions critiques à la demande.

Par ailleurs, des études ont montré que 40 à 60 % de la valeur métier générée par les applications IoT provient de l’interopérabilité entre différentes applications et systèmes IoT. Pour exploiter cette valeur, il est essentiel de disposer de fonctionnalités avancées pour les applications composites, ainsi que de puissantes capacités d’intégration permettant de combiner et de corréler les données issues de différentes sources, et ainsi révéler les informations précieuses cachées dans des ensembles de données disparates.

InterSystems IRIS offre des fonctionnalités permettant de développer et d’exécuter des requêtes et des analyses ad hoc sur les données structurées et non structurées contenues dans la base de données. Elle fournit un accès cohérent et unifié aux données, quel que soit leur type (objet, relationnel, etc.). Les performances des requêtes sur des structures de données objet complexes sont extrêmement rapides — souvent bien plus rapides que celles des bases relationnelles classiques. En plus des avantages de performance inhérents à la base de données multi-modèle, la technologie d’indexation bitmap accélère encore davantage les performances des requêtes sur les données en temps réel.

Les analystes et les data scientists peuvent intégrer un large éventail d’outils d’analyse, notamment la modélisation prédictive, le machine learning, Apache Spark et d’autres, afin d’identifier des motifs, des tendances et des corrélations dans les ensembles de données. Les résultats — qu’il s’agisse d’analyses ou d’algorithmes — peuvent ensuite être intégrés dans les processus métier en temps réel à l’aide d’un environnement de modélisation graphique, pour déclencher automatiquement un processus ou une action dès que des critères spécifiques sont remplis.

InterSystems IRIS fournit un ensemble complet de fonctionnalités pour créer et gérer des processus programmatiques en temps réel, exécutés au plus près des données, dans le même moteur que la base de données, pour une performance maximale.

En outre, les motifs et anomalies dans les données peuvent être détectés en temps réel, et des actions correctives programmées, des processus et des alertes peuvent être déclenchés automatiquement en réponse.

Les principales fonctionnalités incluent:

  • Messagerie et traitement des événements
  • Moteur de règles métier avec un environnement de modélisation graphique
  • Orchestration et gestion des processus métier
  • Moteur de workflow adaptable, prenant en charge les workflows automatisés et humains
  • Développement d'applications composites, pour une utilisation (et réutilisation) au sein des applications InterSystems et avec des applications externes
  • Surveillance de l’activité métier, avec tableaux de bord graphiques et alertes
  • Business Intelligence en temps réel, avec création de modèles de données en glisser-déposer, tableaux de bord en temps réel, et capacité à agir en temps réel sur les informations dans les applications transactionnelles
  • Gestion de bout en bout, incluant une visibilité en temps réel sur les processus métier et les performances du système

Agilité

La plateforme technologique doit être agile et conviviale pour les développeurs, permettant aux organisations de développer et déployer rapidement de nouvelles applications, tout en facilitant l’itération continue en fonction des évolutions des besoins et des exigences métier.

InterSystems IRIS offre un environnement unique et unifié pour le développement, l’exécution et la maintenance des applications IoT. Ainsi, elle élimine le temps et les efforts nécessaires pour apprendre, utiliser et intégrer plusieurs outils, produits et projets open source disparates.

La plateforme propose également un plug-in pour l’environnement de développement intégré Eclipse, très populaire, permettant un développement rapide et ouvert des applications IoT.

Enfin, InterSystems IRIS offre des options de déploiement flexibles, supportant à la fois les déploiements cloud et sur site.

Conclusion

L’Internet des Objets crée des opportunités sans précédent pour les organisations souhaitant transformer leurs activités. Cependant, les technologies et plateformes traditionnelles de gestion des données ne sont pas adaptées pour répondre aux exigences uniques, notamment en termes de débit élevé et d’échelle, propres à ce type d’applications.

InterSystems IRIS est une plateforme de données complète et multi-modèle, idéale pour les applications IoT. Il s’agit d’une plateforme intégrée qui fournit les capacités essentielles pour développer, exécuter et maintenir des applications IoT performantes dans un environnement unique, cohérent et unifié.

Elle repose sur une architecture distribuée capable de gérer des taux d’ingestion et des volumes de données massifs, tout en offrant la flexibilité et la persistance d’une base de données transactionnelle multi-modèle de niveau entreprise, pour travailler avec des données provenant d’un large éventail de dispositifs et dans différents formats.

La plateforme propose un ensemble complet de fonctionnalités d’intégration et de traitement des événements, des capacités analytiques intégrées incluant la prise en charge complète du SQL et du traitement de texte, ainsi qu’un environnement de développement basé sur des standards.

InterSystems est le moteur derrière les applications les plus importantes au monde. Dans les secteurs de la santé, de la finance, de l’administration publique et d’autres domaines où des vies et des moyens de subsistance sont en jeu, InterSystems est la force motrice de ce qui compte. Fondée en 1978, InterSystems est une société privée dont le siège est situé à Cambridge, Massachusetts (États-Unis), avec des bureaux dans le monde entier, et ses produits logiciels sont utilisés quotidiennement par des millions de personnes dans plus de 80 pays.

Plus d’articles sur le sujet :

Source: Plateforme de données InterSystems IRIS pour les applications IoT

Discussion (0)1
Connectez-vous ou inscrivez-vous pour continuer
Article
· Sept 30 7m de lecture

InterSystems IRIS Data Platform For IoT Applications

InterSystems IRIS Data Platform is a comprehensive, multi-model, multi-workload data platform that is ideal for accommodating the challenging requirements of applications for the Internet of Things. It is a complete platform for developing, executing, and maintaining IoT applications in a single, consistent, unified environment. It features a distributed architecture to support massive data-ingest rates and data volumes, while providing the flexibility and durability of an enterprise-grade transactional multi-model database to ingest, process, and persist data from a wide range of devices in different formats. It features a complete set of integration, event-processing, and integrated analytics capabilities, including full SQL support and text processing, business process orchestration, and a standards-based development environment.

Connect to, ingest, and persist a wide range of disparate device data types and formats

The data types associated with IoT applications are often heterogeneous, as they may originate from various devices with diverse functions and manufactured by different vendors. The underlying data platform must be able to ingest and process a wide range of raw data types in their original formats. Many applications also require the data platform to persist all of the disparate source data to detect deviations from normal ranges, accommodate downstream ad hoc analytics, maintain regulatory compliance, and fulfill other purposes.

InterSystems IRIS makes it simple and straightforward to connect to any device, using any protocol, both to consume data from devices and to send data or instructions to devices. It provides a variety of capabilities to ensure inbound and outbound connectivity to any device or protocol. It includes a built-in adapter library that provides connectivity and data transformations for traditional industry standards, protocols, and technologies, such as REST, SOAP, HTTPS, and JMS, as well as newer, IoT-specific protocols, like MQTT. InterSystems IRIS also enables the rapid development of custom adapters (and associated business logic) by application developers to support virtually any device or environment.

InterSystems IRIS incorporates a proven enterprise-grade transactional multi-model database that is designed to work with data on a massive scale and provides the flexibility to store the incoming data in the most appropriate format, including:

  • Schema-free document data models, which can be ideal for storing raw device data (temperature, speeds, etc.) and the associated metadata (timestamp, device ID, etc.) to provide maximum flexibility for performing downstream ad hoc analysis on the data
  • Multi-dimensional arrays, which can be stored with any number of subscripts
  • Relational data structures, for well-structured data types
  • Object-oriented models, for complex data types. The data is described once in a single, integrated data dictionary and is available using object access, high-performance SQL, and multi-dimensional access, all of which can simultaneously access the same data.

Ingest, Process, and Persist Incoming Device Data at High Ingestion Rates

IoT applications must be able to handle massive amounts of data that are being continuously generated by devices, sometimes on the order of hundreds of thousands — or millions — of messages or transactions every second. Traditional databases were simply not designed to accommodate such high data-ingest rates. Consider that fewer than 10 million trades are executed on average each day on the Nasdaq stock exchange. In contrast, a typical smart energy meter application in a small to midsize city must ingest and process more than one billion transactions every day.

InterSystems IRIS is designed to handle incoming data at the extremely high data-ingest rates that are required in IoT environments, in an efficient and cost-effective manner. InterSystems has spent decades optimizing the performance and scalability of its technology to meet the stringent requirements and service-level agreements of its customers.

For example, the European Space Agency uses InterSystems technology to process very large amounts of satellite data at very high ingest rates. Running on one 8-core Intel 64-bit processor, its application ingests and stores five billion discrete Java objects of about 600 bytes each in 12 hours and 18 minutes, at an average insertion rate of 112,000 objects per second.

InterSystems IRIS supports high levels of concurrent access and very large data volumes. Horizontal scaling is available for both on-premise and cloud installations, providing customers with flexible deployment options. Horizontal scaling, where data is shared between nodes, is available via a highly optimized caching protocol that allows the sharing of data among nodes while preserving transactional functionality and integrity.

Integrate Disparate Data, Perform Sophisticated Analyses, and Execute Real-Time Automated Processes

The underlying technology platform must accommodate a range of analytics processing types on the complete, unsummarized historical data, and enable analysts and data scientists to identify correlations among the device data as well as external data sets. This allows the information gleaned from the analyses to be incorporated into real-time programmatic workflows to perform real-time business processes and critical just-in-time actions.

In addition, research has found that between 40 and 60 percent of the business value from IoT applications is gained from interoperability between various IoT applications and systems. Realizing this value requires strong composite application features, as well as strong integration capabilities to combine and correlate data from the different sources to unlock the potential insights that are hidden in disparate data sets.

InterSystems IRIS provides functionality for developing and executing queries and ad hoc analyses on the structured and unstructured data in the database, and it provides consistent, unified access to the data regardless of the object type. The query performance on complex object data structures is extremely fast — typically much faster than relational only databases. In addition to the inherent performance benefits of the multi-model database, bitmap indexing technology further speeds query performance on real-time data.

Analysts and data scientists are able to incorporate a wide range of analytics tools, including predictive modeling, machine learning, Apache Spark, and others, to identify patterns, trends, and correlations in the data sets. The resultant insights or algorithms can be incorporated into the real-time business processes using the graphical modeling environment, to initiate a process or action when specific criteria are met. InterSystems IRIS provides comprehensive capabilities for creating and managing real-time programmatic processes that execute close to the data, in the same engine as the database, for the fastest performance.

In addition, patterns and anomalies in the data can be detected in real  time, and programmatic corrective actions, processes, and alerting can  be initiated in response.

Key capabilities include:

  • Messaging and event processing
  • A business rules engine with a graphical modeling environment
  • Business process orchestration and management
  • An adaptable workflow engine that supports automated and human workflows
  • Composite application development for use (and reuse) within  InterSystems applications and with external applications
  • Business activity monitoring, including graphical dashboards  and alerts
  • Real-time business intelligence, with drag-and-drop creation of data  models, real-time dashboards, and the ability to act in real time on information in transactional applications
  • End-to-end management, including real-time visibility into business  processes and system performance

Agility

The technology platform must be agile and developer-friendly, enabling organizations to quickly develop and deploy new applications, and to easily iterate on the applications as requirements and business demands change.

InterSystems IRIS provides a single, unified environment for developing,  executing, and maintaining IoT applications. As such, it eliminates the time and work required to learn, use, and integrate multiple disparate tools, products, and open-source projects.

It also provides a plug-in to the popular Eclipse integrated development  environment, enabling rapid and open development of IoT applications.

Finally, InterSystems IRIS provides flexible deployment options, supporting both cloud and on-premise deployments.

Conclusion

The Internet of Things is creating unprecedented opportunities for organizations to transform their businesses. But traditional data management technologies and platforms are not equipped to handle the unique requirements, including the high throughput and scale associated with these kinds of applications.

InterSystems IRIS is a comprehensive, multi-model data platform that is ideal for IoT applications. It is a complete platform that provides critical capabilities required to develop, execute, and maintain high-performance IoT applications in a single, consistent, unified environment. It features a distributed architecture to support massive data ingest rates and volumes while providing the flexibility and persistence of an enterprise-grade transactional multi-model database to work with data from a wide range of devices in different formats. It provides a complete set of integration and event-processing capabilities; integrated analytics capabilities, including full SQL support and text processing capabilities; and a standards-based development environment.

InterSystems is the engine behind the world’s most important applications. In healthcare, finance, government, and other sectors where lives and livelihoods are at stake, InterSystems is the power behind what matters. Founded in 1978, InterSystems is a privately held company headquartered in Cambridge, Massachusetts (USA), with offices worldwide, and its software products are used daily by millions of people in more than 80 countries.

More articles on the subject:

Source: InterSystems IRIS Data Platform For IoT Applications

Discussion (0)0
Connectez-vous ou inscrivez-vous pour continuer
Article
· Sept 30 3m de lecture

InterSystems IRIS: o que é, quando usar e um hands-on de 15 minutos

Oi pessoal!  Esse artigo é para quem está começando com InterSystems IRIS. Espero que ajude!

O InterSystems IRIS é uma plataforma de dados unificada: uma base de dados de alta performance com ferramentas de interoperabilidade e análise integradas em um só produto. Você tem SQL e NoSQL na mesma máquina, além de jeitos nativos de rodar Python com seus dados. Em resumo: menos peças móveis, mais capacidade de processamento.

Por que engenheiros escolhem IRIS

  • Multi-modelo, uma máquina. Funciona com tabelas relacionais ,objetos, globais, sem precisar alterar o contexto.
  • Python onde os dados vivem. O “Embedded Python” permite que você escreva métodos Python server-side que rodam dentro do IRIS (e não por um gateway externo). Você também pode chamar o IRIS do Python usando o móduloiris
  • Escalabilidade vertical e horizontal. Inicie em apenas uma instância, e então adicione fragmentos (shards) para dividir o armazenamento e cache horizontalmente em vários nós, para acelerar queries e ingestão.
  • Feito para aplicações em tempo real. Mira em alta performance para altos volumes de cargas de trabalho, mantendo transações e análises juntas.

Quando o IRIS brilha (e quando não)

Use quando você precisa de cargas de trabalho em estilo HTAP (operacional e analítico em um só lugar), consistência estrita e ferramentas de integração incorporadas. Se você simplesmente precisa de um cache ou um sandbox de análise avulso, uma base de dados maias leve pode ser mais simples.


Mão na massa (≈15 minutos)

1) Suba o IRIS localmente (Docker)

# pull e rode Community Edition
docker run --name iris \
  -d -p 52773:52773 -p 1972:1972 \
  intersystems/iris-community
  • Portal de Administração Web: http://localhost:52773/csp/sys/UtilHome.csp
  • As credenciais padrão do container são definidas na primeira execução, siga os prompts da tela.
    (Community Edition é gratuita para desenvolvimento e teste e está disponível no Docker Hub, com a documentação oficial do container de "primeiro contato".) 

2) Crie uma tabela e consulte(SQL)

No Portal de Administração (System Explorer → SQL), execute:

CREATE TABLE demo.orders(
  id INT PRIMARY KEY,
  customer VARCHAR(80),
  total DECIMAL(10,2),
  created_at TIMESTAMP
);

INSERT INTO demo.orders VALUES
 (1,'Kai',125.50,NOW()),
 (2,'Amaka',78.90,NOW());

SELECT customer, total
FROM demo.orders
WHERE total > 100;

(O IRIS SQL lida com conversões de tipos e formatos lógico/display por baixo dos panos.) 

3) Execute o Python dentro do IRIS(Embedded Python)

Crie um método de classe que retorna uma métrica rápida:

Class Demo.Utils
{

ClassMethod BigSpender(threshold As %Numeric) As %Integer [ Language = python ]
{
    import iris
    # Simple count using embedded SQL
    sql = "SELECT COUNT(*) FROM demo.orders WHERE total > ?"
    rs = iris.sql.exec(sql, threshold)
    return list(rs)[0][0]
}

}

Chame-o do terminal:

do ##class(Demo.Utils).BigSpender(100)

Isso é um Python server-side compilado e executado na máquina IRIS (sem ponte externa), e você pode chamar do ObjectScript ou SQL conforme necessidade. 

4) Notas sobre escalabilidade (para quando você crescer)

Se as consultas ou ingestão começarem a atingir os limites, adicione shards. O IRIS particiona dados e cache através de nós, fornecendo escalabilidade horizontal sem reescrever - e você pode misturar escalabilidade vertical e horizontal conforme necessidade.

 

Dicas direcionadas a produção

  • Mantenha unificado. Resista separar OLTP e análises cedo; o IRIS foi feito para mantê-las juntas até que um gargalo real ocorra.
  • Use Python com moderação, mas estrategicamente. Coloque lógica de negócio ou analítica que se beneficie das libs do Python no Embeddede Python; deixe o ETL pesado para trabalhos agendados.
  • Planeje shards antes de necessitar deles. Escolha chaves de shard que você não irá se arrepender (IDs imutáveis, time buckets) e teste em um sandbox de 3 nós.
Discussion (0)1
Connectez-vous ou inscrivez-vous pour continuer
Question
· Sept 30

Move to immutable backups

At the moment, we have 10 HealthShare instance servers (5 x mirrored pairs), where we implement an External Backup approach, using the freeze/thaw commands against whichever server of the pair is the backup mirror member, to complete a VM level backup. These backups are stored to a disk within our control, to purge as required. This approach allows us to deliver a zero downtime backup approach.

Our future backup solution will be storing immutable backups, with the main concern being that the 10 HealthShare instance servers are quite sizable and that our current approach of backing up the current backup mirror member at the time of the backup could generate 10 immutable VM backups and the size this would consume, when the ideal solution would be to only be capturing one side of each pair consistently.

With this in mind, I don't know whether anybody has any experience in a similar situation as to what would be an optimal solution? I have considered the following options and potential concerns:

1) To only ever backup one side of each pair, with a manual effort to keep one server as the preferred primary - the concerns being if it's the current primary that did get backed up, the freeze/thaw will impact system use, or if there was a VM issue for any period of time, we might have no valid backup

2) Implement online backups of the primary member to a centralised drive location and backup the drive, rather than the entire VM - the concerns being the time to complete such large DB backups, the time to restore and in a DR situation, the time to implement a new VM environment to restore the databases

3) Look at implementing a Concurrent External Backup approach - similar concerns to approach 2

Any thoughts are experiences would be gratefully received.

Regards,

Mark

2 Comments
Discussion (2)1
Connectez-vous ou inscrivez-vous pour continuer
Article
· Sept 30 11m de lecture

Implementing a FHIR project - ÚNICAS

Welcome, dear members of the Community!

In this article, we will present an example of a project implementing a FHIR-based solution. This project will be based on the national project (Spanish national project), known as ÚNICAS.

What is ÚNICAS?

In his own words:

A project whose objective is to create an ecosystem of partnerships to improve healthcare for pediatric patients with complex rare diseases (RMDs). This project is being implemented through the network within the National Health System (NHS) to improve the diagnosis and care of patients with rare diseases.

Technical characteristics of ÚNICAS

To summarize, the architecture designed for this project is composed of:

Central node

The objective is to enable the sharing of clinical data on patients affected by these rare diseases. This clinical data will be found in the regional nodes.

Autonomous node

Each autonomous community will have its own node (or more than one, depending on the volume of data). These nodes will be responsible for obtaining patients' clinical information from the various available data sources and providing it to the central node.

Interoperability model

The interoperability model chosen for this case is FHIR (R5), which provides a set of standardized resources for the exchange of clinical data.

Each autonomous community is free to choose how to implement this model, either by deploying a FHIR repository into which to upload clinical data for subsequent sharing or by using a FHIR facade that will transform clinical information into FHIR resources when a sharing request is processed.

FHIR resources in ÚNICAS

Since this project has very specific objectives, it has defined a set of FHIR resources to be used:

Mensajería estandardizada en ÚNICAS en FHIR

While for reports:

Mensajería existente de informes en HL7 CDA

Implementing ÚNICAS with Health Connect

After reviewing the technical aspects of the ÚNICAS project, it's time to examine how it fits with Health Connect and what possible adaptations would be necessary to facilitate implementation.

FHIR R5 Model

In order to implement the interoperability model in FHIR R5, the client has two options:

  • FHIR Facade.
  • FHIR Repository.

FHIR Facade

The FHIR facade will deploy the business components necessary for a REST API within our Health Connect instance that allows us to receive calls from third-party systems with the various FHIR resources. With this option, we can extract the resources in %DynamicObject format and transform them into our specific clinical data format.

Queries received against the FHIR facade must be built manually, since there is no FHIR repository linked below, so we must implement each feature ourselves. For ÚNICAS, this would not be the best option, since the query-level requirements would require us to implement the functionality we already have in the FHIR repository.

FHIR Repository

Health Connect permite (consultar condiciones de la licencia) desplegar un repositorio FHIR R5 de una forma sencilla desde un namespace con la opción de interoperabilidad habilitada desde el menú Health -> FHIR Server Management

Al pulsar sobre la opción de añadir nuevo servidor se nos mostrará una pantalla con las diversas opciones de configuración:

As you can see, we have the option to import Packages into our configuration. This will be useful for importing the ValueSets and Profiles defined by the ÚNICAS IG. We can perform this import at any time by running the following command:

// Rutas a modificar por el usuario
do ##class(HS.FHIRMeta.Load.NpmLoader).importPackages($lb("/iris-shared/packages/hl7.terminology.r5-6.5.0/package", "/iris-shared/packages/hl7.fhir.uv.extensions.r5-5.2.0/package","/iris-shared/packages/full-ig/package"))

In the ÚNICAS IG, there is a small discrepancy regarding the dependencies defined for the full-IG . The following dependencies are listed within the package:

"dependencies" : {
    "hl7.terminology.r5" : "6.5.0",
    "hl7.fhir.uv.extensions.r5" : "5.2.0",
    "hl7.fhir.uv.ips" : "1.1.0"
  }

Version 1.1.0 of  hl7.fhir.uv.ips  is only supported on FHIR R4, and attempting to import it onto an R5 server will result in an error, as it also has several R4 dependencies that will cause incompatibilities. For our implementation, we'll remove it from the dependency list and import only those related to R5...and let God take care of it.

Adapting FHIR repository to ÚNICAS

One of the most interesting features of the FHIR repository in Health Connect is the ability to use the interoperability engine to capture and transform every interaction with the FHIR repository to adapt it to our needs. To do this, we only need to include the business service  HS.FHIRServer.Interop.Service  and the business operation  HS.FHIRServer.Interop.Operation from the production environment linked to the namespace where we have deployed the FHIR repository. This will be the appearance of the production environment for the namespace containing our repository:

With these components in our production, we can specify in our repository configuration that all calls using the REST API will go through this service:

Perfect! Our FHIR R5 repository is ready to begin deploying our ÚNICAS autonomic node. Let's look at an example of how we can leverage interoperability capabilities to transform HL7 v2.5.1 messaging into FHIR R5 resources.

Converting HL7 to FHIR

A typical case to be solved for any FHIR repository implementer is the conversion of HL7 (v2 or v3) messaging into FHIR resources. This conversion is not at all straightforward for two reasons: 

  1. There is no direct equivalence between FHIR resources and HL7 events. While FHIR is used for the exchange of clinical concepts in the form of resources, HL7 represents events, which contain information that may or may not be equivalent to a clinical concept.
  2. The implementation of HL7 v2 or v3 messaging is far from uniform. Each organization adapts messages to its needs, and the same event in different organizations may contain completely different data.

What tools does Health Connect provide us?

Health Connect has a series of predefined transformations that use the SDA3 data model as a gateway, very similar to CDAs. These transformations are currently developed for the R4 version, but it is not very complicated to adapt them to transform R4 resources generated in the R5 version.

So the steps to follow will be HL7 v2.5.1 -> SDA3 -> FHIR R4 -> FHIR R5. 

HL7 v2.5.1 Transformation  -> SDA3

For this first step, we'll use the  HS.Gateway.HL7.HL7ToSDA3 class  and its  GetSDA method  (you can see an example in the documentation here ). This class will transform our HL7 message into a nice SDA.

Transformation SDA3 -> FHIR R4

Once we have obtained our SDA with the clinical information extracted from the HL7 message, we will use the  HS.FHIR.DTL.Util.API.Transform.SDA3ToFHIR class  and its TransformStream  method , which will transform the SDA3 in Stream format obtained in the previous step into a %DynamicObject with a FHIR R4 Bundle resource (more information here ).

Below you can see an example of the code needed to transform an HL7 message into a FHIR R4 bundle in %DynamicObject format:

 do ##class(HS.Gateway.HL7.HL7ToSDA3).GetSDA(request, .SDAMessageTemp)
 set SDAToFHIR = #class(HS.FHIR.DTL.Util.API.Transform.SDA3ToFHIR).TransformStream(SDAMessageTemp,"HS.SDA3.Container","R4")

What about my FHIR R5?

So far we have seen that Health Connect allows for a very simple conversion from HL7 to FHIR R4 (the conversion may not fit 100% of your needs, but it will be easier to edit a resource than to build it from scratch), great, but...WHERE IS MY R5 RESOURCE?

Don't worry, because... the Developer Community is coming to the rescue!

You'll see that attached to this article is an associated OpenExchange application. This application contains the code necessary to transform FHIR R4 resources into the R5 resources used in the ÚNICAS project. To install this code, you can use IPM:

set version="latest" s r=##class(%Net.HttpRequest).%New(),r.Server="pm.community.intersystems.com",r.SSLConfiguration="ISC.FeatureTracker.SSL.Config" d r.Get("/packages/zpm/"_version_"/installer"),$system.OBJ.LoadStream(r.HttpResponse.Data,"c")
zpm "enable -community"
zpm "install spain-unicas"

Once installed, you'll see a package called Spain  appear in the namespace where you installed it. What can you find in this package?

  • Spain.BP : BPL with example of transformation HL7 -> SDA -> FHIR R4 -> FHIR R5.
  • Spain.FHIR.DTL.vR4 : R4 to R5 transformations of the resources used by ÚNICAS.
  • Spain.FHIR.DTL.vR5 : ObjectScript classes that model ÚNICAS R5 resources.
  • Spain.Gateway.HL7 : Minor fix for HL7 to SDA3 transformation.

Transforming R4 to R5

With the tools available in the Spain package  , we can now transform our R4 resources into R5 resources. To do this, we will use the transformations for each resource type as follows:

set obj = ##class(HS.FHIR.DTL.vR4.Model.Resource.Bundle).FromJSONHelper(context.FHIRMessage, "vR4")

for i=1:1:obj.entry.Count() {
    set item = obj.entry.GetAt(i)
    if ($CLASSNAME(item.resource) = "HS.FHIR.DTL.vR4.Model.Resource.Immunization")
    {
      Set status = ##class(Spain.FHIR.DTL.vR4.vR5.Immunization).Transform(item.resource, .immunizationR5)            
      Set item.resource = immunizationR5
      Do obj.entry.SetAt(item, i)            
    }
    elseif ($CLASSNAME(item.resource) = "HS.FHIR.DTL.vR4.Model.Resource.AllergyIntolerance")
    {
      Set status = ##class(Spain.FHIR.DTL.vR4.vR5.Allergy.AllergyIntolerance).Transform(item.resource, .allergyIntoleranceR5)            
      Set item.resource = allergyIntoleranceR5
      Do obj.entry.SetAt(item, i)            
    }
    ...
 }

Remember that we had a %DynamicObject with a bundle of R4 resources, the first step we will take will be to map said %DynamicObject to a  HS.FHIR.DTL.vR4.Model.Resource.Bundle class using the FromJSONHelper method , the objective of this transformation is to be able to later recognize the type of resource present in the bundle by the name of its class.

With the bundle mapped to its ObjectScript class, we will simply go through each entry, identifying the resource. If it corresponds to one of the resources used in ÚNICAS, we proceed to invoke a specific DTL to perform the transformation from R4 to R5.

With the resource already transformed, we reintroduce it into the Bundle replacing the previous R4 resource.

Finally, we will only have to transform the Bundle into a Stream (in this case called QuickStream) and send it within a message of type HS.FHIRServer.Interop.Request  to the business operation HS.FHIRServer.Interop.Operation

 Set context.FHIRRequest.Request.RequestMethod = "POST"
 Set context.FHIRRequest.Request.RequestPath = ""
 Set context.FHIRRequest.Request.RequestFormatCode= "JSON"
 Set context.FHIRRequest.Request.SessionApplication = "/csp/healthshare/fhirserver/fhir/r5"
 Set context.FHIRRequest.Request.IsRecursive = 0
 set qs=##class(HS.SDA3.QuickStream).%New()
 set context.FHIRRequest.QuickStreamId = qs.%Id()
 do qs.CopyFrom(context.FHIRStream)
 

Here we would have the call:

Well, that's it! We now have our small example of HL7 to R5 conversions and its registration in our Health Connect repository. Let's see it in action.

Example of transformation from HL7 to R5

Let's start with an HL7 v2.5.1 message

MSH|^~\&|CLINIC_SYS|HOSPITAL123|EHR_SYSTEM|REGION1|20250826143000||ADT^A08^ADT_A01|MSG00002|P|2.5.1
EVN|A08|20250826143000
PID|1||123456^^^HOSPITAL123^MR||GOMEZ^MARIA^LUISA||19750523|F|||AVDA UNIVERSIDAD 45^^MADRID^^28040^ESP||(555)1234567|||S||12345678Z^^^ESP^NI
AL1|1|DA|70618^Penicillin^RXNORM|SV|Rash|20200115|Clinician noted severe rash after administration
AL1|2|FA|256349002^Peanut protein^SCT|SE|Anaphylaxis|20181201|Reported after accidental exposure
AL1|3|MA|300916003^Latex^SCT|MO|Skin irritation|20191010|Observed during procedure with latex gloves

Here we have a simple ADT_A08 with information related to a patient and their allergies, as diligent implementers that we are, we know that this message will contain (at least) one resource of type Patient and 3 of type AllergyIntolerance. 

We introduced a business service into our production to capture HL7 files and send them to our BPL:

Once captured, we transform our HL7 to R4 through SDA, let's take a look at the AlleryIntolerance resource generated in R4:

{
                                                              
	"request": {
		"method": "POST",
		"url": "AllergyIntolerance"
	},
	"fullUrl": "urn:uuid:4b9f7b2e-9dd0-11f0-870e-c6b593068383",
	"resource": {
		"resourceType": "AllergyIntolerance",
		"category": [
			"food"
		],
		"clinicalStatus": {
			"coding": [
				{
					"code": "active",
					"system": "http://terminology.hl7.org/CodeSystem/allergyintolerance-clinical"
				}
			]
		},
		"code": {
			"coding": [
				{
					"code": "256349002",
					"display": "Peanut protein",
					"system": "http://snomed.info/sct"
				}
			]
		},
		"extension": [
			{
				"url": "http://intersystems.com/fhir/extn/sda3/lib/allergy-discovery-time",
				"valueDateTime": "2018-12-01T00:00:00+00:00"
			},
			{
				"url": "http://intersystems.com/fhir/extn/sda3/lib/allergy-entered-at",
				"valueReference": {
					"reference": "urn:uuid:4b9e97f6-9dd0-11f0-870e-c6b593068383"
				}
			}
		],
		"onsetDateTime": "2018-12-01T00:00:00+00:00",
		"patient": {
			"reference": "urn:uuid:4b9edb58-9dd0-11f0-870e-c6b593068383"
		},
		"reaction": [
			{
				"extension": [
					{
						"url": "http://intersystems.com/fhir/extn/sda3/lib/allergy-severity",
						"valueCodeableConcept": {
							"coding": [
								{
									"code": "SE"
								}
							]
						}
					}
				],
				"manifestation": [
					{
			
						"coding": [
							{
								"code": "Anaphylaxis"
							}
						]
 
					}
				]
			}
		]
	}
}

And now the same resource but adapted to the R5 version:

{
	"fullUrl": "urn:uuid:4b9f7b2e-9dd0-11f0-870e-c6b593068383",
	"request": {
		"method": "POST",
		"url": "AllergyIntolerance"
	},
													  
	"resource": {
		"resourceType": "AllergyIntolerance",
		"category": [
			"food"
		],
		"clinicalStatus": {
			"coding": [
				{
					"code": "active",
					"system": "http://terminology.hl7.org/CodeSystem/allergyintolerance-clinical"
				}
			]
		},
		"code": {
			"coding": [
				{
					"code": "256349002",
					"display": "Peanut protein",
					"system": "http://snomed.info/sct"
				}
			]
		},
		"extension": [
			{
				"url": "http://intersystems.com/fhir/extn/sda3/lib/allergy-discovery-time",
				"valueDateTime": "2018-12-01T00:00:00+00:00"
			},
			{
				"url": "http://intersystems.com/fhir/extn/sda3/lib/allergy-entered-at",
				"valueReference": {
					"reference": "urn:uuid:4b9e97f6-9dd0-11f0-870e-c6b593068383"
				}
			}
		],
		"onsetDateTime": "2018-12-01T00:00:00+00:00",
		"patient": {
			"reference": "urn:uuid:4b9edb58-9dd0-11f0-870e-c6b593068383"
		},
		"reaction": [
			{
				"extension": [
					{
						"url": "http://intersystems.com/fhir/extn/sda3/lib/allergy-severity",
						"valueCodeableConcept": {
							"coding": [
								{
									"code": "SE"
								}
							]
						}
					}
				],
				"manifestation": [
					{
						"concept": {
							"coding": [
								{
									"code": "Anaphylaxis"
								}
							]
						}
					}
				]
			}
		]
	}
}

As you can see, the reaction property has been modified, adapting it to the R5 definition:

Conclusion

As you've seen, Health Connect offers endless possibilities for adapting to the various FHIR-based projects we may encounter. With the interoperability engine, we can adapt our data however we need, taking into account any requirements that deviate from standard behavior.

I hope you find it useful!

2 Comments
Discussion (2)3
Connectez-vous ou inscrivez-vous pour continuer