Hybrid Transactional / Analytic Processing with Temporal Linked Data®

This weblog is about auto-generating a high-throughput, low-latency, resilient, reliable, scale-out, infrastructure-saturating microservice application, with realtime projections for graph analytics, based solely on a set of bespoke RDF data models.  The history of all writes are preserved along side their respective aggregates, providing for a temporal representation of all changes—application time travel for free.  

 

CMB Radiation View of the Universe’s Original State, Courtesy NASA / WMAP Science Team

 

We call this clustered transactional capability Temporal Linked Data® (TLD).  Runtime and persistence is provided through the BEAM ecosystem by way of Docker containers and container orchestration.

 

TLD generates well-written Elixir on world class BEAM web frameworks.  Auto-generating the backend solution is accomplished by reading a set of RDF data models that represent aggregates of OWL Datatype Properties and that contain concrete links between aggregates by way of OWL Object Properties.  These self-describing RDF models enable us to generate: 1) router endpoints, 2) a RESTful API with JSON payloads, 3) Elixir data modules, 4) OTP GenServer and Elixir process registry modules by way of servers and workers in a well-supervised distributed process hierarchy, and 5) change-data projections into scale-out big-data graph analytics platforms.  The result is a highly concurrent, highly reliable enterprise backend.  

 

HTAP via TLD

Iteratively Deployed TLD Microservices with Projections to Big Data Graph Analytics

 

The above sketch depicts autonomous TLD microservices asynchronously projecting writes to a common, scale-out, big-data graph analytics platform.  TLD services are intended to be delivered early and often, over time building up heterogeneous data for unique insights.

 

This is an auto-generated “graph throughout” solution architecture that is best described as a specific type of Hexagonal or Port-and-Adapter architecture where, by default, the transactional graph data structures flow through to the analytic graph structures to provide a realtime 360 view of the business, for The Business.

 

Alan Strait

alans@brsg.io