HTAP – Page 3 – Blue River Systems Group, LLC

18 Aug

Explicit Connections over Implicit Relationships

in HTAP

Low-friction data structures are those that: 1) are self-describing, 2) directly represent that which they model, and 3) do not require transformation between operations and analytics. Our graph-throughout approach to Hybrid Transactional / Analytic Processing system development encourages a direct connection between business and IT where data model creation can be collaborative.

Once the discrete, well-bounded RDF data model is designed, the backend system is generated, ready for: 1) behavioral augmentation where necessary, 2) test, and 3) deploy. Monolith and command-and-control are out, well-bounded microservice, distributed, and peer-to-peer are in.

This low-code approach helps to address the: 1) human resource impact of adopting new technology, 2) communication overhead of translating business concerns to production enterprise software, 3) adaptability of systems to business need, and 4) explicit connection between operational and analytic systems.

In contrast to relational and columnar data management which rely on implicit relationships embedded in SQL, Temporal Linked Data® (TLD) makes an explicit connection between temporal data aggregates.

Likewise, where Event Stores keep all data domain events together, TLD keeps all temporal data alongside the aggregate to which it belongs.

Explicit connections are both hard and soft, technical and social. Hybrid Transactional / Analytic Processing by way of TLD explicitly connects:

Temporal data aggregates,
Operational data with analytic data for realtime analysis,
Business and IT personnel, and
Represents the shortest path between business concept and production deployment.

Amundsen Scott South Pole Station, a low-friction location to view the night sky (NSF Public Domain Image)

13 Aug

Ideas That Scale, HTAP by way of TLD

by Alan Strait

in HTAP, Semantic Web

Comments

Our generation has the unique privilege of being able to see back to within a few million years of the beginning of time. How far back one is able to look depends on the technology that is used, but our Universe’s past is there to be seen. Temporal Linked Data® (TLD) naturally keeps a record of enterprise data changes to enable another kind of time travel, the story of enterprise data, for both operational and analytic purposes.

This series of weblogs introduces TLD as a transactional, low-code, enterprise class compute and temporal data cluster that naturally projects all writes to a world class big data graph analytics platform such as: 1) third-generation graph database for analysis, machine learning, and explainable artificial intelligence by way of TigerGraph, and / or 2) enterprise knowledge graph, ML, and AI by way of ReactiveCore.

For a high-level understanding we will briefly explore these subjects.

For a more concrete understanding we will use a gamification example, described as follows.

A Gamification Example for HTAP / TLD
Agile Gamification
Agile Gamification Model
Augmenting Operational Data with Gamification
Auto-Generated Backend
Temporal Linked Data® as TigerGraph Schema with Temporality
The Value of Temporal Data, Transactionally and Analytically
Reasonable Conclusions about HTAP by way of TLD

The technological innovation represented by the BEAM ecosystem and third-generation graph database allow for the possibility of building enterprise systems that simultaneously account for operational and analytic concerns. We look forward to taking this fast-data, big-data, HTAP, Temporal Linked Data® journey with you.

Our Temporal Universe

13 Aug

Hybrid Transactional / Analytic Processing with Temporal Linked Data®

by Alan Strait

in HTAP

Comments

This weblog is about auto-generating a high-throughput, low-latency, resilient, reliable, scale-out, infrastructure-saturating microservice application, with realtime projections for graph analytics, based solely on a set of bespoke RDF data models. The history of all writes are preserved along side their respective aggregates, providing for a temporal representation of all changes—application time travel for free.

CMB Radiation View of the Universe’s Original State, Courtesy NASA / WMAP Science Team

We call this clustered transactional capability Temporal Linked Data® (TLD). Runtime and persistence is provided through the BEAM ecosystem by way of Docker containers and container orchestration.

TLD generates well-written Elixir on world class BEAM web frameworks. Auto-generating the backend solution is accomplished by reading a set of RDF data models that represent aggregates of OWL Datatype Properties and that contain concrete links between aggregates by way of OWL Object Properties. These self-describing RDF models enable us to generate: 1) router endpoints, 2) a RESTful API with JSON payloads, 3) Elixir data modules, 4) OTP GenServer and Elixir process registry modules by way of servers and workers in a well-supervised distributed process hierarchy, and 5) change-data projections into scale-out big-data graph analytics platforms. The result is a highly concurrent, highly reliable enterprise backend.

Iteratively Deployed TLD Microservices with Projections to Big Data Graph Analytics

The above sketch depicts autonomous TLD microservices asynchronously projecting writes to a common, scale-out, big-data graph analytics platform. TLD services are intended to be delivered early and often, over time building up heterogeneous data for unique insights.

This is an auto-generated “graph throughout” solution architecture that is best described as a specific type of Hexagonal or Port-and-Adapter architecture where, by default, the transactional graph data structures flow through to the analytic graph structures to provide a realtime 360 view of the business, for The Business.

21 Jul

Full Spectrum Analytic Use Cases—No Special Cases

by Alan Strait

in HTAP

Comments

The fact that a chosen analytics platform does not span the entire analytic use case spectrum can be cause for concern and reevaluation. The temptation is to reach for a niche, special case platform or solution, but this may eliminate valuable and desirable forces.

Special case efforts tend to result in one-off projects rather than being available for the ordinary course of business. This tends to reduce the expectation of timeliness and scaleability. It can also unduly limit valuable concepts and algorithms by believing they cannot be used generally.

Page Rank is a good example of an algorithm that has general applicability as a measure of influence in a community, but that can be left out because the current analytic platform does not handle it well.

Full Spectrum Solution

Our own commitment to a graph-throughout architecture with realtime data projection to a third-generation graph analytics platform provides the best chance of being able to use the right algorithm for the job, whether realtime analysis, historical analysis, creating machine learning data sets, or applying explainable AI (as provided by TigerGraph), as well as more RDF-oriented Knowledge Graph solutions (such as our friends at ReactiveCore provide).

The time for graph-based transactional and analytic solutions is here. Please do have a read through our weblogs discussing Hybrid Transactional / Analytic Processing by way of Temporal Linked Data®.

BRSG advocates for business-oriented goals as well as considering lost forces that inhibit the ability to serve The Business well. Please reach out if we can be of service: info@brsg.io, or call 303.309.6240.