sitewhere

SiteWhere is an industrial strength, open source IoT Application Enablement Platformwhich facilitates the ingestion, storage, processing, and integration of IoT device dataat massive scale. The platform leverages a microservices architecture which runs on top ofcutting-edge technologies such as Kubernetes, Istio,and Kafka in order to scale efficientlyto the loads expected in large IoT projects.

SiteWhere embraces a distributed architecture which runs on Kubernetes and providesboth infrastructure such as highly-available databases and MQTT brokers as well asmicroservices to facilitate various aspects of IoT project development. The platform isbuilt with a framework approach using clearly defined APIs so that new technologies may easilybe integrated as the IoT ecosystem evolves.

Deployment and Orchestration

SiteWhere is composed of Java-based microservices which are built asDocker images and deployed to Kubernetes fororchestration. To simplify installation and configuration, Helmis used to provide standard templates for various deployment scenarios. Helmchartsare provided to supply both the microservices and the dependencies needed torun a complete SiteWhere deployment. Infrastructure components includetechnologies such as Apache Zookeeper and Kafka, highly available databases suchas MongoDB, InfluxDB, and Cassandra, and other supporting technologiessuch as MQTT brokers.

Microservices

Rather than using a monolithic approach, SiteWhere is based on many microservicesrunning as a distributed system. Each microservice is a completely self-containedentity that has its own configuration schema, internal components, data persistence,and interactions with the event processing pipeline. SiteWhere microservicesare built on top of a custom microservice framework and run as separateSpring Boot processes, eachcontained in its own Docker image.

Separation of Concerns

Separating the system logic into microservices allows the interactionsbetween various areas of the system to be more clearly defined. It also allowsparts of the pipeline to be shutdown or fail gracefully without preventing otherparts of the system from functioning. The event processing pipeline, which spansmany of the microservices, is buffered by Kafka so that data processing hasstrong delivery guarantees while maintaining high throughput.

Scale What You Need. Leave Out What You Don't

The microservice architecture allows individual functional areas of the system to be scaledindependently or left out completely. In use cases where REST processing tends tobe a bottleneck, multiple REST microservices can be run concurrently to handle the load.Conversely, services such as presence management that may not be required can be leftout so that processing power can be dedicated to other aspects of the system.

Instance Management

SiteWhere supports the concept of an instance, which allows the distributed systemto act as a cohesive unit with some aspects addressed at the global level. All of themicroservices for a single SiteWhere instance must be running on the same Kubernetesinfrastucture, though the system may be spread across tens or hundreds of machinesto distribute the processing load.

Service Mesh with Istio

SiteWhere leverages Istio to provide a service mesh forthe system microservices, allowing the platform to be scaled dynamically whilealso providing a great deal of control over how data is routed. Istio allowsmodern methods such as canary testing and fault injection to be used toprovide a more robust and fault-tolerant system. It also allows for detailedmonitoring and tracing of the data flowing through the components.

Centralized Configuration Management with Apache ZooKeeper

SiteWhere configuration is stored in Apache ZooKeeperto allow for a scalable, externalized approach to configuration management. ZooKeepercontains a hierarchical structure which represents the configuration for one or moreSiteWhere instances and all of the microservices that are used to realize them. Theconfiguration is replicated for high availabilty.

Each microservice has a direct connection to ZooKeeper and uses the hierarchy todetermine its configuration at runtime. Microservices listen for changes to theconfiguration data and react dynamically to updates. No configurationis stored locally within the microservice, which prevents problems withkeeping services in sync as system configuration is updated.

Distributed Storage with Rook.io

Since many of the system components such as Zookeeper, Kafka, and variousdatabases require access to persistent storage, SiteWhere usesRook.io within Kubernetes to supply distributed,replicated block storage that is resilient to hardware failures whilestill offering good performance characteristics. As storage and throughputneeds increase over time, new storage devices can be made availabledynamically. The underlying Ceph architectureused by Rook.io can handle exobytes of data while allowing datato be resilient to failures at the node, rack, or even datacenter level.

High Performance Data Processing Pipeline

The event processing pipeline in SiteWhere uses Apache Kafkato provide a resilient, high-performance mechanism for progressively processing deviceevent data. Microservices can plug in to key points in the event processing pipeline,reading data from well-known inbound topics, processing data, then sending data to well-knownoutbound topics. External entites that are interested in data at any point in the pipelinecan act as consumers of the SiteWhere topics to use the data as it moves through the system.

Fully Asynchronous Pipeline Processing

The SiteWhere event processing pipeline leverages Kafka's messaging constructs to allowdevice event data to be processed asynchronously. If a microservice shuts down and no otherreplicas are available to process the load. The data will be queued until a replica startsand begins processing again. This acts as a guarantee against data loss as data is alwaysbacked by Kafka's high-performance storage. SiteWhere microservices leverage Kafka's consumergroups concept to distribute load across multiple consumers and scale processing accordingly.

Using Kafka also has other advantages that are leveraged by SiteWhere. Since all data inthe distributed log is stored on disk, it is possible to "replay" the event stream basedon previously gathered data. This is extremely valuable for aspects such as debuggingprocessing logic or load testing the system.

API Connectivity Between Microservices

While device event data generally flows in a pipeline from microservice to microservice onKafka topics, there are also API operations that need to occur in real time between themicroservices. For instance, device management and event management functions are contained intheir own microservices, but are required by many other components of the system. Many of theSiteWhere microservices offer APIs which may be accessed by other microservices tosupport aspects such as storing persistent data or initiating microservice-specificservices.

Using gRPC for a Performance Boost

Rather than solely using REST services based on HTTP 1.x, which tend to have significantconnection overhead, SiteWhere uses gRPC to establish a long-livedconnection between microservices that need to communicate with each other. Since gRPC usespersistent HTTP2 connections, the overhead for interactions is greatly reduced, allowingfor decoupling without a significant performance penalty. Istio also allows the gRPCconnections to be multiplexed across multiple replicas of a microservice to scaleprocessing and offer redundancy.

The entire SiteWhere data model has been captured inGoogle Protocol Buffers format so thatit can be used within GRPC services. All of the SiteWhere APIs are exposed directly asgRPC services as well, allowing for high-performance, low-latency access to all APIfunctions. The REST APIs are still made available via the Web/REST microservice (actingas an API gateway), but they use the gRPC APIs underneath to provide a consistent approachto accessing data.

Multitenancy

SiteWhere is designed for large-scale IoT projects which may involve many system tenantssharing a single SiteWhere instance. A key differentiator for SiteWhere compared to mostIoT platforms is that each tenant runs in isolation from other tenants. By default, tenantsdo not share database resources or pipeline processing and have a completely separateconfiguration lifecycles. With this approach, each tenant may use its own databasetechnologies, external integrations, and other configuration options. Parts of the tenant'sprocessing pipeline may be reconfigured/restarted without causing an interruption toother tenants.

Data Privacy

An important consequence of the way SiteWhere handles multitenancy is that each tenant'sdata is separated from the data of other tenants. Most platforms that offer multitenancystore data for all tenants in shared tables, differentiated only by a tenant id. The sharedapproach opens up the possibility of one tenant's data corrupting another, which is notan acceptable risk in many IoT deployments. In addition, each tenant has its own processingpipelines, so in-flight data is never co-mingled either.

Having dedicated resources for tenants can be expensive in terms of memory and processingresources, so SiteWhere also offers the concept of customers within each tenant. Customersallow data to be differentiated within a tenant, but without having a separate dedicateddatabase and pipelines. In cases where colocated data is acceptable, the tenant can haveany number of customers, which shared the same database and processing pipeline. This allowsthe best of both worlds in terms of security and scalability.

Copyright (c) 2009-2019 SiteWhere LLC. All rights reserved.