1. System Design

1.1. Component Overview

1.1.1. System architecture

The FogFlow framework operates on a geo-distributed, hierarchical, and heterogeneous ICT infrastructure that includes cloud nodes, edge nodes, and IoT devices. The following figure illustrates the system architecture of FogFlow and its major components across three logical layers

_images/architecture.png

1.1.2. Three layers

Logically, FogFlow consists of the following three layers:

  • service management: convert service requirements into concrete execution plan and then deploy the generated execution plan over cloud and edges
  • context management: manage all context information and make them discoverable and accessible via flexible query and subscribe interfaces
  • data processing: launch data processing tasks and establish data flows between tasks via the pub/sub interfaces provided by the context management layer

1.1.3. System components

centralized service components to be deployed in the cloud

  • Task Designer: provide the web-based interface for service providers to specify, register, and manage their tasks and service topologies;
  • Topology Master: figure out when and which task should be instantiated, dynamically configure them, and also decide where to deploy them over cloud and edges;
  • IoT Discovery: manage all registered context availability information, including its ID, entity type, attribute list, and metadata; allow other components to query and subscribe their interested context availability information via NGSI9

distributed components to be deployed both in the cloud and at edges

  • Worker: according to the assignment from the topology master, each worker will launch its scheduled task instances in docker containers on its local host; configure their inputs and outputs and manage all task instances locally based on task priority
  • IoT Broker: each broker manages a part of context entities published by nearby IoT devices and also provides a single view of all context entities for IoT devices to query and subscribe the entities they need

external service components to be used in FogFlow

  • Dock Registry: managed all docker images provided by developers;
  • RabbitMQ: The internal communication between topology master and the workers
  • PostgreSQL: the backend database to save the registered context availability information