north.io's Innovative Platform Architecture

Table of Contents

Programming Languages

The Microservice Paradigm

Scalability

Maintainability

Fault Tolerance

Faster Development

Technology Agnosticism

Kubernetes at the Heart

Ensuring Security

Cloud Agnosticism

The ever-increasing volume and complexity of ocean and terrestrial geospatial data pose a significant challenge for organisations in terms of data management, collaboration, and analysis. Our geospatial data platforms, TrueOcean and TrueEarth, bring a multitude of new paradigms into this domain by addressing these challenges with a comprehensive Software-as-a-Service (SaaS) solution for ocean and terrestrial geospatial data management. At the heart of north.io’s cloud platforms effectiveness lies a robust, highly efficient, and scalable architecture. Let’s explore why our approaches are state-of-the-art and why it is essential and unlock the true potential of the vast amount of ocean and terrestrial geospatial data. 

Image: On the right side, a monitor showing the Geospatial Data Platform, TrueEarth, on the left side a monitor showing the Ocean Data Platform, TrueOcean.

Programming Languages 

The robust functionality of north.io’s cloud platforms is underpinned by an architectural approach that allows each service to be built using the most effective language for its specific role. Drawing on the deep expertise of our software, big data, and AI engineers, we can select the best programming language for the task at hand. Below is an explanation of the programming languages we have chosen and why: 
“Golang”, known for its lightning-fast performance and built-in concurrency features, forms the backbone of many of north.io's backend services. The aforementioned properties of the language make it an ideal choice for handling tasks such as high-volume data ingestion and API communication. Golang's ability to efficiently manage multiple requests simultaneously ensures smooth operation even during peak data influxes. For services demanding absolute peak performance, north.io’s platforms strategically inject “Rust”.

Rust shines in raw speed and memory management, making it a perfect choice for specific high-performance tasks within the broader Golang backend. Think of Rust as a special forces unit within the Golang army, deployed for critical missions requiring the ultimate processing power. Meanwhile, “Python” takes the stage for north.io's AI-powered functionalities. Python boasts a vast ecosystem of data science libraries, providing our AI specialists with a powerful toolkit for implementing complex machine learning algorithms that unlock deeper insights from geospatial data.

Our big data environment is powered by “Java” which reigns supreme in Big Data analysis, especially for our “Apache Spark” based Zeus technology. This dominance stems from its JVM compatibility with Spark and a rich ecosystem of big data libraries. While other languages might boast raw speed, Java offers the perfect mixture of performance, scalability, and a mature development environment, making it ideal for Big Data workloads within the architecture of north.io’s platforms. Finally, the user interface relies on “JavaScript”, a language specifically designed for interactivity. JavaScript's ability to manipulate front-end elements on the fly ensures a responsive and user-friendly experience for those interacting with the visualisation and data exploration tools within north.io’s platforms. 

This strategic selection of languages empowers north.io to deliver powerful, adaptable, and high-performance platforms, TrueOcean and TrueEarth, for managing even the most complex ocean and terrestrial geospatial data sets. Unlike monolithic architectures, north.io’s platforms can leverage the strengths of each language. Golang's speed tackles demanding backend tasks, while Python's data science libraries unlock hidden patterns within the data. JavaScript ensures a user-friendly experience for visualising these insights. And for those crucial moments requiring peak performance, Rust, the programming world's special forces unit, swoops in to deliver exceptional processing power. This flexibility and strategic use of languages make TrueOcean and TrueEarth outstanding platforms, perfectly equipped to handle the ever-growing intricacies of ocean and terrestrial geospatial data. 

The Microservice Paradigm of north.io’s cloud platforms

Traditional monolithic architectures, where all functionalities reside within a single codebase, struggle to handle the scalability and agility demanded by platforms like TrueOcean and TrueEarth. As geospatial data volumes grow exponentially, monolithic systems experience performance bottlenecks, become cumbersome to maintain and are reaching its limits very fast. 

A microservice architecture offers a more granular approach to scalability. It decomposes the platforms' functionalities into independent, self-contained services. Each service is responsible for a specific task, such as data ingestion, storage, analysis, visualisation, or access control. This modularity allows for horizontal scaling, a technique where individual services can be scaled independently based on their specific needs. Consider the data ingestion services in north.io. When a surge of geospatial data arrives, only the data ingestion services experiences increased load.

By horizontally adding more instances of these data ingestion services, north.io’s platforms can efficiently handle the influx without impacting other functionalities. This targeted scaling optimises resource utilisation and ensures cost-effectiveness. 

Figure 1: Monolithic vs. Microservice Architecture

These approaches are decomposing TrueOcean and TrueEarth's functionalities into independent, self-contained services. Each service is responsible for a specific task, such as data ingestion, storage, analysis, visualisation, or access control. Until today, north.io’s platforms consist already of more than 65 individual microservices conducting highly specific tasks in a large scale but still modular environment. This modularity brings several key advantages.

Scalability

Microservices enable leveraging cloud-based infrastructure. Cloud providers offer elastic resources that can be provisioned and deprovisioned on-demand. This dynamic scaling capability perfectly complements the microservice architecture. When a specific service experiences a spike in demand, additional resources can be allocated automatically, ensuring seamless performance. Conversely, during low-demand periods, resources can be scaled down, optimising cost efficiency.

The scalability benefits of a microservice architecture are not limited to horizontal scaling. Vertical scaling, where resources like CPU or memory are increased for individual services, is also readily achievable. This allows north.io’s platforms to fine-tune resource allocation for each service based on its specific processing requirements. The microservice architecture empowers north.io’s platforms to achieve unmatched scalability until today, not seen in the field of ocean data processing. By enabling horizontal and vertical scaling of individual services, north.io’s platforms can effortlessly adapt to fluctuating data volumes and processing demands. This ensures that TrueOcean and TrueEarth can handle even the most demanding ocean and terrestrial geospatial datasets with optimal performance and cost-efficiency. 
Traditionally, scaling a system involved a vertical approach, often referred to as "scaling up." This method focuses on upgrading the resources of a single server, such as increasing its CPU cores, memory capacity, or storage space. While vertical scaling offers a seemingly simple initial setup, it encounters limitations as north.io’s platforms tackle ever-growing volumes of complex ocean and terrestrial geospatial data. Upgrading a single server can become an expensive endeavor in the long run. As processing demands surge, reaching a point where adding more power to a single machine becomes impractical or cost-prohibitive. Additionally, vertical scaling creates a single point of failure. If the sole server malfunctions, the entire platform goes down, potentially causing significant data loss and service disruptions. This is where horizontal scaling, also known as "scaling out," shines. This approach involves adding more machines or instances to distribute the workload across them. By distributing tasks across multiple machines, TrueOcean and TrueEarth can achieve several advantages.

Increased data volumes and user traffic become manageable without compromising responsiveness. Horizontal scaling is also more cost-effective in the long run since lower-powered machines can be more economical than continuously upgrading a single, powerful server. Scaling out allows for individual service updates or maintenance on specific machines without impacting the entire platform. Therefore, for a platform like TrueOcean and TrueEarth that anticipate continuous growth and evolving data management needs, the flexibility and resilience offered by horizontal scaling make it the more compelling choice. 

Figure 2: Vertical vs. Horizontal Scaling

Maintainability

Beyond scalability, microservices transform the platforms’ maintainability from a challenge into a strength. Each microservice has its own codebase, independent of other services. This allows developers to focus on specific functionalities without worrying about the intricacies of the entire platform. A change in one service's codebase has minimal impact on others, reducing the risk of regressions and simplifying debugging. Isolated codebases also enable faster development cycles. With independent services, development teams can work in parallel, implementing new features or bug fixes without affecting the development of others. This significantly accelerates development cycles and time-to-market for new features in north.io’s platforms. 

Fault Tolerance

Imagine a scenario where a critical bug hits a core functionality within a monolithic platform, bringing the entire system to a halt. This downtime translates to lost productivity and frustrated users. A microservice architecture offers a critical advantage in such situations – fault tolerance. Since each microservice operates independently, a failure within one service is isolated from the others. This means that if a bug cripples the data analysis service, for instance, users can still access and visualise existing data through the visualisation service. This minimises downtime and ensures a significant portion of the platforms’ functionality remains operational. Furthermore, fault tolerance mechanisms can be implemented within individual microservices.

We apply techniques like circuit breakers which automatically detect failing services and temporarily suspend communication with them. This prevents cascading failures and allows the failing service to recover independently without impacting the entire platform. Another applied approach are health checks implemented to monitor the status of each service and trigger alerts or automated recovery procedures in case of issues. By design, a microservice architecture builds in redundancy. If a single instance of a service fails, additional instances can handle the workload, ensuring service continuity. This redundancy ensures that TrueOcean and TrueEarth remain highly available, even in the face of unexpected failures. 

Faster Development

In a monolithic system, changes to even minor functionalities necessitate modifications to the entire codebase. This tangled web of code is a significant bottleneck in the development process. A change in one section might have unintended consequences elsewhere, requiring extensive testing and debugging. This slows down development cycles and hinders the ability to keep pace with evolving user needs and technological advancements. The microservice structure of TrueOcean and TrueEarth breaks down these barriers. With each service having its own isolated codebase, our highly specialised development work independently on specific functionalities. This enables parallel development, where new features or bug fixes can be implemented in one service without affecting the development of others. This significantly accelerates development cycles and allows TrueOcean and TrueEarth to deliver new features and functionalities to users much faster. Furthermore, microservices foster a culture of ownership and expertise within development teams. Developers can specialise in the intricacies of specific functionalities, leading to deeper understanding and higher-quality code. This not only streamlines development but also simplifies the onboarding process for new team members, as they only need to focus on the codebase of their assigned service. 

Technology Agnosticism

north.io’s platforms interact with a diverse range of ocean and terrestrial data form different domains like hydrography, geography, geotechnics and many more. Handling and analysing massive amounts of raw data requires different storage solutions than processed structured data collected from sensor networks. Additionally, specialised engines might be required for specific analysis tasks.

In a monolithic architecture, selecting the optimal technology for each data type or analysis can be a challenge. The entire platform might be tied to a specific database or programming language, limiting flexibility and hindering performance. Microservices break free from these constraints. By enabling technology agnosticism, each service within TrueOcean and TrueEarth can leverage the most suitable technology for its specific function. The data ingestion service can utilise for example, a specific database approach for efficient storage of raw ocean data, while the analysis service might employ a specialised geospatial database for faster querying and analysis. Developers have the freedom to choose the best programming language and frameworks for each service, ensuring optimal performance and efficient resource utilisation. This technology agnosticism future-proofs north.io’s platforms. As new technologies and advancements emerge in the geospatial data landscape, TrueOcean and TrueEarth can seamlessly integrate them. Individual services can be updated to leverage these advancements without impacting the entire platforms. This ensures that north.io’s platforms remain at the forefront of geospatial data management and analytics, constantly evolving to accommodate the ever-changing technological landscape. 

By leveraging a microservice architecture, TrueOcean and TrueEarth empower organisations to navigate the vast oceans and mountains of geospatial data with confidence. Scalability, maintainability, and agility are ensured, allowing users to unlock the full potential of their geospatial data and make informed decisions that drive success. 

Kubernetes at the Heart of TrueOcean and TrueEarth 

TrueOcean and TrueEarth's microservice architecture thrives on a powerful orchestrator - Kubernetes. Think of north.io’s platforms as a vast, interconnected ecosystem, similar to a thriving coral reef. Each microservice acts as a specialised colony within the reef, with a distinct role in its overall health and functionality. Data ingestion acts as the filter feeder, constantly pulling in new information (plankton) from the environment. Analysis acts as the living coral itself, transforming this raw data into valuable nutrients (insights) that sustain the entire ecosystem. Visualisation, much like the bioluminescent organisms that radiate these insights, illuminates them for users, drawing them into the reef's hidden depths of knowledge. Crucially, Kubernetes acts as the intelligent flow system for this ecosystem. It ensures the seamless exchange of nutrients and information between these specialised colonies.

Just as currents carry vital nutrients to all parts of the reef, Kubernetes facilitates the flow of data between microservices. This enables efficient collaboration and ensures that each colony receives the information it needs to function optimally. Moving away from the analogy, here's why Kubernetes is a perfect fit for north.io’s world: 

• Firstly, Kubernetes excels at container orchestration. Microservices within TrueOcean and TrueEarth are containerised, packaged units that can be easily deployed and scaled. Kubernetes seamlessly manages the lifecycle of these containers, ensuring they're deployed on the right machines, restarted upon failure, and scaled up or down based on demand. This automation frees north.io's developers from tedious manual deployments and allows them to focus on innovation within the services themselves.
• Secondly, Kubernetes fosters service discovery – for seamless communication. Kubernetes establishes a robust service registry, allowing microservices to easily locate and interact with each other, regardless of their physical location within the containerised environment. This eliminates the need for complex configuration management and ensures smooth information flow across the platforms’ functionalities.
• Thirdly, Kubernetes empowers north.io’s platforms with self-healing capabilities. It monitors the health of microservices and automatically restarts any failing containers. This self-healing functionality ensures the platform's high availability and minimises downtime, keeping ocean and terrestrial geospatial data management operations running smoothly.
• Finally, Kubernetes embraces horizontal scaling – as geospatial data volumes surge, the services need to expand. Kubernetes facilitates effortless horizontal scaling by allowing developers to add or remove container instances based on processing demands. This ensures TrueOcean and TrueEarth can efficiently handle growing data loads without compromising performance. 
  

Figure 3: Kubernetes architecture for TrueOcean and TrueEarth

In essence, Kubernetes acts as the invisible conductor, orchestrating the complex interplay between north.io's microservices. This powerful orchestration empowers north.io’s platforms, TrueOcean and TrueEarth, to deliver a highly scalable, resilient, and ultimately, an exceptional platform for managing even the most intricate ocean and terrestrial geospatial data sets. 

Ensuring Security 

As the volume of geospatial data managed by TrueOcean and TrueEarth continues to grow, so does the importance of ensuring the security and integrity of this data. The platforms’ architecture, which employs a combination of microservices, containerisation via Kubernetes, and a polyglot technology stack, provides a robust foundation for implementing comprehensive security measures.  
North.io’s platforms employ end-to-end encryption to safeguard data at rest and in transit. Data at rest within the platforms’ storage solutions, whether in cloud storage or distributed databases, is encrypted using industry-standard cryptographic algorithms. This ensures that data is unreadable without the appropriate decryption keys, which are managed securely and rotated regularly to enhance security. For data in transit, TrueOcean and TrueEarth utilise Transport Layer Security (TLS) to encrypt data exchanges between microservices, as well as between the platforms and end-users. This protects the data from eavesdropping, tampering, and forgery as it moves within the platforms’ ecosystem or across the internet. 
The microservice architecture of north.io’s platforms facilitate fine-grained access control, allowing precise management of who can access what data and functionalities. Each microservice implements its own authentication and authorisation mechanisms, typically leveraging OAuth 2.0 and JSON Web Tokens (JWT) for secure, token-based access control. This ensures that only authorised users and services can access sensitive data or perform certain actions, according to predefined policies. Furthermore, TrueOcean and TrueEarth incorporate Role-Based Access Control (RBAC) within Kubernetes, ensuring that only authorised personnel and services have the operational permissions necessary to interact with the Kubernetes-managed containers and services. 
Kubernetes plays a vital role in managing secure communication between microservices in north.io’s platforms. With its built-in service discovery capabilities, Kubernetes ensures that microservices can securely locate and communicate with each other over encrypted channels. Network policies in Kubernetes are employed to define how groups of pods can communicate with each other and other network endpoints, minimising the risk of internal data leaks and unauthorised access. 

The operational security strategy within north.io’s platforms encompass regular security audits, vulnerability scanning, and penetration testing to identify and mitigate potential security threats. The platforms’ infrastructure is designed to be resilient against DDoS attacks, with scalability features of Kubernetes and cloud providers' security services being leveraged to absorb and mitigate such threats. Compliance with relevant data protection regulations and standards, such as GDPR and ISO 27001, is integral to the platforms’ security posture. TrueOcean and TrueEarth implement comprehensive data governance and privacy practices to ensure compliance and protect users' privacy rights. 

Cloud Agnosticism and the Future 

This meticulously designed microservice architecture, orchestrated by Kubernetes, grants north.io’s platforms a significant advantage: cloud agnosticism. By separating functionalities into independent services, TrueOcean and TrueEarth transcend the limitations of any single cloud provider. Imagine each microservice as a standardised shipping container – its contents (functionality) are self-contained and easily transportable. These containers can be seamlessly loaded onto any compatible ship (cloud platform) without modifying their contents. This allows TrueOcean and TrueEarth to leverage the best of different cloud providers based on specific needs and company restrictions. Legal requirements or data residency regulations might necessitate using a particular cloud provider in a specific region. The platforms’ architectural decisions allow cloud agnosticism and ensure effortless compliance. This flexibility empowers TrueOcean and TrueEarth to optimise costs and legal compliance while ensuring exceptional performance. 

Figure 4: Cloud agnosticism vs. cloud-based and cloud-native applications

Furthermore, cloud agnosticism paves the way for global deployments. North.io’s platforms instances can be easily deployed across geographically dispersed cloud platforms, bringing their functionalities closer to the source of the data. This reduces latency and improves user experience for organisations working in remote locations. Imagine a marine biologist stationed in the Arctic – with TrueOcean instances running in a nearby cloud region, they can access and analyse critical data with minimal delay. These geographically distributed instances can even be interconnected, allowing seamless collaboration between researchers across the globe for worldwide operating companies. 

The modularity of the architecture behind north.io’s platforms extend beyond traditional cloud deployments. Individual microservices could potentially be deployed on edge computing devices, like those found on research vessels. This would enable real-time data processing and analysis at the source, even in remote ocean locations. Imagine a ship equipped with TrueOcean's data analysis microservice – as it collects data, the microservice can perform initial processing and analysis on board, extracting valuable insights before the ship even reaches port. This edge computing capability empowers researchers to make data-driven decisions in real-time, leading to more efficient and productive expeditions. 

Figure 5: Centralised vs. distributed vs. decentralised cloud architecture

By embracing a cloud-agnostic microservice architecture with Kubernetes at their core, TrueOcean and TrueEarth position themselves not just as powerful platforms for today's needs, but as a foundation for a future of decentralised and collaborative geospatial data management, fostering a truly global world of knowledge. 

If you would like to find out more about the capabilities of our geospatial data platforms, TrueOcean and TrueEarth, please contact us or schedule a demo directly with one of our experts.