DIVA: Empowering Secure and Compliant Data Sharing in the Digital Age
Exploring DIVA: A Deep Dive into Secure and Privacy-Aware Data Usage
The DIVA project, developed by Fraunhofer ISST, is designed to address modern challenges in data usage and privacy. This open-source initiative focuses on creating a secure environment for data usage, particularly within the context of increasing regulatory demands and privacy concerns.
What is DIVA?
DIVA, which stands for Data Infrastructure for Visibility and Access, is a framework that facilitates secure data sharing and processing. It is tailored to ensure that data providers retain control over their data while allowing authorized users to access and utilize it securely. This balance is crucial in sectors where data sensitivity and compliance are paramount.
Key Features
- Data Sovereignty: DIVA ensures that data providers maintain control over their data, deciding who can access it and under what conditions. This feature is essential for maintaining trust and compliance with privacy regulations.
- Secure Data Processing: The framework includes robust security measures to protect data during processing. This is achieved through advanced encryption and access control mechanisms, ensuring that data remains confidential and integrity is preserved.
- Compliance: DIVA is designed to help organizations comply with various data protection regulations, such as GDPR. It provides the necessary tools to manage consent, data access, and audit trails, simplifying compliance efforts.
- Interoperability: The framework supports interoperability between different data systems and platforms. This ensures that data can be shared and utilized across various environments without compatibility issues.
- 🏛️ microservice architecture: allows to choose the best technology for solving a problem and a more easy scaling
- 💻 client application: an easy to use web application for managing all kinds of data management related topics
- 🖥️ portal application: simple search for interesting files on different devices (WIP)
- 🐳 docker ready: all microservices and core components are docker ready so you can start them right out of the box.
Applications
DIVA's design makes it suitable for a wide range of applications, particularly in sectors that handle sensitive data. Here are a few examples:
- Healthcare: Securely sharing patient data between hospitals, clinics, and research institutions, ensuring patient privacy and compliance with health data regulations.
- Finance: Enabling secure data exchanges between financial institutions, reducing fraud risk and ensuring compliance with financial regulations.
- Smart Cities: Facilitating data sharing between different municipal departments and private partners, improving city services while protecting citizen privacy.
Technical Aspects
DIVA is built using a combination of modern software technologies designed for scalability and security. It leverages cloud-native principles to ensure flexibility and robustness. Key components include:
- Microservices Architecture: DIVA uses a microservices approach, allowing for modular development and deployment. This ensures that the system can be easily extended and maintained.
- Blockchain Integration: To enhance transparency and trust, DIVA incorporates blockchain technology for logging data access and transactions. This provides an immutable record of data usage.
- APIs and SDKs: The framework provides comprehensive APIs and SDKs for developers, enabling easy integration with existing systems and facilitating the development of custom applications.
Core Technologies and Frameworks used
| Technology | Description |
|---|---|
| Kong | our API gateway that we use to route microservices |
| Kafka | message log for microservice communication |
| node.js | nice JavaScript platform for running server apps |
| Express Framework | helps us building simple microservices |
| Docker | building and publishing images |
| Kubernetes | production-grade container orchestration |
| Airflow | author, schedule and monitor workflows |
| OpenAPI | specification language to describe the HTTP APIs of our microservices |
| AsyncAPI | specification language to describe how Kafka and WebSocket messages look |
| JSON Schema | specification language to describe how an entity is build |
| MongoDB | our main document store that is the single source of truth when it comes to metadata |
| Elasticsearch | our search index used to search for entities and make interesting aggregations |
| Keycloak | Open Source Identity and Access Management |
| MinIO | our object store to save files uploaded by browser (aka diva-lake) |
| neo4j | our graph database to store relations between entities more efficient |
Other Technologies and Frameworks used
| Technology | Description |
|---|---|
| VueJS 2 | component based frontend solution for building robust apps |
| Vuetify | makes frontend beautiful |
| Apache Tika | if you need to take a look into heterogenous data, Tika is your solution |
| Python3 | helps us doing data science and NLP (natural language processing) |
| Kibana | our window into elasticsearch for debugging |
| Filebeat | fills elasticsearch with logs produced in our microservices |
Getting Started with DIVA
To start using DIVA, you can visit the GitHub repository. The repository includes detailed documentation, installation guides, and example use cases. Whether you're a developer, data scientist, or IT manager, the resources available will help you understand and implement DIVA in your organization.
Conclusion
DIVA represents a significant advancement in secure and privacy-aware data usage. By ensuring data sovereignty, facilitating compliance, and supporting interoperability, it offers a comprehensive solution to the modern challenges of data sharing and processing. Explore DIVA today to see how it can benefit your organization.
