Why
The New Mexico Water Data Initiative (NMWDI) has developed multiple core tools:
- AMP API
- Weaver
- Data Integration Engine (DIE)
Each of these tools serves a specific purpose and is designed to work together to provide a comprehensive set of data services to users.
Why AMP API
- Provides a single point of access to all data in NMAquifer
Why Weaver
-
Modern component-based web application using React
- Easy to maintain and extend
- Easy to deploy
-
Accesses data from multiple sources and displays them in a single interface
Why Data Integration Engine
- Open Source, Community Driven
- Technology stack we have experience with
- PIP installable package for easy deployment for users that want to use the engine directly
- Contains all the logic for data integration, data transformation, and data harmonization
- Can be used as a standalone package or as a library in a larger application
- Can be used to integrate data from multiple sources
Weaver vs DIE
Although Weaver and DIE are separate projects, they have overlapping functionality. This presents a challenge for our development team to decide where to draw the line between the two projects. Here is a brief comparison of the two:
DIE:
- service that can be used to integrate data from multiple sources
- written in Python. Can be used as a standalone package or as a library in a larger application
- contains all the logic for data access, transformation, and harmonization
Weaver:
- Web application that uses a variety of services to integrate data from multiple sources and display it in a single interface
- Written in React. Uses vite for compilation and bundling. Uses PrimeReact for UI components and styling
Complication:
Currently, a user can view data in Weaver but not download it and vice versa for DIE. We need to decide how to integrate the two projects so that a user can view and download data from the same interface!!!