Data Intensive Computing
Middleware for Data Intensive Computing (MeDICi)
Principal Investigator: Ian Gorton
Challenge
As the volume, rate and complexity of data increase, scientists and analysts must
- Capture and integrate high-throughput sensor data
- Fuse and analyze in real-time
- Manage diverse data formats and locations
- Integrate heterogeneous hardware
Approach
- Allow scientists and analysts to visually create and modify a pipeline for processing data
- Build on proven standard integration, workflow, and provenance technologies
- Support multiple languages, communication protocols, hardware platforms
- Pass large data by reference
Accomplishments
- Performance: copy latency between components is as low as 0.1 ms
- Graphical Component Builder
- Publicly available on website: http://medici.pnl.gov
MeDICI Architecture Overview