DiDaS

Distributed Data Storage project aims to build an infrastructure of data storage in main nodes of highspeed network Cesnet2.

The main goals of project: deploy an infrastructure of data storage, build up IBP infrastructure over storage servers, develop new applications that can use this infrastructure

Current state of art:

Together, 10 data stores are reckoned with, the total capacity being about 15TB. The stores are placed in six czech cities - Praha - CUNI, Praha - Cesnet, České Budějovice - JCU, Plzeň - VSB, Plzeň, Liberec - VSLIB, Brno - ICS, Brno - ICS, Brno - ICS, and Ostrava - VSB. They are connected with a 100Mbps and 1Gbps network link, and also connected to the high-speed Czech-bone network Cesnet2, which consists of 1 to 2.5Gbps links and features a more-way topology.

These data stores are mainly connected and made available through the Internet Backplane Protocol (IBP). Here can be found L-Bone server that collects particular IBP depots of infrastructure.

We are using low-end PC-class server on the basis of Pentium 4 processor with 1GB of RAM mainly using IDE hard disks.

There are four kinds of data stores:

• eight to ten IDE Serial ATA hard disks in internal hardware RAID 5 array with total capacity of 1.75TB - 2TB
• eight IDE Parallel ATA hard disks in internal hardware RAID 5 array with total capacity of 1.75TB
• eight SCSI hard disks in internal hardware RAID 5 array with total capacity of 511GB
• eight IDE Parallel ATA hard disks in external hardware RAID 5 array with total capacity of 1.75TB

For IBP, new clients are being developed, which will allow access to data and also will be able to exploit the IBP's potential. Currently we have modified transcode to be able read and store files from/to IBP infrastructure. We have also patches to mplayer to be able play content directly from IBP while allowing seeking in content. Both application use our own IBP access library libxio. There is also an HTTP form of access by means of CGI scripts.

Further work:

• creating a filesystem upon IBP
• plug-ins for web browsers
• optionally Java applets to move download directly to clients
• large zoomable picture viewer like Mr.SID

Other points of our interest are experiments with replica management, as well as with metadata and load distribution optimization.

Participating organizations:
Institute of Computer Science MUNI, Brno
Prague supercomputer center CUNI, Prague
Westbohemian supercomputer center ZCU, Plzen
Supercomputing Center VSB, Ostrava
National Library, Prague
Metacomputing Center, Cesnet

Results

Technical reports:
Hejtmánek, Lukáš - Holub, Petr. IBP Deployment Tests and Integration with DiDaS Project Technical report : CESNET z.s.p.o., 2003. 22 p. 20/2003.

Presentations:
Matyska, Luděk. APAN 2003, Hejtmánek, Lukáš - Holub, Petr - Matyska, Luděk. DiDaS/LoCI Infrastructure for Distributed Video Processing, I2 Fall 2004 Meeting, Austin, TX, USA. Hejtmánek, Lukáš - Holub, Petr - Matyska, Luděk. DiDaS - final review