Data Archiving on Jabba

About our data archiving system Jabba

All data you put on our data archiving system will we stored on tape drives. Like this the archive system can hold a large amount of data. The downside is that read and write operations are slow since the tape roboter has to physically grab the right tapes to handle your data. Thus, you should not use Jabba for data you use daily. But if you have a huge amount of simulation data that you might need in some weeks, Jabba would be a good place to take care of them.

Jabba Usage

We will have to give the proper rights to your account before you will be able to use Jabba. Please write an email to support@ee.ethz.ch if you need access to our data archiving system.

As soon as access is granted, you have the following options to get/put data from/to the archiving system:

• NFS (Linux only): /usr/jabba/archive/INSTITUTE/USERNAME or /usr/jabba/backup/INSTITUTE/USERNAME

• Samba: \\jabba\USERNAME

• OpenSSH: scp

Important note

Since data you put on Jabba are stored on tape drives, please follow these rules:

• Only write big archive files (.tar.gz, .zip, ...) to jabba, not many small files.

• The ideal size of these archive files is between 5 and 10 Gigabytes.
• Don't create archive files bigger than 10 Gigabytes.

If you don't obey these rules, backing up and restoring your data will take very, very long.

To backup/archive your files via NFS, do the following:

tar cfv - <Directory> | /usr/isgtc/bin/tubasplit --size 5056 /usr/jabba/backup/tik/hans/file.tar

To archive your files utilizing ssh:
Note: If you have a local jabba-account you should preferably use the NFS-mount. Using ssh creates excessive load on 'jabba'!

tar cfv - [Directory] | ssh jabba "/usr/isgtc/bin/tubasplit --size 5056  /usr/jabba/backup/tik/hans/file.tar"

$ cat  </usr/archive/biwi/<username>/tar-a> </usr/archive/biwi/<usernam>/tar-b>  </usr/archive/biwi/<username>/tar-c> | tar xvf -

* archive : Every file you put into the archive directory (partition) will be stored on two tapes, 5 minutes after the last modification.

* backup : Files write into the backup directory (partition) will only be saved to one tape and 30 minutes after their last modification on harddisk.

In both cases the files on harddisk will be deleted (after storage on tape) only if the threshold of free disk space on the partition is reached. The idea of this two methods of storage is, that you can define how much security you need for your data by choosing the name of the directory. There are several methods to transport the files to the jabba. You can use ftp, ssh (secure shell) or scp (secure copy) or your system manager can mount the directories on jabba using NFS on your own machine. If you want retrieve your files, its only necessary that you call it in the appropriate directory. If the inode leaves on the harddisk (even when the file is stored on tape and deleted on harddisk), the filename leave in the directory and will recopied automatically from tape to the harddisk when you call it.

User commands

-sls an extended version of GNU ls for listing directory information

-sdu an extended version of GNU du

-archive set archive attributes and archive files

-release release disk space and set release attributes

-stage set staging attributes and copy off-line files to harddisk

-ssum set file checksum attributes

Note: You must be logged in on Jabba to use these commands. Certain options are only available to the super user. Please consult the man pages on Jabba for more detailed information.