Data Sharing & Preservation - How to access & FAQ

 

As explained in https://www.ia2.inaf.it/index.php/ia2-services/data-sharing-preservation, IA2 provides storage for “Data Sharing” and “Long Preservation”. The first is essentially linked to Owncloud, while the second refers to saving data on disks and Tape Library.

To harmonize the two parts, a dedicated VOSpace interface is being developed for file ingesting and data retrieval, especially in the "Long Preservation" section.

  

How to access.

Data Sharing:

Each INAF employee and associate can access Owncloud with his IDEM username / password (the same one he uses for mail, card, eduroam, ...). By default, each user has 10 GB of space available, which can be increased upon request.

 

Long Preservation: 

Temporarily, access, ingestion and retrieval of data to the "Long Preservation" section of the IA2 storage takes place through a dedicated personal account by ssh. The procedure indicated below will be updated as soon as the interface is ready.

  • How to request an account:
    • The account is personal and not open to a group. If it is necessary to save data of a group this must be done through an account linked to a contact person.
    • The account must be requested directly from Questo indirizzo email è protetto dagli spambots. È necessario abilitare JavaScript per vederlo..
    • The request must indicate the size of the data to be saved, and the frequency with which you plan to access this data.

  • How to ingest data:
    • Once the account has been created, the user will be allocated a scratch area where he/she can transfer his/her data;
    • The transfer can be done via scp, rsync, gridftp. Any additional software should be agreed directly with IA2 people;
    • Once the transfer of a block of data (directory with sub-directory) is completed by the user, this will be taken over (after communication by the user) and frozen;
    • Note: if in the directories there are too many files (exceeding 2000 files per directory) this can affect both on the efficiency of the file system and data transfer. It is therefore suggested to proceed with a creation of a tar before to transfer it;
    • Once  data has been save on the tape, they can manage by the user in "read-only" mode;
  • Data Retrieval:
    • Retrieval of a single file from the tape is possible. Retrieval of a directory is inefficient from the command line and must be requested to the system administrator.
    • The requested data will be placed in the user's directory with the original path
  • Note:
    • The user does not have direct access to write on the Tape Library

FAQ:

  • Too crowded directory (over 2000 small files on a single directory):
    •  if in the directories there are too many files (exceeding 2000 files per directory) this can affect both on the efficiency of the file system and data transfer. It is therefore suggested to proceed with a creation of a tar before to transfer it;
    • to check it you can use this bash script

 

  • Is there a fee to pay?
    • yes, if the amount of data to import is over 250 TB on Tape