Difference: PetalArchive (1 vs. 11)

Revision 1106 Feb 2019 - StuartBorthwick1

Line: 1 to 1
 
META TOPICPARENT name="ResearchData"
<--===== START PAGE =====================================-->
Line: 41 to 41
  Costs
Changed:
<
<
The cost is approximately £100 per Terabyte or approximately £254 per tape (2.5TB). There is a slight difference in price between charging for use of PETAL on grants that already exist, compared to listing the calculated indexed costs on a grant awaiting future approval. The costs associated with the SRF will be re-assessed each year by the FO and adjusted. The minimum purchase that a user can make is one tape. For further information on the charges, contact the Faculty of Environment Finance Office.
>
>
The cost is approximately £122 per Terabyte or approximately £306 per tape (2.5TB). There is a slight difference in price between charging for use of PETAL on grants that already exist, compared to listing the calculated indexed costs on a grant awaiting future approval. The costs associated with the SRF will be re-assessed each year by the FO and adjusted. The minimum purchase that a user can make is one tape. For further information on the charges, contact the Faculty of Environment Finance Office.
  For further informations please visit the links below:

Revision 1011 Oct 2017 - StuartBorthwick1

Line: 1 to 1
 
META TOPICPARENT name="ResearchData"
<--===== START PAGE =====================================-->
Line: 7 to 7
 
<--===== PAGE TITLE ======================================-->

Revision 907 Sep 2017 - StuartBorthwick1

Line: 1 to 1
 
META TOPICPARENT name="ResearchData"
<--===== START PAGE =====================================-->
Line: 7 to 7
 
<--===== PAGE TITLE ======================================-->

Revision 807 Sep 2017 - StuartBorthwick1

Line: 1 to 1
 
META TOPICPARENT name="ResearchData"
<--===== START PAGE =====================================-->
Line: 7 to 7
 
<--===== PAGE TITLE ======================================-->

Revision 707 Sep 2017 - StuartBorthwick1

Line: 1 to 1
 
META TOPICPARENT name="ResearchData"
<--===== START PAGE =====================================-->
Line: 7 to 7
 
<--===== PAGE TITLE ======================================-->

<--===== PAGE TEXT ======================================-->
Changed:
<
<

Service

The PETAL system provides access to a robotic tape archive for storing petabytes of research data. The service consists of a mixture of disk and tape, providing an active archive. This enables faster retrieval of more recently accessed data as well as the long-term storage of infrequently accessed data. The archive manager software coordinates the movement of data between disk cache and the back-end tape library.

There are 2 service offerings -

  1. 1 tape copy: This is intended for data which can be recreated - e.g. the output from a simulation which could be re-run. Only 1 copy of the data is stored, so there's no possibility of disaster recovery (for example, if a tape fails, or there's a fire or flood in the server room).
  2. 2 tape copies: In this case, a remote copy of the data is kept for disaster recovery - this is for data which can't be easily recreated.

The service appears to the end-user as a filesystem which can be accessed over the network from Linux or Windows clients. The expectation is that users will prepare a final copy of data which is then copied into the PETAL disk cache.

When data appears in the cache, the PETAL system will automatically copy the data to tape, and then potentially delete it from the cache (copy to tape may not happen immediately though).

When a user accesses data in the system, it could be retrieved immediately from the cache, or there could be a delay while it's retrieved from tape.

How to access the service

The PETAL cache is accessible from University Linux clients via kerberized NFSv4. For the School of Earth and Environment, this is at -

/nfs/petal

>
>
Petabyte Environmental Tape Archive and Library
 
Changed:
<
<
There are 2 sub-directories representing the 2 offerings above - near1, near2. When you request storage on the system, a directory with your research project name will appear in one of those 2 subdirectories. You can access and use it with normal Linux commands. For example -

cp -rp ~/data /nfs/petal/near2/ear6stb

Users outside of the School of Earth and Environment will require the PETAL cache to be setup on a machine which they have access to. Typically, an entry would have to be added to local automount maps like this -

petal -fstype=nfs4,sec=krb5,hard,intr petal-fs.leeds.ac.uk:/archive

Costs

The PETAL cache is accessible from University Windows clients at -

\\petal-fs.leeds.ac.uk\archive

>
>

Service

 
Changed:
<
<
There are 2 sub-folders representing the 2 offerings above - near1, near2. When you request storage on the system, a folder with your research project name will appear in one of those 2 sub-folders. You can access and use it with normal Windows commands.
>
>
We offer two specific storage services, depending on the number of copies of the data that are required – either one or two copies of the data. Each group’s can choose the number of copies that they require when filling the storage allocation form, based on their needs and the nature of the data.
 
Changed:
<
<

%T4%

>
>
– Single copy on tape
  • Appropriate for non-critical data
  • Data that can be recreated readily
  • No Backup of this data will be provided
– Two copies on tape
  • Appropriate for data that would be difficult to recreate (time consuming)
  • Appropriate for data that is impossible to reproduce (unique value)
  • Every tape will have a second tape copy synchronized twice a day
Each storage allocation will have a directory created on one of these areas where to store the data.
 
Changed:
<
<
The PETAL system is primarily a WORM (Write Once, Read Many) system. Once data has been committed to tape, it can't be deleted.
>
>
How to access the Service
 
Changed:
<
<

%T5%

>
>
To gain access to the service, you must complete a storage allocation form for each project and agree to the Service Level Agreement and the End User Agreement. This form names the Principal Investigator (PI) who is financially and legally responsible for the data stored and any users who will be given either “read-only” or “read/write” access to the storage area. The System Administrator (SA) will then send you and any other user a welcome email and prepare the Storage Allocation page which can be accessed once logged in on the website using your usual University credential. Once the storage allocation has been approved, an initial purchase of tapes will be required to provide the necessary storage space to deposit data. Storage space must be purchased by submitting a purchasing tapes form. You will need to be logged in the website in order to access the form. The request will be then authorised via the SA who will interact with the Finance Office (FO) and process the transaction. Once authorised, your available storage capacity will be increased and the allocation ready to start copying files to it. You can monitor usage and other information under the Storage Allocations link in the main menu of the website by selecting the name of your folder.
 
Changed:
<
<
There's a one-off cost of £100 for storing one copy of 1TB of data. The cost for storing 2 copies of 1TB of data is £200. Storage can be purchased in multiples of 2.5 TBs. Data is compressed as it's written to tape - the cost is for the volume of compressed data.
>
>
Customers cannot use tapes they have purchased themselves – all tapes must be purchased via the SRF.
 
Changed:
<
<

%T6%

>
>
Costs
 
Changed:
<
<
The URL below has details of current usage.
>
>
The cost is approximately £100 per Terabyte or approximately £254 per tape (2.5TB). There is a slight difference in price between charging for use of PETAL on grants that already exist, compared to listing the calculated indexed costs on a grant awaiting future approval. The costs associated with the SRF will be re-assessed each year by the FO and adjusted. The minimum purchase that a user can make is one tape. For further information on the charges, contact the Faculty of Environment Finance Office.
 
Changed:
<
<

%T7%

>
>
For further informations please visit the links below:
 
Changed:
<
<
Instructions for the pilot.
>
>
Service Level AgreementHow To Use itFAQContact Us
 
<--===== END CONTENT: ===================================-->
Line: 86 to 59
 }% Service
How to access the service
Changed:
<
<
Costs
%T4%
%T5%
%T6%
%T7%
>
>
Costs
 
<--===== START FOOTER ====================================-->

<--===== WEB LINKS ========================================-->

Useful Links

<--===== START PAGE =====================================-->

<--===== BANNER IMAGE =========================================-->

<--===== CONTENT: =========-->

<--===== PAGE TITLE ============================================-->

<--===== PAGE TEXT ============================================-->

<--===== END CONTENT: =========-->

<--===== END PAGE ============================================-->

<--===== IMAGE ===========================================-->

Storage

<--===== END PAGE =======================================-->

<--===== END FOOTER ====================================-->

Revision 610 Dec 2015 - StuartBorthwick1

Line: 1 to 1
 
META TOPICPARENT name="ResearchData"
<--===== START PAGE =====================================-->
Line: 25 to 25
  The PETAL system provides access to a robotic tape archive for storing petabytes of research data. The service consists of a mixture of disk and tape, providing an active archive. This enables faster retrieval of more recently accessed data as well as the long-term storage of infrequently accessed data. The archive manager software coordinates the movement of data between disk cache and the back-end tape library.
Changed:
<
<
There are 4 service offerings -
  1. Long-term archive, 1 copy: This is intended for data which can be recreated - e.g. the output from a simulation which could be re-run. The associated disk cache is relatively small - it's anticipated that the data will not be accessed frequently. Only 1 copy of the data is stored, so there's no possibility of disaster recovery (for example, if a tape fails, or there's a fire or flood in the server room).
  2. Long-term archive, 2 copies: Again, it's anticipated that the data won't be accessed frequently. But in this case, a remote copy of the data is kept for disaster recovery - this is for data which can't be easily recreated.
  3. Near-line, 1 copy: This is intended for data which can be readily recreated. But in this case the disk cache is relatively large - more frequent access of the data is anticipated. Only 1 copy of the data is stored, so there's no possibility of disaster recovery (for example, if a tape fails, or there's a fire or flood in the server room).
  4. Near-line, 2 copies: Relatively large disk cache, and a remote copy of the data for disaster recovery.
>
>
There are 2 service offerings -
  1. 1 tape copy: This is intended for data which can be recreated - e.g. the output from a simulation which could be re-run. Only 1 copy of the data is stored, so there's no possibility of disaster recovery (for example, if a tape fails, or there's a fire or flood in the server room).
  2. 2 tape copies: In this case, a remote copy of the data is kept for disaster recovery - this is for data which can't be easily recreated.
  The service appears to the end-user as a filesystem which can be accessed over the network from Linux or Windows clients. The expectation is that users will prepare a final copy of data which is then copied into the PETAL disk cache.
Line: 39 to 37
 

How to access the service

Changed:
<
<
The PETAL cache is accessible from CentOS 6 clients in the School of Earth and Environment at -
>
>
The PETAL cache is accessible from University Linux clients via kerberized NFSv4. For the School of Earth and Environment, this is at -
  /nfs/petal
Changed:
<
<
There are 4 sub-directories representing the 4 offerings above - deep1, deep2, near1, near2. When you request storage on the system, a directory with your username or groupname will appear in one of those 4 subdirectories. You can access and use it with normal Linux commands. For example -
>
>
There are 2 sub-directories representing the 2 offerings above - near1, near2. When you request storage on the system, a directory with your research project name will appear in one of those 2 subdirectories. You can access and use it with normal Linux commands. For example -
  cp -rp ~/data /nfs/petal/near2/ear6stb
Added:
>
>
Users outside of the School of Earth and Environment will require the PETAL cache to be setup on a machine which they have access to. Typically, an entry would have to be added to local automount maps like this -

petal -fstype=nfs4,sec=krb5,hard,intr petal-fs.leeds.ac.uk:/archive

 

Costs

Changed:
<
<
The PETAL cache is accessible from Windows clients at -
>
>
The PETAL cache is accessible from University Windows clients at -
  \\petal-fs.leeds.ac.uk\archive
Changed:
<
<
There are 4 sub-folders representing the 4 offerings above - deep1, deep2, near1, near2. When you request storage on the system, a folder with your username or groupname will appear in one of those 4 sub-folders. You can access and use it with normal Windows commands.
>
>
There are 2 sub-folders representing the 2 offerings above - near1, near2. When you request storage on the system, a folder with your research project name will appear in one of those 2 sub-folders. You can access and use it with normal Windows commands.
 

%T4%

Changed:
<
<
The PETAL system is primarily a WORM (Write Once, Read Many) system. Once data has been committed to tape, it can't be deleted. It's possible to remove data from the cache filesystem, but this doesn't remove data from tape (and doesn't reduce your used quota).

If data is deleted from the cache before it's been committed to tape, then the data is completely deleted and there's no possibility of retrieval (and in this case it doesn't count against your quota).

There's a possibility of synchronising cache with tape - data is then removed from tape, but this is a lengthy, manual process which can only be carried out by arrangement with IT support.

>
>
The PETAL system is primarily a WORM (Write Once, Read Many) system. Once data has been committed to tape, it can't be deleted.
 

%T5%

Revision 530 Oct 2015 - StuartBorthwick1

Line: 1 to 1
 
META TOPICPARENT name="ResearchData"
<--===== START PAGE =====================================-->
Line: 15 to 15
 
  • Set T4 = Data deletion
  • Set T5 = Cost
  • Set T6 = Statistics
Added:
>
>
  • Set T7 = Pilot
  -->
Line: 25 to 26
 The PETAL system provides access to a robotic tape archive for storing petabytes of research data. The service consists of a mixture of disk and tape, providing an active archive. This enables faster retrieval of more recently accessed data as well as the long-term storage of infrequently accessed data. The archive manager software coordinates the movement of data between disk cache and the back-end tape library.

There are 4 service offerings -

Changed:
<
<
  1. Long-term archive, 1 copy: This is intended for data which can be recreated - e.g. the output from a simulation which could be re-run. The associated disk cache is relatively small - it's anticipated that the data will not be accessed frequently.
>
>
  1. Long-term archive, 1 copy: This is intended for data which can be recreated - e.g. the output from a simulation which could be re-run. The associated disk cache is relatively small - it's anticipated that the data will not be accessed frequently. Only 1 copy of the data is stored, so there's no possibility of disaster recovery (for example, if a tape fails, or there's a fire or flood in the server room).
 
  1. Long-term archive, 2 copies: Again, it's anticipated that the data won't be accessed frequently. But in this case, a remote copy of the data is kept for disaster recovery - this is for data which can't be easily recreated.
Changed:
<
<
  1. Near-line, 1 copy: This is intended for data which can be readily recreated. But in this case the disk cache is relatively large - more frequent access of the data is anticipated.
>
>
  1. Near-line, 1 copy: This is intended for data which can be readily recreated. But in this case the disk cache is relatively large - more frequent access of the data is anticipated. Only 1 copy of the data is stored, so there's no possibility of disaster recovery (for example, if a tape fails, or there's a fire or flood in the server room).
 
  1. Near-line, 2 copies: Relatively large disk cache, and a remote copy of the data for disaster recovery.

The service appears to the end-user as a filesystem which can be accessed over the network from Linux or Windows clients. The expectation is that users will prepare a final copy of data which is then copied into the PETAL disk cache.

Line: 42 to 43
  /nfs/petal
Changed:
<
<
There are 4 sub-directories representing the 4 offerings above - long1, long2, near1, near2. When you request storage on the system, a directory with your username or groupname will appear in one of those 4 subdirectories. You can access and use it with normal Linux commands. For example -
>
>
There are 4 sub-directories representing the 4 offerings above - deep1, deep2, near1, near2. When you request storage on the system, a directory with your username or groupname will appear in one of those 4 subdirectories. You can access and use it with normal Linux commands. For example -
  cp -rp ~/data /nfs/petal/near2/ear6stb
Line: 52 to 53
  \\petal-fs.leeds.ac.uk\archive
Changed:
<
<
There are 4 sub-folders representing the 4 offerings above - long1, long2, near1, near2. When you request storage on the system, a folder with your username or groupname will appear in one of those 4 sub-folders. You can access and use it with normal Windows commands.
>
>
There are 4 sub-folders representing the 4 offerings above - deep1, deep2, near1, near2. When you request storage on the system, a folder with your username or groupname will appear in one of those 4 sub-folders. You can access and use it with normal Windows commands.
 

%T4%

Line: 70 to 71
  The URL below has details of current usage.
Added:
>
>

%T7%

Instructions for the pilot.

 
<--===== END CONTENT: ===================================-->
Line: 85 to 90
 Costs
%T4%
%T5%
Changed:
<
<
%T6%
>
>
%T6%
%T7%
 
<--===== START FOOTER ====================================-->

<--===== WEB LINKS ========================================-->

Useful Links

<--===== START PAGE =====================================-->

<--===== BANNER IMAGE =========================================-->

<--===== CONTENT: =========-->

<--===== PAGE TITLE ============================================-->

<--===== PAGE TEXT ============================================-->

<--===== END CONTENT: =========-->

<--===== END PAGE ============================================-->

<--===== IMAGE ===========================================-->

Storage

<--===== END PAGE =======================================-->

<--===== END FOOTER ====================================-->

Revision 423 Oct 2015 - StuartBorthwick1

Line: 1 to 1
 
META TOPICPARENT name="ResearchData"
<--===== START PAGE =====================================-->
Line: 27 to 27
 There are 4 service offerings -
  1. Long-term archive, 1 copy: This is intended for data which can be recreated - e.g. the output from a simulation which could be re-run. The associated disk cache is relatively small - it's anticipated that the data will not be accessed frequently.
  2. Long-term archive, 2 copies: Again, it's anticipated that the data won't be accessed frequently. But in this case, a remote copy of the data is kept for disaster recovery - this is for data which can't be easily recreated.
Changed:
<
<
  1. Accessible, 1 copy: This is intended for data which can be readily recreated. But in this case the disk cache is relatively large - more frequent access of the data is anticipated.
  2. Accessible, 2 copies: Relatively large disk cache, and a remote copy of the data for disaster recovery.
>
>
  1. Near-line, 1 copy: This is intended for data which can be readily recreated. But in this case the disk cache is relatively large - more frequent access of the data is anticipated.
  2. Near-line, 2 copies: Relatively large disk cache, and a remote copy of the data for disaster recovery.
  The service appears to the end-user as a filesystem which can be accessed over the network from Linux or Windows clients. The expectation is that users will prepare a final copy of data which is then copied into the PETAL disk cache.
Line: 42 to 42
  /nfs/petal
Changed:
<
<
There are 4 sub-directories representing the 4 offerings above - long-one, long-two, short-one, short-two. When you request storage on the system, a directory with your username or groupname will appear in one of those 4 subdirectories. You can access and use it with normal Linux commands. For example -
>
>
There are 4 sub-directories representing the 4 offerings above - long1, long2, near1, near2. When you request storage on the system, a directory with your username or groupname will appear in one of those 4 subdirectories. You can access and use it with normal Linux commands. For example -
 
Changed:
<
<
cp -rp ~/data /nfs/petal/short-two/ear6stb
>
>
cp -rp ~/data /nfs/petal/near2/ear6stb
 

Costs

Line: 52 to 52
  \\petal-fs.leeds.ac.uk\archive
Changed:
<
<
There are 4 sub-folders representing the 4 offerings above - long-one, long-two, short-one, short-two. When you request storage on the system, a folder with your username or groupname will appear in one of those 4 sub-folders. You can access and use it with normal Windows commands.
>
>
There are 4 sub-folders representing the 4 offerings above - long1, long2, near1, near2. When you request storage on the system, a folder with your username or groupname will appear in one of those 4 sub-folders. You can access and use it with normal Windows commands.
 

%T4%

Revision 320 Oct 2015 - StuartBorthwick1

Line: 1 to 1
 
META TOPICPARENT name="ResearchData"
<--===== START PAGE =====================================-->
Line: 8 to 8
 

<--===== PAGE TEXT ======================================-->
Changed:
<
<

Introduction

>
>

Service

 
Changed:
<
<
The PETAL system provides access to a robotic tape archive for storing petabytes of research data. The service consists of a mixture of disk and tape, providing an active archive. This enables faster retrieval of more recently accessed data as well as the long-term archival storage of infrequently accessed data. The archive manager software coordinates the movement of data between disk cache and the back-end tape library.
>
>
The PETAL system provides access to a robotic tape archive for storing petabytes of research data. The service consists of a mixture of disk and tape, providing an active archive. This enables faster retrieval of more recently accessed data as well as the long-term storage of infrequently accessed data. The archive manager software coordinates the movement of data between disk cache and the back-end tape library.
  There are 4 service offerings -
Changed:
<
<
  1. Long-term archive, 1 copy: This is intended for data which can be recreated - e.g. the output from a simulation which could be re-run. The disk cache is relatively small - it's anticipated that the data will not be accessed frequently.
  2. Long-term archive, 2 copies: Again, it's anticipated that the data won't be accessed frequently. But in this case, a remote copy of the data is kept for disaster recovery - this is for data which can't be easily recreated.
  3. Accessible, 1 copy: This is intended for data which can be readily recreated. But in this case the disk cache is relatively large - more frequent access of the data is anticipated.
  4. Accessible, 2 copies: Relatively large disk cache, and a remote copy of the data for disaster recovery.
>
>
  1. Long-term archive, 1 copy: This is intended for data which can be recreated - e.g. the output from a simulation which could be re-run. The associated disk cache is relatively small - it's anticipated that the data will not be accessed frequently.
  2. Long-term archive, 2 copies: Again, it's anticipated that the data won't be accessed frequently. But in this case, a remote copy of the data is kept for disaster recovery - this is for data which can't be easily recreated.
  3. Accessible, 1 copy: This is intended for data which can be readily recreated. But in this case the disk cache is relatively large - more frequent access of the data is anticipated.
  4. Accessible, 2 copies: Relatively large disk cache, and a remote copy of the data for disaster recovery.
 
Changed:
<
<
The service appears to the end-user as a filesystem which can be accessed via NFS or CIFS from Linux or Windows clients. The expectation is that users will prepare a final copy of data which is then copied into the PETAL disk cache.
>
>
The service appears to the end-user as a filesystem which can be accessed over the network from Linux or Windows clients. The expectation is that users will prepare a final copy of data which is then copied into the PETAL disk cache.
 
Changed:
<
<
When data appears in the cache, the PETAL system will automatically copy the data to tape, and then potentially delete it from the cache. (Copy to tape potentially doesn't happen immediately though.)
>
>
When data appears in the cache, the PETAL system will automatically copy the data to tape, and then potentially delete it from the cache (copy to tape may not happen immediately though).
 
Changed:
<
<
When a user access data in the system, it could be retrieved immediately from the cache, or there could be a delay while it's retrieved from tape.
>
>
When a user accesses data in the system, it could be retrieved immediately from the cache, or there could be a delay while it's retrieved from tape.
 
Changed:
<
<

NFS (Linux client access)

>
>

How to access the service

 
Changed:
<
<
The PETAL cache is accessible from CentOS 6 clients in the School of Earth and Environment at -
>
>
The PETAL cache is accessible from CentOS 6 clients in the School of Earth and Environment at -
  /nfs/petal
Line: 38 to 46
  cp -rp ~/data /nfs/petal/short-two/ear6stb
Changed:
<
<

CIFS (Windows client access)

>
>

Costs

  The PETAL cache is accessible from Windows clients at -
Line: 46 to 54
  There are 4 sub-folders representing the 4 offerings above - long-one, long-two, short-one, short-two. When you request storage on the system, a folder with your username or groupname will appear in one of those 4 sub-folders. You can access and use it with normal Windows commands.
Changed:
<
<

Data deletion

>
>

%T4%

 
Changed:
<
<
The PETAL system is primarily a WORM (Write Once, Read Many) system. Once data has been committed to tape, it can't be deleted. It's possible to remove data from the cache filesystem, but this doesn't remove data from tape (and doesn't reduce your used quota).
>
>
The PETAL system is primarily a WORM (Write Once, Read Many) system. Once data has been committed to tape, it can't be deleted. It's possible to remove data from the cache filesystem, but this doesn't remove data from tape (and doesn't reduce your used quota).
  If data is deleted from the cache before it's been committed to tape, then the data is completely deleted and there's no possibility of retrieval (and in this case it doesn't count against your quota).

There's a possibility of synchronising cache with tape - data is then removed from tape, but this is a lengthy, manual process which can only be carried out by arrangement with IT support.

Changed:
<
<

Cost

>
>

%T5%

 
Changed:
<
<
There's a one-off cost of £100 for storing one copy of 1TB of data. The cost for storing 2 copies of 1TB of data is £200. Storage can be purchased in multiples of 2.5 TBs. Data is compressed as its written to tape - the cost is for the volume of compressed data.
>
>
There's a one-off cost of £100 for storing one copy of 1TB of data. The cost for storing 2 copies of 1TB of data is £200. Storage can be purchased in multiples of 2.5 TBs. Data is compressed as it's written to tape - the cost is for the volume of compressed data.
 
Changed:
<
<

Statistics

>
>

%T6%

  The URL below has details of current usage.

<--===== END CONTENT: ===================================-->
Added:
>
>

<--===== WEB LINKS ========================================-->

 
<--===== START FOOTER ====================================-->

<--===== WEB LINKS ========================================-->

Useful Links

<--===== START PAGE =====================================-->

<--===== BANNER IMAGE =========================================-->

<--===== CONTENT: =========-->

<--===== PAGE TITLE ============================================-->

<--===== PAGE TEXT ============================================-->

<--===== END CONTENT: =========-->

<--===== END PAGE ============================================-->

<--===== IMAGE ===========================================-->

Storage

<--===== END PAGE =======================================-->

<--===== END FOOTER ====================================-->

<--===== END PAGE =======================================-->
\ No newline at end of file
Added:
>
>
META TOPICMOVED by="ear6stb" date="1445330919" from="IT.SpectraLogic" to="IT.PetalArchive"

Revision 219 Oct 2015 - StuartBorthwick1

Line: 1 to 1
 
META TOPICPARENT name="ResearchData"
<--===== START PAGE =====================================-->
Line: 17 to 17
 The PETAL system provides access to a robotic tape archive for storing petabytes of research data. The service consists of a mixture of disk and tape, providing an active archive. This enables faster retrieval of more recently accessed data as well as the long-term archival storage of infrequently accessed data. The archive manager software coordinates the movement of data between disk cache and the back-end tape library.

There are 4 service offerings -

Changed:
<
<
  1. Long-term archive, 1 copy: This is intended for data which can be recreated - e.g. the output from a simulation which could be re-run. The disk cache is realtively small - it's anticipated that the data will not be accessed frequently.
>
>
  1. Long-term archive, 1 copy: This is intended for data which can be recreated - e.g. the output from a simulation which could be re-run. The disk cache is relatively small - it's anticipated that the data will not be accessed frequently.
 
  1. Long-term archive, 2 copies: Again, it's anticipated that the data won't be accessed frequently. But in this case, a remote copy of the data is kept for disaster recovery - this is for data which can't be easily recreated.
  2. Accessible, 1 copy: This is intended for data which can be readily recreated. But in this case the disk cache is relatively large - more frequent access of the data is anticipated.
  3. Accessible, 2 copies: Relatively large disk cache, and a remote copy of the data for disaster recovery.
Line: 42 to 42
  The PETAL cache is accessible from Windows clients at -
Changed:
<
<
\\petal-fs-01\archive
>
>
\\petal-fs.leeds.ac.uk\archive
  There are 4 sub-folders representing the 4 offerings above - long-one, long-two, short-one, short-two. When you request storage on the system, a folder with your username or groupname will appear in one of those 4 sub-folders. You can access and use it with normal Windows commands.

Revision 119 Oct 2015 - StuartBorthwick1

Line: 1 to 1
Added:
>
>
META TOPICPARENT name="ResearchData"
<--===== START PAGE =====================================-->

<--===== START HEADER ====================================-->

<--===== START PAGE =====================================-->

<--===== BANNER IMAGE ===================================-->

<--===== CONTENT: =======================================-->

<--===== END HEADER ====================================-->

<--===== PAGE TITLE ======================================-->

<-- 
  • Set THISPAGETITLE = PETAL - Petabyte Environmental Tape Archive and Library
-->

<--===== PAGE TEXT ======================================-->

Introduction

The PETAL system provides access to a robotic tape archive for storing petabytes of research data. The service consists of a mixture of disk and tape, providing an active archive. This enables faster retrieval of more recently accessed data as well as the long-term archival storage of infrequently accessed data. The archive manager software coordinates the movement of data between disk cache and the back-end tape library.

There are 4 service offerings -

  1. Long-term archive, 1 copy: This is intended for data which can be recreated - e.g. the output from a simulation which could be re-run. The disk cache is realtively small - it's anticipated that the data will not be accessed frequently.
  2. Long-term archive, 2 copies: Again, it's anticipated that the data won't be accessed frequently. But in this case, a remote copy of the data is kept for disaster recovery - this is for data which can't be easily recreated.
  3. Accessible, 1 copy: This is intended for data which can be readily recreated. But in this case the disk cache is relatively large - more frequent access of the data is anticipated.
  4. Accessible, 2 copies: Relatively large disk cache, and a remote copy of the data for disaster recovery.

The service appears to the end-user as a filesystem which can be accessed via NFS or CIFS from Linux or Windows clients. The expectation is that users will prepare a final copy of data which is then copied into the PETAL disk cache.

When data appears in the cache, the PETAL system will automatically copy the data to tape, and then potentially delete it from the cache. (Copy to tape potentially doesn't happen immediately though.)

When a user access data in the system, it could be retrieved immediately from the cache, or there could be a delay while it's retrieved from tape.

NFS (Linux client access)

The PETAL cache is accessible from CentOS 6 clients in the School of Earth and Environment at -

/nfs/petal

There are 4 sub-directories representing the 4 offerings above - long-one, long-two, short-one, short-two. When you request storage on the system, a directory with your username or groupname will appear in one of those 4 subdirectories. You can access and use it with normal Linux commands. For example -

cp -rp ~/data /nfs/petal/short-two/ear6stb

CIFS (Windows client access)

The PETAL cache is accessible from Windows clients at -

\\petal-fs-01\archive

There are 4 sub-folders representing the 4 offerings above - long-one, long-two, short-one, short-two. When you request storage on the system, a folder with your username or groupname will appear in one of those 4 sub-folders. You can access and use it with normal Windows commands.

Data deletion

The PETAL system is primarily a WORM (Write Once, Read Many) system. Once data has been committed to tape, it can't be deleted. It's possible to remove data from the cache filesystem, but this doesn't remove data from tape (and doesn't reduce your used quota).

If data is deleted from the cache before it's been committed to tape, then the data is completely deleted and there's no possibility of retrieval (and in this case it doesn't count against your quota).

There's a possibility of synchronising cache with tape - data is then removed from tape, but this is a lengthy, manual process which can only be carried out by arrangement with IT support.

Cost

There's a one-off cost of £100 for storing one copy of 1TB of data. The cost for storing 2 copies of 1TB of data is £200. Storage can be purchased in multiples of 2.5 TBs. Data is compressed as its written to tape - the cost is for the volume of compressed data.

Statistics

The URL below has details of current usage.

<--===== END CONTENT: ===================================-->

<--===== START FOOTER ====================================-->

<--===== WEB LINKS ========================================-->

Useful Links

<--===== START PAGE =====================================-->

<--===== BANNER IMAGE =========================================-->

<--===== CONTENT: =========-->

<--===== PAGE TITLE ============================================-->

<--===== PAGE TEXT ============================================-->

<--===== END CONTENT: =========-->

<--===== END PAGE ============================================-->

<--===== IMAGE ===========================================-->

Storage

<--===== END PAGE =======================================-->

<--===== END FOOTER ====================================-->

<--===== END PAGE =======================================-->
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2014 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.