HPC Policies

Research Access and Accounts

The YCRC operates computing and data resources supporting research across disciplines. Upon request, the YCRC will provide a principal investigator (PI) group account, or a project group account for systems hosting high risk data,  on the YCRC’s resources.  This provisioning will be available to those who meet the PI criteria as defined in 1310 Principal Investigator Eligibility Requirements on Sponsored Projects | It’s Your Yale .  Those who have an exception to the documented eligibility requirements in that policy must provide a copy of their approved Principal Investigator (“PI”) Status Request Form . 1310 FR.04 Principal Investigator (“PI”) Status Request Form | It’s Your Yale . In some instances, a letter from their Dean confirming PI status and duration of that status can substitute for the formal PI status form.

Once a group account has been created, and with the approval of the PI, members of a PI’s research group may request individual user accounts under the group account. The PI is ultimately responsible for all use of YCRC resources by members of the PI’s group, including any costs for data storage and computing that may be associated with such use. 

When PI status ends for an individual, any current Yale students associated with that PI will need to be reassigned to another active PI per the criteria above. The original PI will have 30 days to remove their data from YCRC resources. YCRC will then close the original PI’s accounts and delete any remaining data associated with that individual’s home directory, project directory, and any “rented” PI data storage space. 

Account Deactivation and Removal

The YCRC audits cluster accounts annually in the month of November.  All accounts not associated with a valid NetID will expire at that time. In addition, YCRC needs to be able to contact all account holders for security and communication purposes. Therefore, logins will be disabled throughout the year for accounts that are found not to be associated with a valid email address.  After three attempts to reach the individual associated with a locked account and the associated PI, the account and associated user data will be removed. 

In addition to the automatic expiry of invalid NetIDs, each PI is responsible for and required to validate the status of all users associated with the PI’s research group at least once a year (more frequently for systems hosting high risk data).  If a PI does not respond to this validation request within 30 days, accounts for the PI and group members will be locked.

PIs are also responsible for notifying the YCRC when a user has left the PI’s research group or the University, at which time the user’s account will expire. When accounts expire, PIs are responsible for ensuring all files are properly managed (moved to another active user’s account) or removed within 30 days of user account expiry.   The YCRC reserves the right to delete expired accounts and all files associated with them if the PI does not take action within 30 days of the user account expiry.  Users should arrange to transfer file ownership before leaving the group(s) with which they have been working. It is the PIs responsibility to ensure that members of their group understand this obligation.

Undergraduate Student Access

The YCRC focuses primarily on computational research conducted by faculty, research staff, postdocs, and graduate students. However, the YCRC may provide limited access to undergraduate students when they are part of a PI’s research group (and thus engaged in the PIs active computational research)  or enrolled in a course that uses YCRC resources. In appropriate circumstances, undergraduate students may request use of YCRC resources for independent research under the oversight of a faculty sponsor/advisor. Such requests will be considered on a case-by-case basis, and approval will be at the discretion of the YCRC in consultation with the Chair of the YCRC Steering Committee.

Academic Courses on YCRC Clusters

Instructors of Yale academic courses may request the use of YCRC facilities for their students. Instructors interested in leveraging YCRC resources in their courses are expected to consult with the YCRC and obtain prior approval at least 60 days before the start of the semester.  YCRC will evaluate requests, with fulfillment largely dependent upon the availability of sufficient staff and system resources.   Instructors and their teaching staff are  expected to provide primary support for their students using YCRC facilities, with secondary assistance from YCRC staff during regular Yale business hours as appropriate and as available. The YCRC will do its best to ensure access to its facilities throughout the semester, but scheduled hardware maintenance periods will take place according to the YCRC’s published schedule as listed on the System Status Page on the YCRC website, and instructors should plan accordingly. Instructors should also understand that cluster resources may be unavailable due to unforeseen circumstances.   All accounts and data created for course use will be disabled and removed 30 days after the end of the semester. 

Policies regarding account creation and access to YCRC  resources are subject to change. 

The YCRC’s systems group administers all YCRC hardware, including systems (compute clusters and servers), storage platforms, and related data center networking. Since these systems are intended to support research applications and environments, they are typically designed and operated to achieve maximum performance levels and job throughput. The system team’s primary responsibilities are to maximize system performance, availability, stability, and security. All systems are subject to regular maintenance periods as listed on the System Status Page on the YCRC website, and there is also annual multiday data center maintenance each summer.  During these maintenance periods and, rarely, at other times, some or all of the YCRC compute and data resources may not be available for use.

The YCRC will provide each PI group access to compute and storage resources on one YCRC cluster, subject to reasonable limits and YCRC discretion. In some circumstances, the YCRC may provide a PI group or some of its members with access to additional resources as appropriate for their data risk classification and computational requirements.  (Such additional access may be temporary.) Users are expected to do their best to use the clusters efficiently and release idle resources. 

All users will have access to a baseline amount of storage in home, project, and short-term scratch directories. Quotas limit the amount of storage and the number of files per user and/or PI group. Users are expected to delete files they no longer need. Files in short-term scratch directories are purged automatically after 60 days. Using scratch for long-term file storage (through artificial extension of expiration or other means) is forbidden without explicit approval from the YCRC. PIs are responsible for all storage usage by members of their groups. Additional storage may be provided upon request, subject to YCRC discretion and resource availability,  typically incurring monthly costs.

All computation on the “standard” tier of partitions (e.g. day, week, mpi, gpu) as well as private nodes and scavenge partitions do not incur any charges. Researchers can escalate specific important computations to Priority Tier and run less urgent workloads in the Standard Tier.

The Service Unit rate structure for the Priority Tier is derived to closely match the prorated cost of a similar dedicated node over a 5 year expected lifetime. As such, Priority Tier is an alternative to purchasing dedicated nodes. For the same cost, researchers can realize 100% of the value of their funds (compared to inevitable idle time on dedicated nodes) and always be computing on the YCRC’s latest resources (vs dedicated nodes which get more out of date every year and are decommissioned after 5 years).

Access detailed information on Priority Tier, including how to request access and rates.

Learn how to use the Priority Tier partitions. 

User Account Security

User accounts are personal to each user and may not be shared. Under no circumstances may any individual use another person’s account. Users are expected to follow standard security practices to ensure the safety and security of their accounts and data.

Regulated and Sensitive Data

Users are prohibited from using or storing high risk, sensitive, or regulated data (e.g., NIH controlled access data, HIPAA data, PHI, or PII) at any time on any YCRC facilities other than those explicitly designated for such data.

Data Use Agreements

Regardless of the sensitivity and risk classification of the data, users are prohibited from using or storing, on any YCRC compute or storage system, data covered by a data use agreement (DUA) unless the DUA has been approved by the Office of Sponsored Projects and the University Compliance office, and has been reviewed in advance by YCRC’s Leadership team.  DUAs often have conditions that YCRC cannot meet, thus review is essential prior to data being placed on a YCRC system.   Thus, a copy of the DUA must be provided to the YCRC and, as early as possible in the process; before storing any covered data on any YCRC facility, the YCRC must confirm that it can meet all applicable computing and storage related requirements of the DUA, including, but not limited to, requirements for data encryption, access control, auditing, and special actions to be taken upon removal of the data. Users who expect to start a project that involves the use of YCRC facilities for sensitive data or data subject to a DUA, or who are applying for external funding for such a project, should consult with YCRC early in the planning process. PIs of such projects are responsible for informing YCRC which users are authorized to access which covered data and for notifying YCRC when there are changes to the list of authorized users. PIs are responsible for ensuring that the permissions associated with their data are in compliance with the list of approved users associated with a DUA and/or IRB. PIs are also responsible for ensuring that, at the end of a project or termination of a DUA,  data disposition is consistent with the requirements of the DUA.  

YCRC Security Procedures

The security of YCRC systems and all data stored on them is critical, and the YCRC takes several steps to help ensure a secure computing environment, including: 

• Operating the clusters in secure data centers, with restricted and logged access;
• Using firewalls to allow access only from the Yale campus network or the Yale VPN, and restricting that access to only login and data transfer servers;
• Requiring ssh key pairs instead of passwords  for ssh authentication;
• Requiring SSO authentication with Yale credentials where appropriate (e.g Open OnDemand, VDI access, Globus);
• Requiring Multi-factor Authentication for both VPN and system authentication;
• Keeping operating systems up to date;
• Applying security patches in a timely manner, commensurate with the severity of vulnerabilities that are addressed by patching

In addition, users are responsible for the security of YCRC systems and their data. Accordingly, users are expected to follow standard security practices to ensure the safety and security of their accounts and data. This includes:

• Using strong passphrases on ssh keys;
• Setting permissions appropriately on data files and directories;
• Never sharing private keys, passphrases, or other login information;
• Following the terms of any regulations or Data Use Agreements that cover their data.
• Maintaining awareness of and following Yale’s Minimum Security requirements

Elevated privileges, such as sudo or root access,  on all systems operated by the YCRC are strictly limited to an approved subset of YCRC staff to ensure the security and stability of the systems.

Further information regarding Yale cybersecurity and data classifications can be found at http://cybersecurity.yale.edu. 

Except as described on the YCRC website for specific clusters, only files in users’ home directories are backed up, and then only for a short time (approximately 30 days on most clusters). No other user files are backed up.  Backups are stored in a managed Yale data center, but we strongly recommend that  users also maintain their own copies of critical files at other locations. 

The YCRC’s resources are shared by many users. The YCRC uses a workload management system (Slurm) to implement and enforce policies to provide each PI group with fair, but limited, access to the YCRC  clusters.  Users may not run computationally intensive workloads or compilations on the login or transfer nodes. Instead, users must submit such workloads as jobs to Slurm, specifying the amount of resources to be allocated for the jobs. Jobs running for longer than one week are discouraged. Slurm will terminate jobs exceeding their requested resource amounts with little or no warning. To avoid data loss when jobs terminate unexpectedly, users are strongly encouraged to checkpoint running jobs at regular intervals.

Users are expected to abide by the stated purposes and limits of the cluster partitions and submit jobs in alignment with YCRC best practices, e.g., not running large numbers of very short jobs or workflows that create an excessive number of small files. Jobs making inappropriate use of a cluster may be canceled without prior notice, and repeated offenses after a warning has been issued by YCRC staff can result in account suspension. In extreme cases, where a particular workflow threatens the system’s stability, the YCRC may temporarily lock an account without prior notice, with account restoration requiring consultation with YCRC staff to address the workflow.

The YCRC includes research support staff members who can help users with a variety of needs, including education and training, software installation, and workflow assistance. The YCRC offers a number of classes and workshops on topics relevant to research computing. New users are strongly encouraged to attend one of the introductory training workshops (or watch a recorded training) to learn about the computing and storage resources and to become familiar with the YCRC’s standard operating procedures. 

YCRC staff procure, install, and maintain many standard software tools and applications intended for use on YCRC-managed platforms.  Among these are compilers and languages (e.g., Python, C, C++, Fortran), parallel computing tools (e.g., MPI), application systems (e.g., R, Matlab, Mathematica), and libraries (e.g., Intel Math Kernel Library, NAG, GNU Scientific Library, FFTW). Users requiring additional software on YCRC facilities are encouraged to install their own copies, though the YCRC’s research support staff may be able to assist. For customizable tools such as Python and R, the YCRC has established procedures to enable users to easily install their own modules, libraries, or packages.

YCRC uses the Freshdesk ticketing system to help manage and address user inquiries, requests, and troubleshooting issues related to our services and systems. Users can contact YCRC staff by emailing research.computing@yale.edu. While the time required to resolve particular issues may vary widely, users may expect an initial communication from a YCRC staff member within a reasonable time (typically within one business day). Users may also obtain assistance by arranging appointments to meet with the research support staff or attending  YCRC’s Office Hours.

The McCleary high-performance computing system has specific resources that are dedicated to YCGA users. This includes a slurm partition (‘ycga’) and a large parallel storage system (/gpfs/ycga). The following policy guidelines govern the use of these resources on McCleary for data storage and analysis.

Yale University Faculty User

1. All Yale PIs using YCGA for library preparation and/or sequencing will have an additional 5 TB storage area called ‘work’ for data storage. This is in addition to the 5 TB storage area called ‘project’ that all McCleary groups receive.
2. Currently, neither work or project storage is backed up. Users are responsible for protecting their own data.
3. All Fastq files are available on the /gpfs/ycga storage system for one year. After that, the files are available in an archive that allows self-service retrieval, as described below. Issues or questions about archived data can be addressed to ycga@yale.edu.
4. Users processing sequence data on McCleary should be careful to submit their jobs to the ‘ycga’ partition. Jobs submitted to other partitions may incur additional charges.
5. Members of Yale PI labs using YCGA for library preparation and/or sequencing may apply for accounts on McCleary with PI’s approval.
6. Each Yale PI lab will have a dedicated ‘work’ directory to store their data, and permission to lab members will be granted with the authorization of the respective PI. Furthermore, such approval will be terminated upon request from the PI or termination of Yale Net ID.
7. Lab members moving to a new university will get access to HPC resources for an additional six months only upon permission from Yale PI. If Yale NetID is no longer accessible, former Yale members who were YCGA users should request a Sponsored Identity NetID from their business office. Sponsored Identity NetIDs will be valid for six months. Such users will also need to request VPN access.
8. A PI moving to a new university to establish their lab will have access to their data for one year from the termination of their Yale position. During this time, the PI or one lab member from the new lab will be provided access to the HPC system. Request for Guest NetID should be made to their business office. Guest NetID will be valid for one year.
9. Any new Yale faculty member will be given access to McCleary once they start using YCGA services.

External Collaborators

1. Access to McCleary can be granted to collaborating labs, with the authorization of the respective Yale PI. A maximum of one account per collaborating lab will be granted. Furthermore, such approval will be terminated upon request from the PI. This requires obtaining a Sponsored Netid. The expectation is that the collaborator, with PI consent, will download data from the McCleary HPC system to their own internal system for data analysis.

Non-Yale Users

Users not affiliated with Yale University will not be provided access to the McCleary high-performance computing system.

YCGA Data Retention Policy

YCGA-produced sequence data is initially written to YCGA’s main storage system, which is located in the main HPC datacenter at Yale’s West Campus. Data stored there is protected against loss by software RAID. Raw basecall data (e.g. bcl files) is immediately transformed into DNA sequences (fastq files).

• ~45 days after sequencing, the raw files are deleted.
• ~60 days after sequencing, the fastq files are written to an archive. This archive exists in two geographically distinct copies for safety.
• ~365 days after sequencing, all data is deleted from main storage. Users continue to have access to the data via the archive. Data is retained on the archive indefinitely. See below for instructions for retrieving archived data.

All compression of sequence data is lossless. Gzip is used for data stored on the main storage, and quip is used for data stored on the archive. Disaster recovery is provided by the archive copy.

More information on accessing the sequence data.