Hopper HPC Cluster

Hopper is configured and operated to meet the highest standards of privacy, regulatory, compliance, and security controls. It is specifically designed to handle high-risk data.

Compliance Standards

Hopper is compliant with and/or aligned with stringent information security standards:

• HIPAA: Hopper is a HIPAA-aligned HPC platform. It is designed for the secure computation of sensitive data, including electronic Protected Health Information (ePHI). It meets the rigorous standards required for research involving data protected under HIPAA.

• NIST 800-171: Hopper is compliant with the NIST 800-171 standard. This compliance is necessary for certain government contracts and some NIH controlled-access data sets.

Supported Data Types

Hopper is intended for research involving sensitive or regulated data:

• Electronic Protected Health Information (ePHI).

• NIH Controlled-Access Data (e.g., genomic data from dbGaP).

• Controlled Unclassified Information (CUI).

Use with YNHHS Data

Hopper has been reviewed and approved by IT leadership at Yale and Yale New Haven Health System (YNHHS). YNHHS has approved Hopper for research use with YNHHS data, as it meets HIPAA requirements. Hopper serves as an exclusive technology environment to securely deliver YNHHS health research datasets, provided there is an approved protocol.

 

Hopper is a shared-use High-Performance Computing (HPC) platform available to all researchers at Yale University. It is one of the new secure computing platforms launched to increase Yale’s highly secure research computing infrastructure. Hopper is designed specifically to support large-scale and intensive computational tasks.

Hopper is a fully-managed, secure computing service that is co-managed by Health Sciences IT (HSIT) and the Yale Center for Research Computing (YCRC). The YCRC also manages compute and storage environments for Hopper. The cluster is named after computer scientist and U.S. Navy Rear Admiral Grace Murray Hopper, who earned her Ph.D. in Mathematics from Yale in 1934.

——————————————————————————–

Compliance and Data Security

Hopper is configured and operated to meet the highest standards of privacy, regulatory, compliance, and security controls. It is specifically designed to handle high-risk data.

Compliance Standards

Hopper is compliant with and/or aligned with stringent information security standards:

• HIPAA: Hopper is a HIPAA-aligned HPC platform. It is designed for the secure computation of sensitive data, including electronic Protected Health Information (ePHI). It meets the rigorous standards required for research involving data protected under HIPAA.

• NIST 800-171: Hopper is compliant with the NIST 800-171 standard. This compliance is necessary for certain government contracts and some NIH controlled-access data sets.

Supported Data Types

Hopper is intended for research involving sensitive or regulated data:

• Electronic Protected Health Information (ePHI).

• NIH Controlled-Access Data (e.g., genomic data from dbGaP).

• Controlled Unclassified Information (CUI).

Use with YNHHS Data

Hopper has been reviewed and approved by IT leadership at Yale and Yale New Haven Health System (YNHHS). YNHHS has approved Hopper for research use with YNHHS data, as it meets HIPAA requirements. Hopper serves as an exclusive technology environment to securely deliver YNHHS health research datasets, provided there is an approved protocol.

——————————————————————————–

Technical Specifications and Hardware

Hopper provides both standard computing resources and GPU-enabled nodes. It enables advanced simulations and large-scale data analysis, including the secure use of large language models (LLMs).

Hardware Resources

The platform includes substantial computational and storage resources:

• Initial Configuration: 60 compute nodes and more than 200 GPUs.

• Total Resources (as of Aug 1, 2025): 6,000 CPUs and 212 GPUs.

• GPU Breakdown: GPUs are distributed across multiple nodes, including:

    ◦ 15 NVIDIA H100 nodes (60 GPUs)

    ◦ 8 H200 nodes (32 GPUs)

    ◦ 10 A40 nodes (40 GPUs)

    ◦ 10 L40S nodes (40 GPUs)

    ◦ 10 A5000 nodes (40 GPUs)

• Storage: 2.5 PB of storage.

Storage System

Hopper uses the weston filesystem, a VAST filesystem featuring encryption-at-rest for all user data.

• Users can check current storage usage and limits using the getquota command.

• Shortcuts exist in the home directory to project storage spaces, following the form ~/work_<project> and ~/scratch_<project>.

• Only users in a specific project can access that project’s storage spaces.

• Scratch Storage Warning: Files stored in scratch are purged if they are older than 60 days.

Available Partitions

Jobs run using Slurm. Hopper provides several common use partitions:

Partition	Primary Use	Request Defaults	Available Nodes (Examples)
day	Default for most batch jobs.	--time=01:00:00, --nodes=1, --ntasks=1, --cpus-per-task=1, --mem-per-cpu=5120	52 nodes with 64 CPUs/Node and 976 GiB Memory/Node (cpugen:emeraldrapids)
devel	For ongoing interaction, exploratory analyses, or debugging.	Same as ‘day’	2 nodes with 64 CPUs/Node and 976 GiB Memory/Node (cpugen:emeraldrapids)
gpu	For jobs requiring GPUs (must be requested explicitly with --gpus).	Same as ‘day’	Nodes feature A5000, L40s, A40, and H100 GPUs (4 GPUs per node, 48 CPUs/Node)
bigmem	For jobs with memory requirements exceeding other partitions.	Same as ‘day’	Nodes offering 1953 GiB or 3906 GiB Memory/Node

——————————————————————————–

Access and Usage Policies

Access to Hopper is project-based and subject to rigorous security requirements.

Access Requirements

The platform has specific requirements for access and training:

1. Project-Based Access: Access is granted on a per project basis (<pi_netid>_<project_code>). A single Principal Investigator (PI) may manage multiple projects.

2. Project Approval: A request must be submitted by the PI and approved before user accounts are created. Access requires a Chart of Accounts (CoA) for use of compute and storage resources.

3. Account Approval: All accounts must be associated with an active project and approved by the project’s PI.

4. Required Training: Before access is granted, users must successfully complete a mandatory training program, including NIST 800-171 and HIPAA training. This training must be renewed annually.

5. PI Oversight: PIs are required to conduct a quarterly review of user accounts on their projects; failure to do so results in project deactivation.

Logging In

• VPN Required: Connections can only be made from the Yale VPN (access.yale.edu), even when on campus. VPN connections must be younger than 24 hours.

• Interface: The cluster is accessed through the Virtual Desktop Infrastructure (VDI), which acts as the ‘login node’ to isolate the user from their host computer.

• VDI Access: Navigate to hopper1.ycrc.yale.edu in a web browser to access the VDI. The VDI provides the Open OnDemand Web Portal (coming soon!) and command line terminal access.

Security Restrictions (NIST 800-171 and HIPAA Compliance)

To maintain compliance, Hopper enforces several restrictions:

• The VDI prevents copy/pasting to the host computer.

• Screenshots, screen recording, and screen sharing (e.g., via Zoom) are strictly prohibited.

• If away for more than 10 minutes, users must disconnect from the VDI (by closing the browser tab).

• Access must be from a private location (home or office); access from public locations (coffee shops, transportation hubs) is not allowed.

• Sensitive data or personal identifiers must not be included in directory names or job names.

Data Transfer Procedures

Data may only be transferred using approved methods, and all internet access is blocked except for whitelisted remote sites.

• Low-Risk Data: Use Globus via the “Yale CRC Hopper Low Risk” collection into a user-specific staging directory. Transfers require YCRC staff approval and completion of a request. Large uploads (>200G) require a request prior to uploading.

• All Other Data (Sensitive/High-Risk): Transfers (incoming or outgoing) of non-low-risk data require explicit approval. Outgoing high-risk data transfers require contacting research.computing@yale.edu.

Software and LLMs

All software must be approved and installed by YCRC staff, typically as software modules. Users cannot install software themselves.

• User Scripts: Analysis scripts (Python, R, MATLAB, or bash) do not qualify as software and are permissible to upload and run without approval.

• R and Python Packages: A monitored proxy is set up to PyPI and CRAN, allowing users to install their own Python and R packages using standard methods (pip, install.packages) on the hopper1 login node.

• Large Language Models (LLMs): LLM models (e.g., Llama) qualify as software and must be installed by YCRC staff. Commonly requested LLM models are available as software modules for offline use.

Job Submission

Jobs are run using Slurm.

• All jobs must specify a project account using the -A flag.

• For projects requiring isolation, the -X flag must be used to ensure an exclusive allocation.

——————————————————————————–

Rate Structure and Availability

• Availability: Hopper was available for production use as of July 1, 2025, and fully operational since August 1, 2025.

• Current Usage: Early access usage on Hopper is at no-cost.

• Future Costs: Computations and storage on Hopper will be subject to charges starting January 1st, 2026.

    ◦ Compute Hour (non-GPU) rate: $0.004 per hour.

    ◦ GPU Hour rates range from 1.490/hr (H200).

    ◦ Additional work-style storage beyond the no-cost allocation is $5 per TiB per month.

• Billing: Usage is billed for actual runtime. Compute and storage charges are billed monthly. A Hopper Cost Calculator is provided to assist with cost estimates.

——————————————————————————–

Getting Started

To inquire about using Hopper, researchers can email research.computing@yale.edu.

You can find more information about accessing these new systems by visiting the YCRC’s Hopper user guide.