SUN GPU Cluster::National Center for High-Performance Computing

SUN GPU Cluster

Detail

System Hardware Configuration

1. Sun GPU Cluster Computing Nodes - 44 computing nodes.
Each computing node consists of :

System model : Sun X4170
CPU：Intel X5570 2.93GHz x 2
GPU : Nvidia S1070 1.44Ghz with 4GB GDDR
Memory : 24 GB ECC DDR3-1066
Disk : 300 GB SAS Disk
Network : Intel Corporation 82575EB Gigabit Ethernet
InterConnection : Mellanox MT26428 ConnectX IB 4x QDR

2. Storage Nodes - 2 storage nodes . Each storage node consists of :

System model : Sun X4170
CPU：Intel X5570 2.93GHz x 2
Memory : 24 GB ECC DDR3-1066
Disk card: 8Gbps PCI-E G2 Dual Port Emulex 27001
Network : Intel Corporation 82575EB Gigabit Ethernet
InterConnection : Mellanox MT26428 ConnectX IB 4x QDR
External Disk : IBM DX4700

3. Network

Gigabit Ethernet Switch : 3Com 4200G GE Switch x 3
Infiniband QDR Switch : Voltaire Grid Director 4036 x 3

4. Major Directories

/home: 2 TBytes
/pkg: 2 TBytes
/work: 6 TBytes

5. Job Submission

Interactive: yes
Batch System: SUN Grid Engine 6.2u5
SUN Grid Engine User Guide

Software

1. System Software

OS: Redhat Enterprise Version 5 update 3
Cluster Management: Sun HPC Software 2.02
Developer : Intel Cluster Toolkit 3.2.1 (include Intel Fortran/C/C++ 11.1 and MKL 10.2)
Cuda Version 3.0
MPI : Sun ClusterTooks 8.2.1
Batch system: SUN Grid Engine 6.2u5

2. Application Packages

Most of application packages on system are placed in /pkg directory.
Current packages installed are:

cmake2.8.1
gsl 1.13
namd 2.7b1 (/pkg/chem/namd)
mpich2 (/pkg/chem/lib/npich2, intel, gcc, pgi version)
mvapich2 (/pkg/chem/lib/mvapich2, intel, gcc, pgi version)
fftw215 (/pkg/chem/lib/fftw215, intel, gcc, pgi version)
fftw33 (/pkg/chem/lib/fftw33, intel, gcc,pgi version)
lapack (/pkg/chem/lib/lapack-3.1.1)
scipy (/pkg/chem/lib/scipy-0.6.0)
openMM (/pkg/chem/openmm)

NCHC SUN GPU Cluster User Computing Environment

1. Hardware configuration

2. Queue Policy

Queue Name

Max CPUs

Max CPUs/job

CPU/Run time limit/job

對象

test.q

4

4

1 hour

一般計畫

serial.q

16

1

7 days

parallel.q

64

64

2 days

Queue Name	Max CPUs	Max CPUs/job	CPU/Run time limit/job	對象
test.q	4	4	1 hour	一般計畫
serial.q	16	1	7 days
parallel.q	64	64	2 days

3. Disk Usage Constrained Policy

/tmp : clear files not accessed over 1 day.
/work : clear files not accessed over 2 week.

ATTN: due to disk space limitation, please use /work* instead of /usr/tmp or /tmp
ATTN: NCHC has no obligation to backup files under these working file systems, and reserves the right to change/reconfigure/destroy these working file systems without any backup.

4. Compilers

C/C++ Compiler

Intel C/C++ 11.1 (icc, icpc)
GNU GCC Collection 4.1.2 (gcc, g++)
PGI C/C++ 10.4 (pgcc, pgCC)

Fortran

Intel Fortran 11.1 (ifort)
GNU GCC Collection 4.1.2 (g77)
PGI PGF77 10.4 (pgf77,pgf90)

5. Libraries

SUN ClusterTool 8.2.1 library
Intel MPI library
Intel MKL library
Cuda 3.0 library

6. Queuing System

     SUN Grid Engine 6.2u5

7. File System

     Network File System version 4

NCHC Consulting Window

     (03)5776085 # 442     account@nchc.narl.org.tw