Storage Pools, Filesets and Policies

Objectives:

Create storage pools
Create a file system
Create filesets
Implement a Placement Policy
Define and execute a file management policy

Requirements:

Complete Exercise 1: Installing the cluster
List of available devices
/dev/sd__
/dev/sd__
/dev/sd__
/dev/sd__
/dev/sd__

Step 1: Create A File System With 2 Storage Pools

Storage pools are defined when an NSD (Network Shared Disk) is created. You will use the 4 disks (labelled sd_ through sd_ ) provided by your instructor to create 2 storage pools. Since the storage is all direct attached, no NSD servers are needed. Place the disks into two storage pools and make sure both pools can store file data.

hints:

Create 2 disks per storage pool
The first pool must be the system pool.
The system pool should be able to store data and metadata
Only the system pool can contain metadata # Create a disk descriptor file /gpfs-course/data/pooldesc.txt using the format:#DiskName:serverlist::DiskUsage:FailureGroup:DesiredName:StoragePool
```
/dev/sd__:::dataAndMetadata::nsd1:system
/dev/sd__:::dataAndMetadata::nsd2:system
/dev/sd__:::dataOnly::nsd3:pool1
/dev/sd__:::dataOnly::nsd4:pool1
```
The two storage pools will be system and pool1.

Create a backup copy of the disk descriptor file
```
/gpfs-course/data/pooldesc_bak.txt
```

Create the NSD's using the mmcrnsd command.

> mmcrnsd -F /gpfs-course/data/pooldesc.txt -v no
mmcrnsd: Processing disk sdc
mmcrnsd: Processing disk sdd
mmcrnsd: Processing disk sde
mmcrnsd: Processing disk sdf
mmcrnsd: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.
							

Create a file system based on these NSD's using the mmcrfscommand
* Set the file system blocksize to 64KB
* Mount the file system at /gpfs

Command: "mmcrfs /gpfs fs1 -F /gpfs-course/data/pooldesc.txt-B 64k -M 2 -R 2"
Example:

gpfs1:~ # mmcrfs /gpfs fs1 -F /gpfs-course/data/pooldesc.txt -B 64k -M2 -R2

The following disks of fs1 will be formatted on node gpfs1:
    nsd1: size 20971520 KB
    nsd2: size 20971520 KB
    nsd3: size 20971520 KB
    nsd4: size 20971520 KB
Formatting file system ...
Disks up to size 53 GB can be added to storage pool 'system'.
Disks up to size 53 GB can be added to storage pool 'pool1'.
Creating Inode File
  45 % complete on Wed Sep 26 10:05:27 2007
  89 % complete on Wed Sep 26 10:05:32 2007
 100 % complete on Wed Sep 26 10:05:33 2007
Creating Allocation Maps
Clearing Inode Allocation Map
Clearing Block Allocation Map
  42 % complete on Wed Sep 26 10:05:52 2007
  83 % complete on Wed Sep 26 10:05:57 2007
 100 % complete on Wed Sep 26 10:05:59 2007
  43 % complete on Wed Sep 26 10:06:04 2007
  85 % complete on Wed Sep 26 10:06:09 2007
 100 % complete on Wed Sep 26 10:06:10 2007
Completed creation of file system /dev/fs1.
mmcrfs: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.
							

Verify the file system was created correctly using the mmlsfscommand
```
> mmlsfs fs1
```
Mount the file system using the mmmount command
```
> mmmount fs1 -a
```

Verify the file system is mounted using the df command.

> df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda3             32908108  10975916 21932192  34% /
tmpfs                  2019884         4  2019880   1% /dev/shm
/dev/sda1                72248     45048    27200  63% /boot
/dev/fs1             3989291072 491992640 3497298432 13% /gpfs
							

Verify the storage pool configuration using the mmdfcommand

> mmdf fs1
disk                disk size  failure holds    holds              free KB             free KB
name                    in KB    group metadata data        in full blocks        in fragments
--------------- ------------- -------- -------- ----- -------------------- -------------------
Disks in storage pool: system
nsd1                102734400        -1 yes      yes       102565184 (100%)            90 ( 0%)
nsd2                102734400        -1 yes      yes       102564608 (100%)            96 ( 0%)
                -------------                         -------------------- -------------------
(pool total)        205468800                             205129792 (100%)           186 ( 0%)

Disks in storage pool: pool1
nsd3                102734400        -1 no       yes       102732288 (100%)            62 ( 0%)
nsd4                102734400        -1 no       yes       102732288 (100%)            62 ( 0%)
                -------------                         -------------------- -------------------
(pool total)        205468800                             205464576 (100%)           124 ( 0%)

                =============                         ==================== ===================
(data)              410937600                             410594368 (100%)           310 ( 0%)
(metadata)          205468800                             205129792 (100%)           186 ( 0%)
                =============                         ==================== ===================
(total)             410937600                             410594368 (100%)           310 ( 0%)

Inode Information
----------------- Number of used inodes:            4038 Number of free inodes:          397370 Number of allocated inodes:     401408
Maximum number of inodes:       401408
							

Step 2: Create Filesets

We are going to 5 filesets to organize the data.

Create 5 filesets fileset1-fileset5 using the mmcrfileset command

> mmcrfileset fs1 fileset1
> mmcrfileset fs1 fileset2
> mmcrfileset fs1 fileset3
> mmcrfileset fs1 fileset4
> mmcrfileset fs1 fileset5
							

Verify they were created using the mmlsfileset command
```
# mmlsfileset fs1
```
What is the status of fileset1-fileset5? _________________

Link the filesets into the file system using the mmlinkfileset command

# mmlinkfileset fs1 fileset1 -J /gpfs/fileset1
# mmlinkfileset fs1 fileset2 -J /gpfs/fileset2
# mmlinkfileset fs1 fileset3 -J /gpfs/fileset3
# mmlinkfileset fs1 fileset4 -J /gpfs/fileset4
# mmlinkfileset fs1 fileset5 -J /gpfs/fileset5
							

Now what is the status of fileset1-fileset5? ___________________

Step 3: Create a file placement policy

Now that you have two storage pools and some filesets you need to define placement policies to instruct GPFS where you would like the file data placed. By default, if the system pool can accept data, all files will go to the system storage pool. You are going to change the default and create 3 placement rules. The rules will designate:

Data in fileset1 to fileset4 go to the system storage pool
Data in fileset5 go to the pool1 storage pool
files that end in .dat go to pool1 storage pool

Start by creating a policy file /gpfs-course/data/placementpolicy.txt

/* The fileset does not matter, we want all .dat and .DAT files to go to pool1 */
RULE 'datfiles' SET POOL 'pool1' WHERE UPPER(name) like '%.DAT'

/* All non *.dat files placed in filset5 will go to pool1 */
RULE 'fs5' SET POOL 'pool1' FOR FILESET ('fileset5')

/* Set a default rule that sends all files not meeting the other criteria to the system pool */
RULE 'default' set POOL 'system'
							

Install the new policy file using the mmchpolicy command
```
> mmchpolicy fs1 placementpolicy.txt
```

Step 4: Testing the placement policies

Now you will do some experiments to see how policies work. Use this chart to track the experiments results. You can get the amount of free space by using themmdfcommand.

Experiment	System Pool (Free KB)	Pool1 (Free KB)
Before
Bigilfe1
bigfile1.dat
bigfile2
Migrate/Delete

Record the "Before" free space in the chart

Create a file in fileset1 called bigfile1

dd if=/dev/zero of=/gpfs/fileset1/bigfile1 bs=64k count=10000
							

Record the free space in each pool using the mmdf command (Bigfile1)

> mmdf fs1
disk                disk size  failure holds    holds              free KB             free KB
name                    in KB    group metadata data        in full blocks        in fragments
--------------- ------------- -------- -------- ----- -------------------- -------------------
Disks in storage pool: system
nsd1                 20971520       -1 yes      yes        20588288 ( 98%)           930 ( 0%)
nsd2                 20971520       -1 yes      yes        20588608 ( 98%)           806 ( 0%)
                -------------                         -------------------- -------------------
(pool total)         41943040                              41176896 ( 98%)          1736 ( 0%)

Disks in storage pool: pool1
nsd3                 20971520       -1 no       yes        20969408 (100%)            62 ( 0%)
nsd4                 20971520       -1 no       yes        20969408 (100%)            62 ( 0%)
                -------------                         -------------------- -------------------
(pool total)         41943040                              41938816 (100%)           124 ( 0%)

                =============                         ==================== ===================
(data)               83886080                              83115712 ( 99%)          1860 ( 0%)
(metadata)           41943040                              41176896 ( 98%)          1736 ( 0%)
                =============                         ==================== ===================
(total)              83886080                              83115712 ( 99%)          1860 ( 0%)

Inode Information
----------------- Number of used inodes:            4044 Number of free inodes:           78132 Number of allocated inodes:      82176
Maximum number of inodes:        82176
							

Create a file in fileset1 called bigfile1.dat
```
dd if=/dev/zero of=/gpfs/fileset1/bigfile1.dat bs=64k count=1000
```
Record the free space (bigfile1.dat)
Create a file in fileset5 called bigfile2
```
dd if=/dev/zero of=/gpfs/fileset5/bigfile2 bs=64k count=1000
```
Record the free space (bigfile2)
Questions
Where did the data go for each file?
Bigfile1 ______________
Bigfile1.dat ______________
Bigfile2 ______________
Why?

Create a couple more files (These will be used in the next step)

> dd if=/dev/zero of=/gpfs/fileset3/bigfile3 bs=64k count=10000
> dd if=/dev/zero of=/gpfs/fileset4/bigfile4 bs=64k count=10000
							

Step 5: File Management Policies

Now that you have data in the file system you are going to manage the placement of the file data using file management policies. For this example your business rules say that all file names that start with the letters "big" need to be moved to pool1. In addition, all files that end in ".dat" should be deleted.

To begin, create a policy file /gpfs-course/data/managementpolicy.txt that implements the business rules.

RULE 'datfiles' DELETE WHERE UPPER(name) like '%.DAT'
RULE 'bigfiles' MIGRATE TO POOL 'pool1' WHERE UPPER(name) like 'BIG%'
							

Test the rule set using the mmapplypolicy command
```
> mmapplypolicy fs1 -P managementpolicy.txt -I test
```
This command will show you what mmapplypolicy will do but will not actually perform the delete or migrate.
Actually perform the migration and deletion using the mmapplypolicy command
```
> mmapplypolicy fs1 -P managementpolicy.txt
```
Review:
Review the output of the mmapplypolicy command to answer these questions.
How many files were deleted? ____________
How many files were moved? ____________
How many KB total were moved? ___________

Step 6: Using External Pools

In this step you will use the external pool interface to generate a report.

Create two files : expool1.bash, listrule1.txt:
File /tmp/expool1.bash

#!/bin/bash

dt=`date +%h%d%y-%H_%M_%S`
results=/tmp/FileReport_${dt}

echo one $1 if [[ $1 == 'MIGRATE' ]];then
    echo Filelist
    echo There are `cat $2 | wc -l` files that match >> ${results}
    cat $2 >> ${results}
    echo ----
    echo - The file list report has been placed in ${results}
    echo ----
fi
							

File listrule1.txt

RULE EXTERNAL POOL 'externalpoolA' EXEC '/tmp/expool1.bash'

RULE 'MigToExt' MIGRATE TO POOL 'externalpoolA'
     WHERE FILE_SIZE > 2

Note: You may need to modify where clause to get a list of files on your file system.

Make the external pool script executable
```
chmod +x /tmp/expool1.bash
```
Execute the job
```
mmapplypolicy fs1 -P listrule1.txt
```
It will print output to the screen. When it is done it will print the location of the results file. For example:
The file list report has been placed in /tmp/FileReport_Jul3108-20_15_50