Revision as of 08:29, 4 April 2009

Abbreviations and systems

Quorum

• Quorum: The minimum number of people in a organization to conduct business or in this case the minimum number of resources available to keep the cluster running.

In Redhat/Centos Clustering Quorum is when more than 50% of the votes are available. In a standard setting each node of a cluster have one vote. If you Cluster consists of eight nodes, the Cluster will first be available when five nodes are tied together in the Cluster. See the current operational status with cman_tool status command.
This rule is necessary for example in the case we have eight nodes and a network error cuts the cluster in half, we would suddenly have two clusters with four nodes. The two Clusters would destroy the integrity of databases and storage.

Red Hat Cluster Suite Introduction

Red Hat Cluster Suite (RHCS) is an integrated set of software components that can be deployed in a variety of configurations to suit your needs for performance, high-availability, load balancing, scalability, file sharing, and economy.

RHCS consists of the following major components

• Cluster infrastructure — Provides fundamental functions for nodes to work together as a cluster: configuration-file management, membership management, lock management, and fencing.
• High-availability Service Management — Provides failover of services from one cluster node to another in case a node becomes inoperative.
• Cluster administration tools — Configuration and management tools for setting up, configuring, and managing a Red Hat cluster. The tools are for use with the Cluster Chapter Infrastructure components, the High-availability and Service Management components, and storage.
• Linux Virtual Server (LVS) — Routing software that provides IP-Load-balancing. LVS runs in a pair of redundant servers that distributes client requests evenly to real servers that are behind the LVS servers.

Additional Cluster Components

You can supplement Red Hat Cluster Suite with the following components, which are part of an optional package (and not part of Red Hat Cluster Suite):

• Red Hat GFS (Global File System) — Provides a cluster file system for use with Red Hat Cluster Suite. GFS allows multiple nodes to share storage at a block level as if the storage were connected locally to each cluster node.
• Cluster Logical Volume Manager (CLVM) — Provides volume management of cluster storage.
• Global Network Block Device (GNBD) — An ancillary component of GFS that exports block-level storage to Ethernet. This is an economical way to make block-level storage available to Red Hat GFS.

Cluster management with Conga

Starting luci and ricci

Follow the instructions in chapter 3 - Configuring Red Hat Cluster With Conga - in RedHat 5.2 Cluster Administration manual. Follow the instructions in

Cluster management from the console

Cluster subsystems

There are quite a few subsystems and daemons. See the picture below

CMAN new model, from the aiscman tutorial. (See below)

cman - Cluster Manager

cman consists in CMAN version 2 of OpenAIS See the aiscman tutorial

• aisexec - The actual kernel manager. Communicates among the nodes using a token
• cman_tool - Cluster Management Tool (join,leave,status etc.)
• The service cman status starts the cluster manager services.

ccs - Cluster Configuration System

• ccsd - daemon that manages the cluster.conf file in a cman cluster.
• ccs_tool

clvmd - Cluster LVM daemon

clvmd is the daemon that distributes LVM metadata updates around a cluster. It must be running on all nodes in the cluster and will give an error if a node in the cluster does not have this daemon running.

notes

Services

ricci

ccsd

luci

cman

rgmanager - Resource Group Manager

Running under CMAN (and using the DLM), rgmanager manages and provides failover capabilities for collections of resources called services, resource groups, or resource trees in a cluster. These resource groups are tree-structured, and have parent-child dependency and inheritance relationships within each subtree. See RedHAT rgmanager

ccsd

groupd

clvmd

aisexec

dlm

utilities

• clustat Cluster Status

Files

• /etc/cluster/cluster.conf

hostnames give all the nodes hostnames in /etc/hosts or use dns

• node1.tekkom.dk
• node2.tekkom.dk
• ..

hostname node1.tekkom.dk
vi /etc/sysconfig/network
service network restart

example

[root@node1 ~]# clustat
Cluster Status for webcluster2 @ Sat Apr  4 07:33:26 2009
Member Status: Quorate

 Member Name                                           ID   Status
 ------ ----                                           ---- ------
 node1.tekkom.dk                                           1 Online
 node2.tekkom.dk                                           2 Online, rgmanager
 node3.tekkom.dk                                           3 Online, Local, rgmanager

 Service Name                                 Owner (Last)                                 State
 ------- ----                                 ----- ------                                 -----
 service:apache_service                       node2.tekkom.dk                              started

[root@node1 cluster]# cman_tool status
Version: 6.1.0
Config Version: 16
Cluster Name: webcluster2
Cluster Id: 28826
Cluster Member: Yes
Cluster Generation: 252
Membership state: Cluster-Member
Nodes: 3
Expected votes: 3
Total votes: 3
Quorum: 2
Active subsystems: 9
Flags: Dirty
Ports Bound: 0 11 177
Node name: node3.tekkom.dk
Node ID: 3
Multicast addresses: 239.192.112.11
Node addresses: 192.168.138.157

[root@node1 ~]# cat /etc/cluster/cluster.conf
<?xml version="1.0"?>
<cluster alias="webcluster2" config_version="16" name="webcluster2">
        <fence_daemon clean_start="0" post_fail_delay="0" post_join_delay="3"/>
        <clusternodes>
                <clusternode name="node1.tekkom.dk" nodeid="1" votes="1">
                        <fence/>
                </clusternode>
                <clusternode name="node2.tekkom.dk" nodeid="2" votes="1">
                        <fence/>
                </clusternode>
                <clusternode name="node3.tekkom.dk" nodeid="3" votes="1">
                        <fence/>
                </clusternode>
        </clusternodes>
        <cman/>
        <fencedevices/>
        <rm>
                <failoverdomains>
                        <failoverdomain name="webcluster-failover" nofailback="0" ordered="0" restricted="1">
                                <failoverdomainnode name="node2.tekkom.dk" priority="1"/>
                        </failoverdomain>
                </failoverdomains>
                <resources>
                        <apache config_file="conf/httpd.conf" name="Appache2" server_root="/mnt/iscsi" shutdown_wait="0"/>
                        <script file="/etc/rc.d/init.d/httpd" name="apachescript"/>
                </resources>
                <service autostart="1" domain="webcluster-failover" exclusive="0" name="apache_service"/>
        </rm>
</cluster>

Manuls

Clustering

• RedHat 5.2 Cluster Administration
• Redhat 5.2 Cluster Suite Overview
• RedHat 5.2 Deployment Guide
• Redhat 5.2 Using Device Mapper - Configuring and administration
• RedHat 5.2 GFS Global File System
• GNBD Global Network Block Device - Using with GFS
• SAN Survial Guide
• RedHat 5.2 Cluster LVM (Logical Volume Manager)
• nfscookbook Good example of setup with great explanations (Loads PDF file)
• aiscman tutorial (Loads PDF file)
• FreeNAS Network Attached Storage solution

Backup technologies

• IBM Backup and recovery