JBOD 磁盘配置和 StorCLI 命令使用小结

独家号 HelloDog 作者 王奥(OX) 原文链接

前言

已经实施过2次Hadoop大数据平台的基础环境搭建,对于硬盘的RAID模式都不约而同选择了JBOD,毕竟大数据对于文件系统的管理本身就是基于分布式,由至少3份副本来保障数据可靠性,只是不做RAID后的SAS硬盘读写性能就有点惨不忍睹。这次Hadoop大数据平台工具选择的是Cloudera,大家感兴趣的化也可以浏览下它的官网。本篇文章主要介绍如何实现在不停机状态下在线更换JBOD硬盘。

利用StorCLI命令不停机在线更换JBOD硬盘

更新历史

2016年12月31日 - 初稿

阅读原文 - https://wsgzao.github.io/post...

扩展阅读

Cloudera - The modern platform for data management and analytics - Cloudera

JBOD原理

JBOD ( Just a Bunch Of Disks )不是标准的 RAID 等级,它通常用来表示一个没有控制软件提供协调控制的磁盘集合。 JBOD 将多个物理磁盘串联起来,提供一个巨大的逻辑磁盘。 JBOD 的数据存放机制是由第一块磁盘开始按顺序往后存储,当前磁盘存储空间用完后,再依次往后面的磁盘存储数据。 JBOD 存储性能完全等同于单块磁盘,而且也不提供数据安全保护。它只是简单提供一种扩展存储空间的机制, JBOD 可用存储容量等于所有成员磁盘的存储空间之和。目前 JBOD 常指磁盘柜,而不论其是否提供 RAID 功能。

RAID磁盘阵列配置和调优小结 - https://wsgzao.github.io/post...

参考文档和工具

StorCLI_RefMan_revf
https://www.broadcom.com/supp...

StorCLI (Command Line) Utility for Storage Management v1.07.07 on Linux
https://www-947.ibm.com/suppo...

使用方式

#查询硬件RAID卡信息
[root@localhost Linux]# cat /proc/scsi/scsi |more

Attached devices:
Host: scsi0 Channel: 02 Id: 00 Lun: 00
  Vendor: IBM      Model: ServeRAID M5210  Rev: 4.62
  Type:   Direct-Access                    ANSI  SCSI revision: 05

#解压安装storcli
unzip ibm_utl_sraidmr_storcli-1.07.07_linux_32-64.zip
rpm -Uvh storcli-1.07.07-1.noarch.rpm

#设置软连接
ln -s /opt/MegaRAID/storcli/storcli64 /bin/storcli
ln -s /opt/MegaRAID/storcli/storcli64 /sbin/storcli

#Raid数量及当前状态
[root@localhost Linux]# storcli show ctrlcount
Status Code = 0
Status = Success
Description = None

Controller Count = 2

#查询虚拟磁盘大小,raid结构等
[root@localhost Linux]# storcli /c0 /v0 show
Controller = 0
Status = Success
Description = None


Virtual Drives :
==============

-----------------------------------------------------------
DG/VD TYPE  State Access Consist Cache sCC       Size Name 
-----------------------------------------------------------
0/0   RAID1 Optl  RW     Yes     RWTD  -   278.465 GB      
-----------------------------------------------------------

Cac=CacheCade|Rec=Recovery|OfLn=OffLine|Pdgd=Partially Degraded|dgrd=Degraded
Optl=Optimal|RO=Read Only|RW=Read Write|B=Blocked|Consist=Consistent|
R=Read Ahead Always|NR=No Read Ahead|WB=WriteBack|
AWB=Always WriteBack|WT=WriteThrough|C=Cached IO|D=Direct IO|sCC=Scheduled
Check Consistency
#可以看出此为RAID1,一个虚拟磁盘,大小为278.465 GB 

#查看control0的物理硬盘
[root@localhost Linux]# storcli /c0 /eall /sall show
Controller = 0
Status = Success
Description = Show Drive Information Succeeded.


Drive Information :
=================

-------------------------------------------------------------------------
EID:Slt DID State DG       Size Intf Med SED PI SeSz Model            Sp 
-------------------------------------------------------------------------
252:16    8 Onln   0 278.465 GB SAS  HDD N   Y  512B ST300MP0035    X U  
252:17    9 Onln   0 278.465 GB SAS  HDD N   Y  512B ST300MP0035    X U  
-------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded

#可以看到总计有两颗硬盘,每一颗大小278.465GB,是否在线等

#查看control1的物理硬盘
[root@localhost Linux]# storcli /c1 /eall /sall show
Controller = 1
Status = Success
Description = Show Drive Information Succeeded.


Drive Information :
=================

------------------------------------------------------------------------
EID:Slt DID State DG      Size Intf Med SED PI SeSz Model            Sp 
------------------------------------------------------------------------
8:0      23 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:1      24 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:2      22 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:3      19 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:4      13 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:5      14 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:6      18 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:7      12 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:8      10 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:9      16 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:10     11 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:11     21 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:12     15 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
8:13      9 JBOD  -  1.0810 TB SAS  HDD N   Y  512B ST1200MM0018   X U  
------------------------------------------------------------------------

EID-Enclosure Device ID|Slt-Slot No.|DID-Device ID|DG-DriveGroup
DHS-Dedicated Hot Spare|UGood-Unconfigured Good|GHS-Global Hotspare
UBad-Unconfigured Bad|Onln-Online|Offln-Offline|Intf-Interface
Med-Media Type|SED-Self Encryptive Drive|PI-Protection Info
SeSz-Sector Size|Sp-Spun|U-Up|D-Down|T-Transition|F-Foreign
UGUnsp-Unsupported|UGShld-UnConfigured shielded|HSPShld-Hotspare shielded
CFShld-Configured shielded

#可以看到总计有两颗硬盘,每一颗大小1.0810 TB,是否在线等


#设置硬盘状态
storcli /c1/e8/s0 show all

storcli /c1/e8/s0 start locate
storcli /c1/e8/s0 set offline
storcli /c1/e8/s0 set online
storcli /c1/e8/s0 set jbod

StorCLI Help

#查看StorCLI帮助信息
[root@localhost Linux]# storcli -h
     Storage Command Line Tool  Ver 1.07.07 Nov 15, 2013

     (c)Copyright 2013, LSI Corporation, All Rights Reserved.


storcli -v 
storcli -h| -help| ? 
storcli -h| -help| ? legacy
storcli show 
storcli show all
storcli show ctrlcount
storcli /cx add vd r[0|1|5|6|00|10|50|60]
    [Size=<VD1_Sz>,<VD2_Sz>,..|all] [name=<VDNAME1>,..] 
    drives=e:s|e:s-x|e:s-x,y,e:s-x,y,z [PDperArray=x][SED]
    [pdcache=on|off|default][pi][DimmerSwitch(ds)=default|automatic(auto)|
    none|maximum(max)|MaximumWithoutCaching(maxnocache)]
    [wt|wb|awb][nora|ra] [direct|cached] [cachevd]
    [Strip=<8|16|32|64|128|256|512|1024>] [AfterVd=X] 
    [Spares = [e:]s|[e:]s-x|[e:]s-x,y] [force][ExclusiveAccess]
storcli /cx add vd each r0 [name=<VDNAME1>,..] [drives=e:s|e:s-x|e:s-x,y]
    [SED] [pdcache=on|off|default][pi] [DimmerSwitch(ds)=default|automatic(auto)|
    none|maximum(max)|MaximumWithoutCaching(maxnocache)] 
    [wt|wb|awb] [nora|ra] [direct|cached] 
    [Strip=<8|16|32|64|128|256|512|1024>] [ExclusiveAccess]
storcli /cx add VD cachecade|nytrocache r[aid][0,1,10, 1EC] 
    drives = [e:]s|[e:]s-x|[e:]s-x,y [WT| WB] [assignvds = 0,1,2] [BOOTVOLSIZE=x]
storcli /cx/

开发者头条

程序员分享平台