Dell EMC PowerEdge MX Modular Platform Installation, Configuration, and Management

DELL EMC POWEREDGE
MX MODULAR
PLATFORM
INSTALLATION,
CONFIGURATION, AND
MANAGEMENT
Version 1.0
PARTICIPANT GUIDE
PARTICIPANT GUIDE
Dell Confidential and Proprietary
Copyright © 2019 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC and other
trademarks are trademarks of Dell Inc. or its subsidiaries. Other trademarks may be
trademarks of their respective owners.
Dell EMC PowerEdge MX Modular Platform Installation, Configuration, and Management
© Copyright 2019 Dell Inc. Page i

Table of Contents
Dell EMC PowerEdge MX Modular Platform Installation,
Configuration, and Management ............................................................... 1
Course Introduction ................................................................................................... 2

Course Overview ................................................................................................................. 3
Course Objectives................................................................................................................ 4
Course Agenda .................................................................................................................... 5
MX-Series Introduction ............................................................................... 6
MX7000 Chassis ....................................................................................................... 11

MX7000 Chassis Introduction ............................................................................................ 12
Hardware Comparison ....................................................................................................... 14
MX7000 Chassis - Front View ............................................................................................ 17
Chassis Front Numbering .................................................................................................. 19
MX7000 Chassis - Rear View ............................................................................................ 20
Optional LCD ..................................................................................................................... 22
Left and Right Rack Ear Control Boards ............................................................................ 24
MX9002m Management Module ........................................................................................ 27
Power Supply Units............................................................................................................ 29
Front Loading MX Power Supply Unit ................................................................................ 31
Fans and Airflow Zones ..................................................................................................... 33
Airflow – Sleds and PSUs .................................................................................................. 34
Airflow– Management Modules and IOMs .......................................................................... 35
Blank Filler Module ............................................................................................................ 36
Chassis Tub ....................................................................................................................... 37
Interconnectivity Between Components ............................................................................. 38
Put It Back Together .......................................................................................................... 42
At-the-Box Management .................................................................................................... 43
Servers ...................................................................................................................... 44
Page ii © Copyright 2019 Dell Inc.

MX740c and MX840c Compute Sleds................................................................................ 45
PowerEdge MX740c Key Features .................................................................................... 46
PowerEdge MX740c Front View ........................................................................................ 47
PowerEdge MX740c Interior View ...................................................................................... 48
Single-Width Sled Chassis Right Side View ....................................................................... 49
PowerEdge MX840c Key Features .................................................................................... 50
PowerEdge MX840c Front View ........................................................................................ 51
PowerEdge MX840c Interior View - PEM ........................................................................... 52
PowerEdge MX840c Interior View - System Board ............................................................ 53
PowerEdge MX840c - Stacked Planers ............................................................................. 54
PowerEdge MX740c/MX840c Memory Layout ................................................................... 55
iDRAC Module ................................................................................................................... 56
Storage...................................................................................................................... 57
MX5016s Storage Sled ...................................................................................................... 58
MX5016s layout ................................................................................................................. 59
Storage Fabric ................................................................................................................... 60
Fabric C Switches .............................................................................................................. 61
MX5016s Simplified Block Diagram ................................................................................... 62
Compute Sled Fabric C Mini-Mezz ..................................................................................... 63
PERC/HBA Cards .............................................................................................................. 64
PERC H745P MX Enclosure Support ................................................................................. 65
Drive-Assigned Storage Configuration ............................................................................... 66
Enclosure-Assigned Storage Configuration ........................................................................ 67
Accessing Sled Insertion/Removal Handle ......................................................................... 68
MX5016s Service Tag ........................................................................................................ 69
IDSDM and BOSS ............................................................................................................. 70
Switches ................................................................................................................... 72
I/O Fabrics A and B............................................................................................................ 73
MX7000 Networking Overview ........................................................................................... 74
I/O Modules on Fabrics A and B ........................................................................................ 75
Networking IOMs ............................................................................................................... 76
© Copyright 2019 Dell Inc. Page iii

Compute Node to IOM Connectivity ................................................................................... 77
Fabric Mismatch Behavior .................................................................................................. 78
Simulation Activity .............................................................................................................. 80
End of Module .......................................................................................................... 81

Module Objectives Review ................................................................................................. 82
Knowledge Check Questions ............................................................................................. 83
MX7000 Management ................................................................................ 85
OpenManage Enterprise - Modular ......................................................................... 86

MX Series Management..................................................................................................... 87
Management Module ....................................................................................................... 101
Navigating OpenManage Enterprise Modular (OME-M) ................................................... 105
Simulation and Lab Activities ........................................................................................... 107
Creating Groups............................................................................................................... 108
Simulation and Lab Activity .............................................................................................. 110
Storage Sled Management .............................................................................................. 111
Simulation and Lab Activities ........................................................................................... 116
Extracting Logs ................................................................................................................ 117
Firmware Update ............................................................................................................. 120
iDRAC9 with Lifecycle Controller ......................................................................... 130

Integrated Dell Remote Access Controller 9 (iDRAC9) - 14G ........................................... 131
iDRAC Licensing .............................................................................................................. 132
Accessing the iDRAC....................................................................................................... 133
iDRAC9 Password ........................................................................................................... 134
Dashboard ....................................................................................................................... 135
Unpinned Menu ............................................................................................................... 136
Dashboard Actions........................................................................................................... 137
System Lockdown Mode .................................................................................................. 139
System Lockdown Mode - cont. ....................................................................................... 140
System Lockdown Mode Screenshot ............................................................................... 141
Page iv © Copyright 2019 Dell Inc.

Storage ............................................................................................................................ 142
Configuration - Power Management ................................................................................. 145
Accessing Virtual Console ............................................................................................... 146
Accessing Virtual Console - Plugin Options ..................................................................... 147
Accessing Virtual Console - HTML5 Viewer ..................................................................... 149
Configuration - System Settings ....................................................................................... 150
Configuration - Storage Configuration .............................................................................. 151
Configuration - BIOS Settings .......................................................................................... 152
Maintenance .................................................................................................................... 153
iDRAC Settings ................................................................................................................ 154
iDRAC9 Group Manager .................................................................................................. 156
Group Manager Key Points .............................................................................................. 157
iDRAC9 Service Module (iSM) ......................................................................................... 158
iDRAC9 Service Module - Screenshot ............................................................................. 160
Lab Activities.................................................................................................................... 161
Lifecycle Controller .......................................................................................................... 162
Launching the Lifecycle Controller ................................................................................... 164
Lifecycle Controller Main Menu ........................................................................................ 165
Firmware Update ............................................................................................................. 167
Hardware Configuration ................................................................................................... 168
Operating System Deployment ........................................................................................ 169
Platform Restore .............................................................................................................. 170
Lab Activity ...................................................................................................................... 172
Redfish .................................................................................................................... 173

What is Redfish ................................................................................................................ 174
Redfish Operational Model ............................................................................................... 176
Redfish Tree Structure ..................................................................................................... 177
Redfish Operations .......................................................................................................... 179
End of Module ........................................................................................................ 181

Module Objectives Review ............................................................................................... 182
Knowledge Check Questions ........................................................................................... 183
© Copyright 2019 Dell Inc. Page v

MX Installation and Configuration ......................................................... 185
Initial Setup ............................................................................................................. 186

ESD ................................................................................................................................. 187
Rack Installation............................................................................................................... 188
Cabling and Chassis Deployment .................................................................................... 190
Setting the Initial OME-M IP Address ............................................................................... 193
Steps to Connect to KVM ................................................................................................. 194
Using the Left Rack Ear LCD Panel ................................................................................. 197
Simulation Activity ............................................................................................................ 203
Available RACADM Commands ....................................................................................... 204
Chassis Deployment Wizard ............................................................................................ 205
Simulation ........................................................................................................................ 209
Power on Sequence......................................................................................................... 210
Configuring I/O Modules .................................................................................................. 212
Setting Up iDRAC on Compute Sleds .............................................................................. 215
Identity Pools ................................................................................................................... 217
Create Server Templates ................................................................................................. 219
Lab Activity ...................................................................................................................... 221
Compute Sleds ....................................................................................................... 222

Compute Sled Locations .................................................................................................. 223
Supported Operating Systems ......................................................................................... 227
White Board Activity - Operating Systems Overview ........................................................ 228
End of Module ........................................................................................................ 230

MX Networking ........................................................................................ 234
MX Networking Hardware ...................................................................................... 235
Page vi © Copyright 2019 Dell Inc.

Open Networking Capability ............................................................................................. 236
MX Switch Hardware Naming Convention........................................................................ 237
I/O Modules Connectivity - Chassis Slot Assignments ..................................................... 239
I/O Module Interfaces ....................................................................................................... 240
MX5108n Ethernet Switch................................................................................................ 242
MX9116n Fabric Switching Engine .................................................................................. 243
MX7116n Fabric Expander Module .................................................................................. 244
Pass-Through Modules .................................................................................................... 245
Cable types ...................................................................................................................... 246
Optic Connectors ............................................................................................................. 247
Introduction to QSFP28-DD ............................................................................................. 248
Switch Management Overview .............................................................................. 249

Full-Switch Mode ............................................................................................................. 250
Smart Fabric Services Mode ............................................................................................ 251
Network Switch Management Methods ............................................................................ 252
Scalable Fabric Architecture ................................................................................. 253

Overview.......................................................................................................................... 254
Topology Architecture ...................................................................................................... 255
Chassis Groups and Fabrics ............................................................................................ 257
Initial Switch Setup ................................................................................................ 258

Minimum Requirements ................................................................................................... 259
Making the Initial Connection ........................................................................................... 260
Deploy a SmartFabric ...................................................................................................... 261
Switch Firmware .............................................................................................................. 264
Connecting to UpStream Switches....................................................................... 266

Traditional Mixed modular/rack Networking ...................................................................... 267
Embedded Top of Rack (ToR) ......................................................................................... 268
MX SmartFabric Deployment ................................................................................ 269

MX SmartFabric Deployment Video Introduction .............................................................. 270
© Copyright 2019 Dell Inc. Page vii

SmartFabric Deployment - Introduction ............................................................................ 272
SmartFabric Deployment - MX7000 Initial Configuration .................................................. 276
Deployment Overview ...................................................................................................... 280
Create Multi-Chassis Management Group ....................................................................... 282
Topology: Dell EMC Networking Z9100-ON Leaf Switches .............................................. 283
Z9100-ON Leaf-1 Command Line Interface ..................................................................... 284
Define VLANs in OME-M ................................................................................................. 285
Create the SmartFabric .................................................................................................... 286
Define Uplinks .................................................................................................................. 287
Create a Server Template ................................................................................................ 288
Associate Template with a Network ................................................................................. 289
Deploy Server Template .................................................................................................. 290
Z9100-ON Leaf-1 Command Line Interface ..................................................................... 291
MX9116n FSE on Chassis 1 ............................................................................................ 292
Topology: Cisco 100 GbE Leaf Switches with MX7000 Chassis ...................................... 293
Cisco 100 GbE Leaf-1 Command Line Interface .............................................................. 294
Create a SmartFabric and Define Uplinks, Summary ....................................................... 295
Configuring Switch Features (Simulation Activity) ............................................. 296

Initial Setup Practice ........................................................................................................ 297
Locating Switch Information in OME-M ............................................................................ 298
Configuring Networking Information in OME-M ................................................................ 299
Create Chassis Groups in OME-M ................................................................................... 300
Create a Fabric and Uplink in OME-M.............................................................................. 301
End of Module ........................................................................................................ 302

MX Troubleshooting................................................................................ 306
Minimum Configuration and Population Rules ................................................... 307

MX7000 Minimum Configuration to POST........................................................................ 308
Maximum Population Rules ............................................................................................. 309
Maximum Population Rules (cont.)................................................................................... 311
Page viii © Copyright 2019 Dell Inc.

MX740c/MX840c Minimum Configuration to POST .......................................................... 313
Alerts and Logs ...................................................................................................... 314

Health Status and Alerts .................................................................................................. 315
LCD Alerts ....................................................................................................................... 316
Degraded Health Status Messages .................................................................................. 317
Example – Predictive Failure Detected ............................................................................ 318
Displayed Errors .............................................................................................................. 319
Hardware Error Logs ........................................................................................................ 320
SupportAssist Collection .................................................................................................. 322
Hardware Troubleshooting ................................................................................... 323

Troubleshoot .................................................................................................................... 324
Troubleshooting Management Modules ........................................................................... 326
Management Module Active/Standby ............................................................................... 328
Management Module – Reset .......................................................................................... 329
Profiles and Management Module Failover ...................................................................... 330
Disabling a Forgotten Management Module Password .................................................... 332
Troubleshooting Compute Sleds ...................................................................................... 334
Troubleshooting Compute Sleds (cont.) ........................................................................... 335
Troubleshooting Storage Sleds ........................................................................................ 336
Troubleshooting Storage IOMs ........................................................................................ 337
Troubleshooting Storage IOMs (cont.) ............................................................................. 340
Troubleshooting Power .................................................................................................... 341
MX7000 Power Redundancy Options .............................................................................. 343
Power Supply Unit Indicators ........................................................................................... 345
Troubleshooting Temperature .......................................................................................... 347
Troubleshooting Fans ...................................................................................................... 348
Fan Module Indicator Codes ............................................................................................ 349
Fan Cooling Zones........................................................................................................... 350
Troubleshooting the Tub .................................................................................................. 351
Cold Service and Hot-Plug Components .......................................................................... 353
Lab Activity ...................................................................................................................... 355
© Copyright 2019 Dell Inc. Page ix

iDRAC and Lifecycle Controller Troubleshooting ............................................... 356
iDRAC Health .................................................................................................................. 357
iDRAC Logs ..................................................................................................................... 358
iDRAC Troubleshooting ................................................................................................... 359
iDRAC Troubleshooting – Video Capture ......................................................................... 360
iDRAC Troubleshooting – Post Code/Intrusion/Last Crash Screen .................................. 361
iDRAC Troubleshooting – Diagnostics ............................................................................. 362
iDRAC Troubleshooting – SupportAssist Collections ....................................................... 364
Lifecycle Controller - Platform Restore ............................................................................. 366
Lifecycle Controller – Part Replacement Configuration .................................................... 368
Lab Activities.................................................................................................................... 370
Module Summary ................................................................................................... 371

Summary ......................................................................................................................... 372
Course Conclusion ................................................................................. 375
Course Conclusion ................................................................................................ 376

Course Conclusion........................................................................................................... 377
Review Course Objectives ............................................................................................... 378
Final Questions ................................................................................................................ 379
Thank You for Your Participation! .................................................................................... 380
Page x © Copyright 2019 Dell Inc.

Dell EMC PowerEdge MX Modular Platform
Installation, Configuration, and Management
© Copyright 2019 Dell Inc. Page 1

Course Introduction
Course Introduction
Page 2 © Copyright 2019 Dell Inc.

Course Introduction
Course Overview

Course Introduction
Course Objectives
The course objectives state the minimum levels of knowledge that you should
achieve after successfully completing this course.

Course Introduction
Course Agenda
Introductions

MX-Series Introduction
MX-Series Introduction

The MX-Series is a new design to replace the M1000e. (Note: Sleds and I/O
modules (IOMs) designed for the MX Series are NOT backward compatible with
other Dell EMC modular infrastructure – M1000e, FX2, and VRTX.)
Compute
With performance and capacity options to suit a wide range of demanding
workloads, the PowerEdge MX740c and MX840c compute sleds are well-equipped
to meet the changing data center needs.
 PowerEdge MX740c
 Two-socket, full-height, single-width compute sled offers impressive

performance when paired with the MX-Series system.
 Up to two Intel® Xeon® scalable processors with up to 28 cores per
processor.
 Each M740c supports up to 24 DIMMs of fast DDR4 memory.
 It features generous, scalable onboard storage with up to six 2.5-inch
SAS/SATA (HDD/SDD) or NVMe drives for high performing Direct Attached
Storage requirements.
 The MX7000 chassis supports up to eight MX740c servers.
 PowerEdge MX840c

 A powerful four-socket, full-height, double-width server, features dense
compute, exceptionally large memory capacity, and a highly expandable
storage subsystem.
 Ideal for dense virtualization environments and can serve as a foundation for
collaborative workloads.
 Two or four Intel® Xeon® scalable processors with up to 28 cores per
processor.
 It can also house up to 48 DIMMs of fast DDR4 memory.
 Options for storage on the M840c include up to eight 2.5-inch SAS/SATA
(HDD/SDD) or NVMe drives.
 SAS/SATA drives are hot-swappable.
 Up to four MX740c compute sleds can be installed in the MX7000 chassis.
Storage
The Dell EMC PowerEdge MX5016s storage sled delivers scale-out, shared
storage within the MX-Series architecture. The MX5016s provides customizable 12
GB direct-attached SAS storage with up to 16 HDDs/SSDs.
 You can map individual drives to servers or share drives with multiple compute
sleds. This granular drive-level assignment lets you address databases of any
size without wasting space. Choose the appropriate PowerEdge RAID
Controller (PERC) or HBA option when mapping drives to individual or multiple
servers. Add a clustering operating system or file system of your choice when
mapping to shared drives.
 Both the MX740c 2-socket and the MX840c 4-socket servers can share drives
with the MX5016s using the dedicated MX5000s SAS switch. Internal server
drives may be combined with up to seven MX5016s storage sleds on one
chassis for extensive scalability. Mix and match storage and compute nodes
within the eight available slots depending on your workload requirements.
 Both the MX740c 2-socket and the MX840c 4-socket servers can share drives
with the MX5016s using the dedicated MX5000s SAS switch. Internal server
drives may be combined with up to seven MX5016s storage sleds on one
chassis for extensive scalability. Mix and match storage and compute nodes
within the eight available slots depending on your workload requirements.
 High-availability features such as hot-pluggable drives and hot serviceable
extenders help prevent work interruptions. Further minimize downtime with Dell

EMC validated hard drives that are securely updated through an automated
firmware process. Front access means the MX5016s is always accessible to
swap or add drives as needed.
Networking
The Dell EMC family of networking solutions provides a wide-range of top-of-rack
and embedded switching platforms for server internetworking. Using Open
Networking, Dell EMC leads the industry with the broadest set of software choices
and capabilities to match various use-cases. Dell EMC networking solutions
provide rock-solid dependability that is proven in the most demanding Enterprise
and Service Provider data center environments around the globe.
The new MX-Series Ethernet and Fibre Channel switches are open networking
switches for the MX7000 chassis. These new switches provide customers with a
highly scalable and cost-effective multichassis fabric architecture that uses a broad
set of open networking hardware, software and management options.
 This innovative next-generation family of MX switches delivers key advantages

to customers:
1. Modern design
– introducing a multichassis scalable fabric architecture that can grow with
your needs.
2. Open automation
– giving customers the power of open choice with a broad array of
hardware platform, operating systems and management services.
3. Agile transformation
– by offering optimum connectivity to servers and storage within the
chassis to accommodate demanding workloads and environments.
 The initial set of MX-Series networking switches consist of the following:
 MX9116n Fabric Switching Engine

– 16 x 25 GbE server facing ports, 2 x 100 GbE uplink ports, 2 x
100GbE/32G FC uplink ports, and 12 x 200 GbE Fabric Expander ports.
 MX7116n Fabric Expansion Module

– 16 x 25 GbE server facing ports and 2 x 200 GbE Fabric Switching
Engine ports.
 MX5108n Ethernet switch
– 8 x 25 GbE server facing ports, 2 x 100 GbE uplink ports, 1 x 40 GbE
port, and 4 x 10GBase-T ports.
 MXG610s Fibre Channel Switch
– 16 x 32G FC internal ports, 8 x 32G FC SFP+ ports, and 2 x 100 GbE or
8 x 32G FC uplink ports.

MX7000 Chassis
MX7000 Chassis
Introduction

MX7000 Chassis
MX7000 Chassis Introduction
We begin by looking at the multiple components that make up the MX-series

architecture.
Here is a view of the front of the MX-series chassis. Keep in mind that this image
shows one of a multitude of possible configurations. The MX-series chassis is a
7U, 32-inch deep modular enclosure that combines a customizable pool of server,
storage, and networking resources. The chassis contains eight slots across the
front: Four slots to the left and four slots to the right of the fan column. Customers
can install any combination of compute and storage sleds into the chassis.
 The PowerEdge MX7000 with its MX-series compute, storage and networking
components is the next-generation of modular architecture. Capable of tackling
both the traditional workloads of the past and the new and emerging workloads
of the future, the chassis features:
 Support for 2S and 4S Skylake based servers with massive storage potential
(up to 6 local drives per 2S and 8 local drives per 4S server)
 Support for 16-drive 12 Gb/s SAS storage sleds.
 Comprehensive I/O options including 25Gb Ethernet, 12Gb SAS, and 32Gb
Fibre Channel.
 Three I/O networking fabrics (two general purpose Fabrics A/B and one
storage specific Fabric C supporting either SAS or FC).

MX7000 Chassis
 No midplane design for Fabrics A and B. Fab A/B mezzanine card

connectors dock directly with IOM connectors through the assistance of
mechanical guide pins. This means zero throughput limitations, providing
high-speed technology connections now and well into the future without any
type of midplane upgrade.
 Up to two chassis management modules.
 Up to 6 hot pluggable, redundant 3000W power supplies.
 Two separate cooling zones to provide cooling on demand to the front
components via 5 rear 80mm fans and the rear components via 4 front
60mm fans.
 Third cooling zone exists for the Power Supply Units.
 Front control panel with three hardware options:
– LCD only
– LCD with QuickSync
– LED only

MX7000 Chassis
Hardware Comparison
M1000e MX7000
Rack Height 10U 7U
Sled Vertical Vertical

Orientation
Chassis Dual Chassis Managememt Controllers Dual MM (management

Controller (CMC) modules as a mini-
appliance for
OpenManage
Enterprise - Modular)
Compute Sled M420 MX740c/MX840c plus

Options M520 at least 2 more
M600/M605/M610/M620/M630/M640 compute generations.
M710/M710HD
M805/M820/M830
M905/M910/M915

MX7000 Chassis
Sled Support Up to 16x 'half-height' or 8x 'full-height' Up to 8x 'standard-

or 32x 'quarter-height' blades height' or 4x 'standard-
height/double-wide'
sleds
Local Up to 2x 2.5" drives per 2S blade Up to 6x 2.5" drives per

Compute Sled 2S compute sled or 8x
Storage 2.5" drives per 4S
compute sled in
addition to 2x m.2
devices per compute
sled.
Local Chassis Up to 14x 2.5" iSCSI drives occupying 2 Up to 16x 2.5" DAS
Storage slots drives occupying 1 slot
I/O Modules 3+3 (A/B/C) 3+3 (A/B/C)

Fabric A is Ethernet only. A, B direct-IO
interconnect with
compute sleds.
Fabric C is
interconnect that is
reserved for storage
(SAS, FC-IOM)
PSUs 6x 3 KW 6x 3 KW
Storage PS-M4110 blade array (NAS-based) MX5016s (JBOD SAS-

based)
Quick Sync Not supported Supported
KVM Optional module Integrated to connect

with MM
MX7000 Is/Is Not
IS IS NOT

MX7000 Chassis
M1000e replacement FX2 or VRTX replacement
All new chassis, sled, and I/O Module Backwards compatible with prior
form factors generations
Flexible I/O Fabrics (2 general purpose, 1 Locked in to PCIe internal fabric

storage)
Direct Orthogonal Compute to I/O Fully locked by Midplane architecture

connections enabling for future
technology upgrades (Fabrics A and B)
New Systems Management solution Reuse of CMC solution

including x86 management processor
KVM solution that is integrated into Separate KVM module

management processor
7U Rack Chassis Tower product or tower option
Single Chassis Multiple chassis

MX7000 Chassis
MX7000 Chassis - Front View
1. Left control panel

2. Single-width compute sled
3. Sled blank
4. Front fans (4)
5. Double-width compute sled
6. Single-width storage sled
7. Right control panel
8. Information Tag
9. Power Supply Units (6)
Let's begin by looking at the multiple components that make up the MX-series
architecture.
Here is a view of the front of the MX-series chassis. Keep in mind that this image
shows one of a multitude of possible configurations.
Chassis: 7U, 32-inches deep enclosure sharing power, cooling, and OpenManage
Enterprise – Modular Edition unified management.
Compute: Up to eight single-width 2-socket sleds with up to 6 local

SAS/SATA/NVMe drive bays or up to four double wide 4-socket sleds with eight

MX7000 Chassis
drive bays.
Scalable Networking (rear): Two redundant general-purpose fabrics, MX Scalable

Fabric Architecture for multichassis networking, Open Networking supports third-
party operating systems.
Storage: Up to seven single-width 12GBbs SAS sleds with up to 16 direct –

attached HDD/SDD devices.
Storage Networking (rear): Redundant, highly available Fibre Channel or SAS

storage fabric for high availability, SAS extension for optimal storage scalability.
 Front features:
– Eight single-width or four double-width sleds.

– Double wide sleds in slots 1 - 2, 3 - 4, 5 - 6, 7 - 8.
– Up to six front loading PSU.
– 4x 60 mm IOM cooling ‘Smart Fan’ modules.
– Right rack ear: Power button + LED, two USB Type-A, one USB Type Micro
AB + LED, one display port.
– Left Rack Ear with three options: LCD with QuickSync, LCD without
QuickSync, LED only.

MX7000 Chassis
Chassis Front Numbering
 Numbering:
 From the front, the sleds are numbered from 1 to 8, left to right. For double-
width sleds, the numbers are 1, 3, 5 and 7, left to right.
 The PSUs are numbered 1–6 left to right.
 The front 60 mm fans are numbered 1–4 from top to bottom.

MX7000 Chassis
MX7000 Chassis - Rear View
1. Slot for Fabric A1

2. Slot for Fabric A2
3. Rear fans (5)
4. Slot for Fabric B1
5. Slot for Fabric B2
6. Slot for Fabric C2
7. Power cable connection status LED
8. C22 power inlet connectors (6)
9. Management Module 2
10. Management Module 1
11. Slot for Fabric C1
Rear features:
Three Redundant Fabrics:

A and B use same form factor module, direct orthogonal mate connector (Ethernet,
InfiniBand, and other fabric types). C is unique and supports storage solutions,
connection through chassis board (SAS 12 Gb, 24 Gb, FC16/32).

MX7000 Chassis
Management Controller:
Redundant modules. Each module provides 2 RJ45 connectors, one micro USB
Type B, one System ID button, one status LED, one Power LED.
Cooling:
Five 80mm fan modules for sled cooling.

MX7000 Chassis
Optional LCD
Integrated into the far-left bezel of the MX-series chassis is an optional LCD panel
that can be applied for at-the-box management. This panel displays chassis status,
errors, and the initial IP address.
NOTE: The LCD touch panel is not a hot swappable module. Before you replace
the module, power off the enclosure and remove the AC power from the chassis.
The LCD touch panel enables you to scroll or swipe on the screen. The options
available on the LCD touch panel are as mentioned below:
Welcome Screen Enables you to select your native language and the
default LCD home page.
Main Menu Enables you to access the LCD functionality such as

Identify, Settings, QuickSync, Alerts, Help, and Powered
off.
QuickSync Enables you to connect OpenManage Mobile to the

enclosure.
Alerts Enables you to view a list of all the critical and warning
alerts of the enclosure.

MX7000 Chassis
Network Settings View and configure the chassis management IP address.
LCD Configuration Enables you to configure the LCD options such as View
and Modify, View only, Disabled, Present, and Not
present.
Settings Enables you to edit the Network settings, LCD Language,

and Home Screen.
Service Interaction Displays the impact on drive mapping when a server or

sled is replaced in the enclosure.
System Info Displays the Model number, Asset tag, and Service tag of
the enclosure.
Chassis Power Off Enables you to perform a Shutdown or Graceful

shutdown.

MX7000 Chassis
Left and Right Rack Ear Control Boards
The left ear is also where the optional OpenManage QuickSync hardware is
deployed, enabling IT administrators to connect to the chassis with a smartphone
or tablet using a Bluetooth, secured, wireless connection. With OpenManage
QuickSync, IT Staff can perform a range of tasks, including configuration and
diagnostics, all from a simple mobile application. IT staff can also manage the
MX7000 chassis using a traditional crash cart.
Left rack ear:

1. System health indicator
2. System temperature indicator
3. I/O module health indicator
4. Fan health indicator
5. Stack or group indicator
6. LED status indicator bar
7. System ID button on non-LCD panel
8. System ID indicator on LCD panel
9. Main Menu button
10. QuickSync 2 indicator (optional)
11. System alerts indicator on LCD
12. Activate LCD button/System ID indicator.
13. Error indicator
14. QuickSync 2 wireless status indicator (optional)

MX7000 Chassis
Right rack ear:

1. Power button: With green LED
2. USB 2.0 port (2): Ports for user KVM control support keyboard and mouse only
3. Micro-AB direct management port with Green/Amber LED
4. Mini DisplayPort: Connector for video
Touchscreen LCD: Status, errors, initial IP address
QuickSync (wireless): OpenManage Mobile to chassis using Bluetooth secured

point-wireless connection.
Traditional crash cart: Simple keyboard and display interface (no cable adapter):
access to chassis manager, CLI, and GUI services
STATUS ID BUTTON (TOP) WIRELESS BUTTON

(BOTTOM)
Healthy Replication Manager OFF

internal database Blue
Fault Blink Amber OFF
System ID Blink Blue OFF
Healthy, Wireless ON Replication Manager Replication Manager

internal database Blue internal database White
Fault, Wireless ON Blink Amber Replication Manager

internal database White
System ID, Wireless ON Blink Blue. Replication Manager

internal database White
Healthy, Wireless Replication Manager Blink White

Communication internal database Blue
Fault, Wireless Blink Amber Blink White

Communication

MX7000 Chassis
System ID, Wireless Blink Blue Blink White

Communication
Healthy, Wireless fault Replication Manager Blink Amber

internal database Blue
Fault, Wireless fault Blink Amber Blink Amber
System ID, Wireless fault Blink Blue Blink Amber
Important Points
The right rack ear control panel contains a CMOS chip. In the event of a power
outage, the CMOS is used to store the basic configuration.

MX7000 Chassis
MX9002m Management Module
External Network RJ45 ports

The MM internal network switch supports dual 1000BASE-T uplink ports to the
chassis management network. The management network provides Gigabit
Ethernet (GbE) connectivity to MM, Sleds, and IOMs. Either uplink port may be
used as a stacking port if multiple MMs are daisy-chained to a Top-of-Rack switch.
External Micro USB Serial Port

The MM provides a Micro-USB port for serial access by an external host. The
UART* settings are 115 K baud, eight data bits, one stop bit.
 MM presents a connection option menu on the USB serial port to one of the
following:
– Local MSM controller.

– One of Sleds 1 through 8.
– One of IOMs A1, A2, B1, B2, C1, C2.
 Subsequent to the initial user choice, the communication session varies as
follows:
– Local MM connection – MM CLI presented, login may be required.

– Sled connection – BMC CLI presented, login may be required.
– IOM connection – IOM CLI presented, login may be required.

MX7000 Chassis
System ID Button
The System ID button on the MM rear panel enables for identifying the system in a
rack. When pressed, the front panel of the chassis blinks in the identify pattern.
This button also serves as the Health status LED indicator, displaying blue for
health or blinking amber if the MM has a fault.
* - UART is a digital protocol that we use to transfer data between two devices. A
universal asynchronous receiver/transmitter (UART) is a block of circuitry
responsible for implementing serial communication. Essentially, the UART acts as
an intermediary between parallel and serial interfaces

MX7000 Chassis
Power Supply Units
Let us look at the chassis front again. At the bottom of the chassis front are six
3000W power supplies that provide not only enough power for 14G servers but for
future generations of servers. These can operate in various power supply or grid
redundancy modes.
The MX7000 Power Supplies use a new front-loading form factor that is developed
for the MX7000. Up to six PSUs plug into the front of the chassis. The PSUs link
directly with connectors on the Main DB. The AC connections come from the rear
of the chassis cabling to connections on the PDB.
The Platinum PSUs support 3000 W output with high line AC input. For low line
input the PSUs outputs approximately 1400 W. Mixed high line and low line AC
inputs in the same chassis are not supported. In the case this occurs, the MM
disables the PSUs with the lowest count that have different inputs.
If there are an equal number of high-line and low-line PSUs the high-line PSUs
should be enabled.
The supplies are set nominally to output 12.2 V DC at the output. There is no
remote sense that is implemented in the MX7000 distribution boards, therefore
careful attention to voltage plane layout and power distribution must be applied to
all board designs in the infrastructure.

MX7000 Chassis
Power supply redundancy ensures that if a power supply unit fails, there is enough
power for the chassis to continue running.
Grid redundancy uses two data center grids that are connected in such a way that
if an entire grid goes down, the chassis runs on the second power grid.
No Redundancy
This mode has no provision for redundancy - if a PSU fails the system may fault
due to excess current supplied by the remaining PSUs. This mode is typically used
in cases where the user does not have a sufficient number of PSUs to support the
installed configuration of compute sleds, IOMs, and so on.
Grid Redundancy
This mode provides for redundancy at the grid (i.e., branch) level. Configurations
for this mode are represented by “N+N,” where N is the number of power supplies
on each grid and the total available power is N * PSU capacity. Supported
configurations for this mode should include 1+1, 2+2 and 3+3.
Note: The power load is balanced between 2 grids.
PSU Redundancy
This mode provides for redundancy at the PSU level. Configurations for this mode
are represented by “N+1,” where N is the number of power supplies required to
support the configuration (the total available power is N * PSU capacity).

MX7000 Chassis
Front Loading MX Power Supply Unit
The PSUs are accessed from the front; however, the AC power cable connection is
at the rear of the chassis. There are six C22 inlet connectors, three on the rear left,
and three on the rear right side. Each PSU will also have an associated “AC good”
LED located next to its associated plug.
If a PSU is not present, plugging in an AC cord has no effect. If a PSU is present,

and the AC power source is providing the appropriate power signal, the PSU drives
the LED ON.
The MX7000 Power Supplies use a new front-loading form factor that is developed
for the MX portfolio.
Up to six PSUs plug into the front of the chassis. These connect directly to the main
distribution board. The AC connections come from the rear of the chassis cabling to
connections on the PDB. The Platinum PSUs support 3000 W output with high line
AC input. For low line input the PSUs output approximately 1400 W.
The supplies are set nominally to output 12.2 V DC at the output.
Approximate dimensions of the supply are 70 mm wide x 38.5 mm high x 505 mm

deep (excluding handle). The supplies each have internal fans that are located
toward the front of the module, to push air through the supply.
There are six C22 inlet power connectors located at the rear of the chassis, three

MX7000 Chassis
on the left and three on the right,

The C22 Inlet connectors allow for higher exhaust temperatures. Since these are
different than previous PowerEdge offerings, a C20 to C21 jumper cord will be
necessary to connect to the PDU.
Important Points
Both 220 VAC and 110 VAC are supported but the chassis may still request/require
additional power depending upon installed component power needs.

MX7000 Chassis
Fans and Airflow Zones
The fans in the MX7000 are redundant and hot-swappable. This ensures cold air is
delivered to each component. Max supported fan power is 100 W.
The MX7000 has three distinct airflow paths, or zones, of the multiple airflow zone
infrastructure.
There are two different types of fan modules that are used in the MX7000 chassis,
60 mm fans in the front and 80 mm fans in the rear.
Each fan is enclosed in a mechanical assembly to make up the module.
The fan modules are hot-swappable, with 3+1 redundancy for the front fans, and
4+1 redundancy for the rear fans.
The system should always be populated with the full set of fans to support the
airflow requirements of the chassis.

MX7000 Chassis
Airflow – Sleds and PSUs
The chassis is cooled from front to back, as with typical rack systems. The airflow
for the sleds and PSUs is shown below, as pictured from the side view of the
chassis. There are separate paths for the exhaust, keeping the PSU exit air
isolated from the sled exit air.
The sled air flow is controlled by the rear 80 mm fans, while the PSU air flow is
controlled by the PSU fans located toward the front of the PSUs.
The four front 60 mm fans provide cooling to the rear Fabrics A and B IOMs, Fabric
C IOMs, and the MMs.

MX7000 Chassis
Airflow– Management Modules and IOMs

MX7000 Chassis
Blank Filler Module
The installation of Blank Filler Modules is only enforced through documentation and
factory build rules; there is no mechanism in the system to distinguish between the
presence of a Blank Filler Module or an empty slot.

MX7000 Chassis
Chassis Tub
The chassis consists of two major sections. The Main Chassis provides structure
for mounting PSUs, Sleds, and Front Fans.
The Rear Tub assembly includes the central bulkhead structure and mounting for
main chassis boards, AC plugs, and rear-end modules.
The image in this slide shows the two sections separated.

MX7000 Chassis
Interconnectivity Between Components
Power/Signal 80mm fan (x5)

Vertical PDB cable
Left rack 60mm fan Smart fan

80mm fan
(x4) board
ear interposer
AC Status LED
x6 (not shown)
Flex
cable C22 AC
Busbars plug (x6)
FPC
signal
Right rack ear cable AC cables (x6)
Main
Right rack ear Right rack ear
PDB
cable interposer
High-speed IO for Fabrics A & B is routed directly from the Sled to the IOM via a
direct orthogonal connection.
The Main PDB handles other functions like power distribution and low-speed
signaling, other boards/cables, and FAB C.
The components that are within the Tub assembly are shown on the right of the

MX7000 Chassis
image.
Explanations of the boards are as follows:
Main Distribution Board
 The Main Distribution Board (DB) is the most complex of all the internal
infrastructure boards.
 There are no active components, however there are many signals that are
interconnected on this board. The Main DB sits horizontally near the floor of the
chassis. From the front of the chassis, the six power supplies plug into
connectors on the bottom side of the Main DB. Also, from the front of the
chassis the eight compute sleds plug into this board for power, management
signals, and other fabric connections. From the rear of the chassis the
Management Modules plug in to this board to provide overall chassis
management and control.
 Also, from the rear of the chassis the two fabric I/O Modules (Fabric C IOMs)
plug into this board.

MX7000 Chassis
Item Quantity Description

#
1 6 Fabric C IOM Connectors (3 per IOM)
2 8 Sled Power and Management Signals
3 1 Vertical PDB Signal Connector
4 8 Sled Fabric C Signal Connector
5 1 Right Control Panel Connector
6 8 Enclosure Management Connector (4 per MM)
7 6 Power Supply Connectors
8 2 AC Power Indicator Connector
Vertical Power Distribution Board
 The vertical power distribution board sits above the main distribution board,
which is oriented perpendicular to the chassis floor.
 There are no active components, but many signals connections on this board.
This board connects to the main distribution board with a flex cable, and power
and ground bus bars.
 From the front of the chassis, the four cooling fans plug into the vertical power
distribution board
 Also, from the rear of the chassis the four fabric I/O Modules (Fab-A/B IOMs)
plug into this board for power, management signals, and other fabric
connections
 A cable connects this board to the rear fan board.

MX7000 Chassis
Item Quantity Description

#
1 4 Front 60 mm Fan Connector (x4)
2 2 Bus Bar Contact Points – Power and GND
3 1 FPC Signal Cable Connector
4 4 IOM Fabric A/B Power and Signal Connector
5 1 Fan Interface Cable Connector

MX7000 Chassis
Put It Back Together

MX7000 Chassis
At-the-Box Management
The MX-series ground breaking architecture has removed the need for separate
compute, storage, and networking management consoles used in the past. Instead,
MX-series management provides end-to-end lifecycle management of all
components with the embedded OpenManage Enterprise – Modular console.
Through OpenManage Enterprise – Modular and its onboard chassis management

capabilities, IT staff can manage the MX-series chassis while standing in front of
the chassis or remotely with a web interface.

Servers
Servers
Introduction

Servers
MX740c and MX840c Compute Sleds
To accommodate a wide range of workloads, the MX-series offers 2 modular

compute sleds that slide into the MX-series chassis.
The PowerEdge MX740c and PowerEdge MX840c are full-featured PowerEdge

servers that use 14G server technologies. Each contains a full complement of
memory DIMMs and processors that can handle the entire processor stack
including high-wattage 205 W processors.

Servers
PowerEdge MX740c Key Features
The PowerEdge MX740c compute sled is a two socket, full height, single width
sled. The PowerEdge MX740c provides a cost-effective balance of processor
cores, memory, and I/O making it ideal for virtualization environments, software-
defined storage and as a foundation for collaborative workloads.
This sled has 6x 2.5-inch drive bays in three rows containing two bays in each row.
All drive bays are SAS, SATA, and NVMe capable. The MX740c provides 24
memory slots.

Servers
PowerEdge MX740c Front View

Servers
PowerEdge MX740c Interior View
1. Backplane
2. Backplane cable
3. Processor 1 (heat sink)
4. Processor 2 (heat sink)
5. Mezzanine card A1
6. Power connector
7. Mezzanine card B1
8. Mini Mezzanine connector
9. iDRAC card
10. BOSS connector
11. PERC connector

Servers
Single-Width Sled Chassis Right Side View

Servers
PowerEdge MX840c Key Features
The PowerEdge MX840c compute sled is a four socket, full height, double width
sled. Its additional processing power, exceptionally large memory capacity, and
ability to address more local storage make it ideal for a range of database
applications, heavy transactional workloads, and enterprise applications.
This sled has 8x 2.5-inch drive bays: 4 at the top of the sled, an air intake in the
middle and four bays at the bottom of the sled. All drive bays are SAS, SATA, and
NVMe capable. The MX840c provides 48 memory slots.

Servers
PowerEdge MX840c Front View

Servers
PowerEdge MX840c Interior View - PEM

Servers
PowerEdge MX840c Interior View - System Board

Servers
PowerEdge MX840c - Stacked Planers

Servers
PowerEdge MX740c/MX840c Memory Layout
Channel Processor 1 Processor 2
0 Slots A1 and A7 Slots B1 and B7

Servers
iDRAC Module

Storage
Storage
Introduction

Storage
MX5016s Storage Sled
The MX5016s is a dense, highly flexible, scale-out storage sled with easy front
access that can provide flexible, customizable storage for all the servers in the
chassis.
A chassis can contain up to seven storage sleds taking up seven of the eight
chassis slots. The remaining slot in the chassis must be used for the required
compute sled. Drives can be individually mapped to one or more servers. This
granularity lets your customers add storage as needed without wasting space.
The full-height, single-width sled provides customizable 12 Gbps, direct-attached,

SAS storage with up to 16 SAS HDDs or SSDs. The sled also has hot-serviceable
expanders and hot-pluggable drives. These enable IT staff members to avoid
downtime during servicing because they simply slide out the storage sled and
service the expanders and drives while they are in operation.

Storage
MX5016s layout
The MX5016s storage sled contain the status and activity LEDs for the individual
disk drives, and for the two expanders. The illustrations above show the
approximate location of the drive LEDs, which are physically on the Backplane
Board. The bi-color Green/Amber Status LEDs can be in one of seven different
states.
The Expander devices are the PMC PM8043B-F3EI 12 Gbps SAS expander. The
Expander device is mounted on the back side of the board, facing towards the
inside of the sled. The board incorporates a gold-finger card edge connector for
both signal and power. The signal and power connectors with which the gold finger
contacts connect are physically separate on the Backplane board. A third
connector is used to mate with the SAS Cable assembly.

Storage
Storage Fabric
Now we return to the chassis rear. Located below Fabric B is storage Fabric C with
2 side-by-side switches for redundancy. It has a small mid-plane to enable
connectivity from Fabric C to a storage network. Customers have the option of
using fabric C slots for two redundant SAS switches OR 2 redundant Fibre Channel
switches to connect to a storage network.

Storage
Fabric C Switches
Item Description
1 Information tag
2 External SAS ports (1 & 2)
3 External SAS ports (3,4,5 & 6)
4 Release lever button
5 Release lever
6 LED indicators

Storage
MX5016s Simplified Block Diagram
The illustration in this slide shows a simplified block diagram of the data path for
the MX5016s sled.
The MX7000 chassis Fabric-C interface supports two groups of x4 SAS lanes.
Each group connects to a SAS Expander inside the sled.
These SAS signals are routed through a flexible cable bundle from the Fabric-C
Board to the two SAS Expanders.
The power connector supplies 12V and Ground, as well as some control signals to
facilitate turning the sled ON/OFF and general communication with the sled.
Power and control signals are also passed through the flexible cable to the
Backplane.
The Expanders each connect to ports on all 16 HDDs. Note the first expander
connects to “port A” of the first eight HDDs and “port B” of the second eight HDDs.
The second expander is the opposite and connects to “port B” of the first eight
HDDs and port A of the other eight HDDs.
There is also a x4 link between the expanders to facilitate

communication/synchronization between the two.

Storage
Compute Sled Fabric C Mini-Mezz
The Fabric C Mini-Mezz connects directly to the Min Distribution Board which in
turn connects to the Fabric C IOMs. Within the server, the Fabric C Mini-Mezz
connects to the system board and from there it connects to either the HBA or
PERC card.

Storage
PERC/HBA Cards
The Fabric C Mini-Mezz connects directly to the Min Distribution Board which in
turn connects to the Fabric C IOMs. Within the server, the Fabric C Mini-Mezz
connects to the system board and from there it connects to either the HBA or
PERC card.

Storage
PERC H745P MX Enclosure Support
As shown in this slide, the H745P MX resides in the compute sled and functions as
an interface to the SAS topology of the MX7000. The H745P MX is used to
configure drives that are located in the MX5016s storage sled into RAID volumes.
The H745P MX connects to a pair of MX5000s SAS I/O modules (IOMs) that
manage connections between the compute and storage sleds in the MX7000. For
more information about configuring drive assignments, see the OpenManage
Enterprise Modular User's Guide.
The H745P MX is connected to the SAS IOM, which in turn connects to one of the
SAS expanders in the storage sled. The expander in the storage sled is then
connected to all the physical drives in the storage sled, providing one path between
the H745P MX and a physical drive in the storage sled. With connections to both
SAS IOMs, the H745P MX therefore has two paths to each drive in a storage sled,
allowing for a multi-path configuration. If either a SAS IOM or a SAS expander in
the storage sled is removed, then a path to the physical drive is removed from the
SAS topology.
Important: The I/O modules cannot be configured independently.

Storage
Drive-Assigned Storage Configuration
MX5016s configuration is fairly straightforward in MM.
 Essentially choose drive mappings.

 Two options – Drive-Assigned or Enclosure-assigned.
Drive-Assigned: Individual drives are assigned to an assigned slot in a one-to-one

mapping.

Storage
Enclosure-Assigned Storage Configuration
Enclosure-Assigned: All drives are shared among the compute sleds located in
assigned slots.
Enclosure-assigned maps the entire sled to a compute sled, however also enables
mapping of the MX5016s to more than one compute sled.
 This requires a cluster-aware O/S.

 This mode only supports HBA attach, NOT PERC MX745P.

Storage
Accessing Sled Insertion/Removal Handle
To remove the MX5016s sled from the enclosure:

1. Open the Sled Removal hatch on the front panel of the sled.
2. Press the release button down to release the sled removal handle.
3. Using the sled removal handle, pull the sled out of the enclosure.
To open the MX5016s drive drawer:

1. Slide the drive drawer release latch to unlock the drawer.
2. Pull the drive drawer out of the enclosure.

Storage
MX5016s Service Tag
The Mini Express Service Tag is also referred to as a MEST.

Storage
IDSDM and BOSS
 Boot Optimized Storage Subsystem
 The Boot Optimized Storage Subsystem (BOSS) is offered as a means of

booting PowerEdge MX840c systems to a full OS mode when the:
– Target OS is a full OS and not a hypervisor that may be supported best
by IDSDM.
– Users do not want to trade off standard hot plug drive slots for OS install.
– BOSS supports either Non-RAID disks or a single RAID 1 volume
configuration.
 BOSS is an M.2 SATA/PCIe riser card that uses a connector connection to
draw power and provide SMBUS sideband access into the BOSS.
–
-BOSS drives and riser cards are not hot-plug capable.
 BOSS neither adds nor takes away RAID capability; it is a carrier for
M.2 and passes SATA through.
 Internal Dual SD Module (IDSDM)
 This module contains the Internal Dual SD Module (IDSDM) and vFlash SD
card that are combined into a single card module. There are two options
available:

Storage
– vFlash
– vFlash + IDSDM
– The supported capacity for IDSDM microSD cards are 16 GB, 32 GB, or
64 GB, while for vFlash the capacity is 16 GB only. The write-protect
switch is built onboard the IDSDM/vFlash module.
 The IDSDM card provides the following major functions: Dual SD interface,
maintained in a mirrored configuration (primary and secondary SD)
– The IDSDM card does provide full RAID-1 functionality.
– Dual SD cards are not required; the module can operate with only one
card but will provide no redundancy.
– IDSDM enables support for SDXC (Secure Digital eXtended Capacity)
cards.
– USB interface to host system

Switches
Switches
Introduction

Switches
I/O Fabrics A and B
Each Fabric A and B allows for two IOMs (i.e., A1 and A2, or B1 and B2) for
redundancy. Each IOM connects to all 8 compute nodes through the orthogonal
connectors. The connection from the IOM to each compute node (actually the Mezz
card on the compute node) is a x8 bi-directional link. Depending on the interface,
all the lanes within the link may not be used, however the x8 provides a path for
future high-bandwidth fabric types.
The usage of mezzanine cards enables the end user flexible options from which to
choose for their I/O fabric, such as Ethernet or InfiniBand, for example.

Switches
MX7000 Networking Overview

Switches
I/O Modules on Fabrics A and B
Each Fabric A and B enables two IOMs (that is A1 and A2, or B1 and B2) for
redundancy. Each IOM connects to all 8 compute nodes through the orthogonal
connectors. The connection from the IOM to each compute node (actually the Mezz
card on the compute node) is a x8 bi-directional link.
Depending on the interface, all the lanes within the link may not be used, however
the x8 provides a path for future high-bandwidth fabric types.
Cost-effective, high-performance scalable networking provides 25 GbE and 32G

FC host connectivity with 100 GbE and 32G FC uplinks.
 MX Scalable Fabric Architecture provides:
 low latency 25 GbE connectivity across multiple chassis, managed from

GUI.
 Embedded Top-of-Rack supports rack and servers and capable of
connection directly to the fabric Spine.
 Comprehensive integrated management.
 Open Networking running OS10EE and future-ready for other Operating
Systems.

Switches
Networking IOMs
1. The MX5108n is a basic but powerful Ethernet switch that is targeted for
customers with no more than two MX7000 chassis. This switch can drop to
lower speeds to connect to existing older equipment at a customer site. The
MX5108n requires OS10 and has eight server-facing 25 GbE internal ports.
2. The MX9116n Fabric Switching Engine (FSE) is a high bandwidth converged
solution for a large fabric. This model is also recommended for smaller rollouts
that require Fibre Channel capability. The MX9116n requires OS10 and has 16
server-facing 25 GbE internal ports.
3. The MX7116n Fabric Expander Module (FEM) scales fabric bandwidth across
multiple chassis. The MX7116n is an Ethernet repeater that sends traffic to the
FSE, where it is managed, and all switching happens. The combination of the
MX7116n FEM and the MX9116n Fabric Switching Engine (FSE) is what makes
scalable fabric possible.

Switches
Compute Node to IOM Connectivity
A unique feature compared to the M1000e is that for two of the three networking
fabrics (Fabric A and B), there is no mid-plane. The vertically aligned compute
node mezzanine card connectors dock directly with horizontally aligned IOM
connectors through the assistance of mechanical guide pins. There is no mid-plane
or circuit board between these connectors. This means future networking
technology upgradability is extremely easy. There are no mid-plane bandwidth
limitations, because again there is no mid-plane.

Switches
Fabric Mismatch Behavior
Descriptor File location.

Compute sled DF located within the thermal FRU.
Peripheral sled DF FRU is hosted by peripheral Sled CPLD (I2C over SuperShifty
to FRU).
The DF is intended to be programmed one time during manufacturing, but can be
updated in the field if needed.
Descriptor Files (DFs) are Dell proprietary - not tied to IPMI or FRU format.
Common information for a given IOM type
Each Marketing Name has a DF object which includes templates with objects for
each Fabric Technology and Routing Topology.
 Fabric Consistency Check behavior:
 MM is the authority that makes the Fabric Consistency determination based

on:
– IOMs.
– Sled configuration.
– Chassis configuration if applicable (for example if an interposer is
present).
 Main behavioral difference from previous PowerEdge modular designs:
 MM in the MX7000 will not block power on by default.

Switches
 Can block power in exceptional cases if needed.

 Mismatch is compared only between Mezz and IOM, not Mezz to Mezz This
means sleds installed without IOMs will never cause a mismatch.
 In the past, the IOM was not prevented an IOM from powering on - MM
prevents that when necessary.
Data rate mismatch: Fabric Consistency check guarantees that the endpoints share
a common data rate. It does not guarantee that it can be negotiated. Negotiation
can fail due to configuration settings or other incompatibilities that Fabric
Consistency cannot detect.
If any settings enable compatibility, the Fabric is considered matched. User settings
may still cause incompatibility. The Fabric Consistency check is not designed to
detect/address these cases.
If an IOM supports a mix of fabrics, and two different Mezz cards (with different,
supported fabrics) are connected to the IOM - the Fabric Consistency will not flag a
mismatch.

Switches
Simulation Activity
Overview
MX7000 3D simulation

End of Module
End of Module

End of Module
Module Objectives Review

End of Module
Knowledge Check Questions

End of Module

MX7000 Management
Introduction

OpenManage Enterprise - Modular
Introduction
OpenManage Enterprise - Modular (OME-M)
This lesson covers the following topics:

 Overview of MX Series Management
 Management Module
 Features/Functionality
 Navigating OME-M
 Creating Groups
 Configuring sled or slot profiles
 Extracting Logs
 Firmware Update

MX Series Management
Agile and Converged MX-Series Management
MX Chassis Management Architecture

Important Points
 Redundant Management Modules in each chassis for highest availability

 Private & secure embedded management network for discovery of new chassis
and multi-chassis management communication
 The lead chassis manages all Network IOMs, regardless of chassis. Any
chassis can become the management lead. The lead chassis distributes
automation tasks to management modules in member chassis.
Deployment with Logical Building Blocks

1
Server Template
BIOS
RAID
NIC
iDRAC
2
Virtual I/O Pool
Virtual MAC Address Assign to one or more Server Configuration Profiles
WWN, WWPN servers or slots BIOS
iSCSI Name RAID
NIC
Virtual I/O identities
Virtual MAC Address
WWN, WWPN
3 iSCSI Name
Network
iDRAC
VLAN
Type (QoS)
Important Points
When deploying an MX7000, deployment is completed by associating a profile with

a slot or a physical server. Disaster recovery mobility for stateless server is
accomplished by moving server profiles from one physical server to another.
1. Start with a template of server settings such as BIOS, NIC, and so on that you
can pull off (clone) an existing configured server that is configured.
2. Then optionally add a virtual MAC/WWN identity so that the identity is not
bound to the server shipped from the factory (stateless).
 This is achieved by using the Identity Pool feature which is similar to the
FlexAddress feature found in other modular systems.
– Identity Pools virtualize the network identity for Ethernet, FCoE, iSCSI, or
FC access to the users environment. You can use Identity Pools for
template-based deployment for a set of servers. In user-initiated server

deployment process, the next available identity is fetched from the pool
for provisioning a server from the template description.
3. Network definition rounds out the PROFILE by adding the VLAN and QoS
settings. Historically these settings had to be configured in the switch as a
separate operation for every server and every switch in every chassis.
4. Slot for rip and replace.
iDRAC9 Enhancements on Modular
iDRAC 9
Discussion
Question / Discussion Topic:
How familiar are you with iDRAC9 14G?

Discussion Notes:
iDRAC9 provides:
 A powerful management processor that automates common management tasks
such as deployment, configuration, and, update, slashing server OPEX.
 Innovative agent-free architecture ensures consistent management across any
environment—from containers to OSs to virtual machines.
 HTML5-based GUI for easier, faster navigation without the need for plug-ins
 Powerful scripting APIs to automate remote management including WS-MAN,
RACADM, and iDRAC REST API with Redfish 2016 additions.
 QuickSync 2 for easier “at-the-box” deployment and trouble-shooting using your
Android or Apple mobile device
 “Hi-Rez” monitoring reports detailed health status, connectivity topology, and
performance statistics for deeper insight into remote server issues.
Important Points

Regarding iDRAC9, there is no change from Rack Mount standalone 14G iDRAC.
There are three notable differences from previous generations.
For instance, the Dashboard Health Status is visibly different (vers. 3.21 and later).
Virtual Console is no longer available without the purchase of an Enterprise license
and iDRAC is now a separate component.
Previously, modular severs had the Virtual Console standard with the basic license,
but now, with these nodes, you must have the Enterprise DRAC license to acquire
Virtual Console.
To reiterate, you must purchase the Enterprise license to gain access to the Virtual
Console feature and while it is permitted to have a mix of Express and Enterprise
licenses in the MX7000 chassis, purchased licenses are applied per sled. Meaning,
the purchase of one (1) Enterprise license cannot be used for all sleds installed in
the chassis.
Lastly, the iDRAC on the NGM platform is now a separate card/piece and is no
longer integrated into the system board.
Highlights of iDRAC9 changes:

 Dashboard System Health visually changed with version 3.21 and later.
 Virtual Console as it is now an Enterprise licensed option.
 iDRAC on the NGM platform, which is a separate card and is no longer
integrated into the system board.

Scaling Jobs Across Member Chassis
Representational State Transfer (REST) interface - REST is a web-based API,

which provides a way to interact with a system over a normal web connection. It
supports both HTTPS and HTTP.
Important Points
Scaling jobs across member chassis

1. Within a chassis, the management controller distributes configuration & update
jobs to iDRAC.
2. Lead chassis uses RestAPI to initiate configuration & update member chassis
and their servers.
3. Creating a distributed workload execution.

End-To-End Firmware Update
Important Points
End-to-end firmware update

 Updates validated and packaged together by Dell to eliminate the concern of
interoperability issues.
 Direct download of available updates (requires Internet connection)
– One less tool to install.
– No need to "handle" the download files.
– Offline updates via bundled package can be found at http://support.dell.com.
 Installation of updates is automated (reboot, if required, is scheduled by the
user).
Dell performs over 8 weeks of System tests before a block release to ensure all the
components work together. The user can download as an ISO or .zip file and post
to a share where the management SW can access it, or with Internet access from
the chassis you can access the latest updates from dell.com.

Also, some updates including iDRAC can be completed immediately as it does not
impact the running applications on the server. While other updates that require a
restart can be downloaded and staged in the iDRAC to be applied on the next
scheduled server reboot, typically as part of an OS patch.
OpenManage Enterprise Support for MX-Series
Important Points
OpenManage Enterprise support for MX-Series:
 Manages FX2, M1000e, and VRTX and MX7000

 Manages multiple multi-chassis management groups
 Scales to 8,000 nodes
 No charge download (optional upgrade to deploy servers)
Additional OM Enterprise features:

 Packaged as a virtual appliance

 Comprehensive RestAPI
 Common code base and user experience with OME-Modular
Comparison of OM Enterprise and Modular Edition
OM Enterprise OM Enterprise - Modular
Install appliance on Embedded firmware - just open a browser session.

hypervisor, discover Hardwired discovery and control
devices over the network,
enter credentials for remote
control
Common code shared with Common code shared with OM-Enterprise includes:
OME-Modular includes: directory services, alert management, identity pools,
directory services, alert FW update, and RESTAPI
management, identity
pools, FW update, and
RESTAPI
Additional license per Common code shared with OM-Enterprise, but no

server required for server additional license needed to use server templates
configuration and profiles
*Configuration compliance(drift) detection and
remediation is a future feature

Datacenter scale manages Multi-chassis of MX servers/storage/networking at

up to 8,000 nodes, RTS
12/13/14G racks, towers, *14G rack server management by OME-Modular
FX, VRTX. M1000e, & MX is a future feature
servers
Static and dynamic groups Simple, static roles - chassis admin, compute
with RBAC manager, fabric manager, storage manager, viewer
SupportAssist Enterprise SupportAssist Collector for manual ticket generation

(console appliance)
integration for automatic
ticket generation, etc.
At the Box Management and Wireless Local Access
Note that OMM is wireless/Bluetooth capable with MX7000 only.
When displaying this slide, discuss at the box management options and Wireless
local access and new LCD panel.
Question
What are some of the options available for at the box management?

Answer
Question
What might be some options for wireless local access?
Answer
Question
What are some benefits of wireless access?
Answer
Additional information, such as configuring the IP Address via the LCD

screen will be discussed during the Installation and Configuration module.

QuickAssist Technology
Intel® QuickAssist Technology

Intel® QuickAssist Technology (QAT) provides security and compression
acceleration capabilities that are used to improve performance and efficiency
across the data center. Intel® QuickAssist Technology integrates hardware
acceleration of compute intensive workloads such as cryptographic ciphers, hash,
public key exchange, and compression on Intel® Architecture platforms. Intel®
QAT on Dell EMC PowerEdge MX compute platforms is supported with chipset
integration and is enabled through an optional license. License files are applied to
systems through iDRAC.
For more information about using iDRAC, see Dell's Integrated Dell Remote
Access Controller User's Guide at dell.com/idracmanuals.
Find drivers, documentation, and white papers about Intel® QAT at

https://01.org/intel-quickassist-technology.
QuickAssist Technology
 Enhances security and compression performance in:
 Cloud - Boosts application throughput by adding hardware acceleration for

network security, routing, storage, and Big Data. Resulting in the reduction
of demands on the platform, the support of higher-performance secured
tunnels and a greater number of authenticated clients, along with a
maximization of CPU utilization.

 Networking - Improves efficiency by accelerating SSL/TLS with QAT, which

provides: an increase of performance encrypted traffic via a secured
network, the acceleration of compute-intense symmetric and asymmetric
cryptography, and enhanced platform application efficiency.
 Big Data - QAT reduces overall data size by compressing file system data
blocks, which enables faster analytics. This file compression results in:
faster Hadoop* run times for Big Data, reduces the demands on the
processor and aids in the completion of individual jobs which lowers latency
to boost overall performance.
 Storage applications - QAT enables you to take advantage of more storage
efficiency at every tier, by compressing data in real time.

Management Module
MX9002m Management Module
RJ45 (right side up) SysID LED
Micro USB
Handle latch Handle

Important:
OME-M is completely software based. The lead Management Module
(MM) holds all configuration information on a database that is created
and replicated on the MMs. This configuration is done both for a
single chassis or all chassis in a group.
It is imperative to understand that although there are similarities

between the M1000e and the MX7000 platform, the MX7000 is not
the M1000e. On an M1000e the Control Panel is basically backed up
by the CMC. On an MX7000, there is far too much data on the MM’s
to be backed up by the Control Panel. Therefore, it is critical that a
user, copies/exports backups of their highly complex chassis
configurations.
Also, The MMs default to the installed chassis password after a

racresetcfg is performed. So, if a move takes place, or if a new MM
is installed, the MM takes on the password of the installed chassis
whether that is the luggage tag or root/calvin.
The Management Module, model number MX9002m, essentially controls the

overall chassis power, cooling, and physical user interfaces such as the front panel.
It also contains the Ethernet switch that is used by the management network and
external Ethernet ports. MX7000 supports two MX9002m modules for redundancy.
At least one MX9002m is required to power up the system.
 External Network RJ45 ports

– The MX9002m internal network switch supports dual 1000BASE-T uplink
ports to the chassis management network. The management network
provides Gigabit Ethernet (GbE) connectivity to the management services
module, Sleds, and IOMs. Either uplink port may be used as stacking port if
multiple MX9002m modules are daisy-chained to Top-of-Rack switches.
 External Micro USB Serial Port
– The MX9002m provides a Micro-USB port for serial access by an external
host. The UART settings are 115K baud, 8 data bits, 1 stop bit.

MX9002m presents a connection option menu on the Micro-USB serial port

to:
1. Local MM controller, or
2. One of Sleds 1-8, or
3. One of IOMs A1, A2, B1, B2, C1, C2
 Subsequent to the initial user choice, the communication session varies as
follows (login may be required):
– Local MM connection - MM CLI presented
– Sled connection - BMC CLI presented
– IOM connection - IOM CLI presented
 System ID Button
– The System ID button on the MX9002m rear panel allows for identifying the
MX7000 chassis in a rack. When pressed, the front panel of the chassis will
also blink in the identify pattern. This button also serves as the Health status
LED indicator, displaying blue for health or blinking amber if the MX9002m
has a fault. The rear-mounted ID button ONLY illuminates the
Chassis/Management Module ID button, it does not step through the other
devices including the left Control Panel ID Button.

The chassis supports an “At the Box” capability to use the USB serial port on the
rear of the active MX9002m to manage the MSM, an IOM, or iDRAC by using a
simple menu to select the device to manage.
 To Connect to the MM:
1. Use a standard Android charging cable and plug it into the laptop.
2. Plug the other end of the cable into the back of the MM.
3. Configure PUTTY to connect over Serial.
4. Select the COM port.
5. Change the baud rate to 115200 and will automatically connect up (because
it's over USB it's emulated and will come up on your system).
6. Once all steps are completed, use Ctrl-A or Ctrl-X to get back to the main
Menu.

Navigating OpenManage Enterprise Modular (OME-M)
PowerEdge MX servers are fully managed by the Dell EMC OpenManage

Enterprise Modular (OME-M) edition systems management. OME-M delivers key
functions of the OpenManage Enterprise management platform, within the
PowerEdge MX chassis. OME-M enables IT transformation with:
 A single user interface managing compute, storage and networks simply.

IT Teams are more effective with a unified, simple GUI. Reduce staff learning
curve and wasted effort.
 Simplified administration. Deploy and monitor at scale, from a single chassis
to ten. Expand management easily to thousands of servers with OpenManage
Enterprise. Respond quickly to alerts. Access servers remotely with Dell EMC
OpenManage Mobile and the new QuickSync2 feature. It provides secure
operations, updates and management to protect servers while lowering costs
and risks of downtime.
 Agile, intelligent automation. Roll out changes and templates faster across
hundreds of PowerEdge MX servers. Reduce repetitive tasks. Expand reach
and function with a RESTful API to automate tasks and roll out applications
faster.
Whereas, OpenManage Enterprise is the next generation of OpenManage

Essentials. It simplifies, centralizes, and automates the full span of server lifecycle
management activities. It helps in discovery, configuration, deployment, updates,

and remediation. These tasks are performed within a single console that unifies
management of tower, rack, and modular platforms. OpenManage Enterprise helps
in standardizing and supporting IT management policies and practices.

Simulation and Lab Activities
Overview
Optional OME-M Simulation in place of lab exercise.
Complete the Navigating OME-M lab exercise.

Creating Groups
The Multi-Chassis Management (MCM) group can be of the following type:
Wired group or automatic group—In this type, the chassis is daisy-chained or

wired through a redundant port on the MM. The chassis that you select for creating
the group must be daisy-chained to at least one chassis. You can view a list of
wired chassis and select all or the required number of chassis for creating the MCM
group.
Question
Once the group has been created, what would happen if the Lead chassis fails?
Answer
Important: You must have the chassis administrator privilege to

create an MCM group.

With this configuration you can survive the following failures and retain full
management network access to all nodes in the stack:
 Any single network cable failure

 Any single management module failure
 Power loss to any single chassis in the stack

Simulation and Lab Activity
In this simulation, you walk through the steps to group a pair of chassis.
Once you have completed the Creating Groups Simulation Activity, inform the
instructor before moving on to the Creating Groups lab exercise.

Storage Sled Management
MX5016s Storage Sled
In this simulation, you walk through a 3D removal simulation of the MX5016s

storage sled.
Inform the instructor, once you have completed the MX5016s 3D Simulation
Activity.

Storage Fabric C
Within the MX7000 chassis, the internal connection of the storage subsystem is
referred to as “Fabric C.” This fabric is connected on the Main DB between sleds in
the front of the MX7000 chassis and I/O modules in the rear. The primary use
cases for this fabric will be for SAS or Fibre Channel storage connectivity, however
other technologies could use this interconnect in the future.
There are several connections on the Main DB to enable communication between

the IOMs. Between each pair of IOMs (C1 and C2), there is a link for inter-module
communication. This link is referred to as “Fabric V”. This link supports a x1
connection with each lane operating up to 10Gb/sec per direction. In addition to the
Fabric V, there are also handshake signals between the two modules to facilitate
redundancy/failover or other module to module communication. The usage of these
interconnects depends on the IOM design and its particular requirements.

Drive-Assigned Storage Configuration
MX5016s configuration is fairly straightforward in MM. It essentially chooses drive

mappings. There are two options: Drive-Assigned or Enclosure-assigned.
Drive-Assigned: Individual drives are assigned to an assigned slot in a one-to-one

mapping.
Drive-Assigned Configuration Steps

Drive-Assigned Configuration Steps (cont.)
Enclosure-Assigned Storage Configuration
Enclosure-Assigned: All drives are shared among the compute sleds that are
located in assigned slots.
 Enclosure-assigned maps the entire sled to a compute sled, however also

enables mapping of the MX5016s to more than one compute sled:
– This requires a cluster-aware O/S.

– This mode only supports HBA attach, NOT PERC MX745P.

Enclosure-Assigned Configuration Steps
Enclosure Assigned Configuration Steps (cont.)

In this lab simulation, you walk through configuring Sled or Slot Profiles.
The instructor provides a link to this simulation.
In this simulation, you walk through the steps to group a pair of chassis.
Once you have completed the Creating Groups Simulation Activity, inform the
instructor before moving on to the Storage Configuration lab exercise.

Extracting Logs
Steps to extract the chassis logs:

1. Extract the error logs and save the log to either CIFS or NFS using the
dropdown.
2. Enter the address or hostname of the network share.
3. Enter the network share path.
4. Enter the domain name. (optional)
– The option is displayed only if the Saved File Location is CIFS.
5. Enter the username to access the network share path.
– This option is displayed only if the Saved File Location is CIFS.
6. Enter the password to access the network share path
– This option is displayed only if the Saved File Location is CIFS.

Actions/Buttons:
 Save - Saves the file location details and closes the page.
 Cancel - Closes the page without saving the changes.
Click to play video: Extracting Log Files

Important: When a log file is extracted, it is a .zip file. Once this file
has been extracted and saved, contact engineering (if needed) to
assist in deciphering these logs files.
Important: In addition to the above note and with regard to log

reading, it is recommended to follow your legacy troubleshooting skills
in reviewing:
Alert logs
Hardware logs
Host TSR/SupportAssist logs
While these logs may look new at first glance, they are similar to
those that have always been present in the iDRAC UI and legacy
CMC UI.
Troubleshooting these logs would include reviewing the logs for

events related to the issue reported by the customer, filtering the logs
for specific event types (i.e. Warning, Critical etc.) and exporting the
logs for case management and escalation.
If these logs prove insufficient to determine the point of failure, the

next steps would be to collect the following prior to raising an
escalation (where relevant):
Storage IOM logs
Chassis logs

In this lab simulation, you walk through collecting various logs.
In this simulation, you walk through the steps to gather various logs.
Once you have completed the Combined Logs Simulation Activity, inform the
instructor before moving on to the Log Gathering lab exercise.

Firmware Update
Firmware
As stated in the slide, there are two ways of updating firmware:
1. Apply a single Dell Update Package to a device or component.

2. Create a baseline from a catalog to compare all the devices to the catalog
and return a compliance report to act on. (Slide along with the bold
content has been added to the participant notes.)
Prior to beginning the live demonstration discuss how updating a FW package

might affect the system
Question
What will happen from your viewpoint? Will we lose connection? If so, when?
Answer

Question
If you have two (2) MM modules should you update individually or together?]
Answer
Question
What effect will bundle updates have on users? In addition, should the bundle be
updated immediately, or, scheduled to run at a later time during a maintenance
period?
Answer
There are two ways of updating firmware. The first is to apply a single Dell
Update Package to a device or component. The other is to create a baseline
from a catalog to compare all the devices to the catalog and return a compliance
report to act on.
The following example of a Single device update includes the management module
including the OME-M.
 After logging into the OME-Modular console as root user with password calvin,
there are five steps to a single device update:
1. Start by choosing the target device class, i.e. chassis, IOM, storage sled,
then compute
2. Select target(s) then click “Update Firmware”

3. Select individual package as the update source, browse, and select the
appropriate Dell Windows Update Package (DUP), follow via progress bar
4. Verify the FW version to be applied and select target(s) to be updated
5. Select Run now and then “Finish” for chassis, IOM, and storage sleds. For
servers, stage all but the last update as iDRAC will apply all staged updates
in one reboot
Important: Updating the chassis updates the OME-Modular firmware

on the Management Module. Only the active module is displayed as it
will coordinate the update of the standby module, failing over to the
now updated standby module and updating the remaining module.
The MX9116n and MX5108n Dell Networking switches are not in the
catalog so this is the only way to update their firmware from OME-M
Dell EMC Engineering recommends applying your system updates in the following
sequence. While the file versions may change, the procedures outlined will
continue to stay true.
Update the equipment in the following sequence unless otherwise specified:
1. OME-Modular
2. MX5000s
3. MX5016s

4. MX5108n
5. Compute sleds
Firmware - Add Catalog
As previously mentioned, one of the ways to update firmware is to create a

baseline from a catalog to compare all the devices to the catalog and return a
compliance report to act on. So, before you can create the baseline, you must first
add a catalog. Once the catalog has been added and the bundles discovered,
return to Firmware and Create Baseline which will use your new catalog.
When adding a catalog one of the following three catalog source options can be
used:
 Newest validated stacks of chassis firmware on Dell.com

 Latest component firmware versions on Dell.com
 Network Path
Here we show an example of the Catalog Share Type as HTTPS. Other options
include:
 NFS
 CIFS

 HTTP
 HTTPS
 Updating the firmware using catalog-based compliance method:
1. Download the DUP from the dell.com/support/drivers.

2. Use the Dell Repository Manager (Repository Manager) to create the
catalog.xml file.
3. Place the catalog.xml that you created using Repository Manager, in a
shared location.
4. Navigate to the Configuration Firmware page to create the catalog and
baseline.
5. In the OME–Modular web interface, navigate to the Devices > Chassis
page.
6. Click Update Firmware option. The Select Firmware Source window is
displayed.
7. Select the Baseline option and select the required baseline from the drop-
down.
8. Select the OME–Modular component from the comparison report. The
supported components are displayed.
9. Select the required components, for example: OME–Modular, and click
Update to start the firmware update.
10. Navigate to the Monitor > Jobs page to view the job status.
Tip: Use the Add option on the Configuration > Firmware > Catalog
Management option to download the catalog from dell.com/support.
Refer to dell.com for the most current firmware update information.

Firmware - Create Baseline
1. Firmware > Create Baseline

2. Select Devices or Select Groups
3. Next, click Finish to display the Compliance Report Results.
Compliance Report Results

3. Clicking Finish in Step 2, will scan all selected devices against the catalog
bundles available.
 Click the Catalog checkbox and click on View Report to reveal details.

 Expand each Device Name/Component section to view any components

found to be out of Compliance.
1. Create Catalog
2. Create Baseline from Catalog
Click to play video: Adding a Catalog and Creating a Baseline
Firmware - Rollback
The Rollback Firmware window enables you to roll back a firmware update to the
previous version before the update. The rollback option is available if:

 The device has a rollback or N-1 version.

 The imported catalog contains a reference to the previous version.
 You browse for a firmware package, which has the previous version.
Important Points: The rollback firmware version displayed for

Compute Sleds does not always match with the rollback firmware
version in iDRAC.
Backup Chassis
The Backup Chassis feature enables you to create a backup of the chassis
configuration, including those of its member Compute Sleds, and use it later. You
must have the Chassis Administrator privileges to back up the chassis.
Backups are saved to either an NFS or CIFS share. Select the location type where
you want the chassis configuration backup file.
To restore from Backup: OME-M Home > More Actions > Restore. The Restore
Chassis feature enables you to restore a chassis configuration, if the backup file is
created from the same chassis. The filename extension must be .bin.

Important:
You must have the Chassis Administrator privileges to restore the
chassis configuration. Also, the custom SSL certificate is not restored
after the chassis is restored.

This simulation takes approximately: 20 min.
After the simulation, complete the OME-M Navigation Discovery Hunt lab exercise.
In this simulation, you walk through the steps to update the firmware. (You will be
performing an actual MX7000 firmware update in a later lab exercise.)
Once you have completed the Firmware Update Simulation Activity, inform the
instructor before moving on to the OME-M Scavenger Hunt lab exercise.

iDRAC9 with Lifecycle Controller

Integrated Dell Remote Access Controller 9 (iDRAC9) - 14G
14G PowerEdge servers are equipped with iDRAC9, unlike 13G PowerEdge
servers which used iDRAC8.

iDRAC Licensing

Accessing the iDRAC
 Accessing the iDRAC uses the same methods that have been used on previous
PowerEdge server, both modular and rack/tower systems.
Many new features are introduced with 14G hardware --- here is an overview of
the iDRAC9 changes over the iDRAC8.
 iDRAC9 provides:
 Industry-leading automation of server management via its rich APIs
 A simpler customer support experience with less time required to resolve
issues.
 Best-in-class GUI experience via its new HTML5-based console.
 Simplified methods to deal with tedious, repetitive management tasks by
using simple easy-to-use tools.
 The flexibility to support the diverse management requirements of
different markets and use cases.
 Best in class at-the-box management using superior support of popular
mobile devices.

iDRAC9 Password
Factory Default:
Secure Default Password (14G Default)
Legacy Default Password (Pre-14G Default)
For the 14G PowerEdge servers and iDRAC9, a new option for the factory default
password for iDRAC is offered. This option is referred to as the secure default
password for iDRAC.
The password consists of a randomly generated string of alphanumeric characters.

The password is composed of a fixed length of 12 characters. The alpha characters
are always capitalized. Some alphanumeric characters are omitted from the
random generation set. This has been done to reduce any ambiguity in recognizing
the characters especially when they are translated to written text.
Each server is programmed with the randomly generated string.
It should be noted that there is no guarantee that the randomly generated

password will be unique across all servers shipped.

Dashboard
 Use the Dashboard to:
 Power on the system or shutdown the system

 Identify the system
 Tabs:
 The System tab provides system information and iDRAC details and at a
glance status of the systems. You can access more details about the system
through this tab.
 The Storage tab provides details on the storage components. Summary
information and information about controllers, physical disks, virtual disks,
and enclosures are accessed from here.
 The Configuration tab is where settings for items such as power
management, virtual console, licenses, systems, storage configuration,
BIOS, and server configuration profile may be configured.
 The Maintenance tab includes the lifecycle log, job queue, system update,
system event log, troubleshooting, diagnostics, and SupportAssist.
 The iDRAC tab displays the details of the iDRAC settings. It includes
configuration of the Network Settings, IPv4 settings, and the iDRAC Service
Module. Options for connectivity, services, and users are also available.

Unpinned Menu
The unpinned menu shows the different subheadings under each major menu item.

Dashboard Actions
Use the Dashboard to Power On System or Shutdown the system, Identify System,
and perform More Actions (as in Turns on the System Lockdown Mode) using
iDRAC.
 Power On System
– Power On System allows you to turn on the system. Success message is
displayed after the system is on. You can further shut down the system
using the following options:
 Graceful Shutdown
– Shuts down the operating system and powers off the system. Graceful
shutdown requires an Advanced Configuration and Power Interface (ACPI)
aware operating system, which allows system directed power management.
– NOTE: Configure shutdown option on the operating system before you
perform a graceful shutdown using this option. If you use this option without
configuring it on the operating system, it reboots the managed system
instead of performing a shutdown operation.
 Power Off System
– Powers off the server
 NMI (Non-Masking Interrupt)

– Generates NMI to halt the system operation. An NMI sends a high-level

interrupt to the operating system, which causes the system to halt operation
to allow critical diagnostic or troubleshooting activities.
– NOTE: The halt system operation does not occur on systems running on
Linux operating system.
 Reset System (warm boot)
– Reboots the system without powering off (warm boot).
 Power Cycle System (cold boot)
– Powers off, then reboots the system (cold boot).
– NOTE: There is no Power On System, if the system is already ON.
 Identify System
– Select this option to enable LED blinking to identify the server.
 More Actions
– More Actions allows you to configure the System Lockdown Mode.

System Lockdown Mode
Lockdown mode is a licensed feature, and it is only available for iDRAC enterprise
customers only.

System Lockdown Mode - cont.
After the System Lockdown mode is enabled, users cannot change any
configuration settings. System Settings fields remain disabled with some
exceptions.

System Lockdown Mode Screenshot
After the System Lockdown Mode is enabled, the message in the yellow box is
displayed in the iDRAC UI. Also notice that with the System Lockdown Mode
enabled, access to the Lifecycle Controller is disabled.

Storage
Using iDRAC, the functions that are available in OpenManage Storage

Management can be performed. It includes real-time (no reboot) configuration
commands—for example, create virtual disk. RAID may be completely configured
before installing the operating system.
Controller functions may be configured and managed without accessing the BIOS.
Which includes, configuring virtual disks, applying RAID levels and hot spares for
data protection. Other controller functions such as rebuilds, and troubleshooting
can be initiated from the iDRAC. Data may be protected by configuring data-
redundancy or assigning hot spares.
 The storage devices are:
 Controllers:
– Most operating systems do not read and write data directly from the
disks, but instead send read and write instructions to a controller. The
controller is the hardware in your system that interacts directly with the
disks to write and retrieve data.
– A controller has connectors (channels or ports) which are attached to
one or more physical disks or an enclosure containing physical disks.

– RAID controllers can span the boundaries of the disks to create an

extended amount of storage space—or a virtual disk—using the capacity
of more than one disk.
– Controllers also perform other tasks, such as initiating rebuilds,
initializing disks, and more.
– To complete their tasks, controllers require special software known as
firmware and drivers.
– To function properly, the controller must have the minimum required
version of the firmware and drivers installed.
– Different controllers have different characteristics in the way they read
and write data and execute tasks.
– It is helpful to understand these features to most efficiently manage the
storage.
 Physical disks or physical devices:
– Reside within an enclosure or are attached to the controller. On a RAID
controller, physical disks or devices are used to create virtual disks.
 Virtual disk:
– Is storage created by a RAID controller from one or more physical disks.
Although a virtual disk may be created from several physical disks, it is
viewed by the operating system as a single disk. Depending on the RAID
level used, the virtual disk may retain redundant data if there is a disk
failure or have particular performance attributes. Virtual disks can only be
created on a RAID controller.
 Enclosure:
– It is attached to the system externally while the backplane and its
physical disks are internal.
 Backplane:
– It is similar to an enclosure. In a Backplane, the controller connector and
physical disks are attached to the enclosure, but it does not have the
management features (temperature probes, alarms, and so on)
associated with external enclosures. Physical disks can be contained in
an enclosure or attached to the backplane of a system.
In addition to managing the physical disks contained in the enclosure, other
functions include monitoring the status of the fans, power supply, and temperature

probes in an enclosure. Enclosures can be hot-plugged. Hot-plugging is defined as

adding of a component to a system while the operating system is still running.
The physical devices connected to the controller must have the latest firmware.

Configuration - Power Management
Power Control:
Use this section to view the power status of the server and perform remote power-
control operations.
Power Cap Policy:

Power Cap displays the power capping thresholds for the current system
configuration. This is the maximum AC power load that the system can request
from the data center.

Accessing Virtual Console
Use this page to view or configure the Virtual Console settings. Using the Virtual
Console, the remote system using the keyboard, video, and mouse on the local
management station may be managed to control the corresponding devices on a
remote managed system. Up to a maximum of six simultaneous Virtual Console
sessions may be run. The Virtual Console may be used with the Virtual Media to
perform remote software installations.

Accessing Virtual Console - Plugin Options
Select the type of plug-in to be configured.
ActiveX: Starts the ActiveX viewer on a system running Windows and a Java plug-
in on a system running Linux. ActiveX viewer works only on Internet Explorer.
Java: Starts a Java viewer.
HTML5: Starts an HTML5 virtual console if the virtual console feature is enabled
and the Virtual Console Type is set as HTML5.
NOTE: While using HTML5 to access vConsole, the language must be consistent
across client and target keyboard layouts, client OS, client browser, and the target
OS. For example, all must be in English (US), or any of the supported OS
languages.
 ActiveX:
– Starts the ActiveX viewer on a system running Windows and a Java plug-in
on a system running Linux. ActiveX viewer works only on Internet Explorer.
 Java:
– Starts a Java viewer.
 HTML5:
– Starts an HTML5 virtual console if the virtual console feature is enabled and
the Virtual Console Type is set as HTML5.

– NOTE: While using HTML5 to access vConsole, the language must be

consistent across client and target keyboard layouts, client OS, client
browser, and the target OS. For example, all must be in English (US), or any
of the supported OS languages.

Accessing Virtual Console - HTML5 Viewer
A view of the window that opens when HMTL5 is selected as the viewer type.
 Virtual Media,
– Accessed through the Virtual Console viewer, provides the managed system
access to media connected to a Remote system on the network.
 Using Remote File Share
– Specify an image file on a network share and make it available to the
managed system's operating system as a Virtual drive by mounting as a CD,
DVD, or Floppy using Network File System (NFS) or Common Internet File
System (CIFS).
 Virtual Media and RFS is mutually exclusive.
– Only one feature may be used at a time.

Configuration - System Settings
Use this page to view and filter the events, generate alerts, or specify the actions
when an event occurs. An event occurs when the status of a system component is
outside of predefined thresholds.
If an event matches an event filter and this filter is configured to generate an alert
(Email, SNMP trap, IPMI alert, remote system log, Redfish event, or WS event),
then an alert is sent to one or more configured destinations. If the same event filter
is configured to do a task (such as reboot, power cycle, or power off system), the
action is performed. Only one action may be set for each event.

Configuration - Storage Configuration
Storage Settings are configured from this page.
Storage settings are based on one single controller (at a time), and only one job
per controller can be scheduled at a time
Batch changes may be made into one job by adding them to the Pending
Operations.
Click Apply when ready to start or schedule the job. Pending operations persist
until the job is created or they are discarded.

Configuration - BIOS Settings
Use this page to configure the BIOS settings. The BIOS fields are grouped under
menus.

Maintenance
The iDRAC Maintenance section is covered later in this course.

iDRAC Settings
The following areas are used to configure the iDRAC.
 Overview
– Use this section to view the iDRAC details, network settings, and IP settings.
 Connectivity
– Use this page to configure the network properties, common iDRAC settings,
IPv4 and IPv6 properties, IPMI settings, VLAN settings, and manage SSL
certificates.
 Services
– Services - Use this page to view or change the interface settings for local
configuration, Web server, SSH, Telnet, remote RACADM, SNMP agent,
and automated system recovery agent.
 Users
– Use this page to add or edit users, configure directory services and log in
methods, view active sessions, and so on.
 Settings
– Use this page to create manual backups of server settings, create
automated backup tasks, or to import server profiles
 Management Module

– This page shows the Health Status, the firmware version and the IP Address
of the Management Module.

iDRAC9 Group Manager
With an iDRAC9 Enterprise license, Group Manager provides a simple 1-to-many

management console experience to monitor and manage a group of servers that
are physically connected to the same link local network.
Simplicity and ease of use are key tenets of Group Manager, which provides the
ability to see a list of servers and quickly identify a server with an error and access
that server with a single click.
Group Manager has three primary views: Summary, Discovered Servers, and Jobs.
The Summary view provides a searchable dashboard for the health monitoring and
inventory of all systems in the local group.
The Discovered Servers view shows any iDRACs on the link local network that
have been auto-discovered and have not been on-boarded to a group.
The Jobs view shows the progress and history of group configuration actions.
Group configuration actions include: setting up email alerting, adding a local iDRAC
user, removing a local iDRAC user, changing a local iDRAC user password,
changing the group passcode, changing the group name, and on-boarding newly
discovered iDRACs.

Group Manager Key Points
Display the Group Manager key points slide. Review the key points of Group
Manager as outlined on the slide. Point out the following information and ask the
following question.
Important information:
Group Manager is built into the iDRAC. No software is necessary to install for the
feature to work. It runs completely on the iDRAC.
Question
What do you think up to 100 iDRACs means on the second bullet of the slide?
Answer

iDRAC9 Service Module (iSM)
Question
Does everyone understand what the iSM is?
Answer
The iDRAC Service Module feature is available in the iDRAC Express and
Enterprise licenses.
If OpenManage Server Administrator is installed on the system, then the Lifecycle

Controller log and Automatic System Recovery features are disabled in the iDRAC
Service Module. However, the Automatic System Recovery feature is available in
Server Administrator.

To use this feature, ensure that:

 iDRAC Service Module is installed on the host operating system.
 OS to iDRAC pass-through feature is enabled through the internal USB bus in
iDRAC.

iDRAC9 Service Module - Screenshot
If you access the iDRAC iSM setup section before the iSM is installed, you will see
a screen similar to this slide screenshot.
The iSM can either be installed using the iDRAC option as seen in this screenshot
or you can download the iSM file from the support site and install it from within the
server operating system
Note: Only use an iSM file that is supported with 14G servers.

Lab Activities
In this lab simulation, you will complete lab activities covering Navigating the
iDRAC9 interface and the iDRAC Virtual Console.
You will now complete the Navigating the iDRAC9 and the iDRAC Virtual Console
lab exercises.

Lifecycle Controller
The Lifecycle Controller is the engine that provides advanced system management
functions to perform tasks such as deploy, configure, update, maintain, and
diagnose through a graphical user interface. Like the iDRAC, the Lifecycle
Controller is embedded into the server’s system board and is available outside of
the OS, as it can only be accessed during the power-up sequence.
The Lifecycle Controller and the iDRAC are linked closely together and work side
by side. Some of the Lifecycle Controller functions are linked to the iDRAC license,
so the number of Lifecycle Controller functions are dependent on the iDRAC
license type.
 The benefits of using Lifecycle Controller include:
 Increased availability: early notification of potential or actual failures that

help prevent a server failure or reduce recovery time after failure.
 Improved productivity and lower Total Cost of Ownership (TCO): Extending
the reach of administrators to larger numbers of distant servers can make
the IT staff more productive while driving down operational costs, such as
travel.
 Secure environment: By providing secure access to remote servers,
administrators can perform critical management functions while maintaining
server and network security.

 Enhanced embedded management through Lifecycle Controller: Lifecycle

Controller provides deployment and simplified serviceability through the
Lifecycle Controller GUI for local deployment and remote services (WS-Man)
interfaces for remote deployment that is integrated with Dell OpenManage
Essentials and partner consoles.

Launching the Lifecycle Controller
Accessing the Lifecycle Controller is done by pressing <F10> when the prompt
appears during the power-up sequence. The system will then go through the rest of
the power-up sequence before booting to the Lifecycle Controller.

Lifecycle Controller Main Menu
The features of the Lifecycle Controller in 14G are:

 Enhanced System Erase – supports erasure of HDDs, SSDs, and NVMe
devices
 Support HTTP for network share-based features
 Interfaces
– Pre-boot graphical interface accessible from BIOS for at-the-server
operations; standards-based APIs including WS-Man and Redfish,
supporting complete server lifecycle for remote consoles and scripts
 Core standards
– IPMI, WS-Man, Redfish, HTTP/S, NFS, and SMB/CIFS
 Deploying
– Configure system and storage devices; view and export hardware and
firmware inventory; wizard-guided operating system installation with OS
Driver Packs. Unattended installation for Microsoft -Windows and Red Hat
Enterprise Linux
 Zero-touch auto configuration
– Enabling complete server configuration without human touch
 Updating

– Agent-free firmware maintenance with reduced maintenance downtime.

– Firmware updates from your compliant repositories built with Dell Repository
Manager or access updates from downloads.dell.com.
– Recovery of prior versions with firmware rollback.
 Servicing
– Continuous availability of logs and diagnostics without depending on server
storage.
– Parts Replacement automatically flashes firmware and restores saved state
when replacing server components or system board
 System Erase
– Supports repurposing and retirement of severs by deleting data on selected
server components.
– Enables removal of information from BIOS, Lifecycle Controller, internal
configuration databases, rollback firmware versions, and iDRAC settings.
 Backing up and restoring the server
– Back up the server configuration; restore the server to a previously saved
state; import server licenses.
 Server inventory
– Current and factory system hardware inventory; current firmware inventory.

Once the system has booted to the Lifecycle Controller, the Lifecycle Controller
main menu appears, as shown in the image. There are various options in the
system tree, down the left side of the screen. Also, in the main area of the screen,
under Lifecycle Controller, each of the bullet points is a quick-launch task button.
These save you from navigating through the menus while looking for key functions.
In the lower left-hand corner is the server model and Service Tag.

Firmware Update
While you can update the firmware through the iDRAC interface, you can also
update it using the Lifecycle Controller. Just like the iDRAC, you can roll back to a
previous version. The list of available updates is shown here, but keep in mind that
with the IDRAC, you can access it even while the operating system is running;
whereas with the Lifecycle Controller you have to power-cycle the server. Even
when you update using the iDRAC, it is actually the Lifecycle Controller that
performs the update, as the Lifecycle Controller is running the whole time the
iDRAC is operational. You can obtain the updates from a number of different
sources, including the Dell FTP server, local drives such as USB or DVD, or even a
network share.

Hardware Configuration
Another feature of the Lifecycle Controller is Hardware Configuration. Using the

built-in Configuration Wizards, you can now configure the IDRAC and RAID
Storage. Plus, with the Repurpose or Retire System option, the entire system can
be reset to the factory default. This wipes out all specific customer configuration
information stored in the electronics, such as IP addresses, email addresses, and
trap destinations.

Operating System Deployment
The Operating System Deployment option is a key feature that allows users to
choose from a range of operating systems to install on the managed server. This
includes the ability to configure RAID during the OS installation. The interface is
similar to the System Build and Update Utility DVD that some Dell customers may
be familiar with.
To install an OS, first select the desired one. The Lifecycle Controller will then
check its driver base from its own updatable OS driver pack before instructing you
to insert the OS installation DVD. It will then install the OS and insert the drivers at
the correct time. This results in a fully functional operating system without concern
about which drivers are required or how to get them.

Platform Restore
The Platform Restore option is used to push the configuration back to individual
hardware parts or the entire system board. This is a great option for ensuring that if
things go wrong, you can get your server up and running again as quickly as
possible.
The configuration of every hardware part installed in your server is stored within the
Lifecycle Controller, including the firmware and BIOS code. If something goes
wrong—like if a network card fails and the technician arrives on-site and replaces
the card—then what happens? The card would have to be reconfigured manually—
that is, if the person who knows how it should be configured is around.
That is where the Parts Replacement option comes in. It allows you to decide what
happens if a hardware part fails and needs to be replaced. Instead of manually
reconfiguring the new part, you can set the Lifecycle Controller to automatically
push the configuration back out to the new part. This includes re-flashing the
firmware with the code that was on the old part.
This is a great tool but let us take it one step further. What if the system board
fails? Since the Lifecycle Controller is located on the system board and holds all
the configuration data and firmware versions, you would have just lost all of that
information. But have you?
This is where the Server Profile comes in. When you create a Server Profile, it

backs up all the Lifecycle Controller configuration information, the BIOS, and all the
firmware code onto a USB stick, hard drive, or network share. When the new
system board is installed, you simply access the new Lifecycle Controller and
import the Server Profile. All the backed-up configuration information is now
restored to the new Lifecycle Controller, which in turn pushes all of that information
out to the hardware parts. That saves you time and effort from manually
reconfiguring devices like the iDRAC and the BIOS.
The Lifecycle Controller also pushes out the firmware and BIOS code if required.
However, you must back up the Server Profile before the system board fails. The
Platform Restore function is also dependent on having an iDRAC Enterprise
license.

Lab Activity
You will now complete lab exercise on Navigating the Lifecycle Controller.
You will now complete the Navigating the Lifecycle Controller lab exercise.

Redfish
Redfish
Introduction
Redfish

 Redfish Overview
 Redfish Operational Model
 Redfish Tree Structure
 Redfish Operations

Redfish
What is Redfish
Depending upon the scale of an IT installation, using the Graphical User Interface
(GUI) of OpenManage Enterprise or a single iDRAC may be desirable. However,
when deploying thousands of servers around the world, this mode of operation may
call for the writing of programs/scripts, which can operate on 1000s of servers at a
time. This type of large scale environment would depend on APIs like Redfish.
 What is Redfish?
– It is the first open effort toward building a single API for all hardware. It
replaced IPMI-over-LAN and is a modern, scalable, secure management
API.
– It enables seamless, programmable insight, control, and ultimately
automation of hyperscale architectures.
– A foundational component of Intel Rack Scale Architecture
 Basically, like the CLI, IPMI, and SMNP, Redfish is just another way to
interface with various platforms.
Key benefits include:
 Increased simplicity and usability
 Encrypted connections and heightened security
 A programmatic interface that can easily be controlled through scripts

Redfish
 Ability to meet the Open Compute Project's Remote Machine Management

requirements
 Based on widely used standards for web APIs and data formats
For OpenManage Enterprise and OpenManage - Modular APIs, as

well as additional information on the Redfish resource model, search
Dell.com for the: OpenManage Enterprise and OpenManage
Enterprise - Modular Edition RESTful API Guide.
Alternatively, once on Dell.com, click on the dropdown for Support

and search for Redfish.

Redfish
Redfish Operational Model
Redfish uses JavaScript Object Notation (JSON) and the Open Data Protocol
(Odata). JSON is human-readable and simpler than XML.
Redfish communications are faster in that they can be executed in parallel, in

contrast to IPMI’s communications protocols, which must be executed one-at-a-
time, serially.
Human readable API (Application Program Interface) leads to:

 Easily interpreted scripts and administrator programs
 New programmers writing customized software for server
Important Points: Redfish was introduced with the release of

iDRAC8 but has been enhanced since then.

Redfish
Redfish Tree Structure
The URI is the primary unique identifier of resources. Redfish URIs consist of three
parts as described in RFC3986: Part one defines the scheme and authority of the
URI, part two specifies the root service and version and part three defines a unique
resource identifier.
For example, in the following URI:

https://mgmt.vendor.com/redfish/v1/Systems/SvrID
Redfish Tree Structure/Architect

 RESTful API employed by Redfish is web-based, access is provided using
URIs, which can be typed into a web browser.
 The Redfish API uses a simple folder structure that starts with the Redfish root
at “/redfish/."
 In the case of a PowerEdge server or Compute sled, the root is accessed
through the URI https://<iDRAC IP>/redfish/v1/ - the "v1" at the end of the URI
denotes the version of the API.
The URI is the primary unique identifier of resources. Redfish URIs consist of three
parts as described in RFC3986: Part one defines the scheme and authority of the
URI, part two specifies the root service and version and part three defines a unique
resource identifier.

Redfish
For example, in the following URI:

https://mgmt.vendor.com/redfish/v1/Systems/SvrID
 https://mgmr.vendor.com is the scheme and authority

 /redfish/v1 is the root and version
 /Systems/SvrID is the resource identifier
Note: REST (Representational State Transfer), can be used by

virtually any programming language and relies on a stateless, client-
server protocol, which more often than not, is HTTP.

Redfish
Redfish Operations
 In Redfish, HTTP methods implement the operations of a RESTful API. This

enables the user to specify the type of request being made. It adheres to a
standard CRUD (Create, Retrieve, Update, and Delete) format. Depending on
the desired result, a user can issue the following types of commands:
 GET - View data

 POST - Create resources or use actions
 PATCH - Change one or more properties on a resource
 DELETE - Remove a resource
iDRAC9 supports Redfish 2016 features including:
 BIOS configuration including set attributes, change boot order, enable/disable
boot device state.
 Secure boot and iDRAC configuration.
 Firmware inventory and streamed local updates. To perform a streamed update,
the firmware image must be stored locally on the system where the Redfish
update API is executed.
iDRAC9 includes enhancements to the iDRAC RESTful API for Server

Configuration Profiles (SCP) support and iDRAC configuration including:
 Firmware update via a networked repository during SCP import.

Redfish
 Auto Config, RACADM, WS-Man and Redfish SCP operations via HTTP/HTTPS
in addition to CIFS and NFS.
 SCP operations via local file streaming.
 SCP JSON format for export/import in addition to .XML format.
Important: In the current implementation, HEAD and PUT operations

are not supported for Redfish URIs.
Creation and removal of data is limited depending on the

management characteristics of the resource being targeted.
Generally, viewing and changing settings will be more common.

End of Module
End of Module

End of Module

End of Module

End of Module

MX Installation and Configuration

Initial Setup
Initial Setup
Introduction
Initial Setup

 Setting Initial OME-M IP address
 Steps to connect to KVM
 Using the Left Rack ear LCD Panel
 LED-Only Control Panel
 LCD Control Panel with and without QuickSync
 Setting up KVM
 Chassis Deployment Wizard
 Power-On process
 Configuring I/O Modules in OME-M
 Setting up iDRAC on Compute Sleds
 Identity Pools
 Server Templates

Initial Setup
ESD
Electrostatic discharge (ESD) is a major concern when you handle sensitive

components such as expansion cards, processors, memory DIMMs, and system
board. A slight charge can result in damaging the circuits on these components
causing intermittent problems or shortening the life span.
Due to the increased density of semiconductors used in recent Dell products, the
sensitivity to static damage is now higher than in earlier Dell products. For this
reason, some previously approved methods of handling parts are no longer
applicable.
How to avoid ESD damage:

 Use a wired ESD wrist strap that is correctly grounded.
 Handle all static-sensitive components in a static-safe area. If possible, use
antistatic floor pads and workbench pads.
 Grasp the static-sensitive components by the side and not the top.
 Avoid touching the pins and circuit boards.
 Do not remove components from the antistatic packing material until you are
ready to install the components. Before unwrapping the antistatic packaging, be
sure to discharge static electricity from your body.
 Place the component in an antistatic container or packing before transporting
the components.

Initial Setup
Rack Installation
Weight of Chassis:
Enclosure Minimum Weight Maximum weight (fully

populated)
PowerEdge MX7000 180 lbs. (82 kg) 400 lbs. (182 kg)
Rack installation:
1. Once the Static Rails are installed, with two people, pull the four enclosure
handles on the MX7000 chassis upward, and lift the enclosure.
2. Align the rear of the enclosure with the rails and slide the enclosure into the
rack. When the rear enclosure handles are near the rack flange, press the blue
release buttons on the sides of each enclosure handle, and push the handles
down until it locks into place.
3. Slide the enclosure into the rack until it is firmly seated.
4. Tighten the captive screws on the front panel of the enclosure.
5. Install the sleds, rear modules, power supplies, and fans.

Initial Setup
Removing the enclosure from the rack:

1. As previously mentioned, do not attempt to lift the enclosure by yourself. Dell
recommends that a minimum of two people lift the enclosure.
2. Remove the sleds, rear modules, power supplies, and fans.
3. In order to release the enclosure from the rack, loosen the captive screws on
the front panel of the enclosure.
4. Press the blue release button and pull the enclosure handles.
5. Pull the enclosure until the rear enclosure handles are visible.
6. Press the blue release button and pull the rear enclosure handles.
7. Hold the rear enclosure handles and pull the enclosure from the rail.

Initial Setup
Cabling and Chassis Deployment
Question
How would you normally cable a modular system?
Answer
Question
Would you connect the Management Module (MM) to network cables before you
configure the LCD?

Initial Setup
Answer
Steps to Cable and Deploy an MX7000 chassis:
1. Install the MX7000 chassis in a rack.

2. Install Sleds and I/O modules into chassis.
3. Plug the power cables in.
4. Power the server up by pushing the button on the side of the chassis.
5. Using the LCD panel on the front of the system provide the Management
Module with a static IP Address or configure it for DHCP.
6. Connect to the MM IP Address via web browser.
7. Provide each iDRAC with an IP Address in the MM GUI, or, use the default
setup, DHCP.
8. With the MMs now configured, plug the network cable(s) into the management
module(s).
9. Next, log in through the GUI to the MM and then configure the switches before
connecting the network cables.
10. Power on Sleds and install OS.
Additional discussion notes:

Initial Setup
Important Points: Static rails do not support the ability to service the
enclosure while in the rack and are not compatible with a Cable
Management Arm (CMA). A strain relief bar (SRB) is provided with
each rail kit to organize and secure the power cables exiting the back
of the server.

Initial Setup
Setting the Initial OME-M IP Address

Initial Setup
Steps to Connect to KVM
For the MX7000 chassis front KVM, MM uses a standards-based KVM client to
attach to the iDRAC KVM Server of a requested sled, and then renders the
received video frames within the MMs G200 graphics controller. The MMs USB
host controller receives keyboard/mouse events and redirects them as a virtual
USB keyboard/mouse to the iDRAC of the compute sled.
There is not a KVM selector button (as seen, for example, in the FX2 chassis),
instead it uses an On-Screen Display (OSD) overlay to display a list of the installed
sleds or Chassis Consoles for the user to select from. The MM supports redirecting
the chassis console to the front panel DisplayPort video similar to the M1000e
blade KVM solution. This chassis console is listed as the last node in the OSD for a
user to select it when needed. After the chassis console is selected, the user can
log in to it and execute supported chassis management commands.
KVM can be disabled in the MM GUI through the following options:
 KVM Disable
– KVM is enabled by default, however it can be disabled. If disabled, no KVM
functionality is enabled.
 Chassis Console Disable

Initial Setup
– Enabled by default. If the Chassis console is disabled, it shows disabled in

the OSD and cannot be invoked.
 Individual Sled KVM Disable
– Enabled by default, each iDRAC shall have an attribute to enable or disable

KVM at the front panel. If disabled, that particular sled is not selectable in
the OSD menu.
Important: The sled slot selection is not persistent over MM reset or

reset to factory default. After such an event, none of the sleds will be
selected.
The images shown in this slide depict examples of what you should observe when
connecting to the KVM.
 Steps:
1. Connect to USB keyboard and Mouse, and to display, or DisplayPort dongle.

2. Select from the list/On-Screen Display (OSD) overlay depicting the
available/installed sleds or Chassis Consoles.
Once the KVM display, keyboard and mouse have been connected:

Initial Setup
1. Select the OME-Modular port from the On-Screen Display by clicking with the
mouse or using the arrow keys plus enter.
2. When prompted for credentials, enter the password on the chassis luggage tag
on the front right.
– Default is a random password but calvin can be specified at the time of

order.

Initial Setup
Using the Left Rack Ear LCD Panel
The LED-Only control panel is composed of five Status LEDs, an LED Status Bar,
and a system ID button. The LED behaviors of the five status LEDs are called out
in LED-Only Control Panel slide.
If the customer has the installed LED-Only Control Panel, here are the options to
configure the MX platform:
 Using the Serial Port/USB connector on the controller
 Using the KVM and selecting from the menu in the MM.
– Enter credentials and use the following racadm command to configure:
 racadm setniccfg -s <IPv4Address> <netmask> <IPv4 gateway>
 Through QuickSync when available
The LED Status Bar is illuminated in Blue/Amber based on the chassis state and
behaviors are called out in the following table:
STATUS ID BUTTON (TOP) WIRELESS BUTTON

(BOTTOM)
Healthy Solid Blue OFF
Fault Blink Amber OFF

Initial Setup
Sys ID Blink Blue OFF
Healthy, Wireless ON Solid Blue Solid White
Fault, Wireless ON Blink Amber Solid White
Sys ID, Wireless ON Blink Blue Solid White
Healthy, Wireless Solid Blue Blink White

Communication
Fault, Wireless Communication Blink Amber Blink White
Sys ID, Wireless Communication Blink blue Blink White
Healthy, Wireless fault Solid Blue Blink Amber
Fault, Wireless fault Blink Amber Blink Amber
Sys ID, Wireless fault Blink Blue Blink Amber
When the System ID button is pressed, the identify state alternates between blink
active and inactive. When blink is active, the blue ID LEDs on one of the following
components will blink:
 The chassis LED Status Bar (or circular ID button/indicator on LCD option)
 A sled and associated devices of the same sled
The MM firmware rotates through the installed sleds, moving to the next compute
sled when the button is pressed.

Initial Setup
The optional LCD Panel is a touch screen display (supporting button press, scroll,
and flick gestures), which provides system information, QuickSync 2 buttons,
status, and error messages to indicate if the system is functioning correctly or if the
system needs attention. The LCD is vertically oriented. It supports touch and has
no physical buttons (all user interaction is touch based).
Below the LCD are two status LEDs that light up when the user is not interacting
with the LCD. To extend the life of the LCD backlight, the LCD goes black when not
in use and one of the two LEDs are then turned on. The LCD is not hot-pluggable
and requires the chassis to be fully powered off for servicing.
There are two LED illuminated ICONs located below the LCD (but still part of the
touch panel) which are:
 Activate LCD/Identify/Alert ICON – Circle icon illuminates blue or amber as
follows:
– Solid blue when an alert is not present
– Blinks blue when a chassis Identify is in progress.
– Blinks amber when an alert(s) is/are present.
– When the LCD display area is off, pressing this circled area turns on the
LCD.
 QuickSync 2 Status - White icon is evident when the Bluetooth/WiFi is active. It
is solid when the transceiver is turned on and blinking when there is traffic.

Initial Setup
The Main menu provides access to the LCD functionality. By default, the main
menu is the default home screen, but users can choose an alternative view from
the Welcome wizard or from the Settings page. When users are not interacting with
the LCD, it is turned off. When a user touches the LCD screen while it is off (black),
the LCD turns on.
On initial power up an animation indicating that the LCD is starting is shown until
the chassis manager has initialized and can then respond to user touches. If the
LCD loses communication with the MM, it displays a message on the LCD
indicating it is unable to communicate with the Management Module.
 The MX7000 optional LCD fulfills four primary use cases:
1. Initial deployment (IP, subnet, DHCP, MAC address, etc.)

2. Fault/warning display
3. QuickSync 2
4. Service Interaction
LCD initial setup:
1. A user local to the system will be presented with the Select Language screen on
the LCD
2. The user selects the language
3. The user is presented with a Select Home Page menu, when they select the
style of home page they want to be presented on the LCD (Main Menu, IP,
System Info, etc.).
4. A preview of the home page is displayed, and the user can either choose Save
at the bottom of the page which sets it as the home page, or they can choose "<
Home Pages" at the top of the page which will return them to the selection page
5. The new Home Page is now displayed
The steps to view the (DHCP) IP address are:

1. Select Settings from the main menu.

Initial Setup
2. Select Network Settings. A list is displayed showing the IPv4 address, IPv6
address, DNS name, MAC Address, and VLAN ID of the chassis controller
(MSM)
3. Click the IPv4 address to show the current values for the extended IPv4
network settings. The IPv6, VLAN ID, and DNS name are not selectable.
a. IPv4 address
b. Subnet mask
c. Gateway
Optionally, if a static IP address is desired:
 Select Settings from the main menu
 Select Network Settings
 Next, select Edit (Note: Edit is at the bottom of the network settings list if the
LCD is in View and Modify mode)
 IPv4
 Select Static IP
 While changing from a DHCP address, confirm the change. The IP address
configuration area is displayed
– Select the first through fourth octet one at a time and configure the value
 Select Next
 The Subnet Mask configuration area is shown next
– Select the first through fourth octet one at a time and configure the values
 Select Next
 The gateway address configuration area is shown
– Select the first through fourth octet one at a time and configure the value
 Select Save.
 The IP, subnet mask, and gateway address values are validated.
a. If invalid, an error message is shown to the user
b. If closed, you are returned to the network Settings page
c. Navigate back to correct the error as needed

Initial Setup
 If all values are valid, a wait animation is shown while the network is being
updated
 You are returned to the network Settings screen showing the updated current
values

Initial Setup
Simulation Activity
In this simulation, you walk through the steps to use the Left Rack Ear LCD Panel.
Inform the instructor, once you have completed the Using the Left Rack Ear LCD
Panel Simulation Activity.

Initial Setup
Available RACADM Commands
The subset of RACADM commands available can be displayed via racadm help.
To set a static IPV4 address the command: racadm setniccfg would be used.
Alternatively, a serial interface can be used by connecting the micro USB port on
the rear of the management module to a laptop and opening a terminal emulator
with settings of 115200 baud, 8 data bits, and 1 stop bit.
The initial screen would then enable selection of a login to the management
module, networking I/O modules, or servers (iDRAC). Selecting the management
module would prompt for the credentials on the luggage tag and provide the same
RACADM command options as via the KVM.
Important: It is possible to disable all the interfaces to the chassis

including LCD, web UI, remote RACADM, SSH, and KVM but NOT
the serial interface leaving it the “last resort” recovery access point.

Initial Setup
Chassis Deployment Wizard
When first logging into the web UI, a getting started wizard is displayed for initial
chassis configuration. It is always possible to restart this wizard by selecting Initial
Configuration in the actions bar.
1. Import Profile or join multi-chassis group

2. Select Time Configuration and then select Configure Time Settings.

Initial Setup
Important: If multiple NTP servers are selected, OME-M will

algorithmically select the one to use.
3. Activity and Alerts

– Similar to iDRAC these would be the SNMP and email configuration options.
4. iDRAC - Configure iDRAC and IOM Quick Deploy Settings
– This page specifies the root password and network configuration to use
when a server or IOM is inserted into the chassis.
5. Network I/O Module Configuration

Initial Setup
Important: OME-M will periodically check for updates to the “Newest

validated stacks” and “Latest component firmware versions.” If a new
version is available it will update its copy, run compliance against
baselines that use it and generate an alert if critical or warning
updates are available.
Alternatively, Dell Repository Manager can be used to create a

custom repository that can be mounted on a share to create a
catalog.

Initial Setup
7. Group Definition
 If multiple chassis are wired together before running the Getting Started
Wizard (GSW), a list of available chassis to add to the group is displayed.
After one or more chassis are wired together, you can log in to the chassis
that you want to promote to lead.
The wizard then prompts to name the group plus how and which settings to
replicate from the lead to the members. The recommendation is to
automatically propagate all settings.
Important: Credentials are not required to join as the connection is

validated with factory install certificates.

Initial Setup
Simulation
In this simulation, you walk through the steps to deploy the chassis using the
Chassis Deployment Wizard.
Inform the instructor, once you have completed the Chassis Deployment Wizard
Simulation Activity.

Initial Setup
Power on Sequence
 Fabric Resource Director.
 The MM communicates with IOMs, compute, and storage sleds via the
Fabric Resource Director (FReD).
 FReD is a microcontroller-based reference circuit that provides a consistent
management interface between the MM and the other hardware.
 FReD communicates with MM through an I2C connection.
 FReD provides the following functions:

Initial Setup
– Power state transitions.

– Sensor telemetry acquisition.
– Critical temperature monitoring.
– Failure state behaviors.
– Module LED control.
– Serial connectivity from MM.
– MM <> IOM/Sled message transfer via mailbox mechanism.
 When the MM firmware is updated, the FReD microcode on the IOM/Sled is
updated simultaneously.
– This is automatic and does not require a reboot of the IOM/Sled.

Initial Setup
Configuring I/O Modules
Configuring the IP address for the I/O module:
DHCP:
DHCP is the default IP address mode. When using DHCP, the switch obtains its IP
address, subnet mask, and default gateway from the DHCP server. The DHCP
client can only connect to a DHCP server that is on the same subnet as the switch.
If your DHCP server is not on the same subnet as the switch, use a static IP
address.
Static:
Dell EMC recommends configuring the switch module through the Compute Sled
chassis management GUI, namely, Dell EMC OpenManage Enterprise - Modular.
Use this management GUI to configure the switch IP address and perform
advanced configuration tasks. By default, the IP address mode is set to DHCP. Use
this GUI to assign a static IP address. To configure the FC switch module, follow
these steps:
1. Log in to OME-M
2. Select Devices > I/O modules
– The I/O modules display.
3. Select the I/O module name, IP address, or View Details on the right side of
the display.

Initial Setup
– The Overview tab is displayed. Select the Settings tab.

4. Click and expand the Network option. Select the required check boxes, and
complete the IPv4 or IPv6 settings, DNS server settings, and optionally the
management VLAN ID. Click Apply.
5. Click and expand the Management option. Specify a hostname and root
password. Click Apply.
6. Click and expand the Monitoring option. Select the Enable SNMP check box,
the SNMP version, and enter a read community string. Click Apply.
7. Click and expand the Advanced Settings option. Select the needed check
boxes.
 Replicate Time Configuration from Chassis
 Replicate Alert Destination Configuration from Chassis
8. After clicking Apply, you have configured the IP address and other settings for
the switch module.
Tip: The Fibre Channel Switch Module was used as an example. For
additional information on how to configure a specific switch, and/or
how to configure via Serial connection, refer to the switch model
Installation guide for further instruction.

Initial Setup
Simulation Activity
In this simulation, you walk through the steps to configure the I/O Module in OME-
M.
Inform the instructor, once you have completed the Configuring I/O Modules in
OME-M Simulation Activity.

Initial Setup
Setting Up iDRAC on Compute Sleds

Initial Setup
Simulation Activity
In this simulation, you walk through the steps to configure iDRAC.
Inform the instructor, once you have completed the Setting Up iDRAC Simulation
Activity.

Initial Setup
Identity Pools
Identity Pools
Identity pools are used for template-based deployment on servers to virtualize the
network identity for: Ethernet, iSCSI, FCoE, and FC.
You can create a maximum of 5000 Identity Pools in each of these categories.
The server deployment process fetches the next available identity from the pool
and uses while providing a server from the template description. You can then
migrate the profile from one server to another without losing access to the network
or storage resources in your environment.
You can edit the number of entries in the pool. However, you cannot reduce the
number of entries less than those assigned or reserved. You can also delete the
entries that are not assigned or reserved.
You can perform the following tasks on the Identity Pools page:
 Create Identity pools

 Edit Identity pools
 Delete Identity pools
 Export Identity pools - Exports the selected identity pool in .csv format to the
specified location.

Initial Setup
Important: You must have template management privileges to

manage Identity Pools.

Initial Setup
Create Server Templates
Server Templates are created and deployed after fabrics are created and deployed
to the fabric IOMs to provide connectivity between servers and fabrics.
To create a template, from OME Modular click Home > Configuration > Deploy >
Create Template.
The deploy feature in OpenManage Enterprise Modular enables you to create

server templates and apply them to compute sled or slot. A server template
contains a set of parameters that is extracted from a server and replicated quickly
in multiple servers.
The templates can contain settings for the following categories:
 Local access configuration

 Local configuration
 Power configuration
 Chassis network configuration

Initial Setup
 Slot configuration
 Setup configuration - The setup configuration covers Quick Deploy, Security,
Network Services. Customer Network Access, and Multi-Chassis Management.
 You can also perform the following tasks on the Deploy page:
– Create templates
– Edit templates
– Clone templates
– Export templates
– Delete templates
– Edit network
– Deploy templates
– View template details - Select the check box corresponding to a template
from the list to see View Details on the right side. Click View Details to view
the Template Details page.
Important: You must have the template management privileges to

create templates. Also, you must have the chassis administrator role
with device configuration privilege to deploy templates on chassis,
perform backup and restore operation on chassis, and configure the
auto-configuration settings.

Initial Setup
Lab Activity
You will now complete the PowerEdge Server Templates lab exercise.

Compute Sleds
Compute Sleds

Compute Sleds
Compute Sled Locations
1 2 3 4 5
11 10 9 8 7 6
MX740c:
1. Backplane
2. Backplane cable
3. Processor 1 (heatsink)
4. Processor 2 (heatsink)
5. Mezzanine card A1
6. Power connector
7. Mezzanine card B1
8. Mini Mezzanine connector

Compute Sleds
9. iDRAC card
10. BOSS connector
11. PERC connector
6 7 8 9 10 11 12 13
14
4 15
3
16
2
17
1
18
19
27 26 25 24 23 22 21 20
MX840c:
1. Memory module sockets
2. TPM module connector
3. SATA connector

Compute Sleds
4. BBU power connector

5. USB connector
6. PWRD_EN = System configuration jumper (enabling or disabling the password
settings.)
7. NVRAM_CLR = System configuration jumper (retaining/clearing configuration
settings.)
8. U__USB_OC_FRONT = FIO module connector
9. J_BP_SIG = Backplane signal connector
10. Processor 1
13. J_MEZZ_A1 (CPU1) = Mezzanine card (Fabric A1 card) connector
14. Processor 2
15. UPI cable connector
16. USB 3.0 = USB 3.0 module
17. J_MEZZ_B1 (CPU2) = Mezzanine card (Fabric B1 card) connector
19. System power connector
20. J_MINI_MEZZ_C1 (CPU2) = Mini mezzanine card (Fabric C1 card) connector
21. iDRAC module connector
22. AUX 1 cable connector
23. AUX 2 cable connector
24. BATTERY = System battery
25. BOSS_MODULE = BOSS module connector
26. PERC (CPU1) = PERC card connector
27. Backplane power connector

Compute Sleds
Simulation Activity
In this simulation, you walk through the MX840c platform.
Inform the instructor, once you have completed the MX840c 3D Simulation
Activity.

Compute Sleds
Supported Operating Systems
For an updated version of supported Operating Systems for the MX-Series

platform, refer to www.dellemc.com.

Compute Sleds
White Board Activity - Operating Systems Overview
Discuss OS Installation. Be aware that there are no differences when installing the
OS on the MX7000 platform, as compared to previous 14G modular platforms.
To refresh the participants' memories and to revisit the OS installation procedures,

start the discussion by asking the questions during the next slide, "OS Installation
Review."
Poll the class to ask how many of them have installed an OS on a 14G modular
platform. (Take note of those who have minimal to no experience with hands on OS
installation knowledge.) Take this opportunity to share your OS installation
experience and ask participants to share any good to know, or gotchas that they've
experienced when installing the OS and how they found a work around to their
potential issue (if applicable).
Discussion points: (Guide the participant, but do not provide answers. Also,
participant should be aware of when to use the following, that is, remote or local
install).
Question
When reloading the OS what methods can be used? Ask participants to provide the
answer.

Compute Sleds
Answer
Question
What iDRAC9 licensing changes have been made that affect how the OS is
installed?
Answer

End of Module
End of Module

End of Module

End of Module

End of Module

MX Networking

MX Networking Hardware

Open Networking Capability
MX networking overview:
 MX Ethernet switches are capable of running third-party network operating

systems.
 When running third-party network operating systems, OME-M will only provide
basic hardware health and power control.
 All switch configuration and operations are performed via the third-party
management tools.
Important: Open Networking allows flexibility to incorporate and

switch network operating systems.

MX Switch Hardware Naming Convention
 MX - Modular Series:
 9 = Speed/Type.
 1 = Generation.
 16 = Number of internal ports.
 n = Device Type.
 PowerEdge naming convention:
 C = Cloud - Modular server nodes for hyper-scale environments.

 F = Flexible - Hybrid rack-based sleds for rack-based FX2/FX2s enclosure.
 M/MX = Modular - Servers and other items for the MX7000, M1000e and/or
VRTX enclosure.
 R = Rack-mountable servers.
 T = Tower Servers.
 For three number naming conventions:
1. The first number after the letter indicates the class of the system:
 1-3 being 1 CPU systems.
 4-7 are 2 CPU systems.

 8 can be 2 or 4 CPUs.
 9 is 4 CPUs.
2. The second number indicates the generation:
 0 for 10th generation.
 1 for 11th generation and so on.
3. The third number indicates the make of the CPU:
 0 for Intel.
 5 for AMD.

I/O Modules Connectivity - Chassis Slot Assignments
The illustration in this slide shows the chassis slot assignments:
 The server sleds are installed vertically on the front of the chassis and I/O
modules (IOMs) are installed horizontally on the back of the chassis.
 The connector on the mezzanine card attaches directly to the connector on the
IOM installed at Fabric A and B. No midplane is required.
 The server sleds have two mezzanine cards: One for the A fabric and one for
the B fabric.
 Each mezzanine card has two connectors: The A mezzanine card has
connections to the A1 and A2 fabrics and the B mezzanine card has
connections to the B1 and B2 fabrics.

I/O Module Interfaces

What are the supported I/O Modules for the following Fabrics?
Fabric A and B
1. Ethernet Switch:
2. Fabric Switching Engine:
3. Fabric Expander Module:
Fabric C
1. 12 Gb SAS:
2. Fibre Channel:

MX5108n Ethernet Switch
MX5108n Ethernet switch:
 8 x 25 GbE server-facing ports, 2 x 100 GbE uplink ports, 1 x 40 GbE port and 4
x 10GBase-T ports.
 Supported in Fabrics A & B.
 Supports all PowerEdge MX Ethernet Cards.
 Supports Open Networking, Full Switch, and SmartFabric operating modes.
 < 800 ns latency, 960 Gbps switching fabric.
 NVMe over Fabric Ready.

MX9116n Fabric Switching Engine
MX9116n Fabric Switching Engine:
 16 x 25 GbE server-facing ports, 2 x 100 GbE Uplink ports, 2 x 100GbE/8 x 32G

FC uplink ports, and 12 x Fabric Expansion/Uplink ports.
 Supported in Fabrics A & B.
 Supports all PowerEdge MX Ethernet Cards.
 Supports Open Networking, Full Switch, and SmartFabric operating modes.
 < 450 ns latency, 6.4 Tbps switching fabric.
 NVMe over Fabric Ready.

MX7116n Fabric Expander Module
The MX7116n Fabric Expander Module (FEM):
 16 x 25 GbE server-facing ports, 2x Fabric Expansion ports back to FSE.

 No switching ASIC (application-specific integrated circuit) and no Operating
System means sub ~75 ns latency. All switching done at FSE.
 No port to port oversubscription.
 Nothing to manage, no firmware to update.
Important: The combination of the MX7116n FEM and the MX9116n

Fabric Switching Engine (FSE) is what makes MX Scalable Fabric
Architecture possible.

Pass-Through Modules
Key Capabilities:
 16 x 25 GbE server-facing ports.

 16 x SFP28 or 10G-BaseT external ports.
 Supported in Fabrics A & B
 Supports all PowerEdge MX Ethernet Cards
Why do we use a pass-through module over a basic switch?

Cable types
Key features:
 Direct Attach Cable (DAC) with optical connectors on the end—also called
Twinax. FEC Error Correction adds latency, so these cables are supported at
short lengths of two meters and under.
 Active Optical Cable (AOC) is a fiber optic cable that is optimized for the length
of the cable. AOC is more expensive than DAC but less expensive than
standard transceivers and passive fiber.
 Multi-Mode Fiber (MMF) 50-um Large core fiber 100 m (300 m) reach. Easy to
attach components. Transceivers are low cost. Fiber 3x the cost of SMF.
 Single-Mode Fiber (SMF) 9-um Tiny core fiber. 2/10Km reach. Hard to attach
components. Transceivers are expensive. SMF costs less than dental floss.

Optic Connectors
Optic connectors:
 SFP - Small Form Factor Pluggable

1 Channel
2 Fibers or wires
1-1.5 W
Duplex LC optical connector
MMF or SMF
 SFP = 1G
 SFP+ = 10G
 SFP28 = 25G
 QSFP - Quad Small FormFactor Pluggable
4 Channels
8 Fibers or wires
3.5W-5W
MPO12 8 fiber parallel optical connector
 QSFP+ = 40G
 QSFP28 = 100G
 QSFP-DD

Introduction to QSFP28-DD
Key features:
 QSFP28-DD ports are backwards compatible and support QSFP28 and QSFP+
Optics and cables.
 QSFP28 ports are backwards compatible and support QSFP+ Optics and
cables.
 QSFP28-DD ports are slightly deeper than QSFP28 with a second row of
contacts.

Switch Management Overview

Full-Switch Mode
Full-switch mode (default mode) allows full network customization using

Lifecycle Management Integrated with OME-Modular:
 Configure hostname, SNMP, ports, and so forth.

 Monitor health, logs, alerts, and events.
 Update/manage firmware.
 View physical topology.
 Power management.

Smart Fabric Services Mode
OS10 Enterprise Edition SmartFabric Services for Compute provides:
 Automation for the data center:

– Plug-and-play fabric deployment.
– Network simplification with I/O Aggregation.
– Single pane of glass to manage all switches in the fabric like a single logical
chassis.
 Lifecycle Management:
– Fabric-wide firmware upgrade scheduling and configuration consistency

enforcement.
– Automated or user enforced rollback to last well-known state.

Network Switch Management Methods
Topology validation and self-healing fabric:
 Automatically detects fabric misconfigurations or link level failure conditions.

 Automatically heals the fabric on failure condition removal.
 Ensures compliance with selected physical topology.
 VLAN and Priority assignments drive automated QoS operations.
 Dynamically adjusts bandwidth across all inter-switch links in the event of a link
failure.

Scalable Fabric Architecture

Overview
Scalable Fabric Architecture makes the MX9116n Fabric Switching Engine (FSE)
available to all chassis in the system by linking it to MX7116n Fabric Expander
Modules (FEM).
MX7116n Fabric Expander Modules key features:
 No switching happens in FEM.

 No operating system.
 Low latency.

Topology Architecture
Each Fabric Switching Engine can support up to 9 Fabric Expander Modules in

other chassis.
What is the recommended FSE placement (split across two chassis for
redundancy)?
Choose one configuration and tick the best answer.
Configuration 1: __________
Slot A1: FSE

Chassis 1
Slot A2: FEM
Slot A1: FEM

Chassis 2
Slot A2: FSE

Configuration 2: __________
Slot A1: FSE

Chassis 1
Slot A2: FSE
Slot A1: FEM

Chassis 2
Slot A2: FEM

Chassis Groups and Fabrics
 Scalable Fabric Architecture enables the MX9116n to seamlessly support up to

80 MX compute sleds and 10 MX7000 chassis via the ultralow latency
MX7116n Fabric Expander Module.
 Multiple fabrics are possible within one multichassis management group.
 Management of all servers and fabrics is done from the lead chassis.

Initial Switch Setup

Minimum Requirements
Important: Scalable Fabric Architecture will be unsupported if pairing

is done within the same chassis.

Making the Initial Connection

Deploy a SmartFabric
Define VLANs:
To define VLANs using the OME-M console, perform the following steps:
1. Open the OME-M console.
2. From the navigation menu, click Configuration > Networks.
3. In the Network pane, click Define.
4. In the Define Network window, complete the following:
a. Enter VLAN0010 in the Name box.

b. Optionally, enter a description in the Description box.
c. Enter 10 in the VLAN ID box.
d. From the Network Type list, select General Purpose (Bronze).
e. Click Finish.
Create the SmartFabric:
To define VLANs using the OME-M console, perform the following steps:
2. From the navigation menu, click Devices > Fabric.

3. In the Fabric pane, click Add Fabric.

4. In the Create Fabric window, complete the following:
a.
Enter SmartFabric in the Name box.
c. From the Design Type list, select 2x MX9116n Fabric Switching Engine in
different chassis.
d. From the Chassis-X list, select the first MX7000 chassis.
e. From the Switch-A list, select Slot-IOM-A1.
f. From the Chassis-Y list, select the second MX7000 chassis to join the
fabric.
g. From the Switch-B list, select Slot-IOM-A2.
h. On the Summary page, verify the proposed configuration and click Finish.
Define uplinks:
After initial deployment, the new fabric shows Uplink Count as ‘zero’ and shows a
warning sign. The lack of a fabric uplink results in a failed health check. To create
uplinks, follow these steps:
2. From the navigation menu, click Devices > Fabric.
3. Click on the fabric name, SmartFabric.
4. In the Fabric Details pane, click Uplinks.
5. Click on the Add Uplinks button.
6. In the Add Uplink window complete the following:
a. Enter Uplink01 in the Name box.
c. From the Uplink Type list, select Ethernet.
d. Click Next.
e. From the Switch Ports list, select Ethernet 1/1/41 and Ethernet 1/1/42 for
both MX9116n FSEs.
f. From the Tagged Networks list, select VLAN0010.

g. From the Untagged Network list, select VLAN0001.

h. Click Finish
7. At this point, SmartFabric creates the uplink object and the status for the fabric
changes to OK.

Switch Firmware

Switch firmware overview and update methods.

Connecting to UpStream Switches

Traditional Mixed modular/rack Networking
Traditionally, mixed modular/rack networking can be complex and introduce

latency:
 Rack servers connect to their own Top of Rack (ToR) switches.
 Communication between rack and modular servers must traverse the core.
 The storage array consumes expensive core switch ports.
 This design adds significant cost for multiple switches and latency.

Embedded Top of Rack (ToR)
With embedded ToR, there is a greatly simplified structure:

 Storage and all servers, including modular server and rack, connect to the FSE.
 Use all 10 Gb or all 25-Gb breakout cables to connect to non-MX devices.
 Communication between all devices is kept within the FSE.
 The FSE serves as the single point of management and network security.
 All elements are talking to iDRAC, so alerts come in through OMM.
 This simplification reduces cost and improves performance and latency.

MX SmartFabric Deployment

MX SmartFabric Deployment Video Introduction
Overview
The instructor plays a video of MX7000 SmartFabric deployment and describes

each step as the video plays.
Video Demonstration Overview
This demonstration covers the steps to deploy SmartFabric Services on the

PowerEdge MX platform.
The videos cover all aspects of SmartFabric deployment, including:
The video demonstrates, in the correct order, the steps to deploy SmartFabric
Services:
1. MX7000 chassis access configuration.
2. OME-M Chassis Deployment Wizard
3. I/O Module settings
4. Creating a Multi-Chassis Management group
5. Z9100-ON Leaf Switch configuration
6. Defining VLANs in OME-M
7. Creating a SmartFabric

8. Defining SmartFabric Uplinks

9. Creating a Server Template
10. Associating the Server Template with a network
11. Deploying a Server Template
12. Validating the Z9100-ON Leaf Switch configuration
13. Validating the MX9116n configuration
14. Cisco Nexus 3232C Leaf Switch Configuration
Only the specific Compute Sleds, I/O modules, and switches used in the demo are
shown and described in the video. The equipment used includes:
 Dell EMC PowerEdge MX7000 Modular Chassis
 Dell EMC PowerEdge MX740c compute sled
 Dell EMC MX9116n Fabric Switching Engine (FSE)
 Dell EMC MX7116 Fabric Expander Module (FEM)
 Dell EMC MX9002m Management Module
 Two different Leaf Switch examples:
 Dell EMC Networking Z9100-ON switches

 Cisco Nexus 3232C switches

SmartFabric Deployment - Introduction
 A Scalable Fabric spans multiple chassis and allows them to behave like a
single chassis from a networking perspective.
 A Multi-Chassis Management group allows multiple chassis to be managed like
a single chassis.

 A Scalable Fabric consists of two main components, a pair of MX9116n FSEs in

the first two chassis and additional pairs of MX7116n FEMs in the remaining
chassis.
 Up to ten MX7000 chassis can be connected in a single Scalable Fabric
Architecture.
Click to play video: Scalable Fabric Architecture
Compute and I/O Modules Used In Deployment Example
In this deployment example, we use:

 Four MX740c, each containing single Intel mezzanine cards in slots 1 and 2 of
both chassis.
 Two MX9116n, one in each chassis.
 Two MX7116n, one in each chassis.
Click to play video: Compute and I/O Modules Used In Deployment Example

Leaf Switch Options Used in Deployment Example
In this deployment, two leaf switch options are used:

 The first option includes one pair of Dell EMC Networking Z9100-ON switches.
 The second option includes one pair of Cisco Nexus 3232C Leaf switches.
Click to play video: Leaf Switch Options Used in Deployment Example
Inter-Chassis Cabling

The inter-chassis cabling used in this deployment example is shown in this figure.
Additional inter-chassis cabling provides redundancy to all available MMs in both of
the chassis.
C1MM1 Port GB2 connects to C2MM1 Port GB1.
Using the S3048-ON switch, Port GB1 of C1MM1 and C2MM2, connect to the
management network.
Click to play video: Inter-Chassis Cabling

SmartFabric Deployment - MX7000 Initial Configuration
Out of Box Configuration Using LCD Panel
The initial out of box configuration uses the LCD screen to assist in setting up the
chassis for access.
To activate the LCD touchscreen, gently tap the screen.
Select the Language.
After selecting the language, tap Main Menu, and then Save.

Assign a static IP address, subnet mask, and default gateway.
Click to play video: Out of Box Configuration Using LCD Panel
Log in to OME-M Console
Using the IP address, access the OME-M login screen.
For the user name, enter root.
Use the password printed on the Express Service Tag, which is located on the front
right side of the chassis.
Click to play video: Log in to OME-M Console

Chassis Deployment Wizard
The Chassis Deployment Wizard can be used to perform initial configurations and
set up.
To access Chassis Deployment Wizard, click Configure, Initial Configuration.
Various configurations such as setting the time, importing a profile, network IOM
access settings, and so on can be completed using this wizard.
Click to play video: Chassis Deployment Wizard
Manage I/O Modules

To manage the I/O module, click Devices, then I/O Module.
To change settings, select IOM, then click Settings.
From the Settings screen, choose Network, then click to remove the check from the
Enable DHCP option.
In the required fields, enter the IP address, Subnet mask, and Gateway, then click
Apply.
Click to play video: Manage I/O Modules

Deployment Overview
SmartFabric Deployment
In this section, two short videos cover:
 SmartFabric Description.
 List the SmartFabric deployment steps.
Deployment Steps

Click to play video: Deployment Steps

Create Multi-Chassis Management Group
Create Multi-Chassis Management Group
 This video shows the steps to create a Mult-Chassis Management (MCM) group
using OpenManage Enterprise Modular (OME-M).
Click to play video: Create Multi-Chassis Management Group

Topology: Dell EMC Networking Z9100-ON Leaf Switches
Topology: Dell EMC Networking Z9100-ON Leaf Switches with

PowerEdge MX7000 Chassis
This video shows all the required networking connections to create a SmartFabric:
 Compute sleds to MX9116n
 VLTi connection between Chassis 1 MX9116n and Chassis 2 MX9116n
 Connection between MX7116n Fabric Expansion Modules and MX9116n Fabric
Switching Engines
 Connections between both MX9116n Fabric Switching Engines and Z9100-ON
Leaf Switches
 VLT connection Between Leaf Switch 1 and Leaf Switch 2
Click to play video: Create Multi-Chassis Management Group

Z9100-ON Leaf-1 Command Line Interface
This video reviews the configuration of the Z9100-ON Leaf Switches using the
Command Line Interface.
 Leaf Switch configuration is not part of the MX7000 ICM class. Several switch
models are supported. This information is provided to show one example of
Leaf Switch configuration.
 Leaf Switch configuration is necessary to support the SmartFabric Uplinks.
 To support the uplinks, VLT connections between the Leaf Switches and the
MX9116n FES in each chassis must be prepared.
 VLANs on the Leaf Switches must match the VLANs created on the
SmartFabric.
 For detailed steps to configure Leaf Switches, refer to the Dell EMC PowerEdge
MX SmartFabric Deployment Guide.
Click to play video: Z9100-ON Leaf-1 Command Line Interface

Define VLANs in OME-M
Define VLANs in OME-M
This video shows how to create VLANs in OME-M

 Define VLAN 1 and VLAN 10 in OME-M.
 Set the VLAN name, description, VLAN ID, and Network type for each VLAN.
Click to play video: Define VLANs in OME-M

Create the SmartFabric
Create the SmartFabric
This video shows how to create a SmartFabric in OME-M.
Click to play video: Create the SmartFabric

Define Uplinks
Define Uplinks
 The SmartFabric requires at least one uplink to connect to resources outside of

the MCM Group.
 The uplink connects the MX9116n in each chassis to the Leaf Switches.
 This video demonstrates how to create uplinks for the SmartFabric.
Click to play video: Define Uplinks

Create a Server Template
Create a Server Template
This video demonstrates how to create a Server Template.

 A server template contains a set of parameters that are extracted from a server
and replicated quickly to multiple servers.
Click to play video: Create a Server Template

Associate Template with a Network
Associate Template with a Network
 This video demonstrates how to add VLANs to the server template.

 Add the VLANs to each mezzanine card.
Click to play video: Associate Template with a Network

Deploy Server Template
Deploy Server Template
This video demonstrates how to deploy the server template.
Click to play video: Deploy Server Template

This video shows the commands used to validate the Z9100-ON Leaf Switch
configuration.
Click to play video: Z9100-ON Leaf-1 Command Line Interface

MX9116n FSE on Chassis 1
MX9116n FSE on Chassis 1
This video shows the commands used to validate MX9116n switch operating in
SmartFabric mode.
Click to play video: MX9116n FSE on Chassis 1

Topology: Cisco 100 GbE Leaf Switches with MX7000

Chassis
Topology: Cisco 100 GbE Leaf Switches with MX7000 Chassis
 This section discusses how to deploy SmartFabric using a different Leaf Switch
option.
 In this example, two Cisco Nexus 3232C switches are used as Leaf Switches.
 For additional information, see the Dell EMC PowerEdge MX SmartFabric
Deployment Guide.
Click to play video: Topology: Cisco 100 GbE Leaf Switches with MX7000 Chassis

Cisco 100 GbE Leaf-1 Command Line Interface
Cisco 100 GbE Leaf-1 Command Line Interface
This section reviews the configurations on the Cisco 100 GbE leaf switch.
Click to play video: Cisco 100 GbE Leaf-1 Command Line Interface

Create a SmartFabric and Define Uplinks, Summary
Create a SmartFabric and Define Uplinks, Summary
In this demonstration, the deployment of SmartFabric on Dell EMC PowerEdge MX

platform was carried out using the Dell EMC Z9100-ON, and Cisco 100 GbE leaf
switch options.
Click to play video: Create a SmartFabric and Define Uplinks, Summary

Configuring Switch Features (Simulation Activity)

Initial Setup Practice
In this lab simulation, perform a switch setup walk-through.

Locating Switch Information in OME-M
In this simulation, practice using OME-M to locate switch and port information.

Configuring Networking Information in OME-M
In this simulation, practice using OMEM to configure breakout, VLANs, and Identity
Pools.

Create Chassis Groups in OME-M
In this simulation, practice using OME-M to create a chassis group and review
chassis group information.

Create a Fabric and Uplink in OME-M
In this simulation, practice using OME-M to create a fabric and add an uplink to the
fabric.

End of Module
End of Module

End of Module

End of Module

End of Module

MX Troubleshooting
MX Troubleshooting

Minimum Configuration and Population Rules

MX7000 Minimum Configuration to POST

Maximum Population Rules
Category Maximum Population
Blanks All Empty slots in the MX7000 chassis must be populated with
blanks (Sled, IOM, EC, and PSU). Required for proper cooling of
the chassis and components.
Fans All system fans must be populated in the chassis.
Power The number of power supplies that are required is dependent upon
Supply system configuration and wanted redundancy mode. The minimum
quantity must be 2. For grid redundancy, power supplies must be
populated in the following order: 1+1 (1 and 4), 2+2 (1, 2 and 4, 5)
3+3 (all slots)
Power Cords One C21/C20 power cable must be connected to the C22 plug
corresponding to each populated PSU.

Management At least 1 Management Module must be present to control and

Module manage the chassis.
NOTE: If a single MM fails, the system continues to run. However, it

cannot be managed or controlled until the MM is replaced with a
functional module.
Control Right Control Panel is always required, and at least one left control
Panel panel option (LCD or LED) must be present on the MX7000 chassis.
 Intel's OPA (Omni-Path Architecture) provides:
 Features and functionality at both the host and fabric levels to greatly raise
levels of scaling.
 CPU and fabric integration necessary for the increased computing density,
improved reliability, reduced power, and lower costs required by significantly
larger HPC deployments.
 Fabric tools to readily install, verify, and manage fabrics at this level of
complexity.

Maximum Population Rules (cont.)
Category Maximum Population
Compute Maximum number of 8 single wide or 4 double wide sleds or

Sled combination can be populated. No restriction on order other than
double wide sleds must be in slots 1, 3, 5, 7 due to structure.
Storage Storage sled can be populated in any sled slot. Requires at least one
Sled compute node in the chassis to map storage to. Also requires at least
1 Fabric-C SAS IOM to be present and powered ON.
I/O Module Brocade and SAS IOM only supported in Fabric-C.

Only one type of IOM can be offered in Fabric-C (Fibre Channel or
SAS IOM, not mixed).
Infiniband and OPA only supported in Fabric-B.
Only 1 type of switch can be offered in Fabric-B (HPCC or Ethernet).
At least one Fabric-C SAS IOM must be installed if the chassis
contains a Storage Node.
Mix Speed of pass-through in same fabric is not enabled.

Mezzanine If the chassis contains a storage node, Fabric-C Mini-Mezzanine card

cards (HBA330 or Jumbo PERC) must be installed in at least one compute
node.
Dual Port mezzanine cards must be installed for redundant
IOM/Pass-through configurations.
2nd processor must be installed on the compute node to support
Fabric-B Mezzanine/IOM and Fabric-C Mezzanine/IOM.

MX740c/MX840c Minimum Configuration to POST

Alerts and Logs
Alerts and Logs
Alerts and Logs

Alerts and Logs
Health Status and Alerts

Alerts and Logs
LCD Alerts
Located below the LCD are two LEDs that light up when the user is not interacting
with the LCD. To extend the life of the LCD backlight, the LCD goes black when not
in use and one of the two LEDs are then turned on.
The LCD is not hot-pluggable and requires the chassis to be powered off for
servicing.
 The two LEDs illuminated ICONs located below the LCD are Activate
LCD/Identify/Alert ICON and Circle ICON, which are still part of the touch panel.
 Circle icon illuminates blue or amber as follows:
 Solid blue when an Alert is not present.

 Blinks blue when a chassis Identify is in progress.

Alerts and Logs
Degraded Health Status Messages

Alerts and Logs
Example – Predictive Failure Detected

Alerts and Logs
Displayed Errors

Alerts and Logs
Hardware Error Logs
 The Hardware Logs tab displays the hardware log of the chassis and the
components in the chassis. You can sort the logs that are based on the
following parameters:
 Severity
 Message ID
 Start date
 End Date
 Category
You can export all the logs or selected logs as an excel sheet or file containing
comma-separated values (.csv) to your local system or network drive.
 Severity
 Displays the severity of the hardware log. The possible options are:
 All
 Warning
 Unknown
 Critical
 Fatal

Alerts and Logs
 Info
Message ID
Displays the identification number of the message.
Timestamp
Displays the date and time of generation of the hardware log.
 Category
 Displays the severity of the hardware log. The possible options are:
 All
 Audit
 Configuration
 Storage
 System Health
 Updates
 Work Notes
Description
Displays the description of the hardware log.

Alerts and Logs
SupportAssist Collection
SupportAssist Enterprise is an application that automates technical support for your

Dell server, storage, and networking devices. SupportAssist Enterprise monitors
your devices and proactively detects hardware issues that may occur.
When a hardware issue is detected, SupportAssist Enterprise automatically opens
a support case with Technical Support and sends you an email notification.
SupportAssist Enterprise automatically collects the system state information that is
required for troubleshooting the issue and sends it securely to Dell.
The collected system information helps Technical Support to provide you an
enhanced, personalized, and efficient support experience. SupportAssist Enterprise
capability also includes a proactive response from Technical Support to help you
resolve the issue.
Also, SupportAssist Enterprise can monitor hardware issues that may occur on
devices that you are managing by using OpenManage Essentials, Microsoft
System Center Operations Manager (SCOM), or OpenManage Enterprise - Tech
Release.

Hardware Troubleshooting

Troubleshoot
 List of diagnostic commands.
Commands Function
arp Displays the contents of the

Address Resolution Protocol
(ARP) table. ARP entries cannot
be added or deleted.
ifconfig Displays the contents of the

network interface table.
netstat Displays the contents of the

routing table.
ping <IP Address> Verifies that the destination IPv4

IP address is reachable from OME
Modular with the current routing-
table contents. Enter a valid host
IPv4 address.

ping6 <IP Address> Verifies that the destination IPv6

IP address is reachable from OME
Modular with the current routing-
table contents. Enter a valid host
IPv6 address.
traceroute Tracks the route packets that are

taken from an IPv4 network on the
way to the given host. Enter a
valid host IPv4 address.
traceroute6 Tracks the route packets that are

taken from an IPv4 network on the
way to the given host. Enter a
valid host IPv6 address.

Troubleshooting Management Modules
System ID Button Behavior

The System ID button on the MM rear panel allows for identifying the MX7000
chassis in a rack.
When pressed, the front panel of the chassis blinks in the identify pattern. This
button also serves as the Health status LED indicator, displaying blue for health or
blinking amber if the MX9002m has a fault.
The rear-mounted ID button ONLY illuminates the MM Module ID button, it does
not step through the other devices like the Left Control Panel ID Button.
Rear LEDs
There are two LEDs on each MX9002 at the rear of the chassis: 1) MM Power LED
(green only), 2) MM Status Button/LED (bicolor: Blue and Amber), also serves as
an ID button.
The Power and Status LED (which color depends on the status) states are as
follows:
Chassis Status LED Status
Healthy Chassis (Standby MM) Power LED ON (green), Status LED

OFF.

Healthy Chassis (Active MM) Power LED ON (green), Status LED

Blue-solid.
Healthy Chassis – ID (Active MM) Power LED ON (green), Status LED

Blue-blinking.
Faulted Chassis (Active MM) Power LED ON (green), Status LED

Amber-blinking.
Faulted Chassis – ID (Active MM) Power LED ON (green), Status LED

Blue-blinking.
Failed MM (hardware failure prevents MM Power LED OFF, Status LED OFF.
from uboot on or not able to start).
Failed MM - MMC powers on, uboot starts Power LED OFF, Status LED
boot but is not able to boot operating Amber-solid.
system partitions, or MM fully boots and
detects a failure such as the switch failure,
MM failure, VR failure, and so on).
The combo Status LED/Button on the rear of the chassis changes to AMBER when
any of the Front Panel iconic indicators shows AMBER. When the EC module is in
Identify State, the combo Status LED/Button shall always blink BLUE and overrides
any other Status LED state.

Management Module Active/Standby

Management Module – Reset

Profiles and Management Module Failover
Profiles:
The restore chassis feature enables you to restore a chassis configuration, if the
backup is created from the same chassis. You must have chassis administrator
privileges to restore the chassis configuration.
The backup chassis feature enables you to create a backup of the chassis
configuration, including those of its member sleds and use it later. You must have
chassis administrator privileges to back up the chassis configuration.
Failover:
When a Management Module fails, hardware control from the chassis is
temporarily offline while new Management Module goes active.
Sleds activates failsafe throttling.
While the Management Module is offline or failing over, no new sleds, IOMs, or
PSUs can be powered on.
When a new active Management Module is declared, the chassis returns to normal
operation.


Disabling a Forgotten Management Module Password
Disabling a Forgotten MM Password:
The software security feature of the system includes a Management Module

password. The Management Module password jumper enables or disables
Management Module password features and clears any Management Module
passwords in use.
1. Power off the system, including any attached peripherals, and disconnect the
system from the electrical outlet.
2. Remove the Management Module.
3. Move the jumper on the Management Module from pins 2 and 3 to pin-1 and 2.
4. Replace the Management Module.
5. Reconnect the system to its electrical outlet, and turn on the system, including
any attached peripherals.
6. Power off the system, including any attached peripherals, and disconnect the
system from the electrical outlet.
7. Remove the Management Module.
8. Move the jumper on the Management Module from pin-1 and 2 to pins 2 and 3.
9. Replace the Management Module.

10. Reconnect the system to its electrical outlet, and turn on the system, including
any attached peripherals.
11. Assign a new system and/or setup Management Module password.
The existing Management Module passwords are not disabled (erased) until the
system boots with the jumper on pin-1 and 2.
However, before you assign a new system and/or setup Management Module
password, you must move the jumper back to pins 2 and 3.
Important:
If you assign a new system and/or setup Management Module
password with the jumper on pins 1 and 2, the system disables the
new Management Module password the next time that it boots.

Troubleshooting Compute Sleds

Troubleshooting Compute Sleds (cont.)

Troubleshooting Storage Sleds

Troubleshooting Storage IOMs
NOTE: If storage or other peripheral sleds are installed that do not support Fabric
A/B connections, the corresponding ports on the I/O modules mapping to these
slots are not active. This is also the case for any compute sled that does not have a
Mezzanine that is installed for the particular IOM (A or B) in the system.
 Troubleshooting Storage Sleds Call-Outs:
 MM Alert Logs/Hardware Logs identify events/failures in MX5016s.

 MX5016s Expander logs are also collected through SAS IOM Log
Extraction.
 MX5016s LED indicators help locate failed components:

– HDDs have dedicated LEDs.

– Expanders have health LEDs.
– Front Status LED bar rolls up overall health.
 NOTES:
 If the IOMs are not at an equal firmware level, redundancy will

be lost. Also, redundancy is lost if one of the IOMs is removed.
 If you map a drive from the storage sled using OME-M UI to a
server slot that server, and then you can create a RAID array
using the H745P MX controller with the mapped drives.
 Any failure or removal of a mapped drive from that storage
sled impacts the RAID array. So normal RAID operations
apply.
IOM Redundancy is for multipaths to storage where if an IOM fails the other IOM
will still have a direct path to the drives mapped from the storage array.
MX5000s Locations
Item Description
1 Information tag
2 External SAS ports (1 & 2)
3 External SAS ports (3,4,5 & 6)
4 Release lever button

5 Release lever
6 LED indicators
MX5000s Health Status Indicator
MX5000s
Indicator, Description Condition

Button, or
Connector
Identification Flashes blue for one second and then Indicates that the SAS
indicator turns off. IOM is active.
Note: LED is OFF when no Identification is

in progress.
Diagnostic Green SAS IOM in good

indicator health.
Flashes amber for 2 seconds and turns SAS IOM card failure.
off.
Flashes continuous green for 5 times and SAS IOM firmware

then turns off for 2 seconds. mismatch.

Troubleshooting Storage IOMs (cont.)
 Force Active
– Users can perform a failover of the IOM by performing a Force Active on the
Passive or Degraded Switch.
– Performing a Force Active on the SAS IOM is considered a disruptive
operation and should only be used as necessary.
– When the user performs a Force Active, the SAS IOM is marked Active and
the associated Storage Configuration is applied to the chassis.
 Clear Config
– It is also possible to clear the Storage Configuration of the SAS IOMs by

performing a Clear Config operation.
– Performing a Force Active on the SAS IOM is considered a disruptive
operation and should only be used as necessary as all paths from Compute
sleds to Storage Enclosures will be brought down.
– When the user performs a Clear Config operation, the SAS IOM is marked
Active and the Storage Configuration is cleared from the Chassis.

Troubleshooting Power
Contained on the power page is information about:
Chassis Power Supplies:

- Health
- Power status
- Name
- Presence
- Power supply type
- Capacity in watts
- Input voltage in volts
Power Usage by Device
Sled Overview:
- Slot Number/Type
- Device Name
- Priority
- Power Consumption
- Minimum Power
- Peak Power
Fabric Overview:

- Slot
- Device name
- Power Consumption
- Minimum Power
- Peak Power
 Troubleshooting power source problems
1. Press the power button to ensure that your system is turned on. If the power
indicator does not glow when the power button is pressed, press the power
button firmly.
2. Plug in another working power supply unit to ensure that the system board is
not faulty.
3. Ensure that no loose connections exist. For example, loose power cables.
4. Ensure that the power source meets applicable standards.
5. Ensure that there are no short circuits.
6. Have a qualified electrician check the line voltage to ensure that it meets the
needed specifications.
 Troubleshooting power supply unit problems
1. Ensure that no loose connections exist.

– For example, loose power cables.
2. Ensure that the power supply handle LED indicates that the power supply is
working correctly.
3. If you have recently upgraded your system, ensure that the power supply
unit has enough power to support the new system.
4. If you have a redundant power supply configuration, ensure that the power
supply units are of the same type and wattage.
5. Ensure that you use only power supply units with the Extended Power
Performance (EPP) label on the back.
6. Reseat the power supply unit.

MX7000 Power Redundancy Options
MX7000 Redundancy
MX7000 offers redundancy settings/statistics within the OpenManage Enterprise –

Modular user interface. There is a redundancy policy selection where a user may
set their minimum target redundancy policy. Also, there is a current redundancy
state field that provides the current state. For example, no redundancy or PSU
redundancy could be set, but the current state may be grid redundant if the
appropriate infrastructure is in place and power available.
 The MX7000 also offers more behavioral tenets:
 Get the same power on results, regardless of what is turned on first. After an
AC Cycle, allows all devices that were on to power on again.
 When possible, warn and turn on, rather than denying power-on.
The MX7000 chassis supports various types of redundancy. The user may select
these through the user interface. These are described in the following subsections.
Grid Redundancy
This mode provides for redundancy at the grid (i.e., branch) level. Configurations
for this mode are represented by “N+N,” where N is the number of power supplies
on each grid and the total available power is N * PSU capacity. Supported
configurations for this mode should include 1+1, 2+2 and 3+3.

PSU Redundancy
This mode provides for redundancy at the PSU level. Configurations for this mode
are represented by “N+1,” where N is the number of power supplies required to
support the configuration (the total available power is N * PSU capacity). Supported
configurations for this mode should include 1+1, 2+1, 3+1, 4+1, and 5+1. This is
typically used in cases where the user does not have separate grids to feed the
chassis, or the power that is required by the configuration exceeds the capacity of
three PSUs.
No Redundancy
This mode has no provision for redundancy. If a PSU fails, the system may fault
due to excess current supplied by the remaining PSUs. This mode is typically used
in cases where the user does not have enough PSUs to support the installed
configuration of compute sleds, IOMs, and so on.
PSU Pairing
The PSUs are configured to support dual grid AC sources, and are connected in
such a way that PSUs 1–3 are associated with one grid, while PSUs 4–6 are
associated with the second grid. The AC plugs on the rear of the system are set up
as three plugs on each side of the chassis, for obvious grouping to a rack PDU.
The PSUs support Rapid-ON which enables a PSU to go into a “sleep” mode while
system power consumption is low enough such that its partner provides all the
power. This enables for more efficient power utilization when overall system power
is low. The partner can then “wake” the sleeping PSU by asserting a WAKE signal
when system power draw increases.

Power Supply Unit Indicators
Each PSU has an LED with “power” symbol that lights up when power cable at rear
is inserted. Then there is an LED for AC. Both of those seem to indicate that AC
power is connected, and it is not clear why both are needed.
NOTE: After installing a power supply unit, allow several seconds for the system to
recognize the power supply unit and determine if it is working properly.
Troubleshooting power supply unit problems:

1. Ensure that no loose connections exist. For example, loose power cables.
2. Ensure that the power supply handle LED indicates that the power supply is
working properly.
3. If you have recently upgraded your system, ensure that the power supply unit
has enough power to support the new system.
4. If you have a redundant power supply configuration, ensure that both the power
supply units are of the same type and wattage. (You may have to upgrade to a
higher wattage power supply unit.)
5. Ensure that you use only power supply units with the Extended Power
Performance (EPP) label on the back.
6. Reseat the power supply unit.
PSU Health Indicator State

Indicator

PSU functioning Green

normally
PSU faulty Blinking amber
PSU mismatch ON for one second, five 4 Hz blinks and OFF (nonrepeating
cycle).
PSU firmware Default state, 4-Hz blink during the update, return to default
update state.
AC Indicator Indicator State
AC source available ON
AC source OFF
unavailable or
power cable
unplugged
DC Indicator Indicator State
DC output available ON
DC output OFF
unavailable

Troubleshooting Temperature
Ensure that the following conditions exist:
 System cover, cooling shroud, EMI filler panel, memory module blanks, or back
filler bracket is not removed.
 Ambient temperature is not higher that the system-specific ambient
temperature.
 Ensure that the Sleds, IOMs, and PSU blanks are installed in the empty bays.
 External airflow is not obstructed.
 A cooling fan is not removed or has failed.
 The expansion card installation guidelines have been followed.

Troubleshooting Fans
There are two different types of fan modules that are used in the MX7000 chassis,
60-mm fans in the front and 80-mm fans in the rear.
Each fan is enclosed in a mechanical assembly to make up the module.
The fan modules are hot-swappable, with 3+1 redundancy for the front fans, and
4+1 redundancy for the rear fans.
The system should always be populated with the full set of fans to support the
airflow requirements of the chassis.

Fan Module Indicator Codes

Fan Cooling Zones

Troubleshooting the Tub
The rear fan board, or 80 mm Fan Interposer, sits parallel to the chassis floor. The
rear fans plug into this board from the rear of the chassis. This board connects
internally to the vertical power distribution board, through a signal power cable.
This cable supports power for the five rear fan modules, up to a maximum of 225 W
per fan.
Within the MX7000 chassis, there are several infrastructure boards that are
interconnected through cables to provide the power and communication paths
between modules. These boards are supported within the chassis sheet metal and
are not hot-serviceable.
The Main Distribution Board (DB) is the most complex of all the internal
infrastructure boards. There are no active components, however there are many
signals that are interconnected on this board.
The Main DB sits horizontally near the floor of the chassis.
From the front of the chassis, the six power supplies plug into connectors on the
bottom side of the Main DB. Also, from the front of the chassis the eight compute
sleds plug into this board for power, management signals, and other fabric
connections.
From the rear of the chassis the Management Modules plug in to this board to

provide overall chassis management and control. Also, from the rear of the chassis
the two fabric I/O Modules (Fabric C IOMs) plug into this board.
The vertical power distribution board sits above the main distribution board, which
is oriented perpendicular to the chassis floor. There are no active components, but
many signals connections on this board. This board connects to the main
distribution board with a flex cable, and power and ground bus bars. From the front
of the chassis, the four cooling fans plug into the vertical power distribution board.
Also, from the rear of the chassis the four fabric I/O Modules (Fabrics A/B IOMs)
plug into this board for power, management signals, and other fabric connections.
A cable connects this board to the rear fan board.

Cold Service and Hot-Plug Components
Some redundant components are hot plug serviceable, enabling zero system
downtime for failures and serviceability. The other components are cold service,
meaning that the system must be powered down for the component to be replaced,
creating downtime and impacting availability of the system.
For Hot-Plug components, some behavioral requirements may apply, such as a

prepare-for-removal action prior to removing the item. Two or more items of this
type must exist in the chassis.
Component Redundancy Model Chassis Serviceability Model

Access
Compute Sled None Front Hot Plug
Storage Sled None Front Hot Plug
Front 60-mm Fans 3+1 Front Hot Plug
Power Supplies Various Front Hot Plug
Main DB None Internal Cold Service
Vertical PDB None Internal Cold Service

Rear Fan Board None Internal Cold Service
Right Ctrl Panel None Front Cold Service

Assy
Right CP Cable None Side Cold Service
Left Ctrl Panel/LCD None Front Cold Service
Right to Left CP None Front Cold Service

Cable
Management 1+1 Rear Hot Plug

Module
Fab A/B IOM 1+1 Rear Hot Plug
Fab C IOM 1+1 Rear Hot Plug
Rear 80-mm Fans 4+1 Rear Hot Plug

Lab Activity
You will now complete the PowerEdge MX7000 Troubleshooting lab exercise.

iDRAC and Lifecycle Controller Troubleshooting

iDRAC Health
The iDRAC Dashboard should be one of the first places that you look when
troubleshooting. The System Health section should give you a high-level overview
after which you can then examine to gather more detail.
Also the Virtual Console can be useful for taking control of the server.

iDRAC Logs
The iDRAC creates an error log against any issues it detects in the system. These
error logs are a vital part of any troubleshooting procedure because it allows the
administrator to look back and observe what has been happening within the
system.
For example, a server may have an intermittent issue that never occurs when the
support technician is onsite. Using the logs, the technician can see what errors
occurred together with the time that the error occurred, a message ID and a
summary message. Also, the error can be expanded to display additional
information to help diagnose the issue.
Also, using the icon, the technician can quickly scan down to locate any warning or
critical failures, plus a filter can be used to narrow down the error codes to a
particular field such as storage.

iDRAC Troubleshooting
The iDRAC contains a troubleshooting section which is available when you select
Troubleshooting from the system tree, as you can see from this screenshot.
These features are covered in the following slides.

iDRAC Troubleshooting – Video Capture
NOTE: The Video Capture feature is available with iDRAC Enterprise license.
To configure the boot capture video settings, select one of the following options
and click Apply.
 Disable: Boot capture is disabled.

 Capture until buffer full: Boot sequence is captured until the buffer size has
reached.
 Capture until end of POST: Boot sequence is captured until end of POST.
The boot capture timestamp is the time that the boot capture sequence is
completed - that is either when the boot capture file size has reached 2 MB or
when the host system is rebooted.
The list displays the currently active boot capture file. While the update is in-
progress, click Refresh to view the latest timestamp for the boot capture file.
You can play the files directly from iDRAC or save it to a location on your system.

iDRAC Troubleshooting – Post Code/Intrusion/Last Crash

Screen
Use this page to view the last system POST code (in hexadecimal) prior to booting
the operating system of the managed system. POST Code helps to detect pre-
video stops responding, report fatal errors, and analyze the system failures during
BIOS POST, particularly the No POST No Video situations.
The fatal error codes are used to report all the fatal POST errors.
The Intrusion relates to the chassis intrusion probes (or switches) and provides
information such as the server cover being removed.
The Last Crash Screen displays a view of the most recent crash screen, which
provides information about the events leading to the system crash. Last system
crash information is saved in the iDRAC memory and is remotely accessible. The
Last Crash Screen feature is available with iDRAC Express and Enterprise
licenses.

iDRAC Troubleshooting – Diagnostics
The Reset iDRAC will perform a reset and not lose any settings - A normal reboot
operation is performed on the iDRAC. After reboot, refresh the browser to
reconnect to iDRAC.
The Reset iDRAC to Default Settings gives you the following options:
 Preserve user and network settings

 Discard all settings and reset users to shipping value
 Discard all settings and reset user name and password
You can perform a hard or soft iDRAC restart without turning off the server:
 Hard reset — On the server, press and hold the ID LED button for 15 seconds.
 Soft restart — Using iDRAC Web interface or RACADM.
In the Command field, enter one of the following diagnostic commands and click
Submit. The result is displayed.
Command Function

arp Displays the contents of the Address

Resolution Protocol (ARP) table. ARP
entries may not be added or deleted.
ifconfig Displays the contents of the network

interface table.
netstat Displays the contents of the routing

table. If the optional interface number is
provided in the text field to the right of
the netstat option, then netstat displays
additional information regarding the
traffic across the interface, buffer usage,
and other network interface information.
ping <IP Address> Verifies that the destination IP address

is reachable from iDRAC with the
current routing-table contents. An
Internet Control Message Protocol
(ICMP) echo packet is sent to the
destination IP address based on the
current routing table contents.
gettracelog Displays the iDRAC trace log. It may

take a few seconds to return the trace
log. The command gettracelog -i
returns the number of records in the
trace log. The -A option returns the trace
log without the record numbers. See the
RACADM Command Line Reference
Guide for more information about the
gettracelog command.

iDRAC Troubleshooting – SupportAssist Collections
Use this page to generate a SupportAssist collection of the server and then export
the collection to a location on the management station (local) or to a shared
network location such as Common Internet File System (CIFS) or Network File
Share (NFS).
The collection is generated in the standard ZIP format. You can send this collection
to technical support for troubleshooting or inventory collection.
You can generate the SupportAssist collection in two ways:
 Automatic:
 Using iDRAC Service Module that automatically invokes the OS Collector
tool.
 Manual:
 Using OS Collector tool.

 You must run OS Collector tool on the server OS to export the operating
system and application data. A virtual USB device that is labeled
<devicename> is displayed in the server operating system.
 This device contains OS Collector.

 Run this tool from the server operating system.

 Before generating the SupportAssist collection, ensure:
 Lifecycle Controller is enabled.

 You have Login and Server Control privileges.
An event is recorded in the Lifecycle Controller log each time that the data is
collected.

Lifecycle Controller - Platform Restore
Lifecycle Controller allows you to create a copy (image file) of the server's profile
on the vFlash SD card installed on the server. The server profile that is backed up
on a vFlash SD card contains the server component configuration and firmware
that is installed on various components on the server where the card is installed.
The backup image file does not contain any operating system or hard-disk drive
data.
For better security, Lifecycle Controller allows you to remove the vFlash SD card
and keep it in a safe location, or you can copy the server profile (backup image)
that is stored on the vFlash SD card to any USB drive or an external network share.
Whenever the firmware is corrupted, configuration changes are incorrect, or the
system board is replaced, you can use the backup image to restore the server to its
previously stored profile. The following features are available to maintain a server:
 Backup Server Profile

 Allows you to create the server profile on a vFlash SD card that is installed
on the server. Lifecycle Controller can create the server profile only on the
vFlash SD card.
 Export Server Profile
 Allows you to export the server profile that is stored on the vFlash SD card
to a USB drive or a network share (CIFS, NFS, or HTTP).

 Import Server Profile

 Allows you to restore the backup image from the vFlash SD card, USB drive,
or a network share (CIFS, NFS, or HTTP).
 Import Server License
 Allows you to import an iDRAC license from a network share or a USB drive.

Lifecycle Controller – Part Replacement Configuration
Applying firmware and configuration updates to replaced parts
 Before configuring replaced parts, make sure that the following

prerequisites are met:
 Click the Collect System Inventory On Restart option, so that Lifecycle

Controller automatically invokes Part Firmware Update and Part
Configuration Update when the system is started.
– If disabled, the cache of system inventory information may become stale
if new components are added without manually entering Lifecycle
Controller after turning on the system. In manual mode, press <F10>
after the part replacement during a system restart.
 Make sure that the Disabled option under Part Firmware Update and Part
Configuration Update is cleared.
 The previous component and the new device must be identified as the same
part.

 If the current adapter on the system is NPAR enabled and is replaced with a
new adapter, after the host server is turned on, press <F2> and select
System Setup > Device Settings and ensure that the NPAR is enabled.
NPAR must be enabled on the new adapter before using the Part
Replacement feature.
 To apply part firmware and configuration to replaced parts:
1. Start Lifecycle Controller.

2. In the left pane, click Platform Restore.
3. In the right pane, click Part Replacement. The Part Replacement
Configuration page is displayed.
4. From the part firmware update drop-down menu, select one of the following:
– Disabled
o Firmware update on replaced parts is not performed.
– Allow version upgrade only
o Firmware update on replaced parts is performed only if the firmware
version of the new part is earlier than the existing part.
– Match firmware of replaced part
o Firmware on the new part is updated to the version of the original part
- the default is to match the firmware of the replaced part.
 From the part configuration update drop-down menu, select one of the
following:
 Disabled
– The feature is disabled, and the current configuration is not applied if a
part is replaced.
 Apply always
– The feature is enabled (default setting), and the current configuration is
applied if a part is replaced.
 Apply only if firmware match
– The feature is enabled, and the current configuration is applied only if the
current firmware matches with the firmware of a replaced part.

Lab Activities
Complete the iDRAC Troubleshooting and the SupportAssist Collection lab

exercises.
Please inform your instructor before completing the MX7000 Firmware Update lab
exercise.

Module Summary
Module Summary

Module Summary
Summary

Module Summary

Module Summary

Course Conclusion

Course Conclusion
Course Conclusion

Course Conclusion
Course Conclusion

Course Conclusion
Review Course Objectives

Course Conclusion
Final Questions

Course Conclusion
Thank You for Your Participation!


Dell EMC PowerEdge MX Modular Platform Installation, Configuration, and Management - Participant Guide

Uploaded by

Copyright:

Available Formats

Dell EMC PowerEdge MX Modular Platform Installation, Configuration, and Management - Participant Guide

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Dell EMC PowerEdge MX Modular Platform Installation, Configuration, and Management - Participant Guide

Uploaded by

Copyright:

Available Formats

DELL EMC POWEREDGE

© Copyright 2019 Dell Inc. Page i