Answer: DEL-F10-F1-DEL-DEL-DEL-F8-F10-F8-F1-DEL-F10-F12-DEL-DEL-F11.
In that sequence, to be completed within 3 seconds.


Do you prefer UEFI? Really??
Anyway, racadm, ipmitool, some stuff are probably common to BIOS and UEFI ;-)

Penguin Computing

F2 = Setup
F10 = Display Boot Menu
F12 = Force Network Boot

For BMC/IPMI was said to default to share eth0 with host os when service RJ45 isn't connected.  
However, my systems wasn't setup that way.  
To make IPMI share eth0 with the host os, set it with:
ipmitool raw 0x2e 0xcc 0x5e 0x2b 0 0x0C 0x01 0x02 

Note that ipmi already have a default admin/admin account as user 2.
If setting new admin acc, try to rename the existing one first:
ipmitool user set name 2 oper
ipmitool user list 1


F1 		Setup
F2		Diagnostics
F12		Select Boot Device	# Need legacy mode to do PXE boot


During Post, keys:

tab		Display BIOS POST message

del 		run setup
F11		boot menu
F12		Network boot

RAID controller
AVAGO MegaRAID SAS-MFI BIOS  v5.50.03.0  July 2015)

^P		pause		(disk scanning?)
^V		skip

^H 		WebBIOS
^Y		preboot CLI


F2  = System Setup 
F10 = Lifecycle Controller (BIOS setup)
F11 = Boot Manager
F12 = PXE boot

^A = Avago raid controller

HT - Hyperthreading/Logical Processor

/opt/dell/srvadmin/sbin/racadm get BIOS.ProcSettings.LogicalProc		# query HT status

/opt/dell/srvadmin/sbin/racadm set BIOS.ProcSettings.LogicalProc Disabled	# disable HT
/opt/dell/srvadmin/sbin/racadm jobqueue create BIOS.Setup.1-1			# create job to change BIOS setting
/opt/dell/srvadmin/sbin/racadm serveraction powercycle				# power cycle to effect change

SNC - Sub NUMA Cluster on Skylake

Leave SNC disabled (default).
See Fig 2 of:
Skylake... 14th gen PowerEdge...
"introduce a clustering mode called Sub NUMA clustering (SNC). On CPU models that support SNC, enabling SNC is akin to splitting the single socket into two NUMA domains, each with half the physical cores and half the memory of the socket."

Query Sub NUMA Cluster (SNC) modes 
	- numactl -H 
	- /opt/dell/srvadmin/sbin/racadm get BIOS.ProcSettings.SubNumaCluster
	- lstopo

- SNC=Disabled (default).  ie, two socket show up as two NUMA domains.  
- SNC=Enabled.  within socket has split NUMA domains. two socket system would then have FOUR NUMA domains.  application who does not optimize for memory locality would be worse off.  Thus, not recommended unless benchmark and tuning has been done that benefit from this setting. 

SNC disabled is baseline.  If enable SNC: 
- STREAM, WRF, Fluent, which are highly localized memory optimized, have slight gain of 1-2%
- HPL was ~1% worse off.

	- /opt/dell/srvadmin/sbin/racadm get BIOS.MemSettings.NodeInterleave

- Disabled (default) = system support NUMA (asymmetric) mem config
- Enabled = memory interleave is supported IF symetric memory config is installed.  (SNC will not work when memory configured this way?)
See script that set this and many below values.

Memory Operating Mode

racadm help BIOS.MemSettings.MemOpMode
	OptimizerMode - Optimizer Mode (default)
	SingleRankSpareMode - Single Rank Spare Mode;
	MirrorMode - Mirror Mode;
	FaultResilientMode - Fault Resilient Mode;

	OppSrefEn=Disabled (default)


DAPC profile consume less power than Performance profile, yet said to produce essentially same chrunching power. [my actual HPL test lost 7% performance]
See Fig 3 + 4 of:
Skylake... 14th gen PowerEdge...
- In idle state, Performance Profile consumes ~28% more power than DAPC.
- Peak power consumption in DAPC Profile is ~16% less than in Performance Profile. 
- STREAM, WRF have essentialy same performance in either Performance or DAPC mode.
- HPL is 1.05% better in Performance than DAPC mode.  
- Fluent is ~1.01% better.

/opt/dell/srvadmin/sbin/racadm get BIOS.SysProfileSettings.SysProfile
	SysProfile=PerfOptimized	(default)

/opt/dell/srvadmin/sbin/racadm set BIOS.SysProfileSettings.SysProfile PerfPerWattOptimizedOs

	Using PerfPerWattOptimizedDapc:
	- TBA 

	Using PerfPerWattOptimizedOs:
	- Reduce HPL performance by ~7% on Skylake 6130 2.1 GHz 32cores 96G on Dell C6420
	- Exposes HPL clock calculation bug in random manner (cuz it messes with CPU clock?)

racadm help BIOS.SysProfileSettings.SysProfile  # list available modes (and dependencies):
	- PerfPerWattOptimizedDapc - Performance Per Watt (DAPC)
	- PerfPerWattOptimizedOs - Performance Per Watt (OS)
	- PerfOptimized - Performance
	- PerfWorkStationOptimized - Workstation Performance;Custom - Custom;

Other BIOS param

When CPU doesn't perform, check these params (racadm):
  set BIOS.ProcSettings.LogicalProc      Disabled       # non default, but HPC don't want HT on	
  set BIOS.MemSettings.MemOpMode         OptimizerMode	# default, should not need to change this for HPC
  set BIOS.MemSettings.NodeInterleave    Enabled	# non default, not compatible with SubNumaCluster, not typically recommended
  set BIOS.ProcSettings.SubNumaCluster   Enabled	# non default, only useful if app can better utlize localized mem
  set BIOS.SysProfileSettings.SysProfile PerfOptimized	# default
  set BIOS.SysProfileSettings.SysProfile PerfPerWattOptimizedDapc	# said to save energy
  set BIOS.SysProfileSettings.SysProfile PerfPerWattOptimizedOs		# may save energy, seems to introduce clock/timing bug
  set BIOS.ProcSettings.ControlledTurbo  Enabled 	# allow for external control of when to engage turbo?  Def: Disabled.
  get BIOS.ProcSettings.ProcTurboMode			# def Enabled, changeable only in Custom SysProfile 
  get BIOS.SysInformation.SystemBiosVersion             # BIOS version

  set BIOS.SysProfileSettings.SysProfile Custom		# need custom mode to set the next two option:
  set BIOS.SysProfileSettings.ProcCStates Autonomous 	# def: Disabled. alt: Enabled # to allow proc to operate in all avail power state
  set BIOS.SysProfileSettings.UncoreFrequency DynamicUFS	# def: MaxUFS

  # Perhaps also check:

  get BIOS.ProcSettings.UpiPrefetch			# def: Enabled
  get BIOS.MemSettings.OppSrefEn			# def: Disabled 
  get BIOS.SysProfileSettings.MemFrequency		# def: MaxPerf.  Could choose diff speed such as 2666 MHz, 2400, 1866.
  get BIOS.SysProfileSettings.ProcC1E			# def: Disabled  # but enabled when switch to Custom SysProfile !!
							# Enable = processor is allowed to switch to minimum performance state when idle.
  get BIOS.SysProfileSettings.EnergyPerformanceBias	# def: MaxPower (ie Performance).  
							# alt: BalancedPerformance, BalancedEfficiency, LowPower
Undocumented HPL parameter
# set env var for HPL to properly use AVX
export HPL_HOST_ARCH=3 # AVX2,  eg Hashwell
export HPL_HOST_ARCH=9 # AVX512 eg Skylake

KNL BIOS settings

KNL specific cpu/memory config:
racadm help BIOS.MemSettings...
	Cltt  (def)

	Quadrant (def)	# 2x2

ProcEmbMemMode		# KNL Memory Mode.  affect "free -h", "numactl -H" 
	Cache (def)
	Memory		# ie flat mode, memory seen in "free -h" and malloc-able.
DynamicCoreAllocation=Disabled 	-- This field enables or disables the OS capability to put logical processors in the idling state in order to reduce power consumption.
ProcConfigTdp=Nominal 		-- This field allows reconfiguration of TDP (Thermal Design Power) to lower levels.

Retrieve Serial stored in BIOS

  racadm  get System.ChassisInfo	# service tag of chassis, eg Dell C6420 chassis
  racadm  get System...			# service tag of blade/sledge

  ipmitool fru | grep Serial   						# list all fru and filter for serial of sledge/blade
  ipmitool raw 0x30 0xc8 0x01 0x00 0x0b 0x00 0x00 0x00 | xxd -r 	# get Dell C6320 chassis service tag vintage 2017
More info, eg fetching Dell C6220 II chassis serial, see: ipmi


racadm techsupreport collect 	# start a support collection 
racadm jobqueue view
racadm techsupreport export -f 

racadm racreset		# restart rac, like power cycling the service processor, but not the wipe config and restore to factory default kind of reset, which does have a separate cmd for

Dell Ref


SuperMicro (bios) update manager

SUM = supermicro update manager, not the checksum command from the OS!

  • simple binary to extract out of tgz file, then it just run.
  • write to current dir for a sum.log file.
  • root access required
  • Note that older bios output is simple key-value pair flat file, whereas newer bios is in XML format. (it has nothing to do with SUM version (eg 2.0 vs 2.2, older version with newer bios still produce XML output).
    untar gz, no real need to install
    ./sum -h				# not the checksum command in linux default path... 
    sum -i 10.10.... -u ADMIN -p ADMIN -c GetBIOSInfo	# use IPMI interface to get info
    sum -c GetBIOSInfo			# this find out firmware version info, etc.  run on local machine
    sum -c UpdateBios --file BIOS.rom 	# this update bios, not change config
    sum -c GetCurrentBiosCfg  --file  smBiosCf.txt	# write output of BIOS settings (HT on or off, etc) to file
    sum -c GetCurrentBiosCfg     >    smBiosCf.txt	# should be same, but couple of places have * (default) next to diff entries.  also has SM(c) header, should avoid
    vi smBiosCf.txt		# make desired changes to BIOS use hex code for values. eg turn off Hyperthreading
    			# remove the first two lines of cfg that had the SM copyright info 
    			# in KNL, file was simple text file
    			# in skylake, file was xml (UEFI bios?)
    sum -c ChangeBiosCfg --file smBiosCf.txt --reboot	# update bios config, rebooting host 
    							# --reboot would actually be an soft reboot telling OS to shutdown (via ACPI?)
    sum -h -c ChangeBiosCfg				# get help on how to make changes to bios
    sum -c LoadDefaultBiosCfg		
    sum -c GetDmiInfo
    Select GetCurrentBiosCfg entries
    sum -c GetCurrentBiosCfg | egrep --color 'Hyper-Threading|Turbo|CPU\ C\ State=|Cluster\ Mode=|Memory\ Mode=' 
    [Advanced|Boot Feature]
    Quiet Boot=01                           // Please enter the value in 2 hexadecimal digits. Default value is <<<01>>>
    [Advanced|Processor Configuration]
    Intel(R) Hyper-Threading Technology=01  // 01 (Disable), *00 (Enable)
    [Advanced|Processor Configuration|Advanced Power Management Configuration|CPU P State Control]
    Energy efficient P-state=01             // 00 (Disable), *01 (Enable)
    Turbo Mode=01                           // 00 (Disable), *01 (Enable)
    [Advanced|Processor Configuration|Advanced Power Management Configuration|CPU C State Control]
    CPU C State=01                          // 00 (Disable), *01 (Enable)
    [Advanced|Chipset Configuration|North Bridge|Uncore Configuration]
    Cluster Mode=00                         // *00 (All2All), 01 (SNC-2), 02 (SNC-4), 03 (Hemisphere), 04 (Quadrant), 05 (Auto)
    Memory Mode=01                          // *01 (Flat), 00 (Cache), 02 (Hybrid), 03 (Auto)
    MCDRAM Cache Size=02                    // 01 (25% of MCDRAM size), *02 (50% of MCDRAM size)        Memory Mode = "Flat" or Memory Mode = "Cache" or Memory Mode = "Hybrid" and Memory Mode = "Cache" or Memory Mode = "Hybrid" or Memory Mode = "Auto" and Memory Mode = "Flat" or Memory Mode = "Hybrid" or Memory Mode = "Auto"
    Treat MCDRAM as Hot-Pluggable Memory=00 // *00 (no), 01 (yes)        Memory Mode = "Flat" or Memory Mode = "Cache" or Memory Mode = "Hybrid" and Memory Mode = "Flat" or Memory Mode = "Hybrid" or Memory Mode = "Auto"
    OPIO Parallel Training=01               // *01 (Enable), 00 (Disable)
    OPIO Parallel Training Channel Count=08 // Please enter the value in 2 hexadecimal digits. Default value is <<<08>>>        OPIO Parallel Training = "Enable"
    MCDRAM Repair=01                        // 00 (Disable), *01 (Enable)
    MCDRAM Diagnostics=01                   // 00 (Disable), *01 (Enable)        MCDRAM Repair = "Disable"
    MCDRAM Data in NVRAM=01                 // 00 (Disable), *01 (Store), 02 (Use), 03 (UseStore)
    EDC Demand Scrub=01                     // *01 (Enable), 00 (Disable)
    EDC Patrol Scrub=01                     // 00 (Disable), *01 (Enable)
    [Advanced|Serial Port Console Redirection|COM1 Console Redirection Settings]
    Terminal Type=01                        // 00 (VT100), *01 (VT100+), 02 (VT-UTF8), 03 (ANSI)
    Bits per second=07                      // 03 (9600), 04 (19200), 05 (38400), 06 (57600), *07 (115200)
    Data Bits=08                            // 07 (7), *08 (8)
    Parity=01                               // *01 (None), 02 (Even), 03 (Odd), 04 (Mark), 05 (Space)
    Stop Bits=01                            // *01 (1), 03 (2)
    Flow Control=00                         // *00 (None), 01 (Hardware RTS/CTS)
    Legacy OS Redirection Resolution=01     // 00 (80x24), *01 (80x25)
    Putty KeyPad=01                         // *01 (VT100), 02 (LINUX), 04 (XTERMR6), 08 (SCO), 10 (ESCN), 20 (VT400)
    Redirection After BIOS POST=00          // *00 (Always Enable), 01 (BootLoader)

    Service Processor


    tool to manipulate BIOS using cli tool... TBA

    Lenovo DSA

    Lenovo Dynamic System Analyst (DSA).
    Collect system state/log for support to review. Portable version for rhel7 worked for vNFS n0096.s2
    Download ( ) self extracting elf .bin and right away run the program. don't seems to leave any binary behind when finsih. When finished, leave collected log in /var/log/Lenovo_Support Probably same thing that is invoked by "bios" collect tool, which save result to usb.
    Preliminary okay to give sudo to intern to run... keep a stable binary version in a non-writable place...
    sudo ./lnvgy_utl_dsa_dsala7d-10.3_portable_rhel7_x86-64.bin

    Copyright info about this work

    This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike2.5 License. Pocket Sys Admin Survival Guide: for content that I wrote, (CC) some rights reserved. 2005,2012 Tin Ho [ tin6150 (at) ]
    Some contents are "cached" here for easy reference. Sources include man pages, vendor documents, online references, discussion groups, etc. Copyright of those are obviously those of the vendor and original authors. I am merely caching them here for quick reference and avoid broken URL problems.

    Where is PSG hosted these days?