Differences

This shows you the differences between two versions of the page.

--- mogile_on_tyan_gt86c-b5630 [2025/05/23 05:16] – removed - external edit (Unknown date) 127.0.0.1
+++ mogile_on_tyan_gt86c-b5630 [2025/05/23 05:17] (current) – kenson
@@ Line 1: / Line 1: @@
+====== Mogile on Tyan GT86C-B5630 Servers ======
+Tyan GT86C-B5630 1U LFF 12 bay (3.5") storage server
+  * [[https://www.mitaccomputing.com/en/spec/Barebones/GT86CB5630_B5630G86CV12|Chassis Documentation]] (Note: Use BIOS/FW updates from Motherboard link below instead!)
+  * Motherboard: S5630GMRE-CGN
+    * [[https://www.mitaccomputing.com/Motherboards_S5630_S5630GMRE_EN~Spec|Docs/Downloads]]
+  * Intel C621 SATA controller
+  * 500W PSU: S14-500P1AA
+  * Xeon Gold 5118 2nd Gen Scalable, 12-core 2.3Ghz
+  * 64GB (2x32GB) DDR4
+  * X520-DA2 2x 10GbE SFP+ OCP w/ 2x 10Gtek ASF-GE-T (1000Base-T SFP Copper RJ45)
+  * 1x Micron 60GB SATA SSD
+  * 4x Seagate Exos X16 ST14000NM005G 14TB 7.2K RPM SATA 6Gb/s 512e CMR 3.5in
+Firmware update steps:
+  * Make a FAT32 usb stick.
+  * Load UEFI Shell using steps: [[https://github.com/KilianKegel/Howto-create-a-UEFI-Shell-Boot-Drive]]
+    * UEFI files: {{ :efi.zip |}}
+  * BIOS Upgrade: {{ :s5630gmr_v202.zip |}}
+  * BMC/IPMI Upgrade: {{ :s5630-bmc_v700.zip |}}
+BIOS Changes:
+  * Boot performance to Max Efficiency
+  * Fan control to Manual/30%
+IPMI Notes:
+  * Default login: root/superuser (will prompt for change on first login)
+  * Set PW using internal mnemonic scheme: soup
+  * Also create user dar with same PW
+Ubuntu 22.04.5 custom configs:
+  * To disable cloud-init, create the empty file /etc/cloud/cloud-init.disabled <code>sudo touch /etc/cloud/cloud-init.disabled</code>
+  * If you need to do an offline install of ubuntu 22.04, look at [[https://askubuntu.com/questions/1409135/22-04-offline-installation]]
+Config log:
+<code>
+apt -y install vim
+# temporary allow root access for setup
+vi /etc/ssh/sshd_config
+PermitRootLogin yes
+systemctl restart ssh
+sudo passwd root
+</code>
+<code>
+cd /etc/netplan/
+vi 50-cloud-init.yaml
+</code>
+<code>
+network:
+    ethernets:
+        ens3f0:
+            dhcp4: false
+            addresses: [10.10.22.181/24]
+            gateway4: 10.10.22.1
+            nameservers:
+              addresses: [8.8.8.8,8.8.4.4]
+    version: 2
+</code>
+Turn off kernel audit messages
+<code>
+vi /etc/default/grub
+</code>
+Add "audit=0" to GRUB_CMDLINE_LINUX_DEFAULT
+<code>
+GRUB_DEFAULT=0
+GRUB_TIMEOUT_STYLE=hidden
+GRUB_TIMEOUT=0
+GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
+GRUB_CMDLINE_LINUX_DEFAULT="audit=0"
+GRUB_CMDLINE_LINUX=""
+</code>
+If the server has nvme, proactively do the following to avoid a potential NVMe power saving mode bug.
+<code>
+GRUB_CMDLINE_LINUX_DEFAULT="audit=0 pcie_aspm=off pcie_port_pm=off nvme_core.default_ps_max_latency_us=0 apparmor=0 security=apparmor"
+</code>
+Update grub and reboot
+<code>
+update-grub
+</code>
+Setup Disks
+<code>
+sgdisk -o -n 1:0:0 -t 1:8300 /dev/sdb
+sgdisk -o -n 1:0:0 -t 1:8300 /dev/sdc
+sgdisk -o -n 1:0:0 -t 1:8300 /dev/sdd
+sgdisk -o -n 1:0:0 -t 1:8300 /dev/sde
+mkfs.ext4 -m 0 /dev/sdb1
+mkfs.ext4 -m 0 /dev/sdc1
+mkfs.ext4 -m 0 /dev/sdd1
+mkfs.ext4 -m 0 /dev/sde1
+mkdir /data
+#create a unique device id e.g. dev900
+mkdir /data/dev900
+mkdir /data/dev901
+mkdir /data/dev902
+mkdir /data/dev903
+vi /etc/fstab
+# add the following lines
+/dev/sdb1       /data/dev900 ext4       defaults 1 2
+/dev/sdc1       /data/dev901 ext4       defaults 1 2
+/dev/sdd1       /data/dev902 ext4       defaults 1 2
+/dev/sde1       /data/dev903 ext4       defaults 1 2
+</code>
+Grab Mogile packages
+<code>
+scp -oHostKeyAlgorithms=+ssh-dss -P 75 user@server:/S76usrlocalsrc.tar .
+</code>
+move stuff to /usr/local/src
+Load dependencies
+<code>
+apt -y install build-essential
+apt -y install net-tools
+apt -y install less
+apt -y install libssl-dev
+apt -y install libz-dev
+apt -y install mysql-server
+apt -y install libio-aio-perl
+apt -y install mysql-client
+apt -y install libmysqlclient-dev
+cpan install CPAN
+cpan install Perlbal
+cpan install Danga::Socket
+cpan install IO::Socket
+cpan install Net::SSLeay
+cpan install IO::Socket::SSL
+cpan install Gearman::Client
+cpan install Gearman::Server
+cpan install Gearman::Client::Async
+cpan install Net::Netmask
+cpan install IO::WrapTie
+cpan install DBI
+cpan install BSD::Resource
+cpan install Perlbal::XS:HTTPHeaders
+cpan install YAML
+cpan install DBD:mysql
+</code>
+Add Users:
+<code>
+groupadd mogile
+useradd -m -g mogile mogile
+</code>
+Install mogile
+<code>
+cd MogileFS-Client-1.07
+perl Makefile.PL
+make
+make test
+make install
+cd MogileFS-Utils-2.11
+perl Makefile.PL
+make
+make install
+cd mogilefs-server-2.17
+perl Makefile.PL
+make
+make test
+make install
+mkdir /etc/mogilefs
+cp mogstored.conf /etc/mogilefs/
+</code>
+cat /etc/mogstored.conf
+<code>
+daemonize=1
+docroot=/data
+</code>
+cat /etc/mogilefs/mogilefsd.conf
+<code>
+daemonize = 1
+db_dsn = DBI:mysql:mogilefs:host=10.2.2.19:port=3306
+db_user = mogile
+db_pass = mogile
+listen = 0.0.0.0:7001
+conf_port = 7001
+listener_jobs = 10
+delete_jobs = 5
+replicate_jobs = 0
+reaper_jobs = 3
+old_repl_compat = 0
+rebalance_ignore_missing = 1
+</code>
+SU to user mogile and start daemon
+<code>
+mogilefsd -c /etc/mogilefs/mogilefsd.conf
+</code>
+<code>
+timedatectl list-timezones
+timedatectl set-timezone PST8PDT
+</code>