====== Mogile on Tyan GT86C-B5630 Servers ======

Tyan GT86C-B5630 1U LFF 12 bay (3.5") storage server
  * [[https://www.mitaccomputing.com/en/spec/Barebones/GT86CB5630_B5630G86CV12|Chassis Documentation]] (Note: Use BIOS/FW updates from Motherboard link below instead!)
  * Motherboard: S5630GMRE-CGN 
    * [[https://www.mitaccomputing.com/Motherboards_S5630_S5630GMRE_EN~Spec|Docs/Downloads]]
  * Intel C621 SATA controller
  * 500W PSU: S14-500P1AA
  * Xeon Gold 5118 2nd Gen Scalable, 12-core 2.3Ghz
  * 64GB (2x32GB) DDR4
  * X520-DA2 2x 10GbE SFP+ OCP w/ 2x 10Gtek ASF-GE-T (1000Base-T SFP Copper RJ45)
  * 1x Micron 60GB SATA SSD
  * 4x Seagate Exos X16 ST14000NM005G 14TB 7.2K RPM SATA 6Gb/s 512e CMR 3.5in


Firmware update steps:
  * Make a FAT32 usb stick.
  * Load UEFI Shell using steps: [[https://github.com/KilianKegel/Howto-create-a-UEFI-Shell-Boot-Drive]]
    * UEFI files: {{ :efi.zip |}}
  * BIOS Upgrade: {{ :s5630gmr_v202.zip |}}
  * BMC/IPMI Upgrade: {{ :s5630-bmc_v700.zip |}}

BIOS Changes:
  * Boot performance to Max Efficiency
  * Fan control to Manual/30%

IPMI Notes:

  * Default login: root/superuser (will prompt for change on first login)
  * Set PW using internal mnemonic scheme: soup
  * Also create user dar with same PW

Ubuntu 22.04.5 custom configs:
  * To disable cloud-init, create the empty file /etc/cloud/cloud-init.disabled <code>sudo touch /etc/cloud/cloud-init.disabled</code>
  * If you need to do an offline install of ubuntu 22.04, look at [[https://askubuntu.com/questions/1409135/22-04-offline-installation]]

Config log:
<code>
apt -y install vim
# temporary allow root access for setup
vi /etc/ssh/sshd_config
PermitRootLogin yes
systemctl restart ssh
sudo passwd root
</code>

<code>
cd /etc/netplan/
vi 50-cloud-init.yaml
</code>

<code>
network:
    ethernets:
        ens3f0:
            dhcp4: false
            addresses: [10.10.22.181/24]
            gateway4: 10.10.22.1
            nameservers:
              addresses: [8.8.8.8,8.8.4.4]
    version: 2

</code>

Turn off kernel audit messages
<code>
vi /etc/default/grub
</code>

Add "audit=0" to GRUB_CMDLINE_LINUX_DEFAULT
<code>
GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="audit=0"
GRUB_CMDLINE_LINUX=""
</code>

If the server has nvme, proactively do the following to avoid a potential NVMe power saving mode bug.

<code>
GRUB_CMDLINE_LINUX_DEFAULT="audit=0 pcie_aspm=off pcie_port_pm=off nvme_core.default_ps_max_latency_us=0 apparmor=0 security=apparmor"
</code>

Update grub and reboot
<code>
update-grub
</code>
Setup Disks
<code>
sgdisk -o -n 1:0:0 -t 1:8300 /dev/sdb
sgdisk -o -n 1:0:0 -t 1:8300 /dev/sdc
sgdisk -o -n 1:0:0 -t 1:8300 /dev/sdd
sgdisk -o -n 1:0:0 -t 1:8300 /dev/sde
mkfs.ext4 -m 0 /dev/sdb1
mkfs.ext4 -m 0 /dev/sdc1
mkfs.ext4 -m 0 /dev/sdd1
mkfs.ext4 -m 0 /dev/sde1

mkdir /data

#create a unique device id e.g. dev900
mkdir /data/dev900
mkdir /data/dev901
mkdir /data/dev902
mkdir /data/dev903

vi /etc/fstab
# add the following lines
/dev/sdb1       /data/dev900 ext4       defaults 1 2
/dev/sdc1       /data/dev901 ext4       defaults 1 2
/dev/sdd1       /data/dev902 ext4       defaults 1 2
/dev/sde1       /data/dev903 ext4       defaults 1 2
</code>

Grab Mogile packages
<code>
scp -oHostKeyAlgorithms=+ssh-dss -P 75 user@server:/S76usrlocalsrc.tar .
</code>
move stuff to /usr/local/src

Load dependencies
<code>
apt -y install build-essential
apt -y install net-tools
apt -y install less
apt -y install libssl-dev
apt -y install libz-dev
apt -y install mysql-server
apt -y install libio-aio-perl
apt -y install mysql-client
apt -y install libmysqlclient-dev

cpan install CPAN
cpan install Perlbal 
cpan install Danga::Socket 
cpan install IO::Socket
cpan install Net::SSLeay
cpan install IO::Socket::SSL
cpan install Gearman::Client 
cpan install Gearman::Server
cpan install Gearman::Client::Async 
cpan install Net::Netmask 
cpan install IO::WrapTie
cpan install DBI
cpan install BSD::Resource
cpan install Perlbal::XS:HTTPHeaders
cpan install YAML
cpan install DBD:mysql
</code>

Add Users:
<code>
groupadd mogile
useradd -m -g mogile mogile
</code>

Install mogile
<code>
cd MogileFS-Client-1.07
perl Makefile.PL
make
make test
make install

cd MogileFS-Utils-2.11
perl Makefile.PL
make
make install


cd mogilefs-server-2.17
perl Makefile.PL
make
make test
make install


mkdir /etc/mogilefs
cp mogstored.conf /etc/mogilefs/
</code>

cat /etc/mogstored.conf
<code>
daemonize=1
docroot=/data
</code>

cat /etc/mogilefs/mogilefsd.conf
<code>
daemonize = 1
db_dsn = DBI:mysql:mogilefs:host=10.2.2.19:port=3306
db_user = mogile
db_pass = mogile
listen = 0.0.0.0:7001
conf_port = 7001
listener_jobs = 10
delete_jobs = 5
replicate_jobs = 0
reaper_jobs = 3
old_repl_compat = 0
rebalance_ignore_missing = 1
</code>

SU to user mogile and start daemon
<code>
mogilefsd -c /etc/mogilefs/mogilefsd.conf
</code>

<code>
timedatectl list-timezones
timedatectl set-timezone PST8PDT
</code>