OpenShift Container Platform 4.4 on Synergy

# Operating system deployment

# RHEL deployment using PXE server

In this solution, a PXE server is used for RHEL 7.x OS deployment and is configured on CentOS (version: CentOS Linux release 7.6.1810 (Core)). The PXE server uses the FTP service for file distribution, but can be altered to support HTTP or NFS. This section highlights the steps to configure a PXE server.

# Procedure for setting up PXE server and deploying RHEL OS

  1. Login to the CentOS server to be configured as a PXE server as a user that can run commands as root via sudo.

  2. Use the following command to install packages such as DHCP, TFTP-server, vSFTPD (FTP server), and xinetd.

    > sudo yum install dhcp tftp tftp-server syslinux vsftpd xinetd

  3. Update the DHCP configuration file at /etc/dhcp/dhcpd.conf with the MAC addresses, IP addresses, DNS, and routing details of the installation environment. Domain search is optional. A sample DHCP configuration file is shown as follows.

    ddns-update-style interim;
ignore client-updates;
authoritative;
allow booting;
allow bootp;

# internal subnet for my DHCP Server
subnet 20.0.x.x netmask 255.0.0.0 {
range 20.0.x.x 20.0.x.x;
deny unknown-clients;
option domain-name-servers 20.x.x.x;
option domain-name "twentynet.local";
option routers 20.x.x.x;
option broadcast-address 20.255.255.255;
default-lease-time 600;
max-lease-time 7200;
next-server 20.x.x.x;
filename "pxelinux.0";
}

#######################################
host bootstrap {
hardware ethernet 00:50:56:xx:98:df;
fixed-address 20.0.x.x;
}
host master01 {
hardware ethernet 00:50:56:95:xx:82;
fixed-address 20.0.x.x;
}
host worker01 {
hardware ethernet 00:50:56:xx:ab:82;
fixed-address 20.0.x.x;
}

  4. Trivial File Transfer Protocol (TFTP) is used to transfer files from data server to clients without any kind of authentication. TFTP is used for ignition file loading in PXE-based environment. To configure the TFTP server, edit the configuration file /etc/xinetd.d/tftp. Change the parameter ‘disable = yes’ to ‘disable = no’ and leave the other parameters as is. To edit the /etc/xinetd.d/tftp file, execute the following command.

    > sudo vi  /etc/xinetd.d/tftp

    The TFTP configuration file is as follows.

    service tftp
   {

        socket_type = dgram
        protocol = udp
        wait = yes
        user = root
        server = /usr/sbin/in.tftpd
        server_args = -s /var/lib/tftpboot
        disable = no
        per_source = 11
        cps = 100 2
        flags = IPv4
    }

    Network boot related files must be placed in the tftp root directory /var/lib/tftpboot. Run the following commands to copy the required network boot files to /var/lib/tftpboot/.

    > sudo cp –v /usr/share/syslinux/pxelinux.0 /var/lib/tftpboot

> sudo cp –v /usr/share/syslinux/menu.c32 /var/lib/tftpboot

> sudo cp –v /usr/share/syslinux/memdisk /var/lib/tftpboot

> sudo cp –v /usr/share/syslinux/mboot.c32 /var/lib/tftpboot

> sudo cp –v /usr/share/syslinux/chain.c32 /var/lib/tftpboot

> sudo mkdir /var/lib/tftpboot/pxelinux.cfg

> sudo mkdir /var/lib/tftpboot/networkboot

  5. Copy the RHEL 7.6 (or higher version) ISO files to the PXE server. Mount it to the /mnt/ directory and then copy the contents of the ISO to the local FTP server using the following commands.

    > sudo mount –o loop <OS file name> /mnt/

> cd /mnt/

> sudo cp –av * /var/ftp/pub/

  6. Copy the kernel file (vmlinuz) and initrd file from /mnt to /var/lib/tftpboot/networkboot/ using the following commands.

    > sudo cp /mnt/images/pxeboot/vmlinuz /var/lib/tftpboot/networkboot/

> sudo cp /mnt/images/pxeboot/initrd.img /var/lib/tftpboot/networkboot

  7. Use the following command to unmount the ISO files.

    > sudo unmount /mnt/

  8. For RHEL nodes, create and utilize a new kickstart file under the folder /var/ftp/pub with the name “rhel7.cfg” using the following command.

    > sudo vi /var/ftp/pub/rhel7.cfg

    A sample kickstart file is shown as follows. The installation user should create a kickstart file to meet the requirements of their installation environment.

    firewall --disabled
# Install OS instead of upgrade
install
# Use FTP installation media
url --url="ftp://<FTP_server_IP_address>/pub/rhel76/"
# Root password
# root password can be plaintext as shown below
# rootpw –plaintext <password>
# root password is encrypted using the command “openssl passwd -1 <password>” and resultant output is provided for rootpw as shown below
rootpw --iscrypted $6$uiq8l/7xEWsYXhrvaEgan4N21yhLa8K.U7UA12Th3PD11GOXvEcI40gp
# System authorization information
auth useshadow passalgo=sha512
# Use graphical install
graphical
firstboot disable
# System keyboard, timezone, language
keyboard us
timezone Europe/Amsterdam
lang en_US
# SELinux configuration
selinux disabled
# Installation logging level
logging level=info
# System bootloader configuration
bootloader location=mbr
clearpart --all --initlabel
part swap --asprimary --fstype="swap" --size=1
part /boot --fstype xfs --size=300
part pv.01 --size=1 --grow
volgroup root_vg01 pv.01
logvol / --fstype xfs --name=lv_01 --vgname=root_vg01 --size=1 --grow
%packages
@^minimal
@core
%end
%addon com_redhat_kdump --disable --reserve-mb='auto'
%end

  9. Create a PXE menu. Use the following command to create a PXE menu file at the location /var/lib/tftpboot/pxelinux.cfg/default

      > sudo vi /var/lib/tftpboot/pxelinux.cfg/default

    For each of the OS boot options, provide the following details:

    - MENU LABEL : Custom name of the respective menu label.
- KERNEL : Kernel details of the operating system.
- APPEND : Path of bootloader file along with path of cfg or configuration file (in case of RHEL).

    A sample PXE menu is as follows.

    default menu.c32

prompt 0

timeout 30

MENU TITLE LinuxTechi.com PXE Menu

LABEL rhel76

MENU LABEL RHEL76-Buedata

KERNEL /rhel76/vmlinuz

APPEND initrd=/rhel76/initrd.img inst.repo=ftp://<FTP_server_IP_address>/pub/rhel76 ks=ftp://<FTP_server_IP_address>/pub/rhel76-hcp.cfg

LABEL rhcos-bootstrap

MENU LABEL Install RHCOS4.3 sec-Bootstrap

KERNEL /networkboot/rhcos-4.3.0-x86_64-installer-kernel

APPEND ip=dhcp rd.neednet=1 initrd=/networkboot/rhcos-4.3.0-x86_64-installer-initramfs.img console=tty0 console=ttyS0 coreos.inst=yes coreos.inst.install_dev=sda coreos.inst.image_url= ftp://<FTP_server_IP_address>/pub/rhcos-4.3.0-x86_64-metal-bios.raw.gz coreos.inst.ignition_url= ftp://<FTP_server_IP_address>/pub/sec/bootstrap.ign

LABEL rhcos-master

MENU LABEL Install RHCOS4.2 sec-Master

KERNEL /networkboot/rhcos-4.3.0-x86_64-installer-kernel

APPEND ip=dhcp rd.neednet=1 initrd=/networkboot/rhcos-4.3.0-x86_64-installer-initramfs.img console=tty0 console=ttyS0 coreos.inst=yes coreos.inst.install_dev=sda coreos.inst.image_url= ftp://<FTP_server_IP_address>/pub/rhcos-4.3.0-x86_64-metal-bios.raw.gz coreos.inst.ignition_url=ftp://<FTP_server_IP_address>/pub/sec/master.ign

LABEL rhcos-worker

MENU LABEL Install RHCOS4.2 sec-Worker

KERNEL /networkboot/rhcos-4.3.0-x86_64-installer-kernel

APPEND ip=dhcp rd.neednet=1 initrd=/networkboot/rhcos-4.3.0-x86_64-installer-initramfs.img console=tty0 console=ttyS0 coreos.inst=yes coreos.inst.install_dev=sda coreos.inst.image_url= ftp://<FTP_server_IP_address>/pub/rhcos-4.3.0-x86_64-metal-bios.raw.gz coreos.inst.ignition_url=ftp://<FTP_server_IP_address>/pub/sec/worker.ign

  10. Use the following command to start and enable xinetd, dhcpd, and vsftpd.

        > sudo systemctl start xinetd

    > sudo systemctl enable xinetd

    > sudo systemctl start dhcpd.service

    > sudo systemctl enable dhcpd.service

    > sudo systemctl start vsftpd

    > sudo systemctl enable vsftpd

  11. Configure SELinux for FTP.

    > sudo setsebool –P allow_ftpd_full_access 1

  12. Open ports in the firewall using the following firewall-cmd commands.

    > sudo firewall-cmd --add-service-ftp --permanent

>sudo firewall-cmd --add-service-dhcp --permanent

>sudo firewall-cmd –reload

Note

It is crucial to generate ignition files, copy them to the TFTP server, and update the path in the PXE default file. For more information about generating the ignition files, refer to the Kubernetes manifests and ignition files section in this document.

# Red Hat CoreOS deployment using iPXE server

In this solution, an iPXE Server setup is used for Red Hat Enterprise Linux CoreOS (RHCOS) deployment. This section explains in detail the playbooks required to configure iPXE server on an RHEL 7.6 VM to perform an unattended installation of Red Hat Enterprise Linux CoreOS (RHCOS) for non-ESXi VMs and bare metal servers.

# Prerequisites

  1. RHEL Ansible engine VM with Ansible 2.9.x and Python 3.6.x.

Note

Ansible Engine is configured as mentioned in the Installer machine section.

  1. If the user wishes to setup the iPXE server on a separate RHEL 7.6 VM (other than the Ansible engine VM) preferably or could be bare metal server, they need to have the following minimum configuration:

    • At least 200 GB disk space
    • Two (2) CPU cores
    • 8 GB RAM
    • /var has at least 15 GB disk space allocated while partitioning
    • Static IP on the same network as the RHCOS server
    • Internet access

  2. ​ Disable Selinux using following steps:

    1. Check the status of SELINUX using the command /etc/selinux/config

    2. Disable SELINUX.

      $ sudo sed -i 's/permissive/disabled/g' /etc/selinux/config

      Note

      If the current state of "selinux" is "enforcing," replace the keyword "permissive" in the above command with "enforcing" and then run it.

    3. Reboot the RHEL machine.

    4. Use /etc/selinux/config command to check the status of SELINUX.

# Automated Installation and Configuration of iPXE server

Perform the steps listed to execute the playbooks for installing and configuring iPXE server.

  1. Browse to the following directory on the Ansible engine.

    $ cd BASE_DIR/os_deployment/deploy_rhcos/

    Note

    Refer to Installer machine section to know the "BASE_DIR" path.

  2. Copy Red Hat OpenShift 4.x above install files to directory, preferably under /tmp/ Example: /tmp/image/

  3. Update the inventory file "hosts" with the RHEL machine IP on which the user is trying to setup the iPXE server.

    [ansible_host]
20.x.x.x

  4. Update the secrets.yml file located in BASE_DIR/os_deployment/deploy_rhcos/secrets.yml. Use the following command to edit secrets.yml file.

    $ sudo ansible-vault edit $BASE_DIR/os_deployment/deploy_rhcos/secrets.yml

Note

  • Refer to Installer machine section to know the "BASE_DIR" .
  • The default password for the Ansible vault file is changeme.
 #####################################################################
 # Details of Ansible User Credentials required to set up the iPXE
  ###Common Networking
  # Interface Name Example: interface_name: enp0s10f0
  interface_name: <interface_name>
  
  # Ansible Host IP Example: ansible_engine_ip: 20.0.15.115
  ansible_engine_ip: <ansible_engine_ip>

  # base DNS domain, Example: base_domain: ocp42.local
  base_domain: <base_domain>

  # Openshift sub domain, Example:app_domain: apps.ocp42.local
  app_domain: <app_domain>

  # Network router IP Example: gateway: 192.168.42.254
  gateway: <gateway>

  # Network DNS server Example: dns_server: 192.168.42.252
  dns_server: <dns_server>

  # DHCP range that is used for OCP nodes Example: dhcp_range: 192.168.42.204,192.168.42.210,24h
  dhcp_range: <dhcp_range>
    
  # Subnet Mask Example: net_mask: 255.255.240.0
  net_mask: <net_mask>
    
  ## OpenShift node network
  # mac address of master node 1 Example: master1_mac: 08:00:27:36:0A:01
  master1_mac: <master1_mac>
    
  # master node 1 IP address Example: master1_ip: 192.168.42.204
  master1_ip: <master1_ip>
    
  # master node 2 mac address Example: master2_mac: 08:00:27:36:0A:02
  master2_mac: <master2_mac>

  # master node 2 IP address Example: master2_ip: 192.168.42.205
  master2_ip: <master2_ip>

  # master node 3 mac address Example: master3_mac: 08:00:27:36:0A:03
  master3_mac: <master3_mac>

  # master node 3 IP address Example: master3_ip: 192.168.42.206
  master3_ip: <master3_ip>

  # worker node 1 mac address Example: worker1_mac: 08:00:27:36:0A:04
  worker1_mac: <worker1_mac>

  # worker node 1 IP address Example: worker1_ip: 192.168.42.207
  worker1_ip: <worker1_ip>

  # worker node 2 mac address Example: worker2_mac: 08:00:27:36:0A:05
  worker2_mac: <worker2_mac>

  # worker node 2 IP address Example: worker2_ip: 192.168.42.208
  worker2_ip: <worker2_ip>

  # worker node 3 mac address Example: worker3_mac: 08:00:27:36:0A:06
  worker3_mac: <worker3_mac>
    
  # worker node 3 IP address Example: worker3_ip: 192.168.42.209
  worker3_ip: <worker3_ip>
    
  # bootstrap mac address Example: bootstrap_mac: 08:00:27:36:0A:08
  bootstrap_mac: <bootstrap_mac>
    
  # bootstrap IP address Example: bootstrap_ip: 192.168.42.210
  bootstrap_ip: <bootstrap_ip>
    
  # install media details Example: image_location: "/tmp/image/"
  image_location: <image_location>
    
  # Example: initramfs_name: rhcos-4.2.0-x86_64-installer-initramfs.img
  initramfs_name: <initramfs_name>
    
  # Example: kernel_name: rhcos-4.2.0-x86_64-installer-kernel
  kernel_name:<kernel_name>
    
  # Example: bios_uefi_name: rhcos-4.2.0-x86_64-metal-bios.raw.gz
  bios_uefi_name: <bios_uefi_name>

  1. Run the following Ansible playbook to configure the iPXE server.

    $ ansible-playbook -i hosts master.yml --ask-vault-pass

  2. Figure 17 shows the status of after running the Ansible playbook to configure the iPXE server.

    Figure 17. Status of matchbox,TFTP,Xinetd and dnsmasq Services

  3. Run the following command to test the setup with MAC address.

    $ curl http://localhost:8080/ipxe?mac=08:00:27:36:0A:xx

  4. Copy the ignition files in " /var/lib/matchbox/ignition " and verify the status of ignition file by using MAC address of all nodes.

    $ curl http://192.168.42.200:8080/ignition?mac=08:00:27:36:0A:xx

  5. Boot the VM or bare metal using iPXE.

← User-provisioned DNS requirements →