====== Intro: Minimal CentOS 6.5 ====== The Spectrum XMC1151 digitizer libraries we have require a specific flavour of Linux to operate. Download the specific version by getting the [[http://vault.centos.org/6.5/isos/x86_64/CentOS-6.5-x86_64-minimal.iso]] which is the "minimal" CentOS 6.5. * Actual direct download link for minimal 6.5 CentOS ISO: [[http://archive.kernel.org/centos-vault/6.5/isos/x86_64/CentOS-6.5-x86_64-minimal.iso]] (md5sum is **0d9dc37b5dd4befa1c440d2174e88a87** CentOS-6.5-x86_64-minimal.iso) Kernel from OS install 2016.04.14 Linux lvisf 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux In case your kernel is wrong, you can download the RPM and install directly - Download the correct one (see above) [[http://vault.centos.org/6.5/centosplus/x86_64/Packages/kernel-2.6.32-431.el6.centos.plus.x86_64.rpm]] cd ~/download wget http://vault.centos.org/6.5/centosplus/x86_64/Packages/kernel-2.6.32-431.el6.centos.plus.x86_64.rpm wget http://vault.centos.org/6.5/centosplus/x86_64/Packages/kernel-devel-2.6.32-431.el6.centos.plus.x86_64.rpm - Install sudo rpm -ivh --oldpackage kernel-2.6.32-431.el6.centos.plus.x86_64.rpm sudo rpm -ivh --oldpackage kernel-devel-2.6.32-431.el6.centos.plus.x86_64.rpm ======Kontron BIOS Changes ====== * To make these BIOS changes, we need to get to the UEFI shell. - Reboot the computer - Press key while booting to get to BIOS menu - Boot -> Boot Option #1 -> UEFI: Built-in Shell - Save and Exit (F4) - Press while booting to get to the **Shell>** prompt. - Increase the boot delay so that the Spectrum Digitizers have time to initialise. - Enter the following on the **Shell>** kboardconfig PciCfgDelay 5s - Double check by running without the argument too see what the parameter is set to kboardconfig PciCfgDelay # will see [5s] highlighted in green (and that is the longest delay available) - Disable hyperthreading and whatever VT stands for **NOTE: The non-RT version works great with hyperthreading** (at least so far: --- //[[david.rabine@nasa.gov|David Lloyd Rabine]] 2018/04/27 18:13// ) # ONLY RT version needs this DO NOT DO THIS for normal operations with NORT build (build02 / v2) kboardconfig IntelVT disabled kboardconfig IntelHT disabled - Exit the bios exit * What is SpeedStep? What is CpuTurbo? Sounds like things we don't want on! ====== Base CentOS ====== Using the **500_00746_r100_SDR7000_Linux_SG.pdf** document, set up the core data system. ===== DVD Boot ===== **NOTE:** Press the **Delete** key while booting in order to change the boot to either the USB PLEXTOR drive or the LEXAR CFAST drive * BIOS (delete key after eSATA RAID Marvel BIOS) -> Boot -> Boot Option #1 * Then F4: Save & Exit - Attached the external USB CDROM drive to the computer via the USB3 port (USB stick with UNETBOOTIN did NOT WORK for me?!) (UnetBootin will work with later version of CentOS image... 6.5 is broken) - Insert the Minimal CentOS 6.5 Install Disk - Boot and set the BIOS to boot off the Plextor drive (our USB based DVD drive) - At CentOS 6 Welcome, Click -> Next - Language: English (English) - Keyboard: U.S. English - Storage Devices: Basic Storage Devices - **NOTE:** If another OS detected, select: Fresh Installation (SURE YOU WANT TO DELETE THAT OS!?) - Hostname: lvisf - Timezone: New York - root Password: the usual - Partitioning: Create Custom Layout (bottom option) - Delete any partitions on the new target drive (this 32GB **cfast** is the demo target) - Select the Free disk area -> Create a new partition - Select -> Standard Partition (top option) - Mount Point: **/boot/efi** - File System Type: **EFI System Partition** - Size (MB): **1024** - Additional Size Options: Fixed Size (default, but make sure) - OK - Select -> Standard Partition (top option) - Mount Point: **/** - File System Type: **ext4** - Size (MB): **15360** - Additional Size Options: Fixed Size (default, but make sure) - OK - Next - Select the drive (for this case, the 60 GB (or 60000 MB drive) and put that to the right, then Write Changes - Wait for install to complete (205 packages in ~**4** minutes) - Drive will eject - Remove DVD - Reboot - Load BIOS on reboot and select the **cfast** drive as the standard boot device ====== Hard Drive / Raid ====== * [[subsystem:datasystem:dev:hardware:owc:1uraid:start|OWC 1U Raid]] ====== CentOS Configure ====== ===== Virtual Box ===== Some notes on using virtual box Used **RedHat 64 Bit** as the base machine, worked fine with the CentOS minimal ==== Local Network ==== In order to **ssh** to your host, you can add a rule to port forward into your VM - Network -> Advanced -> Port Forwarding - Add a rules that maps - Name: Forward SSH (whatever you want) - Protocol: TCP - Host IP: 127.0.1.1 - Host Port: 2222 - Guest IP: 10.0.2.15 (or whatever your machine is) - Guest Port: 22 - Now you can **ssh** directly from your host pc to the VM (so I can copy and paste from the wiki, yeah!!) - Example: ssh -CY root@127.0.1.1 -p2222 ==== Screen Scale ==== - Display settings for the host, you can scale the screen directly now, up to 200%. YAY! ===== lvisf Config ===== ==== Filesystem ==== - Add the navigation, internal and external drives to the **/etc/fstab** file - Edit sudo nano /etc/fstab - Add these lines LABEL=LVISF_INTERNAL /mnt/internal ext4 defaults,noatime 0 0 LABEL=LVISF_EXTERNAL /mnt/external ext4 defaults,noatime 0 0 LABEL=LVISF_NAV /mnt/nav ext4 defaults,noatime 0 0 tmpfs /mnt/ramdisk tmpfs nodev,nosuid,noexec,nodiratime,size=1M 0 0 - Make these directories for mount points sudo mkdir /mnt/internal /mnt/external /mnt/nav /mnt/tmp /mnt/usb /mnt/nfs /mnt/ramdisk - **Note:** Ram disk information: [[https://www.jamescoyle.net/how-to/943-create-a-ram-disk-in-linux]] ==== Network Temporary Configuration (DHCP) ==== - Setup the system for DHCP for now - Log in as root - Use **vi** to edit **/etc/sysconfig/network-scripts/ifcfg-eth0** so ONBOOT=yes - Startup network (will take a moment, DHCP) sudo ifup eth0 - Logout and now, you should **ssh** to **gs694-lvisdev.gsfc.nasa.gov** and then to the new machine (whatever its DHCP IP address is now on the local LAN) ==== Prerequisites for Compiling Kernel Modules ==== * [[http://www.tldp.org/LDP/lkmpg/2.6/html/x121.html]] <- build kernel module link for an example - Install the development tools yum groupinstall 'Development Tools' - Get some useful programs yum install ftp -y yum install nano -y yum install wget -y - Grab the ethernet driver while you still have access - Current driver is here: [[https://downloadmirror.intel.com/13663/eng/igb-5.3.5.3.tar.gz]] mkdir ~/down cd ~/down wget https://downloadmirror.intel.com/13663/eng/igb-5.3.5.3.tar.gz ==== Ethernet Driver Install and Configuration ==== - Log in as **root** on the main console since ethernet will be disabled by default - Install Intel ethernet drivers **[Only needed if compiling on Kontron flight hardware]**: - Download {{:subsystem:datasystem:dev:centos65minimal:igb-5.3.4.4.tar.gz|}} to a USB drive - Current driver is here: [[https://downloadmirror.intel.com/13663/eng/igb-5.3.5.3.tar.gz]] - Uncompress in the home directory tar xvf igb-5.3.4.4.tar.gz - Build and install the drvier cd ~/tmp/igb-5.3.4.4/src/ make sudo make install - Modify this module file sudo nano /etc/sysconfig/modules/lvisf.modules - Put the following into this file #!/bin/sh # start intel network driver /sbin/modprobe igb - Change permissions so this script runs at boot time sudo chmod 755 /etc/sysconfig/modules/lvisf.modules **NOTE:** you can just do this sudo cp /opt/lvisf/src/lvisF/scripts/ifcfg-eth* /etc/sysconfig/network-scripts/ - Configure our network device - Change the **eth0** definition - editing the config file vi /etc/sysconfig/network-scripts/ifcfg-eth0 - to the following DEVICE=eth0 HWADDR=00:80:82:1B:E7:04 TYPE=Ethernet UUID=98bd88d8-8fe8-4b8e-bca0-a58bd3306717 ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=static NAME="System eth0" IPADDR=10.4.1.2 NETMASK=255.255.255.0 - Add our default gateway - Edit the network configuration vi /etc/sysconfig/network - End up looking like this NETWORKING=yes HOSTNAME=lvisf NETWORKING_IPV6=no IPV6INIT=no GATEWAY=10.4.1.1 # or 10.4.1.254 if using Dell Laptop Gateway for WiFi still - Ensure our name server can be found - Edit **resolv.conf** vi /etc/resolv.conf - End up with this search gsfc.nasa.gov nameserver 128.183.10.134 nameserver 128.183.50.17 - Turn off IPV6 (found here: [[https://www.centos.org/forums/viewtopic.php?t=7989]]) sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1 sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1 - Change the **eth1** definition - editing the config file vi /etc/sysconfig/network-scripts/ifcfg-eth1 - to the following DEVICE=eth1 HWADDR=00:80:82:1B:E7:05 TYPE=Ethernet UUID=5eb0ea0f-b1e8-43ab-a64d-83d3d9e1dc95 ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=static NAME="System eth1" IPADDR=192.168.0.2 NETMASK=255.255.255.0 - Change the **eth1:1** definition - editing the config file vi /etc/sysconfig/network-scripts/ifcfg-eth1:1 - to the following DEVICE=eth1:1 HWADDR=00:80:82:1B:E7:05 TYPE=Ethernet UUID=5eb0ea0f-b1e8-43ab-a64d-83d3d9e1dc95 ONBOOT=yes NM_CONTROLLED=yes BOOTPROTO=static NAME="System eth1:1" IPADDR=129.100.0.2 NETMASK=255.255.255.0 - REMOVE the keyboard and mouse and never use them again (USB interrupts) - Add **lvis** user to the system (still as root (logged in via SSH for me now)) - Add the user useradd lvis - Set password passwd lvis - Add **lvis** user to the list of **sudo** users - Modify the super user file with the following (as logged in as root) visudo - After the line that has root and ALL and ALL permissions, insert this line lvis ALL=(ALL) ALL ==== File System Extras ==== - Add **ntfs** drive capability - Add the **epel** repository sudo yum install epel-release - Add **ntfs** to our system sudo yum install fuse fuse-ntfs-3g - Add **Exfat** drive capability - Add the **epel** repository and this Nux Dextop repository sudo yum install epel-release sudo rpm -Uvh http://li.nux.ro/download/nux/dextop/el6/x86_64/nux-dextop-release-0-2.el6.nux.noarch.rpm - Add **Exfat** to our system sudo yum install exfat-utils fuse-exfat - An example mounting an Exfat system is sudo mount -t exfat /dev/sdc1 /mnt/tmp ==== X11 ==== - This will allow you to run **X11** # core X11 sudo yum install xorg-x11-xauth xorg-x11-fonts-* xorg-x11-utils xterm -y # required for tuna sudo yum install xauth pygtk2-libglade -y - Install **tuna** tuning package sudo yum install tuna -y - Modify **/etc/ssh/sshd_config** and add this line to forward X11 (or make sure it is not commented out... this was already enabled last I checked) X11Forwarding yes ==== Remove Unused Services ==== - Remove **exim** (NOTE, this was not on the CentOS Minimal I just installed 2018.01.04 (but leaving for historical)) - Stop the service sudo service exim stop - Remove the package from the system sudo rpm -e --nodeps exim - Remove **auditd** sudo service auditd stop # now sudo chkconfig auditd off # after reboot - Remove **postfix** - Stop the service sudo service postfix stop - Remove the package from the system sudo rpm -e --nodeps postfix ==== eSATA Speed Limit ==== Our **eSATA** connection to the raid (and any external ATA devices) was occasionaly running into issues when running at high speed (6.0 GBPS). Dan Caplan found online notes that people put a speed limit on the kernel module that regulates the **ATA** commuincations, and that solved the types of errors we have been seeing. - Modify **/etc/grub.conf** by adding this to the kernel starting command line libata.force=3.0 ==== Kernel Console Redirection ==== If you want to send the kernel messages out to the serial port for logging. - Modify **/etc/grub.conf** so the default kernel command line redirects console to the serial port sudo nano /etc/grub.conf - Change rhgb quiet - To rhgb console=tty0 console=ttyS0,115200n8 === Enable Serial Console Login === And, to make it such that you can log onto the system over that console port... - Modify the grub boot line and add the following lines above all the kernel options - Edit **/etc/grub.conf** sudo nano /etc/grub.conf - Make the top look like this serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1 terminal --timeout=5 serial console hiddenmenu - Add these lines to the kernel that you are using (probably already done, see above!) console=tty0 console=ttyS0,115200n8 - Add **ttyS0** to the **/etc/securetty** file - Edit sudo nano /etc/securetty - Append this to the bottom of the file ttyS0 - Modify **/etc/init/ttyS0.conf** sudo nano /etc/init/ttyS0.conf - And fill it with the following code # ttyS0 - getty # # This service maintains a getty on ttyS0 from the point the system is # started until it is shut down again. start on stopped rc RUNLEVEL=[2345] and ( not-container or container CONTAINER=lxc or container CONTAINER=lxc-libvirt) stop on runlevel [!2345] respawn exec /sbin/getty -L 115200 ttyS0 vt102 ######## ===== Spectrum Config ===== Still as root logged into the main desktop - Disable the Firewall - Turn off the firewall sudo service iptables stop sudo service ip6tables stop - Make this permanent sudo chkconfig iptables off sudo chkconfig ip6tables off - Disable SELinux - Edit the configuration file sudo vi /etc/selinux/config - Modify the file - From SELINUX=enforcing - To SELINUX=disabled - Reboot for all changes to take place ===== Minimize Kernel Modules ===== - Disable **ipv6**, this addition is required along with a line in the modprobe file below - Modifiy **/etc/sysconfig/network** sudo vi /etc/sysconfig/network - And append the following two lines NETWORKING_IPV6=no IPV6INIT=no - Add these two lines to **sshd_config** sudo nano /etc/ssh/sshd_config # lvisf - ipv4 only AddressFamily inet ListenAddress 0.0.0.0 - Blacklist kernel modules we don't require - Edit a file **lvisf-blacklist.conf** sudo vi /etc/modprobe.d/lvisf-blacklist.conf - Add the following lines # lvisf kernel module blacklist # version 1.0 - dlr - 20160418 # ipv6 blacklist ipv6 # NMI watchdog blacklist iTCO_wdt blacklist iTCO_vendor_support blacklist i2c_i801 # blacklist igb blacklist lpc_ich blacklist shpchp blacklist dm_mirror # intel sound blacklist soundcore blacklist snd blacklist snd_seq blacklist snd_hda_intel blacklist snd_hda_codec_hdmi # USB controller (still loads) blacklist xhci_hcd # this is required to REMOVE ipv6 module, I am not sure why # http://www.cyberciti.biz/faq/redhat-centos-disable-ipv6-networking/ install ipv6 /bin/true ==== Extras ==== * These two commands will remove some other modules I couldn't figure out how to make permanently go away # USB driver... if you're using the nousb kernel, this will work sudo modprobe --remove xhci_hcd sudo modprobe --remove dm_mirror ==== Minimal Modules ==== Module Size Used by e1000 170646 0 sg 29350 0 ext4 374902 3 jbd2 93427 1 ext4 mbcache 8193 1 ext4 sd_mod 39069 3 crc_t10dif 1541 1 sd_mod ahci 42215 2 i915 624511 1 drm_kms_helper 44321 1 i915 drm 280012 2 i915,drm_kms_helper i2c_algo_bit 5935 1 i915 i2c_core 31084 4 i915,drm_kms_helper,drm,i2c_algo_bit video 20674 1 i915 output 2409 1 video dm_mod 84209 9 ====== lvisf code ====== ===== SVN Repository ===== :!: **NOTE:** This is obsolete (the repository link), but left here for posterity. * Dave Yancich sent me this note back in May: Repository URL: https://zj.yancich.com:8443/svn/NASA/trunk/lvisF username "lvis" password: standard lvis password This is a cmake project. To rebuild the make files: In "build" directory type "cmake ../" Then "make" to build everything. "make clean" to clean the programs and kernel modules. ==== Build Document ==== (Yancich sent this to me in March 2016) Original document is {{:subsystem:datasystem:dev:rt:configuration_and_build_procedures_for_lvisf_and_support_libraries.docx|}} ==== Requirements ==== - Install **cmake** sudo yum install cmake -y - Install **htop** sudo yum install htop -y - Install **jed** sudo yum install ftp://ftp.pbone.net/mirror/centos.karan.org/el5/extras/testing/i386/RPMS/jed-0.99.18-5.el5.kb.i386.rpm -y - Install **nano** sudo yum install nano -y - Install **p7zip** and **pigz** sudo yum install p7zip pigz -y - Install PCI utilities sudo yum install pciutils -y - Install **screen** sudo yum install screen -y - Install **telnet** sudo yum install telnet -y - Install **wget** sudo yum install wget -y - Install the development package sudo yum groupinstall "Development Tools" - Install development libraries for md5sum and gzip in real time sudo yum install openssl-devel -y - Install the ncurses development library assuming our console gets written using it sudo yum install ncurses-devel -y - Install md5deep so we can check file integrity sudo yum install md5deep -y - Install man pages sudo yum install man -y sudo yum install man-pages -y - Install kernel development source cd mkdir download cd ~/download wget http://vault.centos.org/6.5/os/x86_64/Packages/kernel-devel-2.6.32-431.el6.x86_64.rpm sudo yum --nogpgcheck localinstall ~/download/kernel-devel-2.6.32-431.el6.x86_64.rpm -y - Install **gdisk** for partition manipulation on modern drives sudo yum install gdisk -y - Install the TIFF I/O Library for the CIMG library used for the DEM library sudo yum install libtiff-devel -y - Install the X11 Development library also used for the DEM library sudo yum install libX11-devel -y ==== Debugging ==== * This looks like a great article on remote debugging: [[https://www.cnx-software.com/2012/01/31/debugging-embedded-linux-with-gdbserver-and-insight-gdb-gui/]] - Install **gdb** and **gdbserver** sudo yum install gdb gdb-gdbserver - Install debug information (found this [[https://stackoverflow.com/questions/10389988/missing-separate-debuginfos-use-debuginfo-install-glibc-2-12-1-47-el6-2-9-i686]]) - Add these **yum** utility sudo yum install yum-utils - Modify this file and **enable** the repo by setting enable to = 1 sudo /etc/yum.repos.d/CentOS-Debuginfo.repo # and set # enabled=1 - Install this sudo debuginfo-install glibc ==== 3rd Party Libs to Build from Source ==== === GSL (GNU Scientific Library) === Download version [[http://mirror.rit.edu/gnu/gsl/|2.5]] and follow install instructions. This will build libgsl and libgslcblas need for the //DEM// library. - Grab the library and uncompress cd ~/download wget http://mirror.rit.edu/gnu/gsl/gsl-2.5.tar.gz cd ~/code tar -xvf ~/download/gsl-2.5.tar.gz - Compile cd ~/code/gsl-2.5 ./configure make sudo make install sudo ldconfig ==== Code Base ==== - Create the repoistory home and check out the source code - Create a repository home in **/opt** sudo mkdir /opt/lvisf - Set proper permissions sudo chown -R lvis:lvis /opt/lvisf - **[Optional]** Uncompress our locally attached mount with snapshot (faster than network initial checkout) cd / tar jxvf /media/LVIS-GH-2TB_WD209/lvisf_build/lvisf_sourcesnapshot_20160816_svn439.tar.bz2 - **[Either]** Check out he raw source code from scratch cd /opt/lvisf svn co https://svn.yancich.com:8443/svn/NASA/LVIS/trunk/ src - **[Or]** Update it if you used the tar above cd /opt/lvisf/src svn update - Enter the user home directory and create symbolic links cd /home/lvis mkdir src ln -s /opt/lvisf/src/* /home/lvis/src/ - Add the following lines to the **.bashrc** for the **lvis** user - Edit the file nano /home/lvis/.bash_profile - Add these lines # For SVN commit comment editor export SVN_EDITOR=nano # For the Innovative digitizer export WD_BASEDIR=/home/lvis/src/Innovative/WinDriver # For the SyncClock timer card export PLX_SDK_DIR=/home/lvis/src/PLX/PlxSdk # For the spectrum digitizer export SSP_INSTALL_ROOT=/home/lvis/src/spectrum - Load this environment (re-run this script) source ~/.bash_profile - Edit the file sudo nano /etc/ld.so.conf.d/lvisf.conf - Add these lines /home/lvis/src/Innovative/winCloudDrvr/winclouddrvr_installation/lib /home/lvis/src/MIC3753/linux_driver_source_3.2.7.0_64bit/libs /home/lvis/src/spectrum/shared/linux/pc64/lib - Reload the library path sudo ldconfig ==== SyncClock32 PLX Drivers ==== - Build all drivers - Go to the source code home directory cd /home/lvis/src/PLX/PlxSdk/Driver - Execute the build script ./buildalldrivers - Build the samples - Go to the source directory cd /home/lvis/src/PLX/PlxSdk - Make the programs make clean make - Load the driver - To the binary directory cd /home/lvis/src/PLX/PlxSdk/Bin - Execute the script for the 9030 card (our syncclock32) sudo ./Plx_load 9030 # to just see what boards are supported, run the script with no argument ./Plx_load ==== MIC 3753 DIO Drivers ==== **NOTE** These drivers NEED to be built as root, you cannot **sudo** to build these drivers - Become root all the time sudo bash - Go to the code base directory cd /home/lvis/src/MIC3753/linux_driver_source_3.2.7.0_64bit/drivers/driver_base/src/lnx_ko - Build the drivers make - Go into our specific card directory cd /home/lvis/src/MIC3753/linux_driver_source_3.2.7.0_64bit/drivers/pci1753_mic3753_pcm3753i/src/lnx_ko - Build the drivers # to disable IRQs in this driver, just use this code where the IRQ calls are commented out cp init.c.no_irq init.c make - Make symbolic links to these drivers in the kernel module ln -s /opt/lvisf/src/MIC3753/linux_driver_source_3.2.7.0_64bit/drivers/bin/*.ko /lib/modules/2.6.32-431.el6.x86_64/kernel/drivers/misc/ - Load the drivers (this crashes when i load the bio1753 (IRQ!!! issue) cd /home/lvis/src/MIC3753/linux_driver_source_3.2.7.0_64bit/drivers/bin insmod biokernbase.ko insmod bio1753.ko - Exit the **sudo bash** exit ==== Innovative Digitizer Drivers ==== Described in **LinuxNotes.pdf** (might be this **/usr/Innovative/MalibuRed-1.3/KerPlug/LinuxNotes.pdf**) - Link our Innovative directory to **/usr/Innovative** sudo ln -s /opt/lvisf/src/Innovative /usr/Innovative - Untar the latest driver tar -zxvf /opt/lvisf/src/tars/64WinDriver-12.3-0.X86_64.tgz.tar.gz --directory=/usr/Innovative/ - Build 12.3 Windriver - Link 12.3 as our Windriver cd /usr/Innovative rm /usr/Innovative/WinDriver ln -s WinDriver-12.3 WinDriver - Go here cd /usr/Innovative/WinDriver/redist - Configure ./configure make sudo make install # they say do not do this, we live on the edge - Kerplug - Build (builds in the LINUX directory for the RT kernel) cd /usr/Innovative/KerPlug/linux ./configure make - OK, this SHOULD fail. - Edit the Makefile with nano Makefile - Add seven dot dot slashes to the line: - Convert this MOD_CSRCS = $(WD_BASEDIR)/redist/linux_wrappers.c - to this MOD_CSRCS = ../../../../../../../$(WD_BASEDIR)/redist/linux_wrappers.c - Change 1180 to 1230 - Run **make** again make - Load the driver sudo insmod LINUX.2.6.32-431.el6.x86_64.x86_64/kp_malibu_module.ko - Need to link the generic library name with the compiled library cd /usr/Innovative/KerPlug sudo ./quicklink wdapi 1230 - Final license issue with changed authorization mode - Install qt development sudo yum install qt4 -y sudo yum install qt4-devel -y - Rebuild (and this will also copy the files where they need to go) (NOTE: I believe this is unnecessary, but it does not hurt anything to do this) cd /home/lvis/src/Innovative/Malibu/Qt/Os_Mb qmake-qt4 make === Historical Log === All of this below is just for historical records. Do NOT need to do any more Innovative. Raw code from **/etc/rc.local** from the primary machine to load this sudo bash cd /usr/Innovative/WinDriver-12.3/redist && ./wdreg LINUX.2.6.32-431.el6.x86_64.x86_64/windrvr1230 auto #/usr/Innovative/winCloudDrvr/winclouddrvr_installation/redist/wdreg winCloudDrvr auto #chmod 666 /dev/winCloudDrvr insmod /usr/Innovative/KerPlug/linux/LINUX.2.6.32-431.el6.x86_64.x86_64/kp_malibu_module.ko exit == Field Addendum == In the field we noticed that the driver was saying unregistered, and would only run for 60 minutes (sometimes it ran longer, but this was an issue). Dave found these instructions from an email: FIRST! Make sure this folder exists: /usr/Innovative/Lib/Dll (and it is linked to the Innovative folder in src) Make sure LicenseLoader.so exists in that folder. (I have attached that file and a tar of the 3 others needed to rebuild) Hopefully its just that the .so did not make it to that system disk. If is does exist: Copy all the files in the tar to: /home/lvis/src/Innovative/Malibu Cd to /home/lvis/src/Innovative/Malibu/Qt/Os_Mb “qmake-qt4” “make” This will make the Os library and copy it to the correct folder. “That is all I got” 😊 Dave ==== Spectrum Digitizer Drivers ==== - Create the symbolic link for /spectrum sudo ln -s /opt/lvisf/src/spectrum /spectrum - Enter the code directory cd /home/lvis/src/spectrum/shared/linux/pc64/bin - Copy the drivers for our kernel into the binary directory cp -a /home/lvis/src/spectrum/shared/linux/pc64/bin/`uname -r`/*.ko /home/lvis/src/spectrum/shared/linux/pc64/bin/ - Execute the setup script chmod 755 *.sh # not sure why these were not already executable sudo bash ./setup_xmc1151_dvt.sh - Select the proper option: 1)Install ==== Auto-load MIC-3753 Driver ==== - symlink the drivers into the kernel directory sudo ln -s /opt/lvisf/src/MIC3753/linux_driver_source_3.2.7.0_64bit/drivers/bin/*.ko /lib/modules/2.6.32-431.el6.x86_64/kernel/drivers/misc/ - add to our startup script **/etc/sysconfig/modules/lvisf.modules** - modify the file sudo nano /etc/sysconfig/modules/lvisf.modules sudo chmod 755 /etc/sysconfig/modules/lvisf.modules - add the following # start the MIC-3753 drivers /sbin/modprobe biokernbase /sbin/modprobe bio1753 ==== Auto-load Plx9030 Driver ==== **NOTE: (in lieu of step 2 below)** I had this not work when I manually edited it the first time. A copy of the file is in the repository, you could just copy that file in: sudo cp /opt/lvisf/src/lvisF/scripts/lvisf.modules /etc/sysconfig/modules/lvisf.modules - symlink the driver into the kernel so it will come up on boot sudo ln -s /opt/lvisf/src/PLX/PlxSdk/Driver/Source.Plx9000/Output/Plx9030.ko /lib/modules/2.6.32-431.el6.x86_64/kernel/drivers/misc/ - add to our startup script **/etc/sysconfig/modules/lvisf.modules** - modify the file sudo nano /etc/sysconfig/modules/lvisf.modules - add the following # start the Plx9030 syncclock32 driver /sbin/modprobe Plx9030 - symbolically link in our **rc.local** into **/etc/rc.local** - remove the original # remove the symlink sudo rm /etc/rc.local # move the original to something else sudo mv /etc/rc.d/rc.local /etc/rc.d/rc.local_orig - link in the **lvisf** version sudo ln -s /opt/lvisf/src/lvisF/scripts/rc.local /etc/rc.d/rc.local sudo ln -s /etc/rc.d/rc.local /etc/rc.local - verify that the system will not try to launch on boot (unless that is what you want!) # look for this near the top AUTOSTART_LVISF="false" ===== Installing Boost Library ===== The **boost** C++ library is required for the data system. The tarball of the 1.6.2 version is in the repository. Or download the latest from [[http://www.boost.org/users/download/]]. Here is the latest as of 2018.01.04 [[https://dl.bintray.com/boostorg/release/1.66.0/source/boost_1_66_0.tar.bz2]] - Uncompress the tar into **/opt** cd /opt sudo tar -jxvf /opt/lvisf/src/tars/boost_1_62_0.tar.bz2 - Symbolically link to a generic boost directory sudo ln -s /opt/boost_1_62_0 /opt/boost - Chown it all to **lvis** user sudo chown -R lvis:lvis /opt/boost_1_62_0 - Set the environment variable for the **BOOST_ROOT** variable - Edit the file /home/lvis/.bash_profile nano /home/lvis/.bash_profile - Add the line # Add the boost root directory export BOOST_ROOT=/opt/boost - Run our profile to load this variable source /home/lvis/.bash_profile - Compile the **boost** libraries cd /opt/boost ./bootstrap.sh ./b2 -j $(nproc --all) # should figure out number of cpus automagically ===== Installing Ethernet to Serial MOXA Drivers ===== Follow the instructions here: * [[subsystem:datasystem:dev:hardware:moxa:nport5250a:start|Moxa NPort 5250A Series]] ===== Installing USB Temperature Library ===== Follow the instructions below to compile the USB temperature monitoring device libraries required. * [[subsystem:datasystem:dev:hardware:mccdaq:usbtemp:start|Measurement Computing USB-TEMP-AI code instructions. ]] ===== LVIS User Binary Path ===== - Create this directory mkdir ~/bin - Symbolically link in programs into this path from the **scripts** directory ln -s /opt/lvisf/src/lvisF/scripts/applanix* ~/bin/. ln -s /opt/lvisf/src/lvisF/scripts/build* ~/bin/. ln -s /opt/lvisf/src/lvisF/scripts/javad* ~/bin/. ln -s /opt/lvisf/src/lvisF/scripts/lvis* ~/bin/. ln -s /opt/lvisf/src/lvisF/scripts/query* ~/bin/. ln -s /opt/lvisf/src/lvisF/scripts/run* ~/bin/. ====== Clean Boot Operations ====== ===== Load Drivers ===== - Load the MIC-3753 DIO Drivers sudo insmod /home/lvis/src/MIC3753/linux_driver_source_3.2.7.0_64bit/drivers/bin/biokernbase.ko sudo insmod /home/lvis/src/MIC3753/linux_driver_source_3.2.7.0_64bit/drivers/bin/bio1753.ko - Load the SyncClock32 Timer Drivers cd /home/lvis/src/PLX/PlxSdk/Bin sudo ./Plx_load 9030 ===== Compile Code ===== - Enter the **build** directory for lvisf cd /home/lvis/src/lvisF/build - Use **cmake** cmake ../ - Build code make ===== Run Tests ===== ==== User Space Latency Test ==== - Enter the testing program cd /home/lvis/src/lvisF/build/bin/tests/syncclock - Run the latency program sudo ./galvMove ====== Latency Tweaking ====== * Set CPU affinity to CPU 3 with something like, made it much more consistent since I guess most things go for CPU 0 by default CPU_SET(3, &cpus); * Change the scheduler policy, this information is displayed in the **syncclock** test program sudo chrt --fifo 99 ./galvMove * Isolate a CPU, and then have the code attach itself to that cpu * In **/etc/grub.conf** add this command to the kernel command line, so all user space code will avoid CPU 2 and 3 in this example, leaving 0 and 1 for the OS isolcpus=2,3 * Change the program such that CPU_SET(3, &cpus); * Run the program with **chrt** or not sudo chrt --fifo 99 ./galvMove * Set the timer card interrupt affinity to a single CPU (not the prime) sudo bash -c 'echo 2 > /proc/irq/19/smp_affinity' * Using **2** here put the interrupt 19 servicing on CPU 1 exclusively * More low latency items to add to the kernel command line for **/etc/grub.conf** rhgb nousb quiet isolcpus=2,3 nosoftlockup mce=ignore_ce audit=0 * **nosoftlockup** disables backtrace functionality * **mce=ignore_ce** ignores corrected errors * **audit=0** disables kernel components of the audit subsystem ===== Testing ===== * Testing delays * Install **perf** sudo yum install perf * Run this command to record some latency data cd /opt/lvisf/src/lvisF/build/bin/tests/syncclock sudo perf sched record -o /dev/shm/perf.data chrt --fifo 99 ./galvMove * Run the following to read the data sudo perf sched -i /dev/shm/perf.data latency > /tmp/perf.latency.report ===== Show Interrupt Status ===== * This command shows interrupts for each core / CPU cat /proc/interrupts # notice that CPU1 has all the timer card interrupts (PLx9030) [lvis@lvisf syncclock]$ cat /proc/interrupts CPU0 CPU1 CPU2 CPU3 0: 125 0 0 0 IR-IO-APIC-edge timer 8: 1 0 0 0 IR-IO-APIC-edge rtc0 9: 0 0 0 0 IR-IO-APIC-fasteoi acpi 18: 1968 0 0 0 IR-IO-APIC-fasteoi eth2 19: 0 2021327 0 0 IR-IO-APIC-fasteoi Plx9030 24: 0 0 0 0 DMAR_MSI-edge dmar0 25: 0 0 0 0 DMAR_MSI-edge dmar1 26: 2 0 0 0 IR-HPET_MSI-edge hpet2 27: 0 0 0 0 IR-HPET_MSI-edge hpet3 28: 0 0 0 0 IR-HPET_MSI-edge hpet4 29: 0 0 0 0 IR-HPET_MSI-edge hpet5 34: 235 0 0 0 IR-PCI-MSI-edge i915 35: 0 0 0 0 IR-PCI-MSI-edge ahci 36: 7971 0 0 0 IR-PCI-MSI-edge ahci NMI: 0 0 0 0 Non-maskable interrupts LOC: 20569 29227 10924 50493 Local timer interrupts SPU: 0 0 0 0 Spurious interrupts PMI: 0 0 0 0 Performance monitoring interrupts IWI: 0 0 0 0 IRQ work interrupts RES: 324 194 372 353 Rescheduling interrupts CAL: 47 197 178 211 Function call interrupts TLB: 142 610 312 1 TLB shootdowns TRM: 0 0 0 0 Thermal event interrupts THR: 0 0 0 0 Threshold APIC interrupts MCE: 0 0 0 0 Machine check exceptions MCP: 1 1 1 1 Machine check polls ERR: 0 MIS: 0 ===== Test Results ===== * 2016.04.27: Ran a test for 2.7 hours. We missed 46 shots out of 49097902 at 5khz (200 useconds) so as far as actually reacting to the interrupt, we are in good shape. * Time to make a build 1 data system and get better numbers as we do all our tasks.