• Need to start with ubuntu server, not desktop so the IP address stuff is fixed / static
• OR figure out how to fix this desktop install

Try this as your netplan file:

# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager
  ethernets:
    eno1:
      renderer: networkd
      match:
        name: eno1
      addresses: [10.4.1.10/24]
      gateway4: 10.4.1.254
      nameservers:
        search: [nasa.gov]
        addresses: [128.183.10.134, 128.183.50.17, 8.8.8.8]

• Probably need more swap space than 2GB. Have 64GB of ram so probably should have more like 32GB of swap at least. The multi-threaded zoom 12 tile generation ran into ram / swap issues before finishing with only 2GB available. Wonder if you can add temp swap space?

Could try this: https://linuxize.com/post/how-to-add-swap-space-on-ubuntu-18-04/

1. Turn off swap so we can resize the swapfile

   sudo swapoff -a

2. Create our 32GB swapfile

   sudo fallocate -l 32G /swapfile

3. Offically make this a swap file

   sudo mkswap /swapfile

4. Check /etc/fstab that swap is in there (it is)
5. Can
   1. Turn on manually with

      sudo swapon /swapfile

   2. OR just reboot and it should show up as activated

1. Update

   sudo apt update

2. Upgrade

   sudo apt upgrade

sudo apt-get install openssh-server
sudo systemctl enable ssh
sudo systemctl start ssh

I am still a big fan of this display

sudo apt install kde-full
sudo apt install kdevelop

1. Start with a base version of Ubuntu 20.04 LTS
2. Install base packages

   sudo apt -y install htop jed screen
   sudo apt -y install gimp
   sudo apt -y install net-tools
   sudo apt-get -y install build-essential
   sudo apt-get -y install autoconf automake cmake gdb git libffi-dev zlib1g-dev libssl-dev
   sudo apt -y install ntpdate
   sudo apt -y install unzip pigz md5deep
   sudo apt -y install gfortran
   sudo apt-get -y install libhdf5-serial-dev
   sudo apt -y install hdf5-tools
   sudo apt-get -y install libopenblas-dev
   sudo apt-get -y install libnetcdf-dev libnetcdff-dev
   sudo apt -y install libtiff-dev

3. EXFAT read and write

   sudo add-apt-repository universe
   sudo apt update
   sudo apt -y install exfat-fuse exfat-utils

4. Install python2 and requirements for Sarah's python code

   sudo apt -y install python2

   1. Install python3 modules using pip

      sudo apt -y install python3-pip
      pip3 install numpy --user
      pip3 install h5py --user
      pip3 install scipy --user
      pip3 install matplotlib --user

5. KML browser will want this

   sudo apt -y install firefox
   sudo apt -y install imagemagick

6. Install GIS code

   sudo add-apt-repository -y ppa:ubuntugis/ppa
   sudo apt update
   sudo apt -y install gdal-bin  # I did also do this from source, but not necessary
   # sudo apt install gmt # build from source did not work ubuntu 20.04
   sudo apt -y install qgis

7. Google Earth
   1. Grab the installer

      mkdir ~/down
      cd ~/down
      wget -O ~/down/google-earth.deb https://dl.google.com/dl/earth/client/current/google-earth-pro-stable_current_amd64.deb

   2. Install

      sudo dpkg -i ~/down/google-earth.deb

NOTE: Jason is an excellent programmer and his code rocks!

1. Expand source

   mkdir ~/repos
   cd ~/repos
   tar -zxvf ~/down/dlrdem_src.tar.gz

2. Use cmake to create the compile configurations

   cd ~/repos/dlrdem/
   mkdir build
   cd build
   cmake ..

3. Compile (you can use -jCORES)

   CPUCOUNT=$(grep -c   "^processor" /proc/cpuinfo)
   make -j $CPUCOUNT

4. Link into our ~/bin dir

   mkdir ~/bin
   ln -s ~/repos/dlrdem/build/interpolate_dlrdem ~/bin

5. Copy the DEM folder from another computer

   rsync -vaP lvis@LOCALIP:dem/ ~/dem/

1. Expand source

   mkdir ~/repos
   cd ~/repos
   tar -zxvf ~/down/vegas_source.tar.gz

2. Use cmake to create the compile configurations

   cd ~/repos/VEGAS/code-2021-07-14
   rm -rf build
   mkdir build
   cd build
   cmake ..

3. Compile (you can use -jCORES)

   CPUCOUNT=$(grep -c   "^processor" /proc/cpuinfo)
   make -j $CPUCOUNT

4. Link into our ~/bin dir

   mkdir ~/bin
   ln -s ~/repos/VEGAS/code-2021-07-14/build/vegas ~/bin/

1. Uncompress the source

   mkdir ~/repos
   cd ~/repos
   unzip ~/down/lvisf-master.zip

2. Compile (I should make this part of the cmake stuff)

   cd ~/repos/lvisf-master/rtp/rtp_lvisf2ql2

3. Compile

   make

4. Link the binaries

   ln -s ~/repos/lvisf-master/rtp/rtp_lvisf2ql2/rtp_lvisf2ql2 ~/bin/
   ln -s ~/repos/lvisf-master/rtp/rtp_lvisf2ql2/rtp_ql2filter ~/bin/
   ln -s ~/repos/lvisf-master/rtp/rtp_lvisf2ql2/rtp_ql2vegas ~/bin/

1. Go to

   /home/lvis/repos/lvisf-master/bin/utils/rtpd_nav

2. Compile

   make

3. Link daemon

   ln -s ~/repos/lvisf-master/bin/utils/rtpd_nav/rtpd_nav ~/bin/

NOTES:

• I attempted to get OpenMP working on GMT but it turns out that it doesn't even show up in the nearneighbor module so this really doesn't help out. So multi-thread GMT, not so much, but what I ended up doing is just launching lots of threads, and that works fine.
• Ubuntu gmt was not working so compiling from source is mandatory to get valid results (my range on the data sets made no sense, so some kind of default variable size issue probably)

1. Install some more packages required to compile

   sudo apt -y install subversion build-essential cmake libcurl4-gnutls-dev libnetcdf-dev libgdal-dev libfftw3-dev libpcre3-dev liblapack-dev libblas-dev
   sudo apt -y install libglib2.0-dev ffmpeg graphicsmagick

2. Go to src directory

   mkdir ~/src
   cd ~/src

3. Clone the git repository

   git clone https://github.com/GenericMappingTools/gmt.git

4. Run cmake

   cd gmt
   mkdir build
   cd build
   cmake -D GMT_USE_THREADS=true -D GMT_ENABLE_OPENMP=true ..

5. Build

   CPUCOUNT=$(grep -c   "^processor" /proc/cpuinfo)
   make -j $CPUCOUNT

6. Install

   sudo make install

1. Get the boost development libraries

   sudo apt-get install libboost-all-dev

2. Compile our raw lvis to quicklook program
   1. Get into directory

      cd /home/lvis/repos/lvisf-master/bin/utils/rawlvis2quicklook

   2. Make symbolic links

      ln -s ../../../lib/lvisf_utils/lvisf_utils.* .
      ln -s ../../../bin/utils/parseData/xmcParseMeta.cpp

   3. Compile this code together

      g++ -Wall -O2 \
          lvisf_utils.cpp \
          rawlvis2quicklook.cpp \
          xmcParseMeta.cpp \
         -I../../../include \
         -I../../../bin/utils/parseData \
         -o rawlvis2quicklook

3. Link in the new binary

   ln -s ~/repos/lvisf-master/bin/utils/rawlvis2quicklook/rawlvis2quicklook ~/bin/

• From: https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nfs-mount-on-centos-6

1. Install these packages on the Centos 6 back end by hand (this was a little painful) since end of life for CentOS 6.5

   sudo su  # to get sudo prompt
   rpm -ivh libgssglue-0.1-11.el6.x86_64.rpm 
   rpm -ivh libtirpc-0.2.1-6.el6_4.x86_64.rpm 
   rpm -ivh rpcbind-0.2.0-11.el6.x86_64.rpm
   rpm -ivh keyutils-1.4-5.el6.x86_64.rpm
   rpm -ivh libevent-1.4.13-4.el6.x86_64.rpm
   rpm -ivh --nodeps  nfs-utils-lib-1.1.5-6.el6.x86_64.rpm 
   rpm -ivh nfs-utils-1.2.3-39.el6.x86_64.rpm 

2. Export our internal drive with an /etc/exports file that looks like this

   /mnt/internal               10.4.1.11(ro,sync,all_squash,no_subtree_check)

3. Enable the NFS service to be launched on startup (Turn On a Service for the Default Run Levels (2,3,4,5) from here)

   chkconfig nfs on 

4. Kick off the NFS service (NFS is dependent on RPC)

   service rpcbind start
   service nfs start

5. Show what is being shared up with

   showmount -e 10.4.1.2

• From: https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nfs-mount-on-ubuntu-20-04

1. Install these

   sudo apt update
   sudo apt install nfs-common

2. Make a mount point on our server

   sudo mkdir -p /nfs/lvisf/internal

3. Mount the directory
   1. Add this line to the /etc/fstab to mount this on startup

   2. OR Mount the remote directory manually from the command line

      sudo mount 10.4.1.2:/mnt/internal /nfs/lvisf/internal

Source: https://www.digitalocean.com/community/tutorials/how-to-install-the-apache-web-server-on-ubuntu-18-04-quickstart

1. Install httpd

   sudo apt update
   sudo apt install apache2

2. Adjust firewall if necessary (was not necessary (see link above) but this shows apache already through the firewall

   sudo ufw app list

   1. See apache is on the list

      Available applications:
        Apache
        Apache Full
        Apache Secure
        CUPS
        OpenSSH

3. Create a directory where the output can be written to

   sudo chmod 775 /var/www/html
   sudo chgrp -R lvis /var/www/html
   mkdir /var/www/html/MJD

4. Just need to add this line to each html overlay desired
   1. Leaflet is like this

             // Overlay layers (TMS)
              var lyr = L.tileLayer('http://10.4.1.11/59431/{z}/{x}/{y}.png', {tms: 1, opacity: 0.7, attribution: "", minZoom: 0, maxZoom: 10});

   2. Openlayers is like this

      url: 'http://10.4.1.11/59431/{z}/{x}/{-y}.png'

• https://switch2osm.org/serving-tiles/manually-building-a-tile-server-20-04-lts/ will show you how to setup a local tile server to create tiles in real time based on the OSB planet data base file you are now downloading (PBF snapshot of the Earth file) I wish they had the topo one?
• https://www.postgresql.org/docs/6.3/c0702.htm alternate location of the data base, as it will not fit on the root drive so refer to this link
• Download entire earth PBF file here: https://planet.openstreetmap.org/

1. Create the user account to render the tiles
   1. Open a sudo shell

      sudo -i

   2. Add the user and set permissions

      adduser renderaccount
      # answer the questions and then
      usermod -aG sudo renderaccount
      exit

2. Log in to this account
3. Install some required software

   sudo apt install libboost-all-dev git tar unzip wget bzip2 build-essential autoconf libtool libxml2-dev libgeos-dev libgeos++-dev libpq-dev libbz2-dev libproj-dev munin-node munin protobuf-c-compiler libfreetype6-dev libtiff5-dev libicu-dev libgdal-dev libcairo2-dev libcairomm-1.0-dev apache2 apache2-dev libagg-dev liblua5.2-dev ttf-unifont lua5.1 liblua5.1-0-dev

1. Install data base packages needed

   sudo apt install postgresql postgresql-contrib postgis postgresql-12-postgis-3 postgresql-12-postgis-3-scripts

2. Migrate our database to the other drive as it has way more space
   1. Check WHERE the current data base is located
      1. Fire up psql

         sudo -u postgres psql

      2. Show the directory being used

         SHOW data_directory;

      3. Result for us today is

         /var/lib/postgresql/12/main

   2. Shut down the postgresql server so we can modify the data base location

      sudo systemctl stop postgresql
      sudo systemctl status postgresql

   3. Copy the existing data base to the other drive

      sudo rsync -av /var/lib/postgresql /mnt/rtp

   4. Move the original data just in case

      sudo mv /var/lib/postgresql/12/main /var/lib/postgresql/12/main.bak

   5. Point the database to the new location
      1. Edit the configuration file

         sudo nano /etc/postgresql/12/main/postgresql.conf

      2. Change
         1. Original

            data_directory = '/var/lib/postgresql/12/main'          # use data in another directory

         2. To this

            data_directory = '/mnt/rtp/postgresql/12/main'          # use data in another director

   6. Restart the data base

      sudo systemctl start postgresql
      sudo systemctl status postgresql

   7. Log in and check the data base
      1. Log into the server

         sudo -u postgres psql
         SHOW data_directory;

      2. Results show that we have moved the database!

                data_directory        
         -----------------------------
          /mnt/rtp/postgresql/12/main
         (1 row)

      3. Quit out

         CONTROL-D or \q to quit

3. Create our database
   1. Create the actual database

      sudo -u postgres -i
      createdb -E UTF8 -O renderaccount gis

   2. A bunch of database commands to alter the database

      psql
      \c gis
      CREATE EXTENSION postgis;
      CREATE EXTENSION hstore;
      ALTER TABLE geometry_columns OWNER TO renderaccount;
      ALTER TABLE spatial_ref_sys OWNER TO renderaccount;
      \q
      exit

sudo apt install osm2pgsql

sudo apt install autoconf apache2-dev libtool libxml2-dev libbz2-dev libgeos-dev libgeos++-dev libproj-dev gdal-bin libmapnik-dev mapnik-utils python3-mapnik python3-psycopg2 python3-yaml

1. Install from code

   mkdir ~/src
   cd ~/src
   git clone -b switch2osm git://github.com/SomeoneElseOSM/mod_tile.git
   cd mod_tile
   ./autogen.sh

2. Make and install

   ./configure
   make
   sudo make install
   sudo make install-mod_tile
   sudo ldconfig

1. Grab stuff

   cd ~/src
   git clone git://github.com/gravitystorm/openstreetmap-carto.git
   cd openstreetmap-carto

2. Add some more things

   sudo apt install npm
   sudo npm install -g carto
   carto -v

3. Response should be at least

   1.2.0

4. Use this to convert something Mapnik can understand

   carto project.mml > mapnik.xml

NOTE: This does give you water and land at least the basics, this is probably good enough.

1. Grab some sample data to get a taste and see if we can get this working

   mkdir ~/data
   cd ~/data
   wget https://download.geofabrik.de/asia/azerbaijan-latest.osm.pbf

2. Import the data and use all the threads we have

   osm2pgsql -d gis --create --slim  -G --hstore --tag-transform-script ~/src/openstreetmap-carto/openstreetmap-carto.lua -C 2500 --number-processes 16 -S ~/src/openstreetmap-carto/openstreetmap-carto.style ~/data/azerbaijan-latest.osm.pbf

VanGilst said THIS WILL TAKE A WHILE! You could leave it alone, it looks pretty good as is. But, let's break it.

time osm2pgsql -d gis --create --slim  -G --hstore --tag-transform-script ~/src/openstreetmap-carto/openstreetmap-carto.lua -C 2500 --number-processes 16 -S ~/src/openstreetmap-carto/openstreetmap-carto.style /mnt/rtp/tmp/planet-220117.osm.pbf

If you want a smaller pbf file which should take less time to ingest, you can grab the subdivisions here:

• https://download.geofabrik.de/ ← Get smaller PBF files here!

1. Run this

   cd ~/src/openstreetmap-carto/
   psql -d gis -f indexes.sql

2. And you will get back “CREATE INDEX” 14 times

cd ~/src/openstreetmap-carto/
scripts/get-external-data.py

sudo apt install fonts-noto-cjk fonts-noto-hinted fonts-noto-unhinted ttf-unifont

1. Setup renderd configuration

   sudo nano /usr/local/etc/renderd.conf

   1. NOTE: Since we used renderaccount and have plenty of memory, all the defaults are OK
2. This line, verify this is correct by doing an ls

   XML=/home/renderaccount/src/openstreetmap-carto/mapnik.xml

3. Change the version from 3.0 to 3.1 as of this writing
   1. This

      plugins_dir=/usr/lib/mapnik/3.0/input

   2. Should be this

      plugins_dir=/usr/lib/mapnik/3.1/input

1. Some housekeeping

   sudo mkdir /var/lib/mod_tile
   sudo chown renderaccount /var/lib/mod_tile
   
   sudo mkdir /var/run/renderd
   sudo chown renderaccount /var/run/renderd

2. Edit this file

   sudo nano /etc/apache2/conf-available/mod_tile.conf

   1. And add the following line

      LoadModule tile_module /usr/lib/apache2/modules/mod_tile.so

3. Run this to enable the tile mod

   sudo a2enconf mod_tile

4. Edit this file and add these lines where they go

   sudo nano /etc/apache2/sites-available/000-default.conf

   1. And add the following between the “ServerAdmin” and “DocumentRoot” lines:

      LoadTileConfigFile /usr/local/etc/renderd.conf
      ModTileRenderdSocketName /var/run/renderd/renderd.sock
      # Timeout before giving up for a tile to be rendered
      ModTileRequestTimeout 0
      # Timeout before giving up for a tile to be rendered that is otherwise missing
      ModTileMissingRequestTimeout 30

5. Reload apache2 twice

   sudo service apache2 reload
   sudo service apache2 reload

1. Kick off renderd in the foreground just running it on a console

   sudo mkdir /var/lib/mod_tile
   sudo chown renderaccount /var/lib/mod_tile
   sudo mkdir /var/run/renderd
   sudo chown renderaccount /var/run/renderd
   renderd -f -c /usr/local/etc/renderd.conf

2. Go to your local browser and see if this renders an image: http://localhost/hot/0/0/0.png

After finding the 3.0 to 3.1 change in the renderd configuration file, it worked for me!

1. Check that renderaccount is the RUNASUSER in this file

   nano ~/src/mod_tile/debian/renderd.init

2. Copy this into our init directory

   sudo cp ~/src/mod_tile/debian/renderd.init /etc/init.d/renderd
   sudo chmod u+x /etc/init.d/renderd
   sudo cp ~/src/mod_tile/debian/renderd.service /lib/systemd/system/

3. Start the service

   sudo /etc/init.d/renderd start

4. Set so that it starts on startup

   sudo systemctl enable renderd

Try our locally hosted map now! http://10.4.1.10/rtp_leaflet.html