What started as a project to run "some crypto applications" grew into something much bigger, but also more useful. I learned a tremendous amount about virtual machines, lxc containers and Proxmox. And although I'm just scratching the surface and I don't actually understand it, I know how to do some things. I'll write them down here, partly for myself, but also for you in hopes it will save one of us a lot of time and frustration.
Proxmox has a lot of configuration options. Obviously many of them are dependent on what you want to achieve. In How to install and configure Proxmox I described the most basic configuration options. Here I'll list some more specific ones.
Proxmox config is very delicate. Almost all settings can be configured through the GUI, but sometimes you might have to dig into config files with the CLI. ALWAYS make a backup before doing so! I learned the hard way that messing up only one file (especially related to the cluster) can mean a full reinstall of that cluster!
Go to "Datacenter" -> "Permissions" -> "Groups" and create group
Go to "Datacenter" -> "Permissions" -> "Users" and create user.
Go to "Datacenter" -> "Permissions" click Add -> Group Permissions.
Admins and Role
Open a private browser window and login with your newly created user. Check if you have sufficient permissions (everything should be visible). Logout and close the private browser.
Optional: In your "normal" browser screen logout the root user and login with your own user name. Go to "Datacenter" -> "Permissions" -> "Users" and disable the root user.
This is optional, because some tasks can only be done by the root user. Although this is safer, you have to re-enable the root user each time you need to do one of these restricted/root tasks. I would suggest you disable the root user when you're confident everything is running smoothly.
Go to "Datacenter" -> "Permissions" -> "Users", select the root user and click "TFA". Scan the QR code with your authenticator app.
Repeat for all users, obviously with their own authenticator.
Go to "Datacenter" -> "Permissions" -> "Authentication" and enable TFA for both Realms.
Open a shell to the master node.
Get the private key:
Copy the key to your machine:
C:\users\<username>\.ssh\<hostname>, without file extension.
Open this file with PuTTYgen and save as
C:\users\<username>\.ssh\<hostname>.ppk. The key can now be used with PuTTY.
Store this key somewhere safe.
Disable user-password login (this needs to be done on each machine):
Find and edit lines below:
[...]PasswordAuthentication no[...]UsePAM no[...]
Restart ssh service
systemctl restart ssh
The default file storage location in Proxmox is called
local. VM storage is called
local-zfs when using ZFS or
local-lvm when using LVM. This is fine if you have a single node, but becomes confusing when multiple nodes in a cluster have storage that has the same name. Therefor I renamed it.
Open a shellsession to the Proxmox node.
/etc/pve/storage.cfg and change the names. You cannot rename
local!!! To change, create a new directory storage and disable
If you already have VM's that use this storage (links to ISO on
local and virtual disk on
local-zfs) you have to change them manually.
If you have a NAS than it's almost a must to store VM backups on it. After enabling NFS on the NAS (TrueNas example), you can add the share in Proxmox.
Go to "Datacenter" -> "Storage" and add NFS storage.
Enter details of storage:
<ip address of NAS>
/mnt/pool1/proxmox(this list will show automatically after you enter the server address is entered)
ISO image, Container template, VZDump backup file, Snippets
The NFS store/pool is now available to Proxmox.
After creating an iSCSI target on your NAS/SAN (TrueNas example), you can add the target in Proxmox.
Go to "Datacenter" -> "Storage" and add iSCSI storage.
Enter details of iSCSI target:
<ip address of NAS>:3260
iqn.2005-10.org.freenas.ctl:iscsi-pve1(this list will show automatically after you enter the server address is entered)
The iSCSI target is now available to Proxmox. Click "Add" and choose "LVM".
Enter details of LVM storage:
CH 00 ID 0 LUN 0(in my case)
Disk image, container
You can now add a VM or LXC disk on iSCSI.
Go to "Datacenter" -> "Backup" and add a weekly backup job
If you want to make a backup of TrueNas than you can't store it on a TrueNas network drive. So you have to create a separate backup job for the TrueNas VM that store the backup locally.
This local backup can then be transfered to either another node in the cluster or to the NAS. This can be automated with an rsync cronjob.
Open a shell session to the Proxmox node.
# Every saturday at 0:25 UTC: Copy local backup of TrueNas machine to NAS25 0 * * 6 rsync -avh /var/lib/vz/dump/vzdump-qemu-100* firstname.lastname@example.org:/mnt/pve/nfs-pool1/dump