From VMware to Proxmox VE: How I Migrated a Mixed Windows/Linux Cluster with Ceph, Dual 10Gb Bonds, and PBS

TL;DR: I moved a vSphere cluster (mostly Windows, a few Linux VMs, plus two virtual firewalls) to Proxmox VE. I built a three node Proxmox cluster with out of band IPMI, Ceph for shared storage, redundant Corosync, and VLAN aware networking over an LACP bond to dual 10Gb switches. I finished by standing up a Proxmox Backup Server (PBS) with sane retention & verification. Below is the exact runbook, config snippets, and lessons learned warts and all.

Why Proxmox VE in 2025?

The short version: modern KVM performance, a clean API/UI, tight Ceph integration for shared storage, a now excellent VMware import wizard (added in PVE 8.2), and Proxmox Backup Server for fast, space efficient backups with optional client side encryption. 

The Starting Point

  • Source: VMware vCenter/ESXi
  • Guests: mainly Windows Server (AD/DNS, file/print, IIS, a couple of vendor apps), some Linux VMs (containers, web, logging), and two virtual firewalls
  • Fabric: two 10Gb switches
  • Target: 3× Proxmox VE 8.4.1 nodes, Ceph hyperconverged storage, PBS on a separate box
  • Out of band: IPMI/BMC on its own management network

Network & Storage Design (what I built)

1) IPMI first

Before touching hypervisors, I give each server a static IP on the BMC/IPMI network with unique DNS entries and ACLs. It’s the seatbelt you hope you won’t need.

2) LACP bond to dual 10Gb switches

Each node has two 10Gb NICs in an 802.3ad (LACP) bond. The switches are configured as a single logical entity, so the bond spans both chassis for true switch level redundancy. If your switches don’t support stacking, a single LACP bond cannot be split across two independent switches. (That’s an important gotcha.) 

Proxmox’s Linux bridge supports VLANs natively; making the bridge VLAN aware lets me tag or untag traffic per VM and trunk multiple VLANs to a guest (like pfSense/OPNsense) or any other virtual firewall or virtual network device without extra bridges. 

3) The Proxmox cluster & redundant Corosync

I created the cluster on the first node and joined the others. Corosync got its own VLAN and I added a second Corosync link for failover. Proxmox supports multiple Corosync networks and will switch automatically if one becomes unusable keep latency low and isolate it from storage traffic. 

4) Ceph (hyperconverged)

I deployed Ceph monitors/managers on all three nodes and added OSDs on NVMe SSDs. I used two Ceph networks:

  • public_network (client/front side)
  • cluster_network (replication/back side)

You can run Ceph with one network, but large client traffic benefits from separating the replication network. I mapped each to its own VLAN to keep noise down. 

5) Proxmox Backup Server (PBS)

PBS runs on its own host with ZFS. I configured:

  • Retention: keep‑daily/weekly/monthly via PBS Prune (there’s a handy Prune Simulator in the UI).
  • Verification jobs to detect bit rot.
  • GC (garbage collection) on a schedule.
  • Optional client‑side encryption (AES GCM) for off site sync.

Migration Strategy

I used two paths depending on the VM:

  1. Proxmox VE Import Wizard (PVE 8.2+) the easy button From the Proxmox UI: add ESXi as a storage, pick VM(s), import. Great for most Windows/Linux VMs. 
  2. Manual import – when custom control was needed
    • OVF export from VMware and qm importovf on Proxmox; or
    • Import a VMDK into an existing VM with qm importdisk. qm importovf reads the OVF and converts all VMDKs automatically. 

Example (manual OVF import to storage local-lvm, target VMID 900):

qm importovf 900 MyApp.ovf local-lvm

After import, I adjust machine type, SCSI controller, ballooning, NIC model, and boot order as needed. 

Tip: For large VMs I copy ISOs and OVFs to a local NVMe on the target node, then import from there to keep conversions fast.

Windows VMs: the playbook

Goal: migrate cleanly, then flip to VirtIO everywhere for performance.

  • VirtIO drivers: Windows doesn’t ship VirtIO; mount the virtio win ISO and install the drivers (or use the driver “guest tools” wizard). 
  • Recommended Proxmox settings:
    • Guest OS: “Microsoft Windows 11/2022/2025” (even for Server 2019/2022 is fine)
    • SCSI Controller: VirtIO SCSI (single)
    • Disks: Bus SCSI, enable Discard; optionally enable I/O thread
    • NIC: VirtIO (paravirtualized)
    • QEMU Guest Agent: enabled + install in‑guest These are straight from Proxmox’s Windows best‑practices docs. 

Sequence I follow per VM:

  1. Import VM (wizard or qm importovf).
  2. Boot once with safe defaults if needed (e.g., temporary SATA disk bus) to install VirtIO disk & net drivers.
  3. Switch disk to VirtIO SCSI (single) and NIC to VirtIO.
  4. Install QEMU Guest Agent in the VM. 
  5. Reboot, confirm Device Manager is clean, set balloon and CPU type for your HA/migration strategy.

Note on licensing: Some Windows SKUs will re‑activate because of virtual hardware changes. Plan for that.

Linux VMs: usually “just boot”

Most modern distros already include VirtIO disk/net modules. I still check:

  • /etc/fstab for mount‑by‑UUID (not by old device names),
  • GRUB/EFI boot mode matches the source (UEFI VMs get an EFI disk in Proxmox),
  • install qemu‑guest‑agent package for clean backups and IP reporting. 

Virtual Firewalls (OPNsense / vSRX)

I kept the firewall migration surgical:

  • Backup the config on the old hypervisor first.
  • NIC model: VirtIO is fine on modern FreeBSD, but disable hardware checksum offload in pfSense (System → Advanced → Networking) when using vtnet (VirtIO) to avoid odd traffic issues. 
  • VLANs: I trunk all relevant VLANs to the firewall VM by leaving the VM NIC’s VLAN tag unset in Proxmox (on a VLAN aware bridge). The firewall handles tagging internally. 
  • MACs: Where licensing mattered on the vSRX, I re‑used the guest NIC MACs to avoid surprises.

OPNsense also runs well with VirtIO disks/NICs on KVM/QEMU per their docs. If you hit an edge case during install, temporarily switching the disk to SATA can help. 

Cutover & sequencing

I split the move into waves:

  1. Low risk services first (test storage, networking, backups).
  2. Line of business Windows servers (after nightly maintenance window).
  3. Identity (AD) and the firewalls last, after DNS TTLs were lowered.

I enabled HA for critical VMs after everything lived on shared Ceph RBD pools and qemu guest agent was in place. (HA relies on shared storage and stable Corosync.) 

Backups (PBS): what I set and why

  • Schedules: nightly VM backups to PBS using mode: snapshot, Compression: zstd, and reasonable bandwidth limits.
  • Retention: I used the Prune Simulator to settle on: keep-daily: 7, keep-weekly: 4, keep-monthly: 12.
  • Verification: weekly verify job for the datastore.
  • GC: weekly garbage collection after prune to reclaim chunks.
  • Encryption (optional): enabled client side encryption for any dataset synced off site. All of the above are first class PBS features you can run from the UI or CLI. 

Notes on performance

  • VirtIO everywhere (disk & NIC) on Windows made a visible difference once drivers were in. 
  • The LACP bond spreads flows and gives seamless link/switch failover (with MLAG/vPC on the switch side). 
  • Separate VLANs for Corosync and Ceph traffic produced steadier cluster behavior under load. 

Gotchas I hit (so you don’t have to)

  1. UEFI vs BIOS: Match firmware. If the source VM used UEFI, add OVMF (UEFI) + EFI disk in Proxmox before first boot.
  2. Windows NIC order: When swapping from vmxnet3/E1000 to VirtIO, Windows may re‑index NICs. I tidy up hidden adapters afterward.
  3. pfSense VirtIO quirk: Remember the checksum offload checkbox. It fixes “it boots, but weird traffic dies.” 
  4. Bonds across two switches: Make sure your switch pair is doing MLAG/vPC/stacking; otherwise, a single LACP bond cannot straddle two chassis. 

What I’d do again

  • Use the Import Wizard for the bulk, fall back to qm importovf only when I need to micromanage. 
  • Keep Corosync isolated and redundant from day one. 
  • Stand up PBS immediately and test a full restore (not just a verify). 

Handy reference links

  • Migrate to Proxmox VE (incl. automatic ESXi import, live‑import): Proxmox docs. 
  • Import wizard video (PVE 8.2+): Proxmox. 
  • qm man page (incl. qm importovf): Proxmox. 
  • Windows VirtIO drivers on Proxmox & virtio‑win ISO: Proxmox wiki + Fedora repo. 
  • Windows guest best practices (bus/controller/agent): Proxmox. 
  • QEMU Guest Agent (what/why/how): Proxmox. 
  • Network config (bonds, VLAN‑aware bridges): Proxmox. 
  • Ceph public vs cluster network: Ceph docs. 
  • pfSense VirtIO checksum offload note: Netgate docs. 
  • PBS prune/verify/GC & encryption: PBS docs. 

Copy‑paste checklist

  • IPMI/BMC reachable and documented
  • Proxmox 8.x installed on all nodes
  • LACP bond to MLAG/vPC switch pair; vmbr0 VLAN‑aware
  • Dedicated VLANs: Mgmt, Corosync (x2), Ceph public, Ceph cluster
  • Proxmox cluster created; second Corosync link added
  • Ceph MON/MGR/OSD up; RBD pools ready
  • PBS online; schedules, retention, verification, GC set
  • Import VMs (wizard first, manual if needed)
  • Windows: VirtIO disk/net, QEMU agent, reboots clean
  • Linux: qemu‑guest‑agent installed; fstab/grub OK
  • Firewalls: config backed up; VirtIO + checksum offload disabled; VLAN trunk verified
  • HA enabled (after shared storage confirmed)

Posted

in

, ,

by

ARPHost

Tags:

, , , , ,