I have two HDD 2TB each. I configured RAID1 in my BIOS. I checked if RAID is ok. Ubuntu installation see my RAID. But Proxmox installation don't see my RAID, proxmox see 2 different HDD. All Proxmox VE versions does not support the Linux Software RAID (mdraid).That means, its not tested in our labs and its not recommended, but its still used by experience users.
In tagged / by Proxmox installer is not able to do when you are prompted to software raid, here’s a quick tutorial how to migrate proxmox 3.3 to software raid 1 immediately after installation:. Comment out existing repository, add free of use: # vi /etc/apt/sources.list.d/pve-enterprise.list # echo 'deb wheezy pve pve-no-subscription' /etc/apt/sources.list.d/proxmox.list. Intsall packages that we need: # apt-get update && apt-get install mdadm initramfs-tools screen. Copy the partition table on the second hdd: # sgdisk -R /dev/sdb /dev/sda The operation has completed successfully.
# sgdisk -G /dev/sdb The operation has completed successfully. # sgdisk -randomize-guids -move-second-header /dev/sdb The operation has completed successfully. Change the partition type to raid # parted -s /dev/sdb set 2 'raid' on # parted -s /dev/sdb set 3 'raid' on. Next: # mdadm -zero-superblock /dev/sdb2 mdadm: Unrecognised md component device - /dev/sdb2 # mdadm -zero-superblock /dev/sdb3 mdadm: Unrecognised md component device - /dev/sdb3. Create arrays: # mdadm -create /dev/md1 -level=1 -raid-disks=2 missing /dev/sdb2 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device.
If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use -metadata=0.90 Continue creating array? Y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. # mdadm -create /dev/md2 -level=1 -raid-disks=2 missing /dev/sdb3 mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use -metadata=0.90 Continue creating array? Y mdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md2 started.
Contents. Intro is presently my GUI of choice for using KVM. However, during Proxmox’s ISO install you are only given the choice of what disk to install but not the layout. There are other articles on how to do this but for VE 3.2 there is no single place you can find instructions because the partition type changed from MSDOS to GPT. These instructions only concern fresh installs of Proxmox VE 3.2 from the because if you upgraded — performed a dist-upgrade — your will retain the disk layout of 3.1. Fresh install of Proxmox VE 3.2 My setup Install type Bare metal Proxmox Version 3.2-5a885216-5 built on 2014-05-06 Install disk /dev/sda: a WD 1TB black Mirrored disk /dev/sdb: a WD 1TB black RAID setup mdadm –level=1: mirror setup Step 1: Install and setup Disk prep Before you even start your install of Proxmox I highly recommend — unless you are sure about your disk history — that you boot into something like or and clear the two disks you intend to use for the OS mirror.
It is a good idea when setting up RAID that you use the exact same disk models but it is not a requirement. Once you boot into your live CD or choice use parted or gparted to clear all partitions on the disks you want to use. A second and very important step before setting up RAID is making sure the disks don’t have any hardware or software RAID metadata on them. Rather than worry about if the disks have this data just clear where it lives with these two commands — after you remove all partitions –. The following command will clear the first 512.10 bytes of the disk. This is me being overzealous as the RAID metadata is only suppose to live in the first 512 bytes. Dd if=/dev/zero of=/dev/sdX bs=512 seek=$(( $(blockdev -getsz /dev/sdX) - 1 )) count=1 Make sure you run both of these commands for each sdX where X is your drive (a/b/c/d etc).
Now we can be certain no previous RAID data is going to cause major headaches and confusion with our install going forward. Hint: if you find your /dev/md0 or /dev/md1 showing up as md126 or something similar after a reboot you probably didn’t clear the drives correctly. Proxmox Install Boot from your ISO or and perform the install. Keep track of which /dev/sdX you install to; in my example I am using /dev/sda as my install disk.
Once your installation is completed you’ll want to make sure you have internet connectivity. There is a good chance you don’t have a Proxmox community subscription so if you don’t have a license key follow the instructions in the link below to change over to the “pve-nosubscription” repo. Once you have completed these steps run the following commands to get your Proxmox install up to date. (parted) print Model: ATA WDC WD1002FAEX-0 (scsi) Disk /dev/sda: 1000GB Sector size (logical/physical): 512B/512B Partition Table: gpt Number Start End Size File system Name Flags 1 1049kB 2097kB 1049kB primary biosgrub 2 2097kB 537MB 535MB ext3 primary boot 3 537MB 2000GB 2000GB primary lvm (parted) We can see from this list that partitions 2 and 3 are the ones we need to mirror through MDADM. Remember that in my setup I am using /dev/sda and /dev/sdb but that you may have decided to use other disks so you will need to transpose your device names. Cloning the installed partitions into RAID Because Proxmox VE 3.2 uses GPT instead of MSDOS we have to use the tool sgdisk instead of sfdisk. Run the following commands to prep your blank second disk.
(parted) select /dev/sdb Using /dev/sdb (parted) print Model: ATA WDC WD1002FAEX-0 (scsi) Disk /dev/sdb: 1000GB Sector size (logical/physical): 512B/512B Partition Table: gpt Number Start End Size File system Name Flags 1 1049kB 2097kB 1049kB primary biosgrub 2 2097kB 537MB 535MB primary raid 3 537MB 1000GB 1000GB primary raid (parted) Create your RAID array Since you already installed MDADM earlier on creating the two RAID arrays is simple. However, we are still booted off of a non-raid /dev/sdaX partition so we are going to add a “missing” disk that will later be replaced by /dev/sdaXs. The following commands will create the arrays. Mdadm: /dev/sdd2 appears to contain an ext2fs file system size=1950656K mtime=Tue Aug 19 11: mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use -metadata=0.90 Continue creating array? Ultimately /dev/md0 is going to hold your /boot while /dev/md1 will hold your LVM partition.
Install /boot to your newly created /dev/md0 We are going to format /dev/md0 as ext3, copy grub to it and set it up as our /boot mount. /dev /pve /root / ext3 errors=remount-ro 0 1 /dev /pve /data /var /lib /vz ext3 defaults 0 1 /dev /md0 /boot ext3 defaults 0 1 #UUID=46b4d3d6-fdec-43b6-a4cb-3f8f8a9c6c10 /boot ext3 defaults 0 1 /dev /pve /swap none swap sw 0 0 proc /proc proc defaults 0 0 You can see I commented out the /dev/sda UUID and replaced it with /dev/md0. At this point we are ready to reboot but I must stress that this work requires accuracy and if mistakes are made you might need to start over because the system won’t boot.
Once your reboot is complete typing the following command will show you were /boot was mounted from. Sgdisk -t 3:fd00 /dev/sda mdadm -add /dev/md1 /dev/sda3 At this point /dev/sda3 will start healing to become a clone of /dev/sdb3. If you made it this far you are now running Proxmox VE 3.2 software raid! Conclusion: Proxmox VE 3.2 on software raid The extra work is worth it Proxmox is a great ISO install for running KVM or OpenVZ.
Install Proxmox On Usb
However, the installation does not give you options like software raid — I suspect to make it as simple as possible –. While performing the above steps may seem like a burden it gives you the safety of being able to lose one of your OS drives without having to introduce hardware RAID. Don’t use this for VM storage For several reasons I would recommend storing your VM disk files. It’s good practice to have OS and data storage independent of each other. I find that MDADM mirrored storage is much slower than single disk write performance Below is a test on the array of two 1TB WD Blacks.
Thank you for a great tutorial! I encountered an issue on a completely fresh install of Proxmox 3.3 (both sda and sdb unpartitioned) in the ‘mdadm –create /dev/md0’ step, because a md0 array already existed (probably configured when mdadm was installed).
So I had to do the following steps before continuing with the guide: 1) mdadm –stop /dev/md0 2) mdadm –remove /dev/md0 Also, before the reboot, the following was required to allow success on the boot (otherwise the boot loader complained about an ext2 filesystem issue): 1) mdadm –zero-superblock /dev/sda1 2) remove the md0 array definition from mdadm.conf 3) mdadm –detail –scan /etc/mdadm/mdadm.conf. I’ve extended this method to a LVM on RAID10(mdadm) configuration. Just wanted to say thank you for this tutorial! We have been in the plans of migrating our VMs all to ProxMox and their respective storage to Ceph through ProxMox.
With this I had setup all of the systems with raid 1 on the boot drives and today (about 3 weeks into stability testing) one of the boot drives failed – the server chugged along happily and I was easily able to shut it down (one of the ceph boxes), replace the drive, add it to the array and it is rebuilding itself. Without this guide I think that box would have been lost and I would be rebuilding it from scratch! Can I add my thanks – a really clear tutorial and very helpful! Also just wanted to note for the benefit of others that I did see some warnings when installing grub to sda1 and sda2: root@proxmox:# grub-install /dev/sda Installing for i386-pc platform. Grub-install: warning: Couldn’t find physical volume `(null)’. Some modules may be missing from core image. Grub-install: warning: Couldn’t find physical volume `(null)’.
Some modules may be missing from core image. Installation finished. No error reported. I haven’t chased down the root cause, but it appears to be a similar problem to that discussed at I decided to just risk it and reboot to see if it cleared the warning, as it did for various of the people posting on the page above. The server came back fine and installing grub works cleanly now. So I tried to follow this guide for Proxmox 3.4.
It was pretty similar, except /boot is part of the root partition, and /dev/sda2 is mounted at /boot/efi. So I did the changes accordingly, copied /boot/efi to /dev/sdb2 and set up /dev/md0 and changed /etc/fstab accordingly, rebooted and everything worked. However, when I tried to continue with the LVM partition, everything seems to work until I reboot, and I get thrown into the grub-rescue-prompt saying it can’t find lvmid/long UUID. I’m dead stuck here, and have no idea what to do or how to fix this. I tried using a live Linux Mint, but it’s unable to work with the files inside the raid inside the lvm system.
Any suggestions? I am having problems getting this to work on 3.4-11 I get to the point of editing /etc/fstab and the UUID line is not there (as it was in 3.4-3 which I had this working beautifully on), and if I add the line that used to work before – to boot off of /dev/md0 – after reboot I am told there is no /dev/md0 and am put into a recovery console. So I remove that line from fstab, reboot without any issue, and sure enough there is no md0. Have to put off upgrading all my nodes because this will break them all for sure.