ZFS or Openfiler or Vail

Soldato
Joined
21 Jan 2010
Posts
21,950
Any suggestions?

6x 1TB drives, 250gb VM pool drive - using ESXi. I have vt-d available to me.

I am passing the disk controller to the VM.

Do I use ZFS (OpenIndiana) and present iSCSI to Vail? Or do I install WHS and let that deal with the storage?
 
Associate
Joined
10 Nov 2004
Posts
2,237
Location
Expat in Singapore
What's your goal ?.

Other options may be FreeNAS, UnRaid, Windows striping / spanning, controller raid. Really depends on what you want to get out of it in the end.

I take it you are keeping the VM 250GB drive for ESXi and the 6x1TB drives on a separate controller to the VM.

RB
 
Soldato
OP
Joined
21 Jan 2010
Posts
21,950
Correct -> 250GB is going for VM storage (ESXi is on a usb) and the controller with the 1TB drives on is being passed over to whatever storage VM I chose.

I was happy with WHSv1 to be honest, in the sense that it let me specify from my pooled drives which ones I wanted to backup. However it would be good to sacrifice 1 of the drives to protect all of the data.

I can then run WHS 2011 on the 250GB vmstore, or pass it another drive, for crucial backups from the storage server.

I am now also investigating SnapRAID or similar.

FreeNAS is still viable.

I think ZFS/OpenIndiana is out as I want to add new drives to the pool as and when I see fit.
 
Associate
Joined
25 Nov 2006
Posts
1,906
Location
Birmingham
ZFS is only useful if you have a lot of ram, at least 8GB at minimum IMO if not more. You can also use SSD's for write cache and logs if you have any to spare.

How many vm's do you plan on running?
 
Associate
Joined
3 Oct 2008
Posts
1,890
Location
South London
Are you using ESXi in free mode or with a license?

Before you go any further you should be aware you can't passthrough physical devices in the free version. You'll be met with a "Host does not support passthrough" message once the 60day trial expires.
 
Associate
Joined
10 Nov 2004
Posts
2,237
Location
Expat in Singapore
Are you using ESXi in free mode or with a license?

Before you go any further you should be aware you can't passthrough physical devices in the free version. You'll be met with a "Host does not support passthrough" message once the 60day trial expires.

You get an ESXi free version license (unsurprisingly free once registered on VMWares website) to convert it from trial version to licensed free version. Passthrough works fine with a licensed free version. It can bit a PITA finding the key via VMWares terrible account management system on their website though so I tend to record mine to save trying to hunt for it on-line when needed.

ESXi trial and Free are two separate version licensing levels. The trial gives everything for a limited time (60 days) but the free licensed version gives the basic set of features (up to 32GB Ram, VT-d etc) for unlimited time but restricts things like vMotion, Addition to vCentre etc which are the value adds of the higher level licenses.

RB
 
Associate
Joined
3 Oct 2008
Posts
1,890
Location
South London
You get an ESXi free version license (unsurprisingly free once registered on VMWares website) to convert it from trial version to licensed free version. Passthrough works fine with a licensed free version. It can bit a PITA finding the key via VMWares terrible account management system on their website though so I tend to record mine to save trying to hunt for it on-line when needed.

ESXi trial and Free are two separate version licensing levels. The trial gives everything for a limited time (60 days) but the free licensed version gives the basic set of features (up to 32GB Ram, VT-d etc) for unlimited time but restricts things like vMotion, Addition to vCentre etc which are the value adds of the higher level licenses.

RB
My Lab machines are 'licensed' with the free key (the one they email to you when you register for the download - I assume this is the one you mean?), but they still won't let me do passthrough anymore. Same thing happened on 4.1 too. My live machines with Enterprise licenses one them (Same hardware) do let me passthrough. At least that's the case through the windows client, I haven't tried CLI on 5 because it's different to 4 and I haven't been bothered to learn it yet.

If this isn't normal then methinks it's time to open a VMWare ticket.
 
Last edited:
Associate
Joined
10 Nov 2004
Posts
2,237
Location
Expat in Singapore
My Lab machines are 'licensed' with the free key (the one they email to you when you register for the download - I assume this is the one you mean?), but they still won't let me do passthrough anymore. Same thing happened on 4.1 too. My live machines with Enterprise licenses one them (Same hardware) do let me passthrough. At least that's the case through the windows client, I haven't tried CLI on 5 because it's different to 4 and I haven't been bothered to learn it yet.

Does the server overview screen (with cores, ram, etc) still say it is direct path IO supported as in the pic below ? (Mods I own the photobucket account I am linking too). Yes this is a trial setup as it is a new build for a client using the Supermicro X9SCi-LN4F board with 4 GbE Nics onboard.
ESXivSphere1.png


Does it not let you passthrough new devices or do devices originally passed suddenly revert to no passthrough ?.

What are you passing through ?. I am only passing through NICs and Sas controllers on mine although it seems there are a number fo links in Google relating to USB passthrough and configuration loss.

I have not seen it on my own ESXi server (free licensed version) and the abilities change that occurs between the 60 day free trial and the free version license were logged as soon as I put the license key in to vSphere. Nothing has changed since then.

RB
 
Associate
Joined
3 Oct 2008
Posts
1,890
Location
South London
There weren't any originally, but I noticed it when I was testing if some application servers would virtualise which use USB license dongles after the trial period ended and the advanced features turned off I couldn't pass anything through.
It may well be the case that if I'd configured passthrough before the trial expired they'd have remained.
 
Associate
Joined
10 Nov 2004
Posts
2,237
Location
Expat in Singapore
There weren't any originally, but I noticed it when I was testing if some application servers would virtualise which use USB license dongles after the trial period ended and the advanced features turned off I couldn't pass anything through.
It may well be the case that if I'd configured passthrough before the trial expired they'd have remained.

The thing I am having trouble getting my head around is you mentioning the trial period ending and that your servers have their free license. As soon as you register by inputting the license key, the trial period becomes null in void and the extra features you had in the trial period that are not part of the license you have just registered are turned off.

If you are on the trial then it will say evaluation as it does in the screen shot above. If it is registered with the free license it will say something different (I will post a screen shot when I get home tonight).

RB
 
Associate
Joined
3 Oct 2008
Posts
1,890
Location
South London
The trial period end is when the licenses is entered. This is the point at which the passthrough became unavailable.

does it really matter whether it ended then the license was entered or whether the license is entered within 60 days? Same end result surely.
 
Last edited:
Soldato
OP
Joined
21 Jan 2010
Posts
21,950
The thing I am having trouble getting my head around is you mentioning the trial period ending and that your servers have their free license. As soon as you register by inputting the license key, the trial period becomes null in void and the extra features you had in the trial period that are not part of the license you have just registered are turned off.

If you are on the trial then it will say evaluation as it does in the screen shot above. If it is registered with the free license it will say something different (I will post a screen shot when I get home tonight).

RB

The trial period end is when the licenses is entered. This is the point at which the passthrough became unavailable.

does it really matter whether it ended then the license was entered or whether the license is entered within 60 days? Same end result surely.
Here is a video of me enabling pass through with a licensed 'free' copy of ESXi 5.

http://www.youtube.com/watch?v=CzcWd9veVuQ

You can see as soon as I click configuration it says licensed.
 
Last edited:
Associate
Joined
10 Nov 2004
Posts
2,237
Location
Expat in Singapore
JUst adding the screen shots as promised.

Overview of the server (note Direct path I/O is listed as supported) and ESXi is not in evaluation mode. Yes passthrough works fine.

ESXifreelicense.png


Licensing details;
ESXifreelicensedetail.png


Passthrough working;
ESXifreepassthrough.png


RB
 
Associate
Joined
10 Nov 2004
Posts
2,237
Location
Expat in Singapore
The trial period end is when the licenses is entered. This is the point at which the passthrough became unavailable.

does it really matter whether it ended then the license was entered or whether the license is entered within 60 days? Same end result surely.

Not really as entering the license should not disable VT-d functionality. After the 30 days makes sense which is what you initially said. You then went on to state that your ESXi was licensed which led me to believe you meant that is disables VT-d after 60 days even if your ESXi is free licensed which is not what I am seeing on either my own or my clients machines. This is the confusion. It would also seem that DLockers also is able to use passthrough on a licensed version.

RB
 
Associate
Joined
3 Oct 2008
Posts
1,890
Location
South London
Not really as entering the license should not disable VT-d functionality. After the 30 days makes sense which is what you initially said. You then went on to state that your ESXi was licensed which led me to believe you meant that is disables VT-d after 60 days even if your ESXi is free licensed which is not what I am seeing on either my own or my clients machines. This is the confusion. It would also seem that DLockers also is able to use passthrough on a licensed version.

RB

Just to clarify the confusion earlier, here's my set of screenshots.

http://dl.dropbox.com/u/23261050/license.png <-licensed

http://dl.dropbox.com/u/23261050/NICs.png <- available Intel PRo1000 adapter not bound to any vSwitch
http://dl.dropbox.com/u/23261050/ESX Passthrough.png <- "Passthrough not supported" despite it working on the same hardware that is on a non-free license and it did work before I put the free license key in.

Not sure why, but clearly my free license doesn't do what yours does.
 
Soldato
OP
Joined
21 Jan 2010
Posts
21,950
Just to clarify the confusion earlier, here's my set of screenshots.

http://dl.dropbox.com/u/23261050/license.png <-licensed

http://dl.dropbox.com/u/23261050/NICs.png <- available Intel PRo1000 adapter not bound to any vSwitch
http://dl.dropbox.com/u/23261050/ESX Passthrough.png <- "Passthrough not supported" despite it working on the same hardware that is on a non-free license and it did work before I put the free license key in.

Not sure why, but clearly my free license doesn't do what yours does.

Are you sure your host even supports pass through? On the main screen it should say Direct I/O:
 
Associate
Joined
10 Nov 2004
Posts
2,237
Location
Expat in Singapore
Slightly off topic -

Rimblock - how are you finding the S1200BTL, i'm in the process of speccing a whitebox ESXi build and this board keeps popping up.

I think we are pretty much off topic already, sorry OP. May be worth starting up a generic ESXi thread as there are clearly a number of people using it in home and enterprise.

Back to your question....

I like the BTL in a number of ways. You get the basics and then can choose to add on or not. It comes with AMT 7 (IPMI) but not KVM. Basic KVM is an add-on and KVM with dedicated NIC is another add-on. One other item of note is that you can add a SAS daughter card which does not take up a motherboard slot, has 4 sas ports with differing abilities (depending on which of the 4, IIRC, sas addon cards you choose). This is great for putting in a 1U case with 4 drives. The SAS add-on cards are cheaper than a comparable PCIe SAS card with the same abilities (i.e. the IBM M1015 or Intel equiv) and leave the motherboard slot free for a nic or something else in a 1U.

Mine is running happily with a IBM M1015 (flashed to a LSi 9211-8i), a LSI 3081E-R, An Intel ET quad (found out passthrough does not work with this card :( ), Intel CT nic, Intel GT nic. All are working fine.

Now for the down side. The rear ports for the board are right on the edge of the ATX I/O chassis cut out section. For putting in a case which takes a standard ATX I/O shield, this is fine but putting it in to a 1U case can prove to be problematic. I put it in a 1U Supermicro 813MTQ-600CM case and the motherboard VGA/NIC ports were hard against the side / bottom of the I/O rear hole, so much so that attaching a VGA cable bends the port up a little. Using washers to raise the board up a little may help. The real kicker is that Intel do not supply a 1U I/O shield for this board either with the board or standalone. They do provide the board in their own 1U chassis as a all in one unit but they do not sell the chassis alone and the I/O holes are stamped out of the back of the chassis rather than it using an I/O shield. I had to make a custom rear shield for this combination.

So, in conclusion, great for a standard board you can add options to if you need them. Great for a 1U system with the SAS daughterboard but need to think about the rear I/O port shield. Lots for expansion possibilities with the number of available slots. ESXi certified. Both onboard nics are supported by ESXi but only one is supported for the ESXi management network.

Other alternatives I have build with include the Supermicro X9SCi-LN4F (4 network ports fully ESXi supported inc management network for all) which is in the first screenshot posted above. Supermicro X9SCM-F also works fine. The difference with Supermicro is that you get all the features with the board (and pay in the initial price) with no up/downgrade options afterwards to tailor to your specific needs.

Note, 8GB Unbuffered ECC ram prices have dropped a great deal recently and are pretty easy to get hold of, at least where I am.

RB
 
Back
Top Bottom