JayP-NAS 2.0 - A Real Home Server, Part 2.1: BIOS Updates, IPMI, and other basics

The big reason I picked the motherboard I did was IPMI support. Remote management will allow me to only run power, networking, and a USB device or two into the server, but still allow me to have a visual of what the server would be outputting via iKVM, reboot or shut it down remotely if the OS hangs, control fan speeds at a BIOS remotely, among other things. IPMI can be a bit of a security risk, but it also makes managing things so much easier.

For the most part, if you're building a home server based on what I'm writing here, I expect you to have a decent idea of what you're doing already. I'm really only hoping to provide a bit of guidance if you're trying to do something similar to what I've done. Basically, "I've done the research for this specific manner of doing things. Here it is." As such, I'm not really going to go into detail of setting up everything in the BIOS, or what specifically to do in the IPMI interface. Instead, I'll cover some highlights.

BIOS updates without a CPU? AKA: Getting an IPMI License

One of the coolest features of IPMI is the ability to remotely perform BIOS updates. Specifically, you can even do it without a CPU installed. That's a big deal when a motherboard has a BIOS update to allow newer CPU's. If you got that newer CPU, but the motherboard doesn't support it out of the box, just fire up the IPMI interface and update the BIOS and you're golden. No need for trying to find a supported CPU to borrow temporarily just for the sake of a BIOS update.

With Supermicro, the IPMI interface is mostly supported fully out of the box. I say mostly, because if you need to do a BIOS update via IPMI, you need to get a license for your motherboard. Luckily, it's a one time $20 charge for a life time license. Frankly, for how much these boards costs it should just be included, but at the end of the day what's $20 when I've already spent thousands on this thing.

Supermicro doesn't sell the licenses themselves, you have to get them from your vendor. I bought my board from NewEgg, who apparently does not sell the licenses. So I had to track someone down who could sell it. Thanks to an old thread on the FreeNAS forums I had a pretty fast lead: WiredZone. The way you order this is a tiny bit convoluted but not that bad. You will need some information from your motherboard so don't do this until it's in your hands.

  1. Somewhere on your motherboard there will be a couple stickers with bar codes. Find the one that lists a IPMI MAC Address or BMC MAC Address and write down the entire string.
    1. Protip: I just took a picture of mine. Easier on me in the long run.
  2. Go here: http://www.wiredzone.com/supermicro-software-management-sft-oob-lic-10024441
  3. Order it like anything else. You've bought stuff online before, right?
  4. The product page says to go here: http://www.wiredzone.com/contact but that's apparently broken. Instead I went here: http://www.wiredzone.com/contactus
  5. Fill in the required info and include the following: Your order number for the item you just bought from them, the model of motherboard, and the MAC address we found earlier. Add that to the Contact Us form for WiredZone.
  6. Wait.
  7. I got an email from WiredZone asking for the information I had submitted via their contact us page the next day, so I just replied with the same information. I'm guessing a different group monitor the orders or the contact emails.
  8. Wait.
  9. Get an email with the license key.

The process took only a couple of business days in my case. I was lucky in that my motherboard actually came with a BIOS version that supported my CPU, but there was an even newer BIOS update available so I did the update anyways.

Setting up the IPMI

When I plugged the server in, I basically only needed 3 things: the power cord, an ethernet cable into the IPMI interface, and another ethernet cable into the network port marked 1. I hit the power button and went back over to my desk. Now, the hardest part you're going to have initially is figuring out what IP your router gave the IPMI and NIC for the new server. There are a lot of ways, but here are two I used:

  1. If you've got a decent router, you can probably log in to it's interface and find a list of all connected devices. If it's a quality list, it'll include lots of information about all those devices. Look for anything you don't recognize (hopefully you recognize the vast majority, otherwise do a damn security audit), and if it gives you MAC addresses even better since you hopefully already have that from getting an IPMI license.
  2. Alternatively, I also have an app on my phone called Fing. It scans a network for all active devices. Same thing as doing it on the router, look at the MAC addresses and find your IPMI device. I dunno if there's something similar for iPhone, up to you to figure that out.

Either way you'll get the current IP of the IPMI interface. Connect to that IP in a web browser, and you'll be prompted with a log in prompt. Since Supermicro is Super original, the default is:

Username: ADMIN
Password: ADMIN

Horray for thinking out side the box. Anyways, you will probably immediately be prompted with an error. Supermicro's IPMI interface will show you what would be displayed on a monitor if one was plugged into the server via it's iKVM subsystem, which is fantastic. What's not fantastic is this is a Java based system, and it tries to use Java plugins that are very outdated and seemingly all modern browsers just won't allow it. Now, depending on what your daily use computer is running, getting this working will be different for everyone. At the end of the day, the ONLY feature you'd be using this for is the iKVM stuff via the web browser and there's a better way to do that anyways: Supermicro's IPMIView. For now, try to get used to ignore the crap out of that pop up. What I did was reinstall Firefox on my laptop and set it up to hide those popups. The only thing I use Firefox for is the IPMI web interface, so it's no big deal.

Something I'd recommend up front is poking around the interface first and get an idea of what options are available. Depending on your motherboard different options are available. If you're feeling skiddish just don't actually do anything besides clicking the menus on top or on the left. A lot of what the IPMI interface is for is getting information on the system, so mostly you should poke around looking for where things are so you can come back to it later if/when you need that info.

The main things to look at up front are Users and Fan Control.

  1. Users: Here you'll see a list of users and you can modify their settings. I highly recommend modifying the ADMIN user right away and changing the default password to something more secure.
  2. Fan Control: Each Supermicro board has different fan ports and different settings. Universally, fan ports labeled FAN1, FAN2, etc. are going to focus on the CPU temperatures. Fan ports labeled FANA, FANB, etc. are going to focus on peripheral temperatures. Note, if you custom built your system the peripheral temperature the motherboard is looking at won't be your GPU or hard drives, but instead some other devices. Even a pre-built system might not be able to identify that information since to me it's only looking at the CPU or things on the motherboard. I honestly don't know how it expects you to utilize these, but I imagine if you're using a Supermicro motheboard in a Supermicro chassis with Supermicro fans it makes more sense. As for the settings, most will have Full, Standard, and Heavy IO but some will also have an Optimal setting. Full is obvious, it spins all the fans at full speed. Standard spins the fans at 50% speed but depending on the CPU or peripheral temps will spin them faster if needed. Heavy IO is similar but more aggressive on the peripheral fans (FANA, FANB, etc). Optimal is basically the same as Standard but the fans can spin even slower than 50%, usually 30%.

The problem I have with Supermicro's fan control subsystem is there's no finer level of control for users. That's certainly intentional, these systems are intended for server rooms where the air temperature is controlled and the noise doesn't matter, so who needs to micro-manage it? Home users like me, that's who. No one makes stuff for us. We are an unbelievably niche market, so I don't blame them. Just hoping someone notices and considers us at some point.

Beyond that there isn't a lot of specific guidance I'm comfortable giving you. Everyone's home server purposes are different, and the IPMI interface allows you to do a good bit. The only other thing I'd really recommend for most people is getting Supermicro's IPMIView setup on their desktop/laptop/daily driver/whatever else you want to call the machine that's not your server system. You can grab the latest version from here. This does (almost) everything the web interface does but locally via a Java based client. I know, ew, Java, gross. It does work, though. Since every computer is different I'm not going to guide you through getting it running, it's easier clicking next a few times in an installer or finding a proper JRE that'll work and a bunch of other crap to install it. The advantage of having it is the iKVM feature in IPMIView is far better than the website one. Supposedly they're working on an HTML5 based iKVM for the web based IPMI, but that is still a ways away and might not come to older boards. I believe there are other solutions out there that work with Supermicro's IPMI implantation as it is pretty standardized, but I'm sticking with the more official stuff for now.

An optional thing to address would be the IPMI interface setup. You can change it to a static IP, which will make it easier to find in the future. I prefer using static IP's on any devices that I connect to, not from. Systems or devices that I need to enter in an host name or IP address to talk to from another device get a static IP, devices that only ever connect to the internet or other devices on the network get DHCP. Depending on your network it might not be necessary, totally up to you.

Some Final Setup Notes

So, like I said, I'd rather not make too many suggestions as you should know what you do and don't need from the BIOS and IPMI settings for your specific setup. There are a couple notes I'll make for the benefit of anyone doing something very similar to my setup.

  1. To boot from NVMe you need to use UEFI, no legacy BIOS stuff. This isn't much of a deal for most people, but do make sure your OS install media is setup to boot form UEFI as well. You can setup the BIOS on my motherboard to support both, but personally I'd rather people fully commit to one or the other to avoid issues with some devices or media.
  2. Once I got UEFI setup and the system booting off NVMe, for whatever reason the LSI HBA's long boot process gets completely skipped. With it a reboot could take a several minutes, without it the OS is back up and running within a couple minutes.

Beyond that the world is your oyster or whatever colloquialism you'd prefer to use in this situation. IPMI is fantastic for checking in on your system remotely and taking care of some management tasks on a whim, but personally I'm not punching a hole in my firewall just yet. Instead I'm waiting a bit longer until AT&T GigaPower or Google Fiber are available at home then I'm setting a PFSense firewall and establishing a VPN connection on the whole network. Doing that will let me only allow something to connect to the IPMI interface if it's on the VPN, which will add that extra layer of security that will make me trust it.

This article was updated on 20/10/08 11:43:37