Along with development for Theatre.IO (server management system) (dev. version available for Les Laboratoires Phoenix enterprise clients), I’ve had to look into geo-diversity for the datacenter where the company got ‘silo’/'reference stack’ (a group of servers & devices which are the same, standardized, everywhere).
After a couple of very funny – they must have been a joke) – quotes from well known providers, look like we’ll be doing a dual deployment (Seattle & Chicago). That should cover north-America quite well. The phase 3, our Europe site, being on hold for another 5 months. This is a major investment for Les LabsPhoenix as our reference stack cost are in the 5 digits realm, but with geo-diversity and our own IPv4 and IPv6 ranges, there is much that can be accomplished.
Stay tuned as I’ll soon be presenting that reference stack (the free software which compose it, and the not-so-free-ones). There is still some things that need to be figured out (constant evolution, redesign), such as if the Mikrotik RB1200 will be powerful enough to hold the main traffic.
Geo-diversity
VMWARE vCenter operations
VMWARE recently quietly released a product called “vCenter Operations“. The product helps system administrator get a better view of the general (and specific) health of their infrastructure.
I’d suggest anyone with a vCenter/vSphere setup to try it out. The results are pretty amazing: the graphs and the analytic engine helped me quite a few time to diagnose issues clients have been reporting. Here is a quick screenshot where you can see the default view of one of my Labs environment, configured for testing purpose – a cluster of 3 ESX hosts and 12 VMs:
Ok, this might not be very interesting, but if you click on any items, a datacenter, cluster, esx hosts, vm… you get a screen similar to this one:
Way more interesting data & metrics. You also have a quick analysis of resources in contention, of your current usage and growth/run-way space.
As the software is available for larger environment (package ‘minimum size’ is 50 licenses), this should produce some pretty interesting metrics/data once deployed. I’ll try to do that soon ;-).
vmware labs
I’ve been working intensively with vmWARE products for the last couple of months.
I’ve already wrote about LabsPhoenix’s MYTH cluster entering phase 02 of its development – few months ahead of schedule. It is currently configured as a 3 nodes vSphere Enterprise+ cluster of very modest capacity (Resources: CPU 21GHz, Memory 48GB, Storage 4TB). Next phase is within 60days and will see those resources grow by another 66%. This ‘demo‘ has been so successful that we are already drafting plans for another cluster.
One of LabsPhoenix’s main client also asked me to re-factor its lab environment. Here, we are talking of a 4 nodes vSphere Enterprise cluster, built from scratch, with some very nice capacity (Resources: CPU 95Ghz, Memory 252GB, Storage 2TB).
I’m not throwing those numbers out there to poke anyone, its more of an offer: If anyone got some specific questions about vmWARE deployment, feel free to ping me. I’m often available for a quick chat. My cie, LabsPhoenix, also has some competent sysadmin if the problem get too large / if speed is of the essence.
Sometimes, we get hit by strange errors: Following an upgrade of the Cisco 3750 switches configuration to an higher MTU value (9000bits, to support jumbo frame on the attached iSCSI MSA); The VCENTER process started acting up on the management server. Quick restart of the process worked fined, but nothing in the log shows why the switch’s configuration reload broke that specific service.
Also, changing MTU value in a vmKernel interface is quite easy on vSphere 4.1 – it can even be done through the gui. You might search the option for quite some times though! It is hidden in Home>Inventory>Networking, in the distributed virtual switch configuration (right-click on it, edit setting). If your not using dvSwitch, then, your stuck through CLI commands.
Ahead of IT.
Haven’t been writing much. Not everything get inked easily.
You have some great news and you refrain because your light might shine a bit to bright for some of your entourage. Your news are bad and you wonder if Internet really is the medium to tell / distribute them.
Well I’ve been blessed with good, bad and neutral news in such quantities for the last few weeks that it doesn’t matter much anymore.
From the farthest to the nearest of IT : I’ve sold my condo, bought an home, got very sick then got better. I now have an employee for Les LabsPhoenix and a second one in about a month. I’ve been selected to give a tutorial at Linux Symposium about LabsPhoenix theater system (Puppet + mCollective + DNSMasq + DHCP + tftpd + netboot + drbd + …) – and it should be pretty interesting. I’ve been offered 2 jobs at the same time, one as CTO, on as an Infrastructure Specialist at Nokia in Montreal.
I’ll be here way more often.
Gryphon is live. Long live to myths.
Les Laboratoires Phoenix is pushing a new cluster – MYTH – in production. Normally, this isn’t really worthy of a blog post – nobody really care how many systems I deploy for clients – but since this isn’t for a specific client, it get a bit more interesting:
“Phase 1″ capacity of this multi-tenants (fully redundant) cluster is of 7.18 GHZ, 16GB Ram, 1TB. Phase 2 (next 2 weeks) will see it grow to 21.54ghz, 48gb rams and 10TB. End of February is the target date for the phase 3: 2x on each of those numbers. Everything in this cluster is already redundant – but getting bigger number is always fun.
Tonight, 3 systems get integrated (virtualized). It represent economy of about 600$/month for this client. He won’t need his half rack + bandwidth + power…