Running VMware Server Console on a Mac

This content is 17 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

Last year, I bought a 20″ wide-screen monitor which I run at a resolution of 1680×1050 pixels. Working with all that screen space is fantastic (especially with 4 virtual desktops), except that I’ve got so used to it that the standard 1024×768 pixels on the notebook PC that I use for work seems too small and an upgrade is out of the question as the PC is only 18 months old.

For a while now, I’ve been running the notebook on the desk next to my main display but I’m running out of desk space. As I virtualised my corporate Windows XP build a while back, I thought it would be great if, when I’m working at home, I could run the Linux VMware Server Console on my Mac (which is connected to the large display). The virtual machine would still be limited to 1024×768 but I could access corporate applications in the VMware Server Console and do the big screen stuff (web, e-mail, document edits, etc.) natively on the Mac, using the whole display. Yes, I know that if I used Microsoft Virtual Server I could run it in a browser, but I’d need ActiveX and I’m not using Internet Explorer. Similarly RDP is an option, but I find it to be a bit flaky on an Intel Mac. Anyway, I’m a (pseudo-)geek and so I need to feed on problems like this from time to time!

Actually, much of the hard work has already been done for me – googling for vmware console mac soon turns up Rui Carmo’s article at The Tao of Mac on how to run [the VMware Console] remotely with Apple’s X11; however Rui’s article was written a while ago now and my VMware Server (v1.0.1-build 29996) installation on Fedora Core 5 doesn’t use the command vmware-console – instead I have to use vmware. Nevertheless, it got me 90% of the way there:

  • On the (Linux) VMware Server:
    • Configure SSH and X11 forwarding (my original post used a Windows client and public/private keys but the principles are similar – this time I used password authentication, making sure that the PasswordAuthentication yes and X11Forwarding yes lines were present in /etc/ssh/sshd_config and restarting the SSH daemon with service sshd restart).
    • Locate an appropriate keyboard map in /usr/lib/vmware/xkeymap/, edit the map if necessary (there is a VMware article about keyboard mapping on a Linux host that may be useful – don’t worry that it’s a VMware Workstation document) and edit ~/.vmware/preferences to include xkeymap.language="keyboardmap" (I used gb101 for my Apple UK keyboard).
  • On the Mac:

At this point VMware Server Console ran successfully under X11 on my Mac; however whenever I powered on a virtual machine all I saw was a black screen and a message in the xterm window which read:

X11 connection rejected because of wrong authentication.

After trying a remote VMware Server Console connection to localhost and restarting the Linux host (I’m not sure which, if either, of these made a difference) I found that the virtual machine was actually starting but that for some reason the display wasn’t being repeated in the X11 VMware Server Console on the Mac; however this time there was a different message displayed:

Unable to connect to the MKS: You need execute access in order to connect with the VMware Server Console. Access denied for config file: /var/lib/vmware/Virtual Machines/virtualmachinenname/virtualmachinenname.vmx.

After setting execute permissions to the virtual machine configuration file chmod +x virtualmachinenname.vmx (changing the permission set from 640 to 751), I was able to successfully view the VM on the Mac (and simultaneously on the Linux host) – the only (very minor) issues are that the mouse pointer is solid white when accessing the virtual machine (so sometimes I lose it) and that the sound is not forwarded (no big deal). Now my notebook PC is docked on a shelf away from the desk, with the lid closed, and I’m running the VMware Server Console from the Mac, having reclaimed some space on my desk.

17 thoughts on “Running VMware Server Console on a Mac

  1. OK. So last night this was working. Then I took the notebook PC to work with me and when I returned home this afternoon I had problems connecting. Here’s some extra information that I’ve found…

    Sometimes the VMware Server Console gets upset and throws weird errors. The are not consistent – the most recent was Failed to disconnect from the MKS: Pipe: Write failed but I’ve also had other MKS errors, one warning that the host didn’t have any sound device and a number of plain black screens. What seems to work is this:

    • Log on interactively to the Linux machine before launching the X11 VMware Server Console.
    • If the X11 connection rejected because of wrong authentication error returns, then try xauth generate :0 . trusted (as suggested in Rui Carmo’s original post).
  2. Well, i’ve tried this, and i would like to mention that everthing is dead slow. The mouse lags horribly, just as if you connect to the PC via VNC.
    I’d be glad if you had any solutions for this.

  3. Simon,
    That wasn’t my experience at all. You don’t mention whether it was the console access or the VMs that was slow but if console access in general is slow it may be an issue with the specs. of your PCs and/or network and if the virtual machines are slow are you sure that you have installed the VMware Tools inside each VM?


  4. Just a quick comment for the archives. If you’re stupid like me, and decided to install the bare minimum of packages for your Linux host, you might have overlooked xauth. If, for some reason, you try to “ssh -Y user@host” and DISPLAY is not set properly (localhost:10.0), you’re probably missing the xauth package. On my CentOS 4.5 system, it was an easy fix of “yum install xauth” and we were off to the races. :)

  5. Thanks for the tip Jason – and I wouldn’t consider installing the minimum number of packages to be stupid – it’s good security practice to only install the components of the system that are absolutely necessary.

  6. Hi Mark. Actually, I was stressing the “overlooked xauth” part as dumb, not the “bare minimum” part. But yes, you’re absolutely correct. I install the bare minimum of packages on any system plugged into network, for security reasons. Especially when it’s a Linux server. ;)

  7. I used a simpler solution than an ssh tunnel. My MacBook Pro is using BootCamp with XP installed. I have VMWare Fusion running and it opens my XP BootCamp partition w/o issue. In XP I have quite a few remote access tools installed (I’m an IT consultant so sometimes its RDP, sometimes, VNC, sometimes remote X, and sometimes RAdmin). I installed the VMWare Server Console in XP, then turned on Unity mode and just started using the Windows console natively within OS X. Worked just fine…

  8. @A. Davis – thanks for your comment Andrew – it’s good to know that method works but unfortunately wasn’t an option when I wrote this post back in April 2007 as Boot Camp and VMware Fusion were both beta products back then. Hopefully I’ll be getting a MacBook Pro myself soon, after which I shall pick up a copy of VMware Fusion and give it a go!

  9. Has anyone got the vmware-server-console compiled and running on their Mac natively? I am using Leopard 10.5.1 with MacPorts installed and can get vmware-server-console to run locally via the “ssh -Y” method. However the console keeps crashing when I try to configure the remote VM. I would love to try running the console locally instead to help (potentially) alleviate that issue.

    Thanks Mark for the writeup and to Rui for the original article. Keep the Mac + VMWare Server goodness coming!



  10. My server is a barebones Suse11.0 server console mode only running VMWare Server .
    My main desktop is a Mac Mini. I have Vista in Bootcamp, but this takes forever to power on, even under Fusion.
    My quick workaround is an XP VM workstation on my mac, installed from tinyXP. takes

  11. Using a virtual machine, to control a virtual machine is ridiculous in my eyes. There should be no reason to do so… Sadly we have to. I have had no luck running vmware in X11. My browser comes up, but the login doesnt show, so I cant do anything. I run XP in Parrallels and connect just fine load up the virtual machine… but its kind of silly, Running XP in MAC, to Run and XP virtual machine off a server… silly indeed.

    Im going to try Sun’s Virtual Box and see if its any better.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.