I Have a Blog, And I Must Scream

Administrators new to System Center Configuration Manager (and SMS) are often befuddled by return/error codes from programs delivered using an advertisement.  The common question is typically something like “Where can I find a list of error codes?”  The answer to this question is that you can’t find a list.  Program return/error codes are completely dependant on the program; a developer is free to return any code that they wish using one of many different functions or APIs. 

Having said all of this, Windows does define a well-known list or error codes that many developers do in fact use to return simple error information to the user.  The easiest way to query what each error code translates into is to drop into a command prompt and enter the following:

net helpmsg <errorcode>

That’s it.  As long as the developer chose to use the windows error codes, which most do, you now have an easy way to get to the description of the error code which is usually at least half the battle.

Also note that in general, an error code of 0 means success, everything else is an error.  This is a convention traditionally used by all operating systems and developers.  ConfigMgr also uses this convention to determine the success of failure status of an advertised program.  Every now and then, you will however find someone that doesn’t follow this convention and it will throw ConfigMgr off.  Be aware of this.

ConfigMgr and OpsMgr also use error codes extensively in their logging and alerting facilities, however the following two caveats often apply.

1. Sometimes error codes are HRESULTS; e.g, 0x80040005.  HRESULTS are extended error codes that attempt to provide more information than just a reason for failure.  For a lot more details see Deciphering an HRESULT by Heath Stewart and Tricks for deciphering HRESULTs not listed in my previous 1935 error article by Aaron Stebner.
2. Sometimes HRESULTS are displayed as a negative number; e.g., –2147024891.  This happens because the application displaying the HRESULT does not know to display it as a hex number and converts it to a signed 32-bit decimal.  With signed 32-bit decimals, if the first bit is a one, the the number is negative, thus the negative number.  These simply need to be converted to a hex value to be properly interpreted.  Aaron talks about this in the post referenced above.

Another way to get back information about advertisement status in ConfigMgr/SMS is to use Status Management Information Format (MIF) files.  MIF files are based on a format define by the Desktop Management Task Force (DMTF): the same folks responsible for SNMP and WBEM.  When used with advertisements, these files contain extended error information, not just error codes.  ConfigMgr/SMS uses these files for various things already; e.g., hardware and software inventory status.  Creating one of these files is not difficult and Microsoft has previously published a stand-alone tool to help you with this.  There is also a COM object that you can use to create these file.  To tell ConfigMgr to look for status MIF files, you must also fill in the Reporting tab on the package.

The easiest way, however, to create status MIF files is to use an MSI.  The Windows Installer executable, msiexec, has a specific switch to create a status MIF file: /m.  Most documentation on msiexec does not include this switch, but it’s there.  If you create a package in ConfigMgr using the Package from Definition option and choose an MSI, ConfigMgr will automatically create the the programs with the /m switch and fill in the Reporting tab for you.  Good on ya ConfigMgr team!

There is also a way to produce status MIF files from legacy InstallShield installers, but I dn’t honestly know what it is and don’t feel like looking it up right now.

In troubleshooting an unrelated issue, I learned a lot about Certificate Revocation Lists (CRLs) and how they can impact ConfigMgr and specifically Operating System Deployment (OSD).  I first thought that my problem was the CRL, which would have caused me some pain, but it wasn’t the problem after all.  Below are a couple of great links that I found about CRLs and ConfigMgr.  In short, if you want to deploy a PKI, to support ConfigMgr or otherwise, learn about CRLs and CRL publishing first.  Published CRL locations are very difficult to change once you start issuing certificates so it is imperative that you get them right the first time.

Tips, Tricks & Hints for Native Mode and Internet-Based Client Management - Part 3 of 3

Planning your CRL Distribution Point for Configmgr 2007 Native mode

Hi, my name is Jason and I’ve been an information pack-rat for a long time.  My big problem is, how do I store all of the information I want to keep and how do I get to it quickly and easily.  I recently threw out several large stacks of magazines collecting in my office, must have been over 100 of them from at least five years ago (I have a few more stacks in the garage also).  I’m grateful that most publications are available electronically now and you can also buy CD/DVD compilations.

I also read a lot of Blogs and have been searching high and low to find a solution to save blog postings and make them available to my associates and customers.  OnFolio was the best RSS feed reader I have ever used and it had a great facility for capturing and cataloging web site and posts.  Alas, Microsoft bought OnFolio a few years back and it just died.  Boooo Microsoft.

I finally stumbled upon a new RSS reader that gives me a nice facility called link blogs.  Basically, everything I tag with a category can be synched to a web page (also available as RSS).  This synch happens in the background and the “Link Blog” is publicly available.  The RSS reader is called FeedGhost.  Hoooray FeedGhost!

My only complaint is that the link blogs are hosted on their servers, not a big deal unless they decide not to host them anymore.

Anyway, all of my link blogs are now available via the “Link Blogs” list on the bottom right.

Here is a copy of the list:

While browsing the list of fixes included in Windows XP Service Pack 3, I discovered that the Uberbug fix described in KB931760 is actually included.  I previously stated in a post, Microsoft Deployment and Hyper-V, that this was not the case.  This is puzzling though because the Uberbug still occurred.

Although they’ve been around for a while, certificates are a relatively new thing for administrators to have to deal with.  Configuration Manager in Native Mode heavily relies upon them.   

Although technically not required for Configuration Manager, the only documented way (http://technet.microsoft.com/en-us/library/bb694035(TechNet.10).aspx) to create and support ConfigMgr in Native Mode is with a Microsoft Enterprise Certificate Authority (CA) running on Windows Enterprise.  Notice the use of the word “Enterprise” twice.  The first occurrence indicates a CA integrated with Active Directory.  An Enterprise CA enables auto-enrollment via group-policies, AD based CRL publishing, and certificate template use.  The second occurrence of “Enterprise” indicates that the CA must be installed on Windows Server Enterprise (2003 or 2008).  This enables you to use customize certificate templates.  Although you can create certificate templates on standard versions of Windows, you cannot actually use them.

Certificate templates will make your life much easier – they let you customize certificates that are issued by your certificate authority.  Creating the certificate templates for ConfigMgr is straight-forward when you follow the guide above – note that this guide is for a Windows 2003 based CA, Microsoft has a draft document for using a Windows 2008 CA but it is not generally available yet (http://blogs.technet.com/wemd_ua_-_sms_writing_team/archive/2008/06/30/having-problems-deploying-the-certificates-for-native-mode-with-a-windows-server-2008-ca.aspx).  I’m using this draft guide right now and its works well.

One slight hiccup that you might encounter is that when you create a new certificate template, it is not immediately available when you click on New->Certificate Template to Issue.  Many troubleshooting guides recommend that you ensure that you are running on Windows Enterprise.  They neglect to mention that the template must first replicate to every DC in your forest before it is available.  For smaller forests, no big deal.  For larger ones, this will cause you a delay that may have you pulling your hair out if you don’t know what’s going on.

Windows 2003 introduced the ability to move Active-Directory integrated DNS zones from the default AD partition to dedicated partitions.  You do this through the DNS MMC snap-in by right-clicking on a DNS zone and choosing Properties.  On the General tab of the properties dialog box, click Change next to the Replication line.  This brings up the Change Zone Replication Scope dialog box which allows you to choose one of the following:

• To all DNS server in the Active Directory forest
• To all DNS server in the Active Directory domain
• To all domain controllers in the the Active Directory domain
• To all domain controllers specified in the scope of the following directory partition

So what is the difference between these and why would would want to move your zones?

Each choice specifies which AD partition is used to store the zone’s data.  AD partitions are separate areas of storage within AD itself; each individually named partition can have its own security and replication configuration.  The third choice in the list, To all domain controllers specified in the scope of the following directory partition, is equivalent to how zones were stored in a Windows 2000 AD; the partition used is the default AD partition where all User and Computer objects are also stored.

The first two choices create or use a new application partition in AD and stores the DNS zone there.  The main purpose for this is to prevent the replication of the DNS zone and its records to domain controllers that do not need it, specifically to DC's without DNS installed.  The second option also limits the replication scope by not replicating the zone to DC's outside of the local domain in the forest.  If you have a single domain forest, there is no effective difference between these two.  In smaller network implementations, the reason for doing this is not evident, but when you are dealing with thousands or ten of thousands of dynamically updated records, replication latency and replication bandwidth usage become issues.

The third option allows you to define a custom application partition for advanced control over the replication scope or schedule.

Each zone is configured separately and all zone types have these options available as long as they are AD integrated.

To verify the actual AD storage of a zone, use ADSIEdit (built into Windows 2008 or loaded from the Windows Server 2003 support tools).  Zones are always stored in a parent container called MicrosoftDNS.  The table below contains the DN of this container for the three primary zone locations.  Every zone will have a sub-container that contains all the records for the zone.  The standard caveat for editing AD data using ADSIEdit applies: if you don’t know what you are doing, look, but don’t touch.

I recently installed a new 2008 domain controller in 2003 forest for a customer.  The documentation on TechNet is straight-forward and I didn’t encounter any issues with upgrading the schema or the domain or DCPROMOing the server.  I returned to the customer last week and discovered that Group Policy was not refreshing for the computer account and attempts to update global catalog related DNS entries were also failing. 

GroupPolicy, 1055:

The processing of Group Policy failed. Windows could not resolve the computer name. This could be caused by one of more of the following:
a) Name Resolution failure on the current domain controller.
b) Active Directory Replication Latency (an account created on another domain controller has not replicated to the current domain controller).

Web searches revealed nothing.  At first I though it was my IPv6 configuration on the system. Nothing there. I tried adding the computer account to various groups, still nothing. After a day and a half, I finally felt that the promotion went bad somewhere and demoted the system, let it sit over the weekend and promoted it again. Same problems with different error messages.

GroupPolicy, 1097:

The processing of Group Policy failed. Windows could not determine the computer account to enforce Group Policy settings. This may be transient. Group Policy settings, including computer configuration, will not be enforced for this computer.

Web searches for this didn’t reveal a lot either. A reboot later and the error message changed with the same end result: Group Policy for the system was not refreshing.

GroupPolicy, 1006

The processing of Group Policy failed. Windows could not authenticate to the Active Directory service on a domain controller. (LDAP Bind function call failed). Look in the details tab for error code and description.

This time, however, a web search turned up a KB and a hotfix: Events 1925, 1006, 1645, 1055, 40961 on new Windows Server 2008 domain controllers or error message: "No authority could be contacted for authentication" in Windows Vista when you try a Remote Desktop Connection. A quick call to Microsoft scored me the hotfix which has cleared up the problem: Yipee! Because this is at a customer site, I cannot verify the exact cause of the issue or whether it lines up with the cause described in the KB. I’m just glad that it worked.

So I've noticed this "problem" on our internal Exchange system and at every customer site that I deal with that has Exchange 2007: when you send e-mail to an external recipient via OWA, it is sent as plain text.  This is sometimes very annoying but I could never find a solution or even anyone asking the question.  After some (more) hunting the other day I actually came up with a possible solution: http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=2012151&SiteID=17.  I had our internal IT folks implement the fix and it's now working.  It's a quick command to run in Powershell and completely reversible:

set-remotedomain -Identity Default -ContentType MimeHtmlText

You can verify the setting on an Exchange 2007 server by running the following:

Get-RemoteDomain -Identity Default

The documentation for set-remotedomain indicates that the default setting for the ContentType option is MimeHtmlText; however, this is not (based on my experience with multiple production Exchange installations) the default Exchange setting.

I just finished my first production deployment of Configuration Manager SP1 on a Windows Server 2008 system. It is fully supported and I didn’t run into any issues. The Techcenter for ConfigMgr has a page with all the “special” configuration details: http://technet.microsoft.com/en-us/library/cc431377(TechNet.10).aspx.

There were only two things that didn’t work immediately: reporting and remote console access. Reporting not working was my fault: I forgot to enable ASP via the IIS role. Remote console access was a firewall issue; I simply went into the Windows Firewall with Advanced Security management snap-in and enabled the three Windows Management Instrumentation rules.

Installation was a bit more involved, but everything was well documented in the page linked above. Good on ya ConfigMgr Team!

I just hit an interesting snag when deploying XP using source files and a capture and create task sequence. In the process of customizing the task sequence, I forgot to change the User Name and Organization name for the Apply Windows Settings task. Not a big deal except that the username defaults to the currently logged on user; in this case, I was logged on as Administrator. When XP setup reached the username and organization input screen, it stopped and threw up an error saying that you cannot use Administrator for the username. As many times as I’ve installed XP, I guess I never tried to use Administrator as the username. It let me manually change it and hit OK, so, no harm, no foul I guess.

Strictly speaking this should actually be defined as an XP unattended setup quirk, but I hit it during OSD.

Server Manager is the much talked about new, all-in-one, handy-dandy MMC snap-in to manage Windows Server 2008 systems.  It provides most of the same functionality of the Computer Manager snap-in that we are all used to in previous versions of Windows.  There are some notable changes and difference however:

• Server Manager does not allow connections to remote systems. Some functionality of Server Manager, specifically Roles and Features, can only work on the local system so allowing it to connect to a remote system doesn't make sense.
• Server Manager can be launched by right-clicking on My Computer and selecting Manage.  This isn't a difference between these two snap-ins, but is changed functionality in Server 08: previous versions of Windows launched Computer Manager using this shortcut.  A shortcut to the Computer Manager snap-in is still available via the Administrative Tools Start Menu folder(which is also reachable via the Administrative Tools item in Control Panel).
•  
• Server Manager is automatically launched upon administrative user login by a scheduled task.  A new choice for scheduled tasks in Vista and Server 08 is to run them at login and this is how the server manager launch is handled.

The main impact for me so far is a combination of numbers one and two above.  I commonly right-click on My Computer, select Manage, and then connect to another system to view logs, services, etc. instead of remoting into that system.  Now I just have to remember to go to the Admin tools to do this same task. 

This really comes into play for Hyper-V systems installed on Windows Core.  Because Windows Core has no GUI admin interface of its own, it is very difficult, if not impossible, to perform some tasks on the Core console; e.g., disk management, event log review, and device manager.  Thus, the best way to accomplish these taks on a Hyper-V core system is to connect using the Computer Management snap-in.

Nothing earth-shattering here, just a few observations and differences that are not immediately obvious and caused me a small amount of web searching.

Thick vs. Thin Images.  The distinction between these two is somewhat subjective so I will start off with my simplistic and generic definitions.  Both are generic descriptions of what should be included in an image.

Thick Image:  An image that includes the OS, OS updates and patches, miscellaneous components, drivers and applications.

Thin Image: An image that contains only the OS and a very minimal set of updates and patches.

So why, in theory, is a thin image better?  Because it is easier to maintain; it contains a minimal set of components and thus a smaller set of components that must be updated.

I don't agree with this thin vs. thick theory in whole however.  Like many theories, they sound great, but reality gets in their way.  Maintenance of images should be automated and should be a minor concern: see part 1 of this series for a further explanation.

There are a few goals for the master deployment image(s) and ease of maintenance is not one of them:

1. Hardware agnostic.  Few organizations are actually able to standardize on a single hardware system for al of their desktops, so this goal should be obvious.  What might not be readily obvious is that it is achievable.  The main obstacles to this goal are drivers and the Hardware Abstraction Layer (HAL) in XP.  Part 3 will cover this one.
2. Universal.   Images should be a baseline for all deployments in an organization, they should contain the least common denominator of all the desktop needs in an organization.  If not everyone in the organization needs an application or component, it shouldn't go into the image and instead should be layered on after the image is deployed.  Part 4 will cover this one.
3. Deployment Speed.  This is not as important as one and two, but is still worth including and does become very important if network is not as fast as it should be or a WAN is involved.  Applications and components included in an image only slightly increase the time it takes to deploy a system because they are already installed and do not have to be pulled across the network separately.  Applications and components layered on after the deployment increase the time significantly because they must be installed and pulled over the network.  Typically installations include some files that aren't even installed on the system, so this can have a greater impact than is first realized.

Ultimately, I honestly feel the thin vs. thick is a moot argument.  Every deployment image will probably be somewhere in the middle and what is right for one organization may not be right for another.  However, I do feel strongly that having a thin image, just for the sake of having a thin image, should not be a primary goal.  Maintenance of images, if automated and done correctly, is a minor concern.

Often also called "Gold Images", these are the images of reference systems that are created for deployment to production systems.  In this series of posts, I will go through my best practices, tips, and advice.

The first issue I would like to address is the process of updating images and how painful this is especially when there is a need to maintain multiple images.

So why would anyone want to update their master image(s)?  There are a lot of potential reasons; here's a few I came up with off the top of my head:

1. Adding or removing a standard line of business application
2. Adding or updating a common denominator component like the .Net Framework or Internet Explorer
3. Refreshing the patch level or updating the OS service pack
4. Add new or updated drivers

Definately not a exhaustive list -- I'll go into thick vs. thin images in the next part.

If you are using a "legacy" tool from a non-Microsoft vendor, this process could definitely be painful.  First you have to update your documentation (15 minutes), find the appropriate hardware (??), deploy the image (20-25 minutes), then you have to verify the state of the image (5 minutes), update the image (5-30+ minutes), then you have to re-capture the image (20-25 minutes), re-deploy to test (20-25 minutes), test (5-10 minutes) plus any coffee breaks, user interruptions, OpsMgr alerts, etc.  All told at least 90 minutes and more like 2 hours.  Some folks even do this using ConfigMgr OSD and MDT.

 My first reaction is to put up a bulls-eye on the wall that reads "Bang Head Here".  One of the beauties of System Center Configuration Manager (ConfigMgr) Operating System Deployment (OSD) and the Microsoft Deployment Toolkit (MDT) is that they are more than imaging tools.  They are complete processes for defining an image, creating and capturing an image, deploying an image, and configuring a system after it is imaged.  You should never manually load a system to create an image; you should never manually do anything that will be done more than one or two times.  IT is about automation, so why would an IT administrator, the master of automation, not automate?

Both MDT and ConfigMgr provide facilities to automatically deploy Windows from source files (XP and 2003) or the source image (Vista and 2008), add baseline applications, add standard configurations, update the OS, and automatically capture the final state to an image.  In my experience, this usually takes about 45 minutes with very limited (if any) manual intervention.  Updating an image is simply adding the application, component, update etc. to the MDT workbench or ConfigMgr console and kicking off the automated image build.  This of course has exponential benefits if you are for some reason maintaining multiple images (and there are valid reasons for doing so).

Maintaining images, if done correctly with the LAB build in MDT or Image Creation & Capture task sequence in ConfigMgr is painless and requires very little hands-on time.  These build processes are also extensible, easily customized and scale easily to support multiple images without any duplication of effort.  Both tools are also hardware agnostic, but I'll talk about that in a future post.

I just ran into another bizzaro problem.  Here's the low-down:

• Clean OpsMgr 07 SP1 environment
• Installed agents on two DCs, no problems encountered
• Installed Windows base MP, no problems encountered
• Installed AD MP, problems encountered.  The DCs never showed up in the DC State view and the following warning was recorded in OpsMgr and in the OpsMgr Event Log on both DCs.
  

  Event Type:    Warning
  Event Source:    Health Service Modules
  Event Category:    None
  Event ID:    21405
  Date:        6/10/2008
  Time:        3:18:55 PM
  User:        N/A
  Computer:    XXXX-DC-02
  Description:
  The process started at 3:18:55 PM failed to create System.Discovery.Data, no errors detected in the output.  The process exited with 1
  Command executed:    "C:\WINDOWS\System32\cscript.exe" "C:\Program Files\System Center Operations Manager 2007\Health Service State\Monitoring Host Temporary Files 1\2170\ADLocalDiscoveryDC.vbs" 0 {42180558-CF7E-7292-C345-CF1B3F5362DD} {24A4E1CD-9AB6-51C7-BB4C-B6A8E3BE69B9} xxxx-dc-02.YYY.ZZZ.COM XXXX-DC-02
  Working Directory:    C:\Program Files\System Center Operations Manager 2007\Health Service State\Monitoring Host Temporary Files 1\2170\

  One or more workflows were affected by this. 

  Workflow name: Microsoft.Windows.Server.2003.AD.DomainControllerRole.DCDiscovery
  Instance name: xxxx-dc-02.YYY.ZZZ.COM
  Instance ID: {24A4E1CD-9AB6-51C7-BB4C-B6A8E3BE69B9}
  Management group: XXXX-MG-01

My first step was of course searching the web: surprisingly almost nothing to be found.  Some references to oomads, the Operations Manager Helper Objects for AD.  So I checked out one of the DCs and they're not there.  These are usually installed automatically when the MP rules are pushed to a system, but I gave it a shot.  After a manual install and re-installation of the MP, same issue.

Next, I logged into the DC to try to run the script manually: Can't find script engine "VBScript" for script...  Now that's something I've never seen before.  Back to my favorite search engine and lots of hits.  The solution appears simple: regsvr32 %systemroot%\system32\vbscript.dll.  This time, after re-installing the AD MP, everything works as expected.

So what's unique about these DCs?  The only thing is that they are 64-bit Windows 2003.  I've installed lots of agents on 64-bit Windows 2003 before without issues, but this is the first time on 64-bit 2003 DCs.  Without further testing, I can't verify if this is indeed the case, but it's something to note. 

This was definitely not an OpsMgr problem, but I could see OpsMgr getting blamed for it -- a classic example of shooting the messenger.  I'm sure that all OpsMgr administrators are used to this though.

Oh yeah, this was another "I am better than you [you stupid system]" moment.

That's what I like to tell a system when I solve a particularly troublesome problem; sometimes I even pound on my chest.  I know: what a geek.

The problem this time had to do with sending e-mail through Outlook 2007 via a third-party application; in this case, the application was FeedDemon.  The problem was that when I clicked on the in-application link to forward a post via e-mail, the e-mail was created in plain text -- this of course caused a loss of the post's formatting and pictures which resulted in an ugly e-mail to say the least.  What was really annoying was that this worked fine last week and I didn't change anything (that I know of).

Another reason that this caused me so much anguish is that I've experienced this issue before with another RSS feed reader: OnFolio.  I never got that fully resolved either.  There are two primary commonalities: Outlook 2007 and Internet Explorer 7.  The problem never manifests itself with Internet Explorer 6 and goes away if I load Outlook 2003.

After a lot of searching I finally came upon a promising potential fix: the file association for URL:MailTo.  Upon investigation, it was missing.  Resetting Outlook as the default program did not restore this file association.

I downloaded a reg file to add it back, updated the file to match my configuration, and imported it (contents copied below).  Like magic, the send to from FeedDemon produced a pretty HTML message again.

How did the file association get deleted?  I have no idea.

Why doesn't resetting the default program recreate this key?  Not a question I can answer.

Would this have fixed my issue with OnFolio? I don't know because that was at a previous employer. 

I am now happy again and can resume spamming my colleagues and friends with "useful" blog posts.

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\mailto]
"EditFlags"=hex:02,00,00,00
"URL Protocol"=""
@="URL:MailTo Protocol"

[HKEY_CLASSES_ROOT\mailto\DefaultIcon]
@="\"C:\\Program Files (x86)\\Microsoft Office\\Office12\\OUTLOOK.EXE\",7"

[HKEY_CLASSES_ROOT\mailto\shell]

[HKEY_CLASSES_ROOT\mailto\shell\open]

[HKEY_CLASSES_ROOT\mailto\shell\open\command]
@="\"C:\\Program Files (x86)\\Microsoft Office\\Office12\\OUTLOOK.EXE\" -c IPM.Note /m \"%1\""

Now that I'm up and running with Hyper-V on my laptop, I decided to put together a LiteTouch demo using the latest Microsoft Deployment Toolkit (MDT).  In the process, I discovered two annoying things concerning the so-called Uberbug.

1. The Hyper-V RC1 emulated BIOS for Windows XP is susceptible to the Uberbug. 
2. Windows XP SP3 does not include the published fix from Microsoft for the Uberbug in Windows XP SP2: KB931760.  The article specifically identifies only SP2 so I doubt that the patch would work on SP3 without breaking other things.

While neither problem is a show-stopper and both are easy to fix via the Set Diskpart BIOS Compatibility Mode task sequence task, it is still annoying that they were not accounted for or addressed.

Other than that, no issues.  Hyper-V rocks BTW.

As most folks know, the RTM version of Windows Server 2008 includes a beta version of Hyper-V. In March, Microsoft released Hyper-V Release Candidate Zero (RC0). The update from the beta to RC0 is described in KB 949219. Now the thing I discovered about 949219 is that it doesn't only apply to the Hyper-V hypervisor itself, it must also be applied to any 2008 Server systems where the Hyper-V tools were installed in order to connect to an RC0 Hyper-V server – this unfortunately takes a reboot. Additionally, 949219 must also be applied to virtual guests running Windows Server 2008. By default, 2008 Server has the integration components installed; the problem is that it has the beta integration components and these must be updated to the RC0 integration components. Without the RC0 update, the mouse will not work via a terminal session and the NIC drivers will not be present (among other things).

So how do you get 949219 to the guest if it doesn't have a network connection? The easiest way (for me) is to create an ISO using a tool like ImgBurn and then mount it in the guest. You can even get fancy and add an autorun.inf to the ISO so it launches the update automatically. I actually had to do this because I'm RDPed into a 2008 server where the Hyper-V tools are installed to connect to a server 2008 core Hyper-V system and without the mouse integration working, I can only use the keyboard to control the guest.

I also discovered a great way to enter product keys: Entering Product Keys into Virtual Machines.

In the process of installing Hyper-V for a customer, I made two discoveries. The first discovery was that the only way to manage Hyper-V on a Server 2008 Core system is from another, remote 2008 system or a remote Vista SP1 box. In both cases the Hyper-V Management Tool (MMC) must be installed separately. For Vista SP1, it's a free download. For Server 2008, it can be installed via the Features. This would of course pose an issue in an XP only environment with a single Server 2008 system. Of course, the customer that I at was XP only; luckily, they did have another Server 2008 system.

Now for the second discovery: instead of installing the Hyper-V tools from features, I installed the Hyper-V role because I didn't know about the feature. Not a big deal, everything installed fine, rebooted, and everything was still fine. Then I discovered that I could just install the tools via Features so I decided to remove the Hyper-V role. Again, not a big deal. Wrong. In the process of starting up (yes, a reboot was required), the server hung at Installing Updates – 75% for 20+ minutes. Uh-oh. I could connect to the server remotely and look at the logs, but there was nothing significant or out of the ordinary in them. Time to turn to my trusty web search tool of choice and bam: http://support.microsoft.com/default.aspx/kb/950792. Not sure who to blame this one on, but at least I wasn't the first to experience it.

I took the plunge and reloaded my laptop, a Dell Latitude D830, with 64-bit Server 2008 Enterprise with Hyper-V. I experienced no problems during actual installation and it even found most of my drivers. I did download all the Vista x64 drivers from the Dell site and installed those just because. They all worked great including the wireless: I'm attributing this to the fact that Server 08 and Vista SP1 share the same code base and presumably the same driver model.

I have experienced a few little "differences" that have caused me to do a little research though.

• The wireless feature is not installed by default, you have to go into the features and add it.
• The latest versions of Live Messenger won't load. From what I've read and seen, it's not a technical problem, rather an imposed limitation. The work-around to this is to download the Live Messenger 8.5 msi; it installs fine.
• After installing Hyper-V, the power management no longer allows standby and hibernation. From various posts, this appears to be a design choice because very few folks, if any will actually be running a production Hyper-V on a laptop or where these features are needed and so it just wasn't worth the cost and effort to make it work.
• As with Vista, if you are remote and need to log on with a domain account that is not yet cached, you must log in with another valid account, most likely a local one. You can then connect however you connect to the corporate network and then do a switch user. This will bring you back to the logon screen while still being connected to the VPN.
• Virtualization must be enabled in the BIOS as this was not the default for this particular laptop. Windows will warn you that virtualization must be enabled, but it will not prevent you from actually installing Hyper-V.

I'm sure there will be other little things, but for now, I'm satisfied.

Operations Manager Service Pack 1 is finally here. I've installed it into production twice already and from my perspective, it is a lot better. The UI is much snappier and the script error noise is sigficantly less. The upgrade was painless in both cases and actually solved a major issue for one of the installations: a particular remote server agent refused to talk to the RMS. I manually upgraded the agent to SP1 and without out any other intervention, the agent starting talking.

I hated giving the answer "wait for SP1" to customers, but it definatley lived up to the hype even though it was long overdue.

There were two minor gotchas in the upgrade:

1. Increase the Operations Manager DB log size before the upgrade. This is documented in the release notes.
2. Reboot the RMS after the upgrade. The upgrade does not prompt you to do this, but in both cases, the RMS issueing alerts and just acted weird in general.

Another known noise issue is documented and being actively worked in Redmond: http://blogs.technet.com/momteam/archive/2008/03/01/performance-module-could-not-find-a-performance-counter.aspx

Overall, I'm very pleased with this service pack so far.

In the course of installing and configuring Audit Collection Services (ACS) I had to troubleshoot a permissions issue connecting to the database.

Specifically, the ADTServer application service running on the collector was attempting to connect to the ACS database on a separate system.  The service runs as the local network service on the collector which means that it uses the computer account to perform network authentication: DOMAIN\SYSTEM$.  This account was added to SQL Server and granted dbo privileges by the ACS installation.  However, every time the service attempted to connect to the database, authentication for the account was rejected with the below events.

I went in a lot of different directions but ended up looking at the local system policies and finally found one: Access this computer from the network.  By default, this setting is set to Administrators, Backup Operators, Power Users, Users, and Everyone.  On the database server, it was set to Administrators and Backup Operators only.  Thus in order to connect to a database on this system, a user must be an administrator (or Backup Operator) on this system.  By seemingly reducing the access to this system it is actually less secure because it requires database users to be administrators. 

Leaving this setting at its default setting does not pose a security risk, it simply allows users to be authenticated and access resources that they are authorized to use; in this case, SQL Server is controlling authorization.  Note that authentication and authorization are two distinctly different things.  The setting identified above is limiting authentication not authorization and this ultimately decreases the effective security of this system.

Event Type:      Error
Event Source:   AdtServer
Event Category:            None
Event ID:          4618
Date:                1/8/2008
Time:                9:25:10 AM
User:                N/A
Computer:         Collector
Description:
Error occured on database connection:
 Status: 0x02200000
 ODBC Error:    18452
 ODBC State:    28000
 Message:         [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for user ''. The user is not associated with a trusted SQL Server connection.
 Database:        Register
 Connection:     ComplianceTest
 Statement:       -
 

Event Type:      Error
Event Source:   MSSQL$ACS
Event Category:            (4)
Event ID:          17806
Date:                1/8/2008
Time:                9:25:10 AM
User:                N/A
Computer:         DATABASE
Description:
SSPI handshake failed with error code 0x8009030c while establishing a connection with integrated security; the connection has been closed. [CLIENT: 10.32.9.69]
 

Event Type:      Failure Audit
Event Source:   MSSQL$ACS
Event Category:            (4)
Event ID:          18452
Date:                1/8/2008
Time:                9:25:10 AM
User:                N/A
Computer:         DATABASE
Description:
Login failed for user ''. The user is not associated with a trusted SQL Server connection. [CLIENT: 10.32.9.69]

I finally found a fix for the problems that I've experienced with the DNS Management Pack in Operations Manager 2007; actually it's more of a work-around: add the Data Reader account to the Operations Manager Administrators group and all is well.  Note that this should be the group designated during installation as the Operations Manager Administrators; the setup program adds this group to the Operations Manager Administrators role.

The problem occurs when the DNS Management Pack is added to Ops Manager 07 and generates the following alert: Data Warehouse failed to deploy reports for a management pack to SQL Reporting Services Server.  Additionally, an error is logged in the Operations Manager event log (a sample of which is copied below).

I've experienced this problem both in the lab and in multiple production environments.  This problem does not affect any other management packs and I have no idea what it is uniquely doing to cause this issue.  I do think it's an installation only issue and if you remove the account from that group after the MP has been successfully deployed to all agents, everything will work fine – I haven't actually tested taking it back out though.

             Event Type:        Error

Event Source:    OpsMgr SDK Service

Event Category:  None

Event ID:              26319

Date:                     12/20/2007

Time:                     2:50:34 PM

User:                     N/A

Computer:          MOM

Description:

An exception was thrown while processing GetRelationshipTypesByCriteria for session id uuid:2856e71b-a9a4-4518-ba41-151504c5e7d1;id=19.

 Exception Message: The creator of this fault did not specify a Reason.

 Full Exception: System.ServiceModel.FaultException`1[Microsoft.EnterpriseManagement.Common.UnauthorizedAccessMonitoringException]: The creator of this fault did not specify a Reason. (Fault Detail is equal to Microsoft.EnterpriseManagement.Common.UnauthorizedAccessMonitoringException: The user DOMAIN\scomreader does not have sufficient permission to perform the operation.).

As Microsoft begins incorporating PowerShell into more and more products, most admins are wondering how to get their feet wet. The way that I've been recommending is to stop using cmd: install PowerShell on your desktop and don't ever launch cmd.exe again. PowerShell has aliases built in so that most of the commands available via cmd work without any changes. Thus, instantly, you're using PowerShell without actually having to learn anything new. Then, slowly you can start investigating some of the awesome power available in the aptly named PowerShell.

By default SMTP relay is disabled in Exchange 200x.  This of course is a good thing because it prevents Exchange systems exposed to the Internet from being unwittingly used by spammers to “relay” their e-mail to other systems.

Often, organizations have an application that sends alerts or notifications via e-mail so they assume that they have to add an exception to allow that application to send e-mail via the SMTP service on the Exchange system.  This is only partially correct: the exception only needs to be added if the e-mail is being sent to externally hosted addresses.  If the mail is to be delivered only to mailboxes hosted by the organization that the Exchange server is part of, then this e-mail is for local delivery and not relayed.  Thus it is not subject to the relay restrictions in the SMTP service and no exception is needed.

This makes sense, because ultimately, the internal application server is no different than an external SMTP server trying to deliver e-mail.  If the address is valid and local, you would never want Exchange to reject it regardless of the source.

Active Directory sites are considered empty if they do not contain a domain controller. This immediately begs the question: why would anyone want an empty site? The answer is that there are more applications than just Active Directory that are site aware. This, albeit short list, includes DFS, SMS 2003, SCCM 2007, and Exchange 2007. Thus, just because a site does not have a domain controller, does not mean that the boundary defined by the site is not important for another application.

So just how do clients determine which site that they are in? Sites are of course are defined by IP subnets and at the time a client is joined to the domain, it performs a lookup in Active Directory to match its IP address to a site which is then stored in the registry. This information is updated every time a client logins into the network.

Why do some consider empty sites bad? This stems from the mis-perception that clients in an empty site will not be able to choose an optimal domain controller. Clients find domain controllers by looking them up in DNS based upon the site that they are contained in. Active Directory creates DNS records in empty sites for domain controllers according to the topology generated by the KDC. This can also be manually controlled via Group Policy or the registry on specific domain controllers: How to optimize the location of a domain controller or global catalog that resides outside of a client's site.

Empty sites do need a little extra thought, but they do have a purpose and should not be discarded. For detailed information about how clients find domain controllers, see the Locating Active Directory Servers from the Windows 2000 Resource Kit.

This week, I learned some more about Exchange 07 senders, specifically the Intra-Organization SMTP Send Connector.  I'm working at a client that has two Exchange 07 boxes, one is supposed to be a hot spare (XYZExchange02) with no live mailboxes on it.  In the process of testing mail flow, I discovered that no mail is flowing from the hot spare to the live system and the following error appears in the Application Log:

Event Type:        Error
Event Source:    MSExchangeTransport
Event Category:                SmtpSend
Event ID:              2017
Date:                     11/27/2007
Time:                     5:01:22 PM
User:                     N/A
Computer:          XYZEXCHANGE02
Description:
Outbound authentication failed with error TargetUnknown for Send connector Intra-Organization SMTP Send Connector. The authentication mechanism is ExchangeAuth. The target is SMTPSVC/email.xyz.com.

Web searches turned up absolutely nothing except that the Intra-Organization SMTP Send Connector is created by default to handle all internal routing between hub transport servers based upon the AD topology, that it is hidden and can't be modified in any way, and that there is no logging for it (until SP1).

After staring at the above error message way to long, I realized that it was looking for a Service Principle Name (SPN) and I found a KB article detailing similar problems in Exchange 03. I ran the following command on a DC to set the SPN and lo and behold, everything started working.

setspn -a SMTPSVC/email.xyz.com XYZExchange01

The big question to me still is why is it trying to use email.xyz.com instead of XYZExchange01 for the SPN? My only answer is that the receive connector on XYZExchange01 is configured to identify itself as email.xyz.com.

Looks like I have some research to do on SPNs.

I visited a customer once who felt that creating partitions ended up creating problems; he felt that limiting the C drive to a specific size would eventually lead to the C drive running out of space.  Thus, most of his servers had 200 GB+ C drives and no other partitions.  To me, this is definitely the wrong answer.  His problem wasn't that his C drive wasn't big enough; it was that he had no control over his servers, what was on them, and where it was placed. 

Creating separate partitions requires some planning, but that's a good thing -- too many folks just stick the CD in and go.  Separating data types on different drives adds a layer of fault tolerance and allows you to easily manage those different types of data differently via permissions, quotas, backup policies, etc.  It also allows you to distribute the workload between different disk spindles which will result in performance gains.This leads to the obvious question, what is the best practice for drive partitioning?  As usual, best practices are subjective and are meant to be flexible depending on the exact situation but here are my general rules:

1.  Create partitions at the hardware level.  Use the built-in array tools, like HP's ACU, to create RAID sets and then carve those into containers that will be seen as physical drive partitions by Windows.  This eliminates any ambiguity as to what partition is on what disks.

2.  Create a minimal C drive (boot partition).  Not much should actually go on this drive, just the OS and any minor support applications like the Windows Support tools.  The partition should be large enough though to handle Windows Update uninstall files and of course Windows itself.  Right now with Windows 2003 Server, I like 16GB as a C drive size.  This gives plenty of overhead while also being relatively small.

3.  Create a separate partition for the paging file.  Paging files today are often 2-4GB, a significant chunk of 16GB.  Creating a separate paging drive tidies things up and potentially increases performance.  The maximum paging file size is 4096kb, so I generally create a partition of 8192kb (8GB).  This prevents the partition from showing up on any low drive space reports.  I then also use this as temp space (instead of the popular C:\temp) going as far as changing the system environment variables (%TMP% and %TEMP%) to use this partition.

4.  Additional partitions for applications and data.  Using mount points, these can actually be accessed via the C drive letter or they can be assigned a new drive letter.

To some, this may seem a bit of overkill, but if done up front with planning and consistency, is easy to accomplish and makes working with your systems that much easier.

Early in my career, I knew a UNIX admin that liked to tell me "vi is your friend". I eventually came to believe that statement and although I rarely use vi anymore, I still think that vi is my friend.

This brings me to some of my best friends today: Google and Wikipedia.

Google is often my best friend: anytime something doesn't work (stupid anti-virus) and I'm just staring at a cryptic error message, I type it into Google and, in my experience, 95%+ I get lots of hits. I may not find the exact answer that I need to solve the problem, but I'll get lots of useful information that puts me on the right path to fixing the issue. The premise behind this for me is that there is no way I'm the only one that has ever dealt with this issue. And, as has been proven time and time again when I do this, I'm right.

One important resource that many people who also use Google for this purpose miss out on though is Google groups. Google groups is an indexed collection of the former Usenet bulletin boards. If I don't find my answer via the Google web search, the group search often does the trick.

Wikipedia is also another great friend to have when you want to know about something, anything, everything. The reference material on Wikipedia is nearly endless; it’s always a great starting point to learn about a topic.

One of the keys, IMHO, to being a good system administrator and system troubleshooter is not necessarily knowing all of, or even most of the answers, its knowing where and how to find the answers.

As an administrator of a lot of Windows systems, I can get by with using all of the built-in tools.  There's very little that I cannot do using them.  However, there are a lot of third-party tools that I like to use to make my job easier.  Here's a list and the benefits that they provide me.

•  RunIT - This little tool is a very simple application launcher.  I put a shortcut to it in my start up group so that it runs every time I log in.  Additionally, I set the shortcut to run with alternate credentials and then use my admin account to launch it.  This way, anything that I launch using RunIT, launches with my admin credentials.  This is similar to what a lot of Unix admins do by opening a second command shell and then su-ing to root.  Now when I want to hop into ADUC using my admin credentials, I simply hit F4, type AD and there it is.  This definatley makes running as a normal user (a security must) much easier to deal with.
• xplorer² - This is a dual pane file management tool.  Lots of great features and functions that Windows Explorer just cannot match.
• ADVrunas - This small tool replaces the built-in RunAs dialog box with one that remembers who you last ran a program as.  It also allows you to create shortcuts that automatically run as another user without prompting for a password, and it adds a small line of text to title bar of most windows indicating which user account ran the program.  This last feature is great when you start opening a lot of different windows, some with admin privileges, some without.
• PromptPal - This is an alternate command line with a tabbed interface that allows easy cut/copy/paste as well as command favorites and command completion.
• ClipMate - An excellent ClipBoard utility that has way too many features to list.  By far the best supplemental clipboard utility out there.
• SnagIt - The best screen capture utility.  Also includes way too many features to list.  Easy to use and configure, includes an graphic editor that allows you to easily add callouts, arrows, highlights, circles, etc.
• PrimalScript - A code script editor. Will handle any kind of script that you can throw at it and includes great debugging abilities as well.  If you're an occasional or heavy script writer, this one is a must.
• Terminals - This little gem is a tabbed RDP/Remote Desktop/Terminal Services client.  A little bit like the older MMC based Terminal Services client, but much better for usability.  It allows groupings, favorites, and password caching.  Great for when you have more than session open and need to switch back and forth quickly.

Of all the tools above, only RunIT and Terminals are free, but they are all (IMHO) well worth the small amount that I paid for them.

As I'm sitting at my desk the other, showing a couple of co-workers our patch management process, I tried to open a document from our Sharepoint site: IE Crash!  That was weird.  Let's try that again: same result.  Reboot, open IE, go to SharePoint, click on document, crash.  Must be that new McAfee that was pushed to my desktop.  Disable McAfee, repeat process, crash.

Time to do some searching.  Lo and behold, it turns out that this is a known issue if you have components from both Office 2003 and Office 2007 loaded: Description of the 2007 Office hotfix package: June 17, 2007. Everything works great after the patch and a reboot.  The issue just now reared its ugly head because I just loaded Pro