Summary: my copies of the DVDs arrived today. It's a chance to enjoy the conference again (and in some ways for the first time).

Many of us have received the MMS 2008 DVDs this week, and others will soon, so now we can finally catch up on the bits we missed. (In fact, this was my worst year ever for attending sessions. Not because I was enjoying the 'activities' (there was some of that), but simply because I had a lot of side meetings and conversations.)

So I loaded up my Zune for tomorrow's hike (Granite Mountain - about 10 miles roundtrip, 3800 feet of elevation game). Despite the speculative aspect of them, I like keynotes, so those are a natural addition. And I'm still keeping an open mind for Powershell, so a couple of those sessiosn are loaded. The State of the Union would be my first choice, but that is only available in PowerPoint form - what happened there? Next week I'll check out the 'meatier' sessions (on the road to a Judas Priest concert).

If you didn't attend the conference and would like to order one, check out:
http://shop.ecompanystore.com/mseventdvd/MSD_productdetail.asp?EventID=70873&TYPE=MMS+2008

So which sessions will you be checking out? (and don't feel obliged to say SI21 and SI22)

p.s. Am I the only one that wishes they'd add in the 'promo' videos? The comedian doing his cheesey bits, the attendees, the rock & roll, and the pre-keynote vidoes? I admit to being shallow enough to being inspired by them, and I'd like to continue that enthusiasm throughout the year. Why do they never make it to the DVD?

Posted by pthomsen | with no comments

Summary: one of our most popular stories is how Microsoft IT does patch management (software update management, or SUM). The stories have become a bit stale, so we've finally updated them.

Thanks to the folks at IT Showcase, and especially Partha Chandran (presenter) and Satish Petwe (contributor to the presentation, and handling the Q&A), we have an updated patch management story:
http://msevents.microsoft.com/cui/WebCastEventDetails.aspx?culture=en-US&EventID=1032374744&CountryCode=US

As always, we'd appreciate your feedback (good, bad, or indifferent). Our story has its own nuances, but hopefully it gives you some ideas for how to make your ConfigMgr patching experience even better.

I used to be the patch management service lead, but times change, of course. I'm more than pleased that Partha, Satish, and other members of our patch mgmt team have taken it to the next level. Security management must evolve - our foes are also evolving. For background, over two years ago we did a webcast on how Microsoft IT does patch management. I've updated that posting with a link to an even older version of the story.

p.s. My apologies for not blogging much lately. I've got lots of topics in mind, but I've been busy catching up after the MMS conference and then a business trip to India and England. Not to mention getting back to the mountains, now that the Pacific Northwest weather is finally decent...

Posted by pthomsen | with no comments

Summary: A small, non-technical point today, but interesting (I hope): it looks like System Center has a new logo.

As Stefan Stranger points out, it looks like System Center has a new logo:

Historically, when System Center documents or web pages included a logo, it was on what I would call 'the brick' theme:

Back in the BackOffice days there was a similar theme based on a horizontal brade.

So does the change mean anything? I'm not privy to marketing's thoughts, so I can't say whether the change is supposed to mean anything. But my first impression is that I like it. Even my wife once commented on the brick theme that it seemed like someone was trying to put up a wall. Not very inclusive.

Does the new logo mean anything to you?

 UPDATE (6/18/08): the System Center Team Blog has more details. Be sure to check out their link to a SQL Server team blog that goes into even more details.

Posted by pthomsen | 2 comment(s)
Filed under:

Summary: I get that question a lot. Well except from my mother, who long ago gave up on trying to understand what I do... But for you guys the answers used to be easy: I do what you do. Computer management. SMS administration of various sorts. ConfigMgr 'engineering'. That sort of thing. But things change.

Sure, working here I get to work very closely with the ConfigMgr product team. And I do that 'dogfooding' stuff. And we have 250,000 clients. And our networks are wonderful. So there are differences, but every organization has its differences. Basically we all did the same thing. Now my story is a little more complicated.

For a few years now the broad team I work on has been learning how we can help some customers by doing their computer management for them. That's been done by a bunch of people, but originally I stuck to the Microsoft IT side of the story. In the last year I've been drawn into the external-facing efforts. Especially for the client health part.

So what does it mean for Microsoft to do computer management for some organizations? Basically, it's a software-as-a-service scenario, which you've probably heard a lot about lately. Our marketing guys put SaaS in context like so:

 

And how is that good?:

In fairness, the leap between those two parts of the story is large. How does SaaS "accelerate" the journey to goodness? They've obviously got thoughts on that, but the real explanation will come in the delivery. We look forward to proving the point (as our peers in Exchange Online have already done).

In the meantime, the answer to the question of what I do is basically service building, especially in the client health space. After all, would anyone pay for a computer management service without a high degree of accountability for all the clients? As per my recent presentation at MMS, that builds on years of client health efforts throughout the industry and recent improvements, and will lead to great solutions and guidance for all of us. So it's an honor to have this job.

But at the same time I help out with related tasks such as dogfooding, proving our solution works, documentation (I have some background as a tech writer...), presentations, and misc. other bits. Fun stuff!

Posted by pthomsen | with no comments

Summary: now that we're home, it's time for general observations and other follow-up. Actually, I've been meaning to do that for at least a week now, but it's been busy times around here (and the weather has finally been decent).

The Conference Bag 

On the myITforum WIKI, I have maintained a history of SMS/ConfigMgr bags. I have to figure out my account/password for that one, but in the meantime, here's a picture:

There's actually a lot of pictures of the bag out there. Someone even does a complete 'unboxing': http://www.myitforum.com/absolutevc/?v=910

I'd say this year's event bag was unique - it's clearly a backpack bag. Only one past bag had shoulder straps, and they were optional (the bag could also be used as a suitcase). But as a mountaineer I take backpacks seriously and the best I can say about this one is that it's a modern traveler's bag. It has no provision for a laptop, is too large for business meetings, and has no lap belt for serious loads, but does have provisions for devices. Most people did carry them around throughout the event, so I suppose they had some utility, but I'm not sure of the ongoing value. The intel logo is prominent, so I suppose the sponsor is happy.

Technical Updates

In my "Going Big" presentation I mentioned that when going to ConfigMgr one thing you should watch out is 'rogue' WSUS installations in your environment that will conflict with SUM (or which SUM will conflict with). Fortunately there's an easy way to avoid the grief - get your current SMS infrastructure to look for them. That's done quite easily with a hardware inventory extension for WSUS. We do that by collecting this registry key: HKLM\SOFTWARE\Policies\Microsoft\Windows\WindowsUpdate\WUServer

On the client health front, there have been a few updates (which I will also incorporate into the relevant posts):

  • Greg Ramsey has shared their ConfigMgr Client Troubleshooter. I've used it a bit and it has some handy features. It has a bit of 'many clients' functionality but it's not meant to be an enterprise-scale solution. Manual inspection and repair is always going to be a part of any client health strategy, and this tool really helps in those situations. So it is a good addition to your client health toolkit.
  • And there's the similar SMS Client Center from Roger Zander.
  • Chris Stauffer has begun an ambitous effort to produce a client-side client health checking script. It's a script that could be run as a scheduled task, or as a regular advertisement (for partially healthy clients). There's a lot of functionality to be packed into a vbscript, but I know Chris can do it, especially with some help from the community.
  • In case you missed it, John Nelson added a great comment to one of my earlier postings about WMI reliability hotfixes. In particular, on how to verify they've been applied to your clients. I haven't tried his idea yet, but it looks like a good one to me.
  • One suggestion I heard was for a Wiki on client health. That way we can work together to fill out the client health story, and provide the "guidance" that I often talk about to each other. Fortunately, myITforum has a wiki: http://www.myitforum.com/myITWiki/. I'm trying to figure out how to make it work.
  • Dan Thomson and I had a great conversation about many client health topics, and one of his suggestions was that a computer startup script could also be set up to run as a scheduled task on each client. That would mitigate the main limitation of computer startup scripts, which is that they only run as often as users reboot (especially while on the corporate network). Somone else said they also send there's out as regular software distribution. That only helps with clients that are healthy enough to run a software distribution, but since the content and policy can be cached, that's actually more clients than you might expect. I'm embarrassed to say I didn't have either of these ideas years ago Embarrassed
  • One organization I've talked to recently has developed a 'control' that sits on each of their clients and reports (via hardware inventory) the response times of their DPs and MPs. I'm hoping to encourage them to share that with us.
  • A manager that I talked to at the conference said that there was an upside to his staff doing a lot of manual client health remediation: they come to understand the clients well and get really good at client-side troubleshooting. He even observed a significant improvement in morale - techies like tricky technical challenges!
  • Of course I see a lot of specific client health issues in the course of my work, or in the course of client health discussions in the community. WMI issues may be the most classic example. But in the course of conference convesations (or conference-inspired conversations), I've heard some new issues:
    • Disk corruption: if the disk is having problems then BITS may fail to download content, or there may be other kinds of client health issues. Running chkdsk a few times, even without the switch for fixing the problems, often helps. The customer that reported this one sees this daily.
    • Another customer reports the startup details for BITS were disappearing on many of his computers. We did a little poll of other customers and none had seen this, but maybe they weren’t looking.
    • Someone suggested that one measure of a healthy client is one that has a current patch mgmt catalog, since patching is a core computer management service and many parts have to work well for the catalog to be up-to-date. So it makes sense to watch for old catalogs.
    • One idea that should be easy to verify, but that I haven't had time to do just yet, is that if the PATH is messed up then ccmexec won’t start. For example, if there is a simicolon in an inappropriate place, or two backslashes when there should be one.
    • Another suggestion for verifying a client is healthy is that the SMS/ConfigMgr WMI namespace has classes for recent content and policy downloads, and if those are old, incomplete, or have errors then the client is unhealthy
    • Running out of disk space (anything less than 90 meg) is an obvious way to cause client health problems, but what can you do to automatically but safely fix that problem? Deleting the restore details for old patches, and clearing the SMS/ConfigMgr cache of old content are good suggestions
    • One customer observed that clients with a  missing RSOP namespace seem to cause problems, but this problem also easy to fix (recompile the MOF). I haven't verified the details on this one, but I do recall seeing a the RSOP namespace
    • An up-to-date SMS_Def.mof version would also be a good indicator of a health client
    • Another way for WINS or DNS to have issues is for DHCP to be set up incorrectly. Watching the details at the client side would help to identify these issues

General Observations

The unfortunate reality is that many of the most interesting things I get to do at the conference are not bloggable. Not things that would change the world, but discussions with people that expect some discretion, personal discussions, or discussions that could be easily misunderstood when taken out of context. I suppose the point is that there is a lot of 'backchannel' activity that happens at the conferences. And even if you're out of those loops, you're actually involved, in that your comments clarify positions and give credibility to the whole market/community. I know I'm out of the serious loops, but there's a lot of those backchannel conversations at the conference. So there's a lot I've been thinking about since the conference but I can't get into much of it here.

Overall I can say that this was one of my best MMS conferences. I attended fewer sessions than ever before - in fact only one (I'll listen to many more on my Zune later) . But the conversations were non-stop and highly varied. Every year I feel more "at home" at the conference. The food-for-thought was nonstop.

At a personal level, I can say that I feel energized in a way that I haven't felt in about a year. It's like youth has returned. I have to think that has to do with the quality of the conversations. But maybe the quality accomodations, good food, nice trip, etc. helped. In any case, I am inspired. Now to get back to work!...

Posted by pthomsen | with no comments

Summary: home again. And thanks again for the opportunity to talk with you (if you were at the conference)!

Fridays at MMS are usually relatively useless but this one was much better. I got to breakfast about 8AM and was pleasently surprised to find many people moving about, mostly going to sessions. Apparently there was feedback that people would like sessions starting around 8:30 Friday morning, and that was done this year. In the past the sessions started closer to 10:00, on the assumption that people would need more rest Friday (for some reason...). But that meant there was only time for two sessions on Friday mornings, and with many people catching flights, the attendance was very thin for the last session. They're all available on the DVD, so the sessions were worth presenting, but it didn't feel the same.

The day started with a good breakfast conversation, followed by my session on how we do ConfigMgr at Microsoft IT, and then I sat in on a session on Asset Intelligence 1.5. More good conversations. From there it was a leisurely trip home. It was nice to finally read some newspapers and think about the week.

My presentation today seemed to work well. I felt the audience was more engaged than during my client health presentation. The evaluation numbers back that up - they were the kind of numbers we all want - in the top 15 overall. My previous session this week had a little less than average numbers, so passable in a sense, but not as good as you deserve.

I must admit that one of the ironies of presenting has been that the topics I'm most passionate about are not necessarily the ones that get the best response. For example, a few years back I gave a presentation on local policies, which I think have the potential to revolutionize how we do computer management in several key scenarios. It even had cool demos. That presentation even lead to Jeff Tondt writing a TechNet magazine article on the subject. But much of the audience didn't buy into the concept, and the review scores weren't great for that presentation - not bad but not as good as they should be. Another presentation I gave that year got better scores. This year I presented on client health, which I'm even more passionate about, but it got the middling scores. I can see how I can improve that presentation (thanks for the feedback everyone), but I liked the content... The really good news is that the client health presentation led to many great conversations on the topic of client health, and I look forward to postings based on those observations. Even more importantly, I really believe those conversations will lead to better solutions in the future - so keep them coming. But maybe the lesson is that real-world results are more relevant than specific-case concepts. Something like that...

Anyways, it was sad to see the end of the week, as always. Not as sad as in some years, but only because I know we'll be back together before we know it (even if it is a year). There should be some kind of final goodbye speech, but since many people are gone, I suppose that wouldn't work. Maybe at the Thursday night party?

This weekend I'll post some general observations, and make some updates based on some notes. For now it's nice to get back to normal life...

p.s. The blogosphere had a bit of observations about my client health presentation: Aaron Czechowski and Sherry Kissinger. But nothing on any of my potentially more controversial statements (maybe that's a good thing...)

 

Summary: It was another good day of client health meetings. Some very interesting thoughts, some of which will be reflected in postings next week.

Otherwise a relatively quiet day for me, which was much needed. I even managed to get a decent meal in (but nothing special).

MMS 2009 is confirmed for April 27 to May 1, 2009, in Vegas! Start planning now (time flies...)

Posted by pthomsen | with no comments

Summary: client health, baby!  :-)

It was another very busy day, with a variety of meetings and one-on-one discussions about client health. Everyone acknowledged client health was a crucial issue that they were addressing or needed to address. About half put it at the top of the list of their issues and were very passionate about it. All of them helped me greatly to confirm and/or clarify my thinking.

The keynote was more relevant today, and I was particularly intrigued by the emphasis on a changing world in which devices, mobile users, and technically demanding users are supposedly going to revolutionize IT and thus computer management. I can see some truth in that, but details were thin on what exactly that means. I do believe we should all give this possibility some serious thought and discussion.

The day was so busy that there was little opportunity to eat properly, but I did manage to attend the 1E party. It was great as usual. The band was first-rate, though too loud for my liking. Fortunately there was an outside deck where we could talk. Even better was that everyone was keen to talk about technical issus - perfect for me!

Posted by pthomsen | with no comments

Summary: this is the first serious day, and it went well.

The BobMu keynote was inspirational, and while intersting was very datacenter-centric. I suspect I'll relate to tomorrow's keynote more. The State of the Union is always one of my favorite sessions, but there wasn't a lot of news to share, and (I hate to say it Bill) the top 10 list was not as funny as usual. The Brady Richardson demo was definitely a conference highpoint.

My client health presentation went smoothly. It led to lots of great conversations and questions, both at the end of the event and later. The evaluation scores were middling - not bad but not great. Clearly I should have figured out how to do some kind of demos. But I really want to see where the client health discipline goes from here - my ultimate goal is to stimulate its evolution.

Later we had a discussion of issues that affect large customers, and that was very interesting. It was intentionally free-form, and ended up covering a lot of topics with participation from everyone. There was a lot of passion on various topics. We'll have to see what we can do to keep the momentum going...

The myITforum party was wonderful, as always. A little smaller than I expected, but it was great to see everyone.

Dinner was a very enjoyable eclectic event. Great in many ways, with wonderful conversation, but the washrooms were the most out-of-the-box experience I've had in awhile.

Posted by pthomsen | with no comments

Summary: activities got a little more 'official', and the community events were great.

Now that we're all settled, I had a few private meetings. All were very constructive and gave good food for thought.

The freshman orientation was good and gave a chance to say Hi to some newbies (which are something like 60% or 70% of the attendees). Welcoming new members is key to keeping the community strong, so we should all make an effort to say Hi to them. Apparently Denmark has about 220 attendees alone (with wonderful red jackets). My home country of Canada had about 200. I believe I heard that Sweden had even more than Denmark.

The welcome reception was the highlight of the day, other than there were so many great conversations that there was no time for swag or food. And given the number of attendees I may not find any good swag this year. Fortunately I have some friends who may take mercy on me...

The 1/4 mile lineup to rush for myITforum party buttons is the best evidence that all other vendors should pay some kind of commission to myITforum for bringing people into the Exposition area.

And breakfast was basic, lunch was light but good, and dinner was wonderful - a fine Italian meal. Intersting vegetables, followed by fine clams and spaghetti-like noodles that worked very well with the wine. Then finished with a delicate quale. Conferences are hard work, but there are rewards...

 

Posted by pthomsen | with no comments

Summary: we're off to a great start. We're ready to get serious...

The whole point of arriving Sunday is to get settled and catch up with old friends. Both were done in grand fashion. The trip from door to door went according to plan. The only problem was when registering - because the event sold out, none of us Microsofties get a bag. So I don't have a picture for you this year (the first time in 11 years, but maybe I can correct that problem later). It's backpack style, which is unique, and seems to be large enough and good quality. I have no details on the contents either, of course.

The first (informal) community event (La Scena) was booked with a tournament, so we met at Zeffirino a little early, which was almost as good. Twitter proved to be valuable in getting the word out. The main community event at Zeffirino was very nice - thanks to SCCM Expert and Adaptiva.

All told I probably saw 30 friends. We didn't have much time to get into serious conversations, but there's plenty of time in the week for that.

In addition I got out for a couple of light meals. Lunch was 5 interesting Italian cheeses with 3 sauces. 2 of the cheeses stood alone well and a third cheese was wonderful with one of the sauces. The rest were ok. Dinner was DIY Kobe beef slices. That was a new experience for me and exquisite. I'm a huge fan of beef but this took beef to a new level for me. That's part of the fun of Vegas - new experiences.

 

Posted by pthomsen | with no comments

Summary: a new ConfigMgr DCM module is available for beta testing. It's supposed to be packed with goodness to ensure your Windows computers are secure (and compliant with relevant standards).

The Security Compliance Management toolkit provides you with best practices from Microsoft about how to plan, set, get and remediate a security baseline, along with tools that you can use to verify the implementation of recommended security baselines from Microsoft for Windows Vista, Windows XP SP2, and Windows Server 2003 SP2.  

   
Security Compliance Management – Beta Now Available!
 

 

 

 

 

 

 

 

 

 

 

I haven't tried it myself, but I figure at least some of you will find this useful. More details are included in the download. And we'd like your feedback to make it even better.

p.s. AFAIK it doesn't work with SMS 2003 DCM.

Posted by pthomsen | with no comments
Filed under: ,

Summary: MMS 2008 is less than a week away! It's time to finalize the preparations.

The excitement is palpable around here. We've got 13 people coming from my team this year (what might be thought of as the service provider for 'Microsoft IT'). If you think of us as a customer (which we largely are), we are probably the best represented customer of all. And yes, we do have to pay the full cost like any other customer. That's up from 6 people last year, which I thought was quite good. So we highly value this opportunity, especially because it gives us a chance to talk to you (yes, I'm serious).

I did my annual internal briefing today on how to get the most from the conference. It's essentially the same as last year, so I refer you to those postings:

During my briefing I add a few points for Microsofties, but the only point I would generally add is: use your biggest piece of luggage. You shouldn't need it on the way there, but on the way back it will be stuffed full of swag, hand-outs, the conference bag, etc. So you'll need the space.

Rod Trent always does a wonderful job of preparing everyone for MMS, so I encourage you to check out his blog for the last few months. That especially includes: http://www.myitforum.com/absolutenl/nls/myITforum_MMS2008.htm

 

Posted by pthomsen | with no comments

Summary: the SMS Client Health Tool (CHT) is being updated in the upcoming ConfigMgr 2007 R2 release, and renamed as Client Status Reporting (CSR). There are improvements big and small that you should consider.

Rob Stack, one of the ConfigMgr product group technical writers reviewed the CSR a few weeks ago. That's a great summary, so there's no point in me doing a similar summary here. But there are a couple of subtleties that Rob alludes to but are worth highlighting:

  • The tables now include complete client activity history - I've always been an advocate of judging client health in the context of 'normal' activity. Historic data makes that possible.
    • the history data is at the hierarchy level, so my earlier posting on saving your own historic data will still apply if you want site-level client activity reporting
  • FSP data - FSP data is great in its own right, as is client activity data - combining the two is wonderful. The CSR service does that, so that way we have richer client health data.

And there other points worth making:

  • CSR is implemented very differently from the CHT, so you can readily run them in parallel. You can run CHT for your existing SMS 2003 and ConfigMgr hierarchies, and CSR for your ConfigMgr SP1 hierarchies from the same server. (of course that assumes you have multiple hierarchies, and a single server that you would like to be both your CHT and CSR server)
    • when you're done with CHT, you can simply de-install it. In fact, if you don't, both will continue to run, unnecessarily. When you install CSR it will not remove CHT.
  • Once your central site is upgraded to ConfigMgr SP1, you can use the CSR for all the sites in the hierarchy, even if some are still SMS 2003 sites or ConfigMgr sites without SP1. The core functionality is the same for both tools, so CSR can access the same MP logs and clients that CHT would, regardless of site or client version.
  • The new CSR database views are v_CH_ClientSummary and v_CH_ClientSummaryHistory (there's also v_CH_PolicyRequestHistory). There's some common ground with the CHT tables, but there's enough differences that if you've created reports based on the CHT tables then you'll have to rewrite them to run against the CSR views. 
  • Even though the CSR data is stored in the ConfigMgr database, that doesn't mean you can create collections based on it.
  • One of the ongoing challenges of the client health art is terminology. I like "client activity" for client health data that is collected at the SMS/ConfigMgr servers to show which clients are able to reporting client-side activity all the way through to the servers. That excludes clients that are broken at the ccmexec level, the client-side level generally, the environmental level (networking, basically), or the server-side level (broken MPs, etc.). So "activity" is the best we can see with such server-side solutions - we can't say anything about the health of the clients themselves. And "status" is a multi-purpose word, so it doesn't clearly say what CSR is reporting, IMHO. So I think of CSR as actually providing "client activity reporting"
  • ConfigMgr R2 enables SQL Server Reporting Services (SRS) reporting. I'm optimistic that CSR data will be readily reportable via SRS.

There are other subtlites to CSR, but I'll save those for when R2 releases, or is close to releasing. That level of subtlety can change as the product is finalized.

Overall, CSR is much easier and much more meaningful than CHT for client activity reporting, and the upgrade path is quite easy. So you should use Client Status Reporting as soon as you can.

p.s. While we're talking client health, I like this recent Jason Milley myITforum posting on DCOM issues (particularly as they relate to CPI, but I would consider that a client health issue).

Posted by pthomsen | with no comments

Note that the rooms for my presentations at MMS have been upgraded. We now have capacity for 1,130 people in each session!

So rather than being in Titian 2303 and Murano 3201A respectively, we're now in Veronese 2401B for both. Update your calendars accordingly, but if you're using the scheduling tool at the MMS-2008 website then I'm sure it will automatically reflect the change.

Be sure to schedule your sessions. It's a handy way for you to plan your week, but it also helps the conference organizers to adjust room capacity as needed. It's also an unofficial way to cast your vote for what's important. I like to think client health is especially on that list!

Posted by pthomsen | with no comments

Summary: organizations with large numbers of SMS or ConfigMgr clients have special challenges. Why don't we talk at MMS, informally and confidentially, to share our solutions to those challenges?

My group's main customer is Microsoft IT, with 250,000 clients. I know there are plenty of other big SMS and ConfigMgr customers - that's one of the great strengths of SMS and ConfigMgr. I'm pleased to share our experiences (and often do), and I hope you would as well (if you're a large customer, or plan to be).

This suggestion is inspired by a thread on myITforum's forums: http://www.myitforum.com/forums/m.asp?m=176737.

I'm not sure how many people would like to participate, when they're available, how much there is to discuss, etc. So for now please e-mail me if you're interested, and I'll coordinate something based on the responses. That's at pthomsen@microsoft.com. In particular, let me know:

  • your organization and number of clients
  • when you're available at MMS 2008 (Sunday afternoon through to Friday noon)
  • whether an NDA or similar agreement is important to you (formalized confidentiality)
  • what topics you would like to discuss
  • how many people from your organization would like to attend our get-together
  • what kind of venue you would prefer (bar, lunch, dinner, meeting room, etc.)

And don't forget my Friday morning session on using ConfigMgr on our 250,000 clients at Microsoft IT, as described in this blog posting.

p.s. How large is "large"? I'm flexible on the point. My guess is above 50,000 clients, and certainly above 100,000 clients. Especially if at least one of the physical locations is large (well above 25,000 clients). For now we can self-select on this question.

Summary: here's a technique for collecting details about the HAL used by your computers. Do you need to know which Hardware Abstraction Layer (HAL) your clients are using? For example, you may wish to ensure you're using all features (such as multiprocessors).

This solution is from a couple of years ago, and I don’t know if I had the optimal method (or whether it’s still optimal), but research at that time indicated that the following was necessary, and I don't have reason to think there's a better solution:

1. Do a binary file open of %windir%\system32\hal.dll
2. Find the original HAL name in the file
3. (optionally) translate that into a meaningful name (if I remember right, there were only a couple that were common, and of course things may have changed over the last couple of years) (code snippet below)
4. Write the results to a custom WMI data class (or a registry entry)
5. Collect the results using typical SMS/ConfigMgr hardware inventory extension methods

Tricky, but not bad.

                'HAL descriptive name
                if HALname="halmacpi.dll" then HALdescr = "ACPI Multiprocessor PC"
                if HALname="halaacpi.dll" then HALdescr = "ACPI Uniprocessor PC"
                if HALname="halacpi.dll"  then HALdescr = "Advanced Configuration and Power Interface (ACPI) PC"
                if HALname="halsp.dll"    then HALdescr = "Compaq SystemPro Multiprocessor or 100% Compatible"
                if HALname="halapic.dll"  then HALdescr = "MPS Uniprocessor PC"
                if HALname="halmps.dll"   then HALdescr = "MPS Multiprocessor PC"
                if HALname="hal.dll"      then HALdescr = "Standard PC"
                if HALname="halborg.dll"  then HALdescr = "SGI mp"

p.s. Here’s some resources on the topic:

• http://support.microsoft.com/default.aspx?scid=http://support.microsoft.com:80/support/kb/articles/Q237/5/56.ASP&NoWebContent=1
• A script class to read file version information: http://www.jsware.net/jsware/scripts.php3 (under File Version Information Class) (I don’t know if there are licensing issues)
• http://www.microsoft.com/resources/documentation/WindowsServ/2003/all/deployguide/en-us/Default.asp?url=/resources/documentation/WindowsServ/2003/all/deployguide/en-us/acicc_ris_jfxo.asp  (you’ll have to dig a bit)

Summary: did you feel a disturbance in the computer management force this week? Some of us serving Microsoft IT felt it. A moment of silence is in order.

You might think I jest, but as a fellow techie you probably know that we develop a respect and even affection for hardware that sees a lot of history, doing a lot of important of work with us. It's much like we respect great software (OpenVMS anyone?) or great books{"VAX/VMS Internals and Data Structures" still does it for me, but Charles Petzold's "Programming Windows" is a close second). Similarly, we respect great (even if rarely famous) techies of all sorts (too numerous to even begin listing) and great tools.

Over the years any techie will spend longs hours with their main servers. Often in the middle of the night and/or under stressful circumstances. No matter how good the hardware, failures will occur, and we have to wrestle the situation under control. If anything, we're amazed such problems don't occur more often. In so doing, we come to respect our allies - the hardware. It's easy to spend more time with the server than our spouses.

Here at Microsoft IT we've had such a server, with sitecode "RDM" and server name "B11ITGSMS01". It has been the central site of our main hierarchy for many years.

One of my coworkers, Edward Bell, has written the following words in honor of RDM, which lost the last of its child sites (and thus clients) on Friday (March 28th, 2008). I'm pleased that he has allowed me to share those words with you:

"RDM is a battle tested warrior and should be retired with full military honors. The server has been in service longer than a number of the people working on SMS today have been at Microsoft.

Did you know…

• The RDM SMS central site server was placed into service around 2000 running SMS 2.0 with a pre-release version SP2. 

• RDM replaced a previous SMS 2.0 server with site code GBL for Global.

• RDM has been upgraded to following SMS versions: SMS 2.0 SP2, SMS 2.0 SP3, SMS 2.0 SP4, SMS 2.0 SP5, SMS 2003, SMS 2003 SP1, SMS 2003 SP2, and SMS 2003 SP3.

• In 2002, RDM server hardware was an enterprise class SQL server with 8 CPUs and 8GB memory. I ordered and configured the server and it cost around $40,000 dollars. Yes, [even then it occasionally] still had SMS inbox backlogs.

• [At its peak] RDM managed more than a quarter of a MILLION (250,000) computers.

• RDM SQL database size was approximately 250GB.

• RDM has deployed millions of software package intances to Microsoft desktops.

Top 10 suggestions on what to do with RDM:

1. Declare RDM a god. Start a new religion. Rev Paul Thomsen will hold services Sunday from 10am to 12noon. We will be reading from the book of SMS 2003 Concepts, Planning & Deployment, Chapter 14, Upgrading to SMS.

2. Launch RDM on a Delta rocket into outer space. Maybe in a few hundred or thousand years, some alien civilization will encounter the server and marvel at our advanced technology.

3. Put RDM out to pasture. Charge a $100 stud fee to produce baby RDM offspring servers.

4. Donate the server to the Smithsonian Institute.

5. Use RDM as an the world's largest artificial heart.

6. Place RDM in the Microsoft Museum. Tourists can learn about system management [as it was done] in the good old days.

7. On Halloween, drive RDM around to geeks' homes. Yell 'trick or treat' and frighten them with old server technology.

8. Use RDM as a DVD player to view Mitch Groeneveld’s DVD office collection [which is huge].

9. Convert RDM to a slot machine and sell RDM to a casino in Las Vegas.

10. Display RDM outside of the Tuk5 Datacenter as modern art. Just remember, to power it down and unplug it.

Finally, we should retire the RDM Site code. No Microsoft SMS server can ever use the RDM site code again. This works for athletes. Why not servers?"

Another long-time colleague, Brian Wyne, shares those sentiments "And part of me dies with it. I will miss it forever. Goodbye RDM."

UPDATE: I almost forgot these words, from Mike Church, another of our MSIT colleagues (and who spent many years in the SMS Product Group):

"RDM we’ve loved you so
But now it’s time for you to go.
You’ve been with us for oh so long
To say good bye to some seems wrong.

But as they say all things must end
So here’s so long to you old friend.
It’s out to pasture now for you
We need to move to something new.

Some are glad to see you gone
Others wish you’d just hang on
But out the door, you got the boot
Get along it’s time to scoot.

As all us old farts will some day
It’s time to retire, you just can’t stay.
Don’t go mad, just go, get out
And as the door slams HOORAY we shout."

And the person that predates all of us as a Microsoft IT SMS administrator, Cutter Smith"

"Ah the memories indeed.
Seems so long ago I drew the Visio for the infrastructure wrote up the long Project plan and placed that fateful purchase order for good old RDM…."

Posted by pthomsen | with no comments

Summary: we talk about client health a lot on this blog, but ultimately we all want solutions. What solutions are available?

In a past posting I listed the solutions Microsoft IT uses for client health management. And occasionally I've talked about how to build reports. But of course there are more client health solutions. I'm sure I'll miss some, so please let me know what I'm missing. Here's my current list of computer management client health solutions:

  • Reports
    • The first step in solving a problem is understanding the extent of it. Reports address that task
    • From this blog you can find queries that can report client counts and client activity. They can be rendered as SMS or ConfigMgr web reports
    • If you're using the Client Health Tool (below), then it has a web report and Excel spreadsheet. Its data is also fairly good for distinguishing online broken clients and approproximating offline client counts. It's ConfigMgr R2 equivalent even has standard ConfigMgr web reports ready to go!
    • From this blog you can also get details to save the various pieces of data and thus build sophisticated reports
  • Computer startup scripts (or logon scripts)
    • As your computer starts up you can have them run a script that checks the health of your clients and tries to repair them if need be
    • A computer startup script is preferred over a logon script because it will run in the system context, and as soon as the computer starts up (but asynchronously and in the background, so it doesn't delay user log in).
    • Brian Mason has a great background article on this concept: http://www.myitforum.com/articles/8/view.asp?id=10774
    • As Brian says, DudeWorks (Rob Olsen et al) has a free download for the purpose, and they have a support forum to discuss it (myITforum.com occasionally has threads on it as well)
  • Client Push Installation
    • This is a standard client deployment method for SMS and ConfigMgr
    • If you know which clients might be unhealthy, you can try pushing the client at them to see if re-installation helps (it does sometimes)
  • Remote Scripts
  • Manual Remediation
    • Yes, this is what we're all trying to avoid. It's expensive because it's time consuming and may involve travel. But it works, so for completeness we must keep it in mind
    • Most often you will find that the root cause is not in SMS or ConfigMgr itself. For example, it might be a WMI problem (see below). Fixing those issues will also help other software that depends on those components
    • Checking the state of the SMS or ConfigMgr service itself (ccmexec) is a good starting point. It may be stopped or disabled, which is easy to correct. Finding out why it was stopped or disabled may be trickier, as is correcting that problem
    • If the computer seems fine but ccmexec is not working, running the command ccmexec /repair may help
    • If the repair doesn't work, a simple client re-installation may help. Deinstalling and re-installing is another variation that sometimes helps
  • Manual Investigations
    • This is all the usual technical troubleshooting we all do, including log analysis (which can be partially automated)
    • Again, manual investigation is not our favorite solution, but it's the ultimate method to take the mystery out of anything you can't explain by other means
  • The SMS Client Health Tool
    • It's just a reporting tool (no repairs), but it provides some valuable data that is not available elsewhere. In particular, it scans management logs for client policy requests (which should happen hourly, by default). For those clients that don't request policies, it pings them to find if they're online, and if possible it will try to retrieve a few core details about the state of the client.
    • This is a free download available from Microsoft since September 2004:
      http://technet.microsoft.com/en-us/sms/bb676776.aspx
    • It was originally for SMS 2003 SP1, but I've used it successfully with every version of SMS and ConfigMgr since then
    • A new version will be available with ConfigMgr R2, called Client Status Reporting. I'll blog more about that soon.
    • Brian Tucker has a great blogcast on it at the BlogCastRepository:
      http://www.blogcastrepository.com/blogcasts/folders/sms/entry5439.aspx
  • Collection-based Targeting
  • Fallback Status Point (FSP) data
    • FSP is a new ConfigMgr system role that collects data on clients that have problems during installation or (in some cases) start failing to communication with management points
    • In addition to the core product documentation that refers to FSPs and the in-the-product reports, you can check out a blog posting I wrote previously on FSP data.
  • Maximize the number of clients that are online
    • If you want more clients to be active (to apply patches tonight, for examples), then you can use IBCM and/or WOL
    • Internet-based Client Management (IBCM) is a ConfigMgr feature that allows you clients to be managed when they're outside of your corporate network, as long as they can access the Internet
    • Wake-on-LAN (WOL) is available in ConfigMgr and from third parties. Computers that are powered down can be woken up remotely
  • Anything that improves server and environmental health
    • If your SMS or ConfigMgr servers are working less than perfectly, or the environment your clients are working in has issues, then your clients will be less active, and thus less healthy, than they could be
    • Consider using MOM's or OpsMgr's SMS or ConfigMgr management packs
    • Use good ITIL or MOF (Microsoft Operations Framework) policies and procedures to keep your servers running smoothly
    • Microsoft Consulting Services offers an "SMS Health" engagement where they'll assess your SMS or ConfigMgr server-side health and provide advice for improvements. More details are are available at:
      http://www.microsoft.com/downloads/details.aspx?FamilyID=C52E06CF-F702-4490-9A6A-3C98F309D4C9&displaylang=en
    • No doubt other consulltants offer similar services, as may your Technical Account Manager or Premier Support specialist, if you've signed up for such services
    • The community, including myITforum.com, the Microsoft forums and newsgroups, blogs, and other web sites offer advice on a wide variety of issues 
  • Anything that improves computer health
    • If the client computers themselves are running well, then that maximizes the opportunity for the SMS or ConfigMgr client to run well
  • WMI Improvements 
  • Guidance
    • Understanding the world of client health can be challenging. We all have an intuitive understanding, but the more we dig into it, the more challenges we find. So reading about client health will help to get you comfortable with this world
    • Rick Jones' and Chris Stauffer's documentation based on Chris Sugdini's collection-based solution (above)
    • Don Hite's blog occasionally has articles on client health
    • I hope this blog is useful
    • My presentation at MMS 2008 on client health will take a 'start at the beginning' approach and dive into as much detail as time allows. So I hope that will make for a cohesive story
      • check it out at presentation SI21, "Advances in SMS 2003 and Configuration Manager 2007 Client Health Management", which is Tuesday, April 29 11:45 AM - 1:00 PM in the Titian 2303 room
      • or see it on the DVD, when that's available
    • If you think more is needed, say so (to anyone that will listen, including in the comments of this blog)

    Whew - that's a lot of options. You don't have to use all of them. Once you understand your needs, you can pick and choose the solutions that are appropriate for you.

    And I hope you noticed that a lot of people have been working on client health, including Microsoft since September 2004. Together we are beating this issue.

    Posted by pthomsen | 6 comment(s)
    Filed under:

    Summary: patch scanning is normally a quiet behind-the-scenes activity that computer managers don't have to worry about. But that doesn't mean we shouldn't proactively look for worst-case scenarios.

    Those of us that have been in the patch management business a couple of years or more will recall that sometimes patch scanning can be less quiet and behind-the-scenes than it should be. So we know that it's wise to watch patch scanning times. Even if there isn't a widespread issue, maybe there are some corner case scenarios we can identify and improve.

    The following SQL script calculates the scan time for SMS 2003 clients:

    -- get a sample set of relevant records to work with - for large hierarchies the whole table would be too large
    SELECT top 100000 machinename, time, messageID into #temp
    FROM v_StatusMessage s3 LEFT OUTER JOIN v_StatMsgAttributes AS att ON s3.RecordID = att.RecordID
    WHERE att.AttributeID = 401 AND att.AttributeValue = '<patching advertisement ID>' AND messageID in (10005,10009)
    group by machinename, time, messageID

    -- put the 10005's (advertisement started) into a seperate table, so the max time select won't be confused with the 10009's max time select
    select machinename, time, messageID into #temp5 FROM #temp where messageID=10005
    -- same for 10009's (advertisement successfully run, with details returned via status MIF)
    select machinename, time, messageID into #temp9 FROM #temp where messageID=10009

    --get the most recent records for the 10005's
    select t1.machinename, t1.time, t1.messageID into #temp2
    from #temp5 t1 join #temp5 t2 on t1.machinename=t2.machinename
    group by t1.machinename, t1.time, t1.messageID
    having t1.time=max(t2.time) order by t1.machinename
    --same for the 10009's, and put them into the same temp table
    insert into #temp2 (machinename,time,messageID)
    select t1.machinename, t1.time, t1.messageID
    from #temp9 t1 join #temp9 t2 on t1.machinename=t2.machinename
    group by t1.machinename, t1.time, t1.messageID
    having t1.time=max(t2.time) order by t1.machinename

    --build a new temp table with just the times, so that the datediff calculation is easy
    SELECT machinename,
    ( select time from #temp2 s1 where messageID=10005 and s1.machinename=s3.machinename) 'Start',
    ( select time from #temp2 s2 where messageID=10009 and s2.machinename=s3.machinename ) 'End'
    into #temp3 FROM #temp2 s3 order by machinename

    --look at the results, without the exceptional data (with is a smallish percentage, like 15%)
    select datediff(s, [start], [end]) from #temp3
    where [start] is not null and [end] is not null
    and datediff(s, [start], [end]) >0 and datediff(s, [start], [end]) <2000
    order by datediff(s, [start], [end])

    --the all important average
    select avg(datediff(s, [start], [end]) ) from #temp3
    where [start] is not null and [end] is not null
    and datediff(s, [start], [end]) >0 and datediff(s, [start], [end]) <2000

    This script is also a good example of the usefulness of temporary tables. I don't pretend to be a SQL guru, but I like techniques that allow non-guru's to accomplish complex tasks using SQL alone.

    As usual, I can't guarantee that this is the only or best way to accomplish this goal. But it has worked well for me and seems to return accurate results. Now I have to come up with a ConfigMgr equivalent...

    Posted by pthomsen | with no comments

    Summary: The ConfigMgr software development kit is finally released to a public URL.

    Actually, it hasn't been hard to get ahold of the SDK (via the Connect (beta) program). But that can be a bit of a hassle, and while the SDK is being worked on there's obviously going to be some content that is missing or tentative.

    In this case I know the SDK has received a LOT of work from some top-notch people. I'm sure you'll be impressed by the sheer size of it. There's got to be lots of good stuff in it.

    I have a bit of programming experience (well, 5 years as a professional programmer, at a few companies). And I certainly like to get creative when solving computer management problems (thus the blog, and similar efforts in the past). An SDK is the ultimate inspiration for techniques to address things that I would like to improve. So this is very exciting for me - I hope it is for you too.

    http://www.microsoft.com/downloads/details.aspx?FamilyId=064A995F-EF13-4200-81AD-E3AF6218EDCC&displaylang=en%20

     

    Posted by pthomsen | with no comments
    Filed under: , , ,

    Summary: SMS 2003 and ConfigMgr 2007 have an often useful feature called protected distributed points. But a site can have many protected DPs, and the locations that correspond with each DP will be of various sizes, so how do you know how many clients you have for each DP? How do you know if any of them are possibly supporting too many clients?

    With protected DPs, you can't just divide the site size (in client count) by the number of DPs in order to get the clients per DP ratio, as you would with regular DPs. Each protected DP only serves the clients within its boundaries, and that will vary widely. So you'll need a query that relates the clients in the boundaries to the boundaries for each of the DPs. If you're using ConfigMgr, that's relatively easy:

    (The ideal way to determine excess clients per DP is to monitor relevant performance counters during your worst-case deployments, but that's very labor intensive and has to be timed just right, so a cruder approximation is useful).

    If you are using AD sites as your boundaries for the protected DPs, this query will give you the answer:

    select servername, sitecode, count(distinct name0) 'clients'
    from ProtectedSiteSystem_ARR PSS join v_BoundaryInfo bound on pss.BoundaryID=bound.BoundaryID join v_R_System sys on bound.value=sys.AD_Site_Name0
    where client0=1 and obsolete0=0
    group by servername, sitecode
    order by count(*) desc

    If you're using IP subnets as boundaries, this will be the query:

    select servername, sitecode, count(distinct name0) 'clients'
    from ProtectedSiteSystem_ARR PSS join v_BoundaryInfo bound on pss.BoundaryID=bound.BoundaryID join v_RA_System_IPSubnets subs on subs.IP_Subnets0=bound.value join v_R_System sys on sys.resourceID=subs.resourceID
    where client0=1 and obsolete0=0
    group by servername, sitecode
    order by count(*) desc

    If you're using both kinds of boundaries, the query is left to the reader ;-)

    Of course, no solution is perfect. The subnets or AD site data for each client is dependent on the latest discovery data, so if you're running discovery infrequently then you may count some clients that have moved away, or not count some clients that have moved into the boundaries.

    UPDATE: If you're using SMS 2003, then the above technique won't work - it doesn't have the ProtectedSiteSystem_ARR table (thanks to Yanze pointing that out in the comments (I am rather ConfigMgr-centric these days)). Russ Slaten's blog has a script that looks up protected DPs and their boundaries via the site control file. You could fairly easily extend that script to pull in the number of clients in each boundary, thus getting the same result as my queries above. That will work for both SMS 2003 and ConfigMgr.

    p.s. A query best practice is to always use the views. But in this case the most useful information seems to be in the ProtectedSiteSystem_ARR table - I couldn't find an equivalent view. So I've broken with best practice in this case. In future versions of ConfigMgr we may have to tweak these queries.

    p.p.s. As is almost always the case, this approach is based on my own analysis of the possible solutions, so I (and Microsoft) don't guarantee this is the best possible solution for this problem. But it seems to be working well so far. Your thoughts are greatly appreciated - I'm always looking for better solutions.

    Posted by pthomsen | 2 comment(s)

    Summary: users have good reasons to keep their e-mail distribution list memberships up-to-date, so they're a great way to accurately target software distributions. Here's how you can create collections based on DLs (my which I mean Active Directory distribution lists, as would be used by Microsoft Exchange and Outlook).

    I've always been a fan of distribution lists for keeping track of users. People care a lot about receiving e-mails that are applicable to the groups and teams they belong to, but they don't want spam, so they'll add and remove themselves to DLs as needed, without a long delay. Any other targeting mechanism is going to be dependent on details that users don't care a lot about (like OU's, or hardware configuration), lists made up by third parties, broadcasting, or being reactive ("I need this software, please send it to me now!").

    So if you can do software distributions to collections based on DLs, you're very likely to hit the right people (more exactly, the machines of the right people). Similarly, any reports based on those collections will accurately portray the relevant machines.

    Here’s the query for a collection that targets a DL close to my heart:

    select * from SMS_R_User where SMS_R_User.UserGroupName like "%SMS Admins & Engineers - Desktop Mgmt"

    You have to use the display name of the DL, not its alias. And I use the “%” so that I don’t have to worry which domain the group is in.

    That’s for the users in the DL, of course. If you want to target the computers of the users, you can build a second collection using a query like so:

    select * from SMS_R_System where LastLogonUserName in (select UserName from SMS_R_User where Name in (select Name from SMS_CM_RES_COLL_CEN00315))

    So that’s a lookup via a subselect to the collection above and then a little translation of username format (via SMS_R_U