Analyzing a lot of logs
Summary: SMS and ConfigMgr, and possibly other systems, make a lot of logs available in case you have to troubleshoot a tricky problem. If you need to look at some of those logs on a lot of clients or sites, then doing so manually will be very labor intensive. Why not do it via script?
There are a wide variety of problems where you have to look at a lot of logs. For example, the problem might be rare but nasty. Or it might be fairly common but counterintuitive - some people may think that the problem is so unlikely that the small subset of logs you manually look at are not representative of the rest of your clients or servers. Maybe the problem comes and goes. Maybe it occurs under special circumstances but you don't know which ones, yet. Or maybe you're just a perfectionist and you want to find every last problem.
Some relatively simple scripting will allow you to get a computer to do the log analysis.
The core part of the code is:
set fso = CreateObject("scripting.filesystemobject")
contents = fso.OpenTextFile( logfile ).readall
lines = split( contents, vbCRLF )
linecount = ubound(lines)
for i=0 to linecount
if instr(1, ucase( lines(i) ), "ERROR") then wscript.echo "error found in " & logfile & " at line " & i & ": " & lines(i)
next
The "if" statement is actually where you'll probably do most of the coding. There will likely be a lot of lines with the word "error" (or whatever else you're looking for) that aren't relevant to your issue. So you'll need to add a little logic to filter those out. Or maybe you just want to see the first occurence of each error (some errors repeat MANY times).
The next question is: how do we get the large number of logs we need? There are two approaches to that:
-
create a small script to write the log back to a common share. The script should make a folder for each client or include the computer name in the log name so the logs don't get overwritten (and so you can further investigate any suspicous clients). Even a .BAT file can do that. Then advertise the script to all your clients (or a relevant subset). The only trick in this case is to spread the workload over time so you don't have 10,000 clients trying to copy a log to the share all at once. And if the share is on a workstation (as opposed to a server), then remember the 10 connection limit.
-
create a read-only share on all clients that points to the %windir%\system32\ccm\logs folder. You might as well restrict access to just you and your fellow admins, and maybe even make it a hidden share.
For option 1, something like this batch file will do (substitute in your own server, share, and folder names, of course):
if not exist "\\<server>\<share>\<folder>\%computername%" mkdir "\\<server>\<share>\<folder>\%computername%"
copy %windir%\system32\ccm\logs\execmgr.log \\<server>\<share>\<folder>\%computername%
For option 2, I don't have a script that I can share at this time. So I'll leave that as an exercise for you or the community. It is on my ToDo list.
If you go with option 1, then once you've got that share with a bunch of folders and logs, the next trick is to get your script to work through the folders and logs, so it can do the above checking. Code like this will do that:
Set mainfolder = fso.GetFolder( path )
Set folders = mainfolder.SubFolders
for each folder in folders
set folder = fso.GetFolder(path & "\" & folder.name)
Set files = folder.Files
For each file in files
wscript.echo path & "\" & folder.name & "\" & file.name
'parse it, as above
next 'file
next 'folder
If you go with option 2, then you'll first need a list of the relevant clients. I do a SQL query and save the results to a file named temp.txt. Then the following logic will check the logs on each of those clients:
set infile = fso.opentextfile("temp.txt")
computers = split(infile.readall,vbcrlf)
for each computer in computers
on error resume next
set folder = fso.GetFolder( "\\" & computer & "\" & logshare )
if err<>0 then
wscript.echo " " & computer & " online but the " & logshare & " share is not available: " & err
on error goto 0
else
filetoget = "\\" & computer & "\" & logshare & "\" & filename
on error resume next
test = fso.GetFile( filetoget )
if err=0 then
on error goto 0
'parse it, as above
else
on error goto 0
end if 'get file
end if 'got folder
Either way, you've soon got your computer reading lots of logs. You can concentrate on the serious business of analyzing the results.
In the 'credit where credit is due' department, I can't claim credit for the idea of creating a share for the ccm logs folder on all clients (option 2 above). That was Levi Stevens. And my coworkers and I have written a bunch of scripts for log collection ever since I joined Microsoft IT, so some parts of the other ideas are no doubt inspired by coworkers as well.
p.s. Bonus material 1: SMS and ConfigMgr logs turn over so that they don't grow to consume the whole disk. That complicates log reading a bit because the current log might be very short, and thus not tell you the whole story for that client or component. You can open the previous log the same way we did in the first code snippet above, and prepend the contents before doing the split (i.e. contents = contents_prev & contents). If the logs end in ".log" and ".lo_" then that's trivial. But what if there's multiple previous logs, as with Configmgr clients, then you can use this logic:
Function Get_Previous_Log( path, filename )
set folder = fso.GetFolder(path)
Set files = folder.Files
for each file in files
if ucase( left(file.name, len(filename) ) ) = ucase( filename ) then
if ucase(file.name)<>ucase(filename & ".log") then if ucase(file.name) > ucase(prevfile) then prevfile = file.name
end if
next
if prevfile<>"" then Get_Previous_Log = prevfile
end Function
Bonus Material 2: if you use option 2 to grab logs from the clients, you'll find that many clients aren't online, or are on slow links from where you are. The code above will work in those cases, but it's even better to ping the clients first, so you can filter out those that aren't online or are too slow. This will do that:
Set loc = CreateObject("WbemScripting.SWbemLocator")
Set WbemServices1 = loc.ConnectServer( "", "root\cimv2" )
tooslow = 100 'ms
Set PingStatus = WbemServices1.Get("Win32_PingStatus.Address='" & computer & "',Timeout=1000")
if PingStatus.StatusCode = 0 then
if debugmode then wscript.echo " " & computer & " is online"
if pingstatus.responsetime < tooslow then <try to get the folder, as above>
end if
UPDATE1: some people have pointed out another similar solution (well, partially similar):
http://geekswithblogs.net/drewby/articles/LogParser.aspx
That's based on the Microsoft Log Parser tool:
http://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en
Someone has even created a GUI front-end for it:
http://www.softhypermarket.com/Visual-LogParser-download_46061.html
And Jamie Moyer gives a good summary of those solutions:
http://blogs.msdn.com/jamoyer/archive/2008/01/28/analyzing-sms-sccm-log-files-the-easy-way.aspx
UPDATE2: for the "option 1" method of copying files, the sample script I gave was for 32-bit clients. In particular, it finds the files at %windir%\system32\ccm\logs. But on 64-bit clients, the normal location is %windir%\sysWOW64\ccm\Logs. And 64-bit clients are getting more common these days (aren't they?). So some extra logic should be added to that process to copy from the appropriate location.
"Option 2" above doesn't suffer from that problem because it's copying from a pre-existing share. But the logic that makes that share available should also allow for 64-bit and 32-bit clients.
Either way, here's some vbscript that should help (but I don't guarantee that it works in 100% of the cases):
Set oFSO = CreateObject("scripting.filesystemobject")
sRoot=oShell.ExpandEnvironmentStrings("%SystemRoot%")
sTEMP=oShell.ExpandEnvironmentStrings("%TEMP%")
'what kind OS is this (probably)?
If oFSO.FolderExists(sRoot & "\sysWOW64\ccm\") Then
sOS = "Windows64bit"
ElseIf oFSO.FolderExists(sRoot & "\system32\ccm\") Then
sOS = "Windows32bit"
Else
sOS = "Unknown"
End If
If sOS = "Windows64bit" Then
'logs will be at sRoot & "\sysWOW64\ccm\Logs"
ElseIf sOS = "Windows32bit" Then
'logs will be at sRoot & "\system32\ccm\Logs"
Else
'if you need to put logs somewhere, this should do: sTemp
End If