Number2 Blog - myITforum

If you read my previous post on the blood-boiling comparison between SMS/SCCM & Promisec Spectator, you've likely developed all kinds of responses in your head about how they're so wrong on SMS/SCCM.  I couldn't believe it so I just had to respond. 

THE REBUTTAL

I'd like to address their points to make sure we're being evaluated in the proper light, so the tone of this is simply an honest rebuttal of their points to them.  This is delivered with no disrespect or malice, but with correction in mind.

(BTW, we're in the process of upgrading SMS to SCCM for the enterprise in the coming weeks and months…SCCM meaning System Center Configuration Manager from Microsoft, also abbreviated CM.  SMS, SCCM & CM are really used interchangeably in this doc)

My comments in RED

Intelligent Search and Reporting capabilities:

1.       SMS returns large amounts of data per endpoint, which can be up to 1,000 lines per endpoint or more. Furthermore, filtering through this data intelligently requires exporting it to an SQL database. Once installed, a number of filters need to be built and maintained in order to retrieve the relevant information needed. These tasks are time consuming and create work rather than alleviating some of the administrator’s workload.
While it's true that several "lines" exist in SCCM per computer (I'm assuming they mean lines of data), it amounts to less than 1 MB per computer, and it's already in SQL and requires no exporting like they suggest.  Furthermore, instead of sending a full manifest back to our servers every time inventory runs like they imply, our CM client only sends the deltas.  If there's no change in inventory from the last time it was run, there's no data to send back. 

I'm not sure what "filters" they think need to be built and maintained to retrieve relevant information…we have canned reports and we have custom reports and we can also provide data feeds/exports to groups that require data right from SMS.  I'm not aware of any filter building/maintenance tasks that we need to perform.

Also, in the < 1MB worth of data that's in the CM database,  is rich information that's being used for lots of different groups around the enterprise.  We keep Add/Remove Programs data, software deployment status, patch compliance, information on motherboards & system chassis, make/model, auto-start programs, logged on users, monitors, disk drives (hardware), disk space, DTC/WFDC build numbers, installed files/folders, network adapters, network configuration (like IP addresses, subnets, default gateways, DNS search order, DNS domains, etc), OS version & service packs/patches, partition info, BIOS info, keyboard/pointing device info, CDROM drive info, processors, RAM, SCSI controllers, Video Controllers, WFDC Installed packages, client health, etc.  In addition to that we have the ability to show software usage, Service States, Running Programs, Desired Configuration Management (drift management), the values of registry keys and just about any custom piece of information about a machine that can be gathered from Active Directory, WMI, the disk or the registry.  And we also have the ability to link to other external datasources to make the CM data even more useful and relevant.

Lastly, we currently keep 60 days worth of history too for any of the above items.  If RAM goes from 4GB down to 2GB, we  have a record when it was 4GB and we have the new record that says 2GB.  So there's much more to it than saying SMS has large amounts of data.

2.       Spectator Professional, as a clientless endpoint security management product, only returns the deviations from a pre-defined policy. With Spectator, the report is gradually reduced as items are either acknowledged or eliminated. However, with SMS they remain and are reproduced every time a report is generated. The only way to reduce the report is by way of filters that a user has to configure. Furthermore, there is no ability to remove acknowledged items from the reports resulting in a large report, full of line items every single time.
I'd like to know what their source is for this...  If there isn't one already, I could create a report within minutes which dynamically returns only those machines which meet whatever criteria we want.  If the machines get remediated and have no more threats, they fall off the report.  There's no "large report, full of line items every single time".  Also, using the desired configuration management (DCM) module in SCCM, you define a baseline and deviations from that baseline show up in reports.  But it's not just security related baselines, you can manage any kind of drift…if any file/folder gets created/removed, if the MD5/SHA1 hash of a file changes, any security setting changes, any registry key gets added/modified/removed, anything in WMI changes, anything in SQL, any XML gets modified, if the IIS Metabase gets changed, or any number of custom baseline items, SCCM can know about it and it will show up on a report.  If the drift gets remediated, it drops off the report.

3.       Additionally, the reports SMS provides are disparate and not necessarily consolidated into a single view of all the threats for the entire user group or company. This does not offer a very clear ‘big picture’ of the internal organization’s position. Only through checking lists of queries for numerous issues can the relevant report be generated
If the canned reports don't meet the business needs and we need a different single view of all the threats for the entire user group or company, I need only 15 minutes to create a dashboard (a built-in feature of SMS/SCCM, not some rogue, custom one-off app) that consolidates any reports into a big picture view, or create drill-down reports that start at the summary level and allow a click-through to the details of a particular threat or a particular OU.  There's no "checking lists of queries for numerous issues"...whatever that means.  We provide insight into the data, the business uses it, we move on.

Furthermore, we have the ability to include in our detail reports any of the previously mentioned data including the user info (address, phone number, etc), Active Directory info like the OU or group membership or the OS build date or any of thousands of pieces of data making our data much more robust and useful.

4.       Spectator on the other hand was developed from the point of view of the CSO, auditors and security managers who could manage and control their endpoints from a single location. Spectator also enables the workload to be split between several consoles that report to a single centralized management console that consolidates reports onto a single view - Another feature lacking with SMS.
This is false.  SMS/SCCM is centrally managed as well.  Yes, there are multiple servers in our hierarchy, but they all are centrally managed from the top down and data from all the clients trickle back up to the top so there's one management console (which can be customized for particular roles and secured accordingly) and one reporting point where everyone goes to get their data.  This hierarchy gives us the ability to scale up to the hundreds of thousands of clients we have now, or grow to millions of clients like other organizations have.

5.       Spectator is also able to take a snapshot of a clean ("healthy") host and then compare it against all of the endpoints in the network to easily identify any differences and irregularities. Spectator also enables the building of a baseline of approved applications, processes, services, start-up commands and toolbars in minutes without the need to write any scripts or define any filters; which makes it much easier to identify deviations. With SMS, the user is able to produce a complete inventory scan but is then required to build a number of filters to retrieve the relevant information.
This is true, but misleading. The implication is that we or any of their other customers will be able to use their product without modification. I imagine we'll have to tell it what subnets to use, what account(s) to use when accessing a machine remotely, configure the console security,  The extensibility of SMS is one of its strengths. The promise that Spectator will be used without any custom scripts or filters is a promise that nobody can make without knowing our requirements.

6.       Even items that are not present on an endpoint are still included in the SMS report but have no values in the fields, which may be seen as a false positive.
This has nothing to do with the data SMS provides, it is entirely based on the way a report is written. You would have to go out of your way to create such a poorly written report that shows such false positives.   This is not something inherent to SMS/SCCM, this is something inherent to unskilled IT people which our team doesn't have.  In fact, to make this claim at the same time as the previous claim is flawed logic. They say, in effect, “SMS users have to create their own reports and those reports are written poorly.”

Accuracy

1.       SMS gathers its information from locations on the OS that is not reliable. For example, SMS can inform you that a bad application is installed only if it has been registered on the "ADD\Remove application” field. Unfortunately, applications can be removed from this list by design, or in purpose which means that you will not get accurate information.
This is false.  Add/Remove Programs is not the only place SCCM gets it's information.  It also scans the hard disks for file/folder information, and scans WMI for any number of things including the registry, services, running processes, and so on.

2.       Another example is checking the existence of a third party AV and its condition. SMS will check again for the “ADD\Remove application” list, which and if you have sophisticated SMS operators they will also check on the “Service” field. Then they will also make another effort to check for the DAT entry on the registry field (This will need to be maintained on a daily basis.)
As previously stated, SMS/SCCM can check any number of places including the registry, in WMI, in files themselves, and yes, in Add/Remove Programs.

"DAT entry on the registry field" that "will need to be maintained on a daily basis"?  I'm not sure how to respond to that…it's like saying "If you take your car to that mechanic instead of this one, you'll have to adjust the Johnson-Rod more often"

3.       …although many efforts are being made, you will not receive sufficient and accurate information.
…says the salesman to the customer about his competition…saying it does not make it true :)

4.       For Spectator Professional, which was designed to perform as a security tool, these checks are built in, maintained by the vendor and do not require any efforts from the operators. Spectator Professional’s inspection interrogates the OS from several different APIs, makes comparisons and  correlations of the information it collects, and provides you with 100% accurate information.
Granted, some work will be required to setup the desired configuration baseline, but it's 100% configurable, will provide history as well as the ability to remediate, and there are already vendor and community created baselines to use as a starting point.  Implementing yet another 3^rd party suite would likely be much more work than creating the baseline, plus THAT has a cost, whereas SMS/SCCM are already licensed under an enterprise agreement.

5.       SMS will not report on Machines which his agent is not installed on while Spectator in clientless in order to “catch” anything on the network.
SCCM isn't intended as an application to detect rogue machines.  We already have entire groups dedicated to security including intrusion detection and rogue machine management and everything that is security.  If groups want to manage configuration, inventory, drift, OS deployment and dozens of other things, they get managed with SMS/SCCM.  If groups don't want that, their machines stay hands-off.  If they fall into the rogue category, the enterprise already has facilities in place to eliminate or sequester the threat. So unless Promisec Spectator is going to replace those processes/products, this is an invalid argument. In addition, the inherent limitation of “agentless” systems is completely missed. A secure system will not allow scanning by an agentless mechanism. Robust discovery and inventory of systems requires either an agent or root level access, something that rogue machines rarely offer.

Speed and Performance

1.       SMS scanning process is very slow and can overload your machines and your network bandwidth.
I'd like to see the source cited for this. There are machines with Windows 2000 and as little as 128 MB of RAM running the SMS agent with no reported issues.  If it were "overloading" the machines, we would know.  There are 10's of millions of these clients installed at companies all over the world.  You don't get that big by overloading machines and the network, you get there through competence and value.

Also, no bandwidth is used during the inventory/scanning processes in SMS/SCCM.  The only bandwidth that is used is the amount needed to send any inventory deltas.  DELTAs, not full inventory.  If there are no changes, there's nothing to send back but a datestamp of last inventory.  That's, something like 20-40 bytes which I wouldn't think could overload the network.

The times SMS/SCCM does tax the network is with software/patch DISTRIBUTION.  This is a separate issue that occurs because of things like having 250MB of patches that need to go out to 200K machines…that means you're moving something like 48 Terabytes of data to the clients.  Of course that's going to tax the network.  Although with SCCM and the recent Nomad Enterprise purchase from 1E, that 48TB will be reduced dramatically because the 250MB package only needs to go out once per subnet and all the other clients will get their package from that one machine locally.

2.       Most of the time organizations will choose to run SMS off-hours and by doing that, they will “miss” many machines and will receive partial information.
This is another statement that I'd like to see cited.  Our engineering team is very active in the SMS/SCCM community (2 of our engineers are  Microsoft MVPs in SMS/SCCM and 2 of us were asked to present on SMS/SCCM/SQL issues at the Microsoft Management Summit this year)…we have all been at this for 8-10 years so we know what "most organizations" do with regards to SMS & SCCM.  SMS is a 24/7 application.  There's no running it "off-hours".  It's not like there's some master on/off switch that we throw before we go home. Processes are going on all the time…inventory is coming in all the time.  We schedule inventory every 24 hours, but that's a simple schedule who's start time is determined at the client, not initiated by the servers like a clientless solution would be…if the client is off the network, inventory still happens (because our agent is still installed and running) and when the machine comes back into the office, SMS sends all of the delta inventories that have queued up while it was offline (it uses BITS over HTTP distributed to regional management point servers so it's polite on the network, even if there happens to be a lot of data in this case).  If a client is powered off at night, the SMS client will initiate any overdue actions once it's back up, but nothing is missed!  You cannot do that with a clientless solution. 

3.       Spectator runs around the clock and will never impact your network bandwidth. Also, Spectator Professional is transparent to the machines on the network…
I can't see how something that is clientless will "never impact your network bandwidth".  When you scan a machine remotely, you have to tell the machine what to scan, and you have to tell it every time.  That means your server has to send a list of registry keys, files, folders, WMI classes, etc.  that the client needs to look for and report on.  So, if there's 1500 items to look for, each averaging 35 bytes long, that's 52K worth of data just by itself, for one machine.  Include the overhead for instantiating objects/APIs remotely and the associated acknowledgements/return values/error handling, etc. and we're talking, what, 100K? I'd guess that's much lower than the real number, but let's say 100K per machine * 200K machines…that's 19GB of data just to scan the machines, and because nothing's stored on the clients, that needs to be sent EVERY SINGLE TIME a scan is run.  Now add to that the data that actually comes back from these scans. Let's say that's 100K per machine.  Now we're talking 38GB of data.  Also, because it's clientless, it has to do a ping-sweep of all addresses in all subnets.  That's going to add slightly more traffic.  So I don't see how can that "never impact your network bandwidth".  To be fair, most network links can handle that, but some of the worst sites (like a 56K frame relay to a remote site in another country) might have some issues and to pretend there's no impact on the network for clientless (but is for SMS) is disingenuous.   

Contrast that to CM…because there is a client on the endpoints, CM is able to compress and encrypt the inventory/baseline policy from the server to the clients (resulting in lower bandwidth and higher security) and only needs to transfer the policy once and get another policy only if there is a change.  It also uses BITS, bandwidth throttling and checkpoint restarts to be as polite and reliable as possible on the network.  What does a clientless solution use?  I would venture to say there's actually a significant IMPROVEMENT in network bandwidth usage with SMS/SCCM over a clientless solution and some quick network captures could easily confirm that.

I think of an agentless solution as an agent that installs, runs, and then uninstalls.

4.       …which is why organizations choose to use Spectator during working hours and sometimes even around the clock; to get full and complete visibility.
The latest news report I saw, there were around 100K endpoints being managed with Spectator.  When you add up all of the SMS/SCCM endpoints, there are 10's of millions including many large sites with 100K, 200K, 300K, 1M.  I don't see a clientless scanning agent scaling well to that size…either it will tax the network or it will take so long to complete that it will be worthless. We have been on many enterprise-wide pilots and rollouts of systems management software and various companies (SMS/SCCM/Tivoli/Altiris/Opsware/etc.) and something most people overlook is the scalability of an application.  With only 100K or so endpoints TOTAL over ALL companies, I have to wonder if there's been enough scalability testing.  If there aren't any companies with 200K clients trying to run a clientless scan, how will we know it works well?

5.       Another important fact is that because SMS is a heavy tool, its operators run inspections (max) once a week, which from a security standpoint is definitely insufficient.
Again, something that needs citing.  People choose to inventory every 24-48 hours (depending on the type of inventory).  But there are companies who run inventory 4 times per day if they want to. The average seems to be about very 1-2 days, not "(max) once a week"  Any more than this creates more work for very little gain. (remember, even for a clientless solution that will supposedly "never impact your network bandwidth" there's at least 19-38GB of data, plus the time and processing horsepower needed for the server to process the scans of 200K machines, plus a database to store the data.  If you scan 4 times per day, you multiply that by 4.  How does that scale?)  There are also different kinds of data that come back in different intervals.  Desired Configuration Management data (drift management) comes in on-demand, meaning if a DCM scan happens on a client, the data immediately gets sent to the management points to be sent on up the hierarchy into the DB.  Hardware inventory has its schedule, software inventory has its schedule. So to make a blanket statement like operators run "inspections" weekly isn't right.

Prevention

1.       An ounce of prevention is worth a pound of cure and that is why Promisec built a number of prevention capabilities into its software. These include the ability to prevent usage of external storage devices or the usage of any type of modem. SMS does not have any of the prevention capabilities as it is designed as a system tool rather than a complete security solution.
That's fair.  SMS/SCCM isn't designed as a complete security solution.  Surely, however, highly targeted apps that just do USB/Firewire monitoring/restricting can be integrated into the infrastructure in a way that meet that need but don't overlap so heavily with the other things that SMS/SCCM does or can do or we should be looking into something like Microsoft Forefront (Stirling) which, being built by the same vendor as builds the configuration management suite and the desktop/server OS, is positioned to integrate better than anyone into our existing products.

There's also the Windows Feature Pack coming out which apparently will enable us to restrict access to portable devices such as USB flash drives via a certificate or password authentication based on IEEE 1667 standard specification. http://connect.microsoft.com/site/sitehome.aspx?SiteID=434

Database of Potential Threats

1.       Spectator Professional provides a thorough database (1500+) of P2P, Remote PC and Synchronization Applications that could endanger the organization. SMS does not have a database of potential threats such as P2P, Remote Control or Synchronization applications. Therefore, SMS increases the workload of the administrator requiring around the clock research of potential risky applications that could put organizations at risk.
That's completely false.  In fact SMS & SCCM have an Asset Intelligence database which has a database of over 400K categorized applications.  Including 6000+ apps that fall under P2P, Adult, Games, Security Threat, Security Risk, Virus and Spyware.  More are on the way, and SCCM also allows you to update the list from Microsoft on a regular basis as well as modify the list using the CM gui to add your own custom software and categories/families.  That makes the database of potential threats many times larger than Spectator's and does not in fact require "around the clock research of potential risky applications"

2.       Promisec has a research team dedicated to this research, updating Spectator Professional with any new potential risky application.
Microsoft bought a whole company, Asset Metrix, who is dedicated to this kind of work.  The output of that is in SMS/SCCM's Asset Intelligence database.

Identifying the Usage of Unauthorized Devices

1.      Identifying the usage of unauthorized devices within the organization is essential. Spectator Professional returns the full name and vendor of every device that can be attached to a workstation by a user. For instance, if a USB Mass storage device is attached to a workstation, for SMS only to identify it will require research and building a new query. Once identified SMS cannot remove this value as part of a system clean up so a security administrator would never know if the user attached the device again.
I'm not sure what "system clean up" means...but it would be pretty easy to tell SMS/SCCM to pull this same information.  If a clientless solution can get the information, SMS & SCCM can certainly get the information.  And an argument could be made that it could do it better and more often.  If the machine is off the network, but still on, our agent will still inventory and send back results when it connects again.  A clientless solution can't see a machine that's disconnected like that.

2.       With Spectator Professional you are able to "Remove indications of USB Mass storage Device", so you will know if the user attached the device again after being warned not to.
Sure, SMS/SCCM is not a USB monitoring/restricting application today, but it would appear that this may soon be a part of the OS from Microsoft: http://connect.microsoft.com/site/sitehome.aspx?SiteID=434

3.       Another important example is the existence of split tunneling inside your organization. Spectator Professional was designed to detect two network cards enabled together (e.g. Ethernet and Wi-Fi, or cellular), or even an authorized modem that was installed creating back door. With SMS it will be impossible to detect these hot security features.
There are already services that perform this split-tunneling check to make sure no network device is creating a bridge the network.  Some of them free, some of them homemade by the companies themselves.

Repeat Violations are Color Coded

  .      Reports can be made to use whatever colors the business needs.

1.       SMS supports only Microsoft platforms, while Spectator can also provide you with information regarding your UNIX boxes.
With the Quest Management eXtensions add-on, we have to ability to manage Unix, Linux, Mac, AIX, etc.  Also, Microsoft just began supporting Unix/Linux in SCOM (Operations Manager) and are actively working on adding Unix/Linux/Mac support for SCCM as well.

Privileges and Front-End

1.       Spectator can be used by security engineers, auditors and help desk personnel. Promisec understand that every division mentioned above requires different functionality, that is why Promisec provides the flexibility of having a different frontend for every division with the exact required needs. With SMS these divisions will not be able to operate the solution themselves but to trust the SMS operators to provide them with accurate and complete results.
There are many different security options and customizable console options available to us.  We make a point to give as much access to reports and consoles as is possible without sacrificing the infrastructure (you don't give a help desk person the ability to delete your primary sites and wipe the DB) so this is another false claim.

1.       In the event that the administrator needs to run a scan, Spectator will be able to complete the scanning in a minutes (rather than days like SMS) without affecting the network and the endpoints being inspected.
First of all, SMS doesn't take days.  If something needs to be done on all clients as soon as possible, it doesn't take days.  All clients who are turned on will receive a new policy within 15 minutes and take the appropriate action.  If there are machines that are offline, it will take care of them when they power back up and get on the network.  If it takes days to reach everyone, it's because machines were off or disconnected from the network  But a clientless solution can't reach a powered off machine any more than SMS can.  Secondly, I seriously doubt Spectator will be able to "complete the scanning in a minutes".  If you're scanning 200K machines remotely on 10,000 different subnets over various fast and slow network links, you are not going to complete them any faster than SMS does.  If you do, it will be at the cost of network reliability (SMS is polite, if another app wants to hog the bandwidth, SMS will let it, and thus SMS will take slightly longer to do its work)

2.       In the event that a problem is found, Spectator will remediate the problem manually or automatically, while with SMS you will need to build another rule and then run an inspection yet again.
I don't know what additional inspection would need to be done, but SMS/SCCM can manually or automatically remediate problems as well.  Is the implication that Spectator

3.       Spectator maintains a database of threats which are constantly being researched and updates and enables customers to run security checks on the latest threats. SMS cannot support it, which will require a company to research and update their own database of threats,
As stated earlier, this is not true and in fact the Asset Intelligence database is substantially larger than theirs.

4.       Security engineers and auditors do not have real visibility and control of their network because they are being “fed” by the systems engineers who provide them with the “required” information. SMS can only be operated by its administrators. The administrator produces reports for the auditors and security engineers, which can be very problematic. Spectator enables Security engineers and auditors to have complete control of their network by allowing them to use a dynamic, easy to use application that can be installed on a single machine within your network.
Yes, that's how the world works.  If you need access to something, you request it and it's granted to you.  If it's not what you were looking for, you make a more specific request and it gets remediated.  I can't see that as a reason SMS is bad.  Besides, if you're not allowed to have full control to all workstations through SMS, you certainly won't be allowed to buy a 3^rd party product and gain full access to them with that instead.  We have no desire to be a big bureaucracy that everyone has to bow down to in order to get data or access. So if it's visibility and control you want, it's happily granted when there's a business need. 

I guess the only benefit I see to Promisec Spectator is its claimed ability to monitor and/or restrict usb device access.  However, since it's clientless, I seriously doubt its ability to scale up to large-enterprise size and restrict access to devices at the level that is needed, especially the mobile client base.  As a clientless solution, it would need to be scheduled to run continuously against all machines which I can't believe is feasible given the 19-38GB of data that I estimate would be required each round for 200K machines.  If USB monitoring and restricting is what we need, we would probably get as much use out of modifying our SMS/CM inventory to look for USB controller devices and get the info ourselves.  Or I would assume there are some best-in-breed (client-based) USB monitoring apps  that do just that and do it well, or we could create a vbscript, powershell script or .NET app that subscribed to the Win32_USBController events and report on that.  There are so many more options than this.

Couple that with this uninformed or untruthful (I can't judge the motives) SMS vs. Promisec document and I've got serious reservations…

Number2 (John Nelson)
MyITForum - Forum Posts
MyITForum - Blog