In our environment, we recently added two new child sites to our System Center Configuration Manager 2007 hierarchy. The parent site had been running for several months prior to the addition of the child sites.
While that parent site was up and running, we used it to perform typical software update tasks such as the packaging and deployment of updates. We would create a new update list and deployment for each monthly issuance of updates from Microsoft. Rather than make monthly deployment packages, the patches were downloaded to a single annual package (thus 2007-10, 2007-11, and 2007-12 were sourced out of the 2007 deployment package).
All was working just fine until I added the child sites.
The child sites are automatically configured to have WSUS synchronize from the parent site’s WSUS server. This part worked out just fine. The child WSUS server obtained the list of updates successfully. The break in the process, however, came with the synchronization of the SCCM database.
With the parent site, the SCCM database gets its data from the local WSUS server. The child sites, however, get their database data from the parent SCCM database rather than the WSUS server. This means that the child sites separate the WSUS synchronization from the database synchronization.
The SCCM database table that holds the lionshare of the data for the software updates process is the CI_ConfigurationItems table. This table holds records for each update known to the system as well as each of the update lists, deployments, and deployment packages definitions. Part of these definitions indicate when a particular update is superseded by another and when an update is expired.
There does seem to be a mechanism to tombstone entries so that they are properly replicated, but the tombstone lifetime does not appear to be sufficient. It is unclear which Delete Aged XX Data site maintenance task is responsible for getting rid of the data, but in my case, the expired data had been removed from the system.
Now because I had been using the parent site for longer than the tombstone lifetime, I still had packages, deployments, and update lists that referred to data that was no longer present (or at least would not replicate to the child site). Because of this, the child site would not correctly replicate the package or deployment. Even though all other updates were available and valid, the child site would not allow the replication to succeed. This was a huge problem.
At first, I tried forcing the CI_ConfigurationItems table to re-replicate (I updated the LastModifiedBy text field on all rows). The replication process was kicked off successfully (it took 4-5 hours to complete), but I was still missing the data. On the child site, the objreplmgr.log file would contain records similar to Referenced configuration items are not available yet and list the unique identifiers for the item that was needed. In addition, the inboxes\objmgr.box\INCOMING\Retry folder would have a large number of .CID files in it. Obviously there was some problem here.
Turns out, the problem was the expired updates. On the parent site, I went through all of the deployment packages, deployments, and update lists and removed the expired updates from them. Once that was completed, the replication was successful and the child sites now had fully registered versions of the lists/deployments/packages.
So, the lesson here is to find those expired updates and remove them from the referencing lists before you add any child sites. Theoretically, the tombstoning should catch it properly when a child site exists, but that still bears some discovery to see if that part works (I have my doubts as I have the system downloading definition updates every few hours, which would expire previous updates and the IsTombstoned flag is never set in the database table).
Long post, but hopefully this will help someone else out there.