SCCM 1806 Upgrade fails

While working at a client site I was assisting onsite guys to upgrade SCCM 1802 to 1806.

They did the 1710 to 1802 a few weeks back without any issues.

The prereqs passed and they started the upgrade. It failed and they decided to remove AV as they thought it might be a contributing factor. 

Tried upgrade again still fails on Install Update Files step in the setup process. I started to review the CMUpdate.log to see what errors was being written. 

I could not copy the content of the file but below is an example of what was in the log:

Cannot create D:\Program Files\Microsoft Configuration Manager\tools\UploadOfflineFeedback, [error code: 5, error message: Access is denied.]. CONFIGURATION_MANAGER_UPDATE 9/26/2018 13:35:43 1796 (0x0704)

ERROR: Failed to create directory D:\Program Files\Microsoft Configuration Manager\tools\UploadOfflineFeedback CONFIGURATION_MANAGER_UPDATE 9/26/2018 13:35:43 1796 (0x0704)

Setup has encountered fatal errors while performing file operations. CONFIGURATION_MANAGER_UPDATE 9/26/2018 13:35:43 1796 (0x0704)

Failed to install update files. CONFIGURATION_MANAGER_UPDATE 9/26/2018 13:35:43 1796 (0x0704)

So compared their setup to my LAB and everything seemed in order. Did some research and came across a Technet forum post about permissions missing on the folder.

Asked the team to confirm if Builtin\Administrators and SYSTEM had full permission on the Tools folder. They confirmed that both entries were missing. Not sure who changed permissions on the Tools folder but after adding the accounts back and retrying the update, 1806 installed without any issues.

SCCM Update Sync issue

UPDATE 15-06-2018: Microsoft has confirmed issue is resolved. We tested and our catalog has now synced successfully. If you still have issues post on the following link.

So yesterday morning we were greeted with SUP Sync failures at one of our clients.
We could not understand as the sync the previous afternoon was working fine but with the release of patch Tuesday's patches it started failing.

We tried WSUSUtil Reset to see if that might fix the issue as it looked like an update missing metadata in the SUSDB. After numerous attempts it still kept on failing. We then un-ticked all the Classifications and the sync was successful. We then added everything back and it failed again.

This was the error we saw in WSUS console:

Then we looked at the SCCM log (Wsyncmgr.log) and there we saw the following error:

Sync failed: ImportUpdateError: Source: Microsoft.SystemsManagementServer.SoftwareUpdatesManagement.WsusSyncAction.WSyncAction.SyncWSUS

We then turned to the Softwaredistribution.log on the WSUS server and saw the following error:

System.Data.SqlClient.SqlException (0x80131904): Cannot insert the value NULL into column 'RevisionID', table '@AtLeastOneBundle'; column does not allow nulls. INSERT fails.

We tried to find those UpdateID's at other clients but was unable to.

We then opened a support call with Microsoft as we did not want to redo our SUP server as that felt like an extreme measure. Within 5 minutes of the call the engineer confirmed there is an issue with multiple clients at the moment regarding sync issues and they are awaiting a fix from the product group. 

As a temporary workaround we were instructed to deselect the Definition Updates classification.

We attempted the sync and it worked. We do use SCEP for AV so in the meantime we are deploying the definitions via normal software distribution method in SCCM. You can reference this blog as it is still relevant: Deploying Endpoint Protection Updates Offline Using SCCM 2012 R2

SCCM Software Update Scan Error 0x80240fff on Windows 10 (1511) Clients and more

**Another Update - Even more resource regarding WSUS and cleanups that should be used.
Fixing WSUS - When the Best Defense is a Good Offense

*Update - With all the current WSUS issues I suggest folks also have a look at this new article from Microsoft, we are currently investigating its affects as well.
High CPU/High Memory in WSUS following Update Tuesdays

With the WannaCry crisis currently happening around the world I think everyone is in patching mode.

While working on a client I started picking up weird errors when looking at Scan Reports as well as Deployment Error reports.

We had lots of machines failing mostly 1511. Now that most machines are sorted out we saw 1607 Windows 10 also giving errors.

The first set of errors where related to 1511 which was that 0x80240fff which is discussed in the System Center Dudes post:
SCCM Software Update Scan Error 0x80240fff on Windows 10 Clients
I found another article on Technet related to the error which pointed us in the direction of KB4015219:
April 11, 2017—KB4015219 (OS Build 10586.873)

In the improvement and fixes portion there was a particular line that they mentioned in the Technet post:

• Addressed issue that might sometimes lead to updates not getting installed on machines due to file corruption.

With this new knowledge we went and downloaded the patches and ran them manually on a few machines. After the reboot the 0x80240fff scan error went away and it installed the May release of patches. We did not decline any updates in WSUS yet as mentioned in the articles above.

So another workaround was to package the updates and use an application to deploy it which also worked. (Reminds me of the Windows 7 update agent issues 2 years ago.)

Our Compliance started to go up nicely but plateaued 2 days later. After more investigation this morning I started seeing the following in the UpdatesDeployment.log file on a bunch of machines:

Endless Google searches did not give anything that would help.I found an article about Application Deployments failing with same error code and one of the comments was to untick the "Delay enforcement of this deployment according to the user preferences, up to the grace period defined in the client settings" as our Client Setting is set to 0 on the Grace period.

So I changed the deployment setting of the update groups, did a policy refresh and deployments evaluation cycle on the machine and the updates installed without any issues.

Windows Updates failed 0x80244022

**Another Update - Even more resource regarding WSUS and cleanups that should be used.
Fixing WSUS - When the Best Defense is a Good Offense

*Update - With all the current WSUS issues I suggest folks also have a look at this new article from Microsoft, we are currently investigating its affects as well.
High CPU/High Memory in WSUS following Update Tuesdays


So this morning the Desktop guys started complaining about updates on some machines not working.

I started looking at the machine in question and SCCM seemed fine on the PC.
I checked the WUAhander.log and WindowsUpdate.log and came across the following errors:

WUAhander.log

WindowsUpdates.log

I tried all the Windows update agent tricks in the book as I thought the issue was on the machine.
Then I checked my own workstation and it was doing the same so now start to worry that the WSUS box is on the fritz.

I log in and I am greated with the reset node mmc error when trying to open the WSUS console.
So I restarted the server in the hope that it might do the trick. Upon rebooting it looked fine and then the console stopped working again.

So next I try to figure out whats happening in the event viewer:

Does not mean a lot me so I start to research some of the WSUS error codes. Then one stands out HTTP error 503 and I remember something I read on Eswar's blog
Configmgr OnSearchComplete – Failed to end search job Error 0x80244022 WUAHandler.log

So I checked my Wsuspool in IIS and it was stopped. Starting it only works for a while then it stops again.

So I set out to find out more about HTTP error 503 then I came across the technet blog article:
ConfigMgr 2012 Support Tip: WSUS sync fails with HTTP 503 errors

So I did the changes on my Wsuspool in IIS as follows:
Private Memory Limit: 4194304 (4GB)

When I tried to start the apppool again it gave a error so I did an IISreset and after that everything was working as it should again.

I know this looks like a repeat of the other articles but had to add my experience onto it as well.

SCCM 2012 SP1 CU5 Upgrade

So this past weekend I had to perform an upgrade from CU2 to CU5.

The last upgrade I did was way back with SCCM 2007 R2 to R3, and thinking back it was a tedious process starting from your primary then going down the hierarchy.
After some research I found out that there is no more need to install it on every site system manually on CM2012 :)

So I proceeded with my Primary Site first. Installation took about 20min to complete and all were in working order or so it seems.

First thing I saw was packages being distributed after the update.

Only one of my Secondary sites did not get the Configuration Manager Client Upgrade Package (which is hidden from the console btw). The Configuration Manager Client Package I had to prestage the content to the other sites and extract it locally which worked great.











So I remembered the issue I had with a Secondary Site that failed and had to be rebuilt. I used this nifty trick again:
http://scexblog.blogspot.com/2014/04/configuration-manager-client-upgrade.html

Only downside was I had to prestage the Configuration Manager Client Package again as the version updated when using the Client.acu file.

So when that was finally done. I started with the upgrade of the Secondary sites. Which was as easy as deploying a software package to a machine. I also saw an update for the console. So I created 2 Collections. One based on Secondary Sites and one with all machines which had the Console installed but not on version .1600 (CU5).
As a test I made the Server Update available to one server and just ran it from Software Center. As I was looking at the log file it detected that the server had a Console installed an proceeded to update that as well so no need to deploy the Console only update afterwards it seems.
I saw that in the log files that gets created in %windir%\temp folder cm12-sp1cu5-kb2978017-x64-enu.log as well as configmgr2012adminui-sp1-kb2978017-i386.msp.log.

Here are the 2 collection queries I used:

Workstations with a Console installed (I know you can use the ARP displayname as well but more than one way to skin a cat right :))
(You can just change the %Workstation% to %Server% to get all servers with the Console installed)
Consoles:
select SMS_R_SYSTEM.ResourceID,SMS_R_SYSTEM.ResourceType,SMS_R_SYSTEM.Name,SMS_R_SYSTEM.SMSUniqueIdentifier,SMS_R_SYSTEM.ResourceDomainORWorkgroup,SMS_R_SYSTEM.Client from SMS_R_System inner join SMS_G_System_SoftwareFile on SMS_G_System_SoftwareFile.ResourceID = SMS_R_System.ResourceId where SMS_G_System_SoftwareFile.FileName = "Microsoft.ConfigurationManagement.exe" and SMS_G_System_SoftwareFile.FileVersion < "5.0.7804.1600" and SMS_R_System.OperatingSystemNameandVersion like "%Workstation%"

All Secondary Sites:
select SMS_R_SYSTEM.ResourceID,SMS_R_SYSTEM.ResourceType,SMS_R_SYSTEM.Name,SMS_R_SYSTEM.SMSUniqueIdentifier,SMS_R_SYSTEM.ResourceDomainORWorkgroup,SMS_R_SYSTEM.Client from SMS_R_System  inner join SMS_G_System_ADD_REMOVE_PROGRAMS_64 on SMS_G_System_ADD_REMOVE_PROGRAMS_64.ResourceId = SMS_R_System.ResourceId  where SMS_G_System_ADD_REMOVE_PROGRAMS_64.DisplayName = "Microsoft System Center 2012 Configuration Manager Secondary Site Setup"


As an added check I also installed Site Servicing Console Extension to verify that all the sites were on the correct version.

http://blogs.technet.com/b/configmgrteam/archive/2014/12/09/now-available-microsoft-system-center-2012-configuration-manager-servicing-extension.aspx

https://www.microsoft.com/en-us/download/details.aspx?id=45033