Windows server 2016 Operating System is not supported for SCOM 2012 SP1. So I would recommend to upgrade the OS to Windows server 2012 R2.

 

https://docs.microsoft.com/en-us/previous-versions/system-center/system-center-2012-R2/jj656654(v=sc.12)

 

Before proceeding with any of the below action plan, please make sure you have full backup (working) of both the SCOM databases (OperationsManager and OperationsManagerDW).

 

Action Plan 1:

 

For MS:

 

  1. Please change the primary management server to another server for the agents which are reporting to the management server.

 

You can of course configure all agents to report to the other MS, so that you can avoid the small downtime they have until the fall back by default to any other MS. Of course, this is the case only if you haven’t done any failover config with PowerShell. Otherwise you must use PowerShell again to assign them to their new MS:

 

Set Primary and Failover Management Servers for Agent Managed Computers with Powershell

 

So, just move you agents to the other Management Server (if they are “push” installed, via the console). If not (if they are installed manually, GPO or SCCM), then you can either make them remotely managed and then change their MS or you can just let them do the failover after they find out their Primary MS is down (this is associated with a small agent downtime).

  1. If MS plays as watcher node, redirect watcher node role into another machine
  2. If you have configure email notification, remove present MS from notification resource pool
  3. Do a in place upgrade for the OS Windows server 2008 R2 to OS Windows server 2012 R2 data center (Please refer Windows Upgrade)

 

for gateway:

 

  1. Please change the primary gateway server to another server for the agents which are reporting to the gateway server.

 

You can of course configure all agents to report to the other MS, so that you can avoid the small downtime they have until the fall back by default to any other MS. Of course, this is the case only if you haven’t done any failover config with PowerShell. Otherwise you must use PowerShell again to assign them to their new MS:

 

Set Primary and Failover Management Servers for Agent Managed Computers with Powershell

 

So, just move you agents to the other Management Server (if they are “push” installed, via the console). If not (if they are installed manually, GPO or SCCM), then you can either make them remotely managed and then change their MS or you can just let them do the failover after they find out their Primary MS is down (this is associated with a small agent downtime).

  1. Now do a in place upgrade for the OS Windows server 2008 R2 to OS Windows server 2012 R2 data center (Please refer Windows Upgrade)

 

Action Plan 2:

 

For MS:

 

  1. Build a new server with the same name which you have for the existing management server with the OS Windows server 2012 R2 data center.
  2. Now unjoin the old server from domain and reboot the machine and join the new server to the domain and reboot the server.
  3. Now install the pre-requisites as mentioned in the below.

 

  1. Add the .NET 3.5.1 feature to windows. Use the Server Manager UI, or use PowerShell:

Open PowerShell (as an administrator) and run the following:

Import-Module ServerManager

<then>

Add-WindowsFeature NET-Framework-Core

 

  1. Install .NET 4.0 to all servers

  2. Install the Report Viewer controls to all Management Servers. Install them from http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=6442

  3. Install all available Windows Updates.
  4. Add the “SCOMAdmin group” domain global group to the Local Administrators group on each server.
  5. Install IIS on any management server that will also host a web console:

 

Open PowerShell (as an administrator) and run the following:

Import-Module ServerManager

Add-WindowsFeature NET-Framework-Core,Web-Static-Content,Web-Default-Doc,Web-Dir-Browsing,Web-Http-Errors,Web-Http-Logging,Web-Request-Monitor,Web-Filtering,Web-Stat-Compression,Web-Mgmt-Console,Web-Metabase,Web-Asp-Net,Web-Windows-Auth -Restart

 

  1. Reboot the server.
  2. Now copy the installation media and run the below command.

 

Setup.exe /silent /AcceptEndUserLicenseAgreement

 /recover

  /EnableErrorReporting:[Never|Queued|Always]

  /SendCEIPReports:[0|1]

  /UseMicrosoftUpdate:[0|1]

  /DatabaseName:<OperationalDatabaseName>

  /SqlServerInstance:<server\instance>

  /DWDatabaseName:<DWDatabaseName>

  /DWSqlServerInstance:<server\instance>

  /UseLocalSystemDASAccount

  /DatareaderUser:<domain\username>

  /DatareaderPassword:<password>

  /DataWriterUser:<domain\username>

  /DataWriterPassword:<password>

  /ActionAccountUser:<domain\username>

  /ActionAccountPassword:<password>

 

 

You must use the same parameter values for account credentials, management group, and database names as the old server you are trying to recover.

 

https://docs.microsoft.com/en-us/previous-versions/system-center/system-center-2012-R2/hh531578(v=sc.12)

 

https://kevinholman.com/2018/10/29/recovering-a-scom-management-server/

 

 

for gateway:

 

  1. Please change the primary gateway server to another server for the agents which are reporting to the gateway server.

 

You can of course configure all agents to report to the other GW, so that you can avoid the small downtime they have until the fall back by default to any other MS. Of course, this is the case only if you haven’t done any failover config with PowerShell. Otherwise you must use PowerShell again to assign them to their new MS:

Set Primary and Failover Management Servers for Agent Managed Computers with Powershell

 

So, just move you agents to the other Management Server (if they are “push” installed, via the console). If not (if they are installed manually, GPO or SCCM), then you can either make them remotely managed and then change their MS or you can just let them do the failover after they find out their Primary MS is down (this is associated with a small agent downtime).

  1. Now upgrade the OS (Please refer Windows Upgrade)

 

 

Action Plan 3:

 

For MS:

 

  1. Build a new server with the OS Windows server 2012 R2 data center.
  2. Now install the pre-requisites as mentioned in the below.

 

  1. Add the .NET 3.5.1 feature to windows. Use the Server Manager UI, or use PowerShell:

 

Open PowerShell (as an administrator) and run the following:

Import-Module ServerManager

Add-WindowsFeature NET-Framework-Core

 

  1. Install .NET 4.0 to all servers

  2. Install the Report Viewer controls to all Management Servers. Install them from http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=6442

  3. Install all available Windows Updates.
  4. Add the “SCOMAdmin group” domain global group to the Local Administrators group on each server.
  5. Install IIS on any management server that will also host a web console:

Open PowerShell (as an administrator) and run the following:

Import-Module ServerManager

Add-WindowsFeature NET-Framework-Core,Web-Static-Content,Web-Default-Doc,Web-Dir-Browsing,Web-Http-Errors,Web-Http-Logging,Web-Request-Monitor,Web-Filtering,Web-Stat-Compression,Web-Mgmt-Console,Web-Metabase,Web-Asp-Net,Web-Windows-Auth -Restart

 

  1. Reboot the server.
  2. Now copy the installation media and run the set up file.

 

https://kevinholman.com/2013/10/18/scom-2012-r2-quickstart-deployment-guide/

 

 

For Gateway:

 

  1. Please change the primary management server to another server for the agents which are reporting to the management server.

 

You can of course configure all agents to report to the other MS, so that you can avoid the small downtime they have until the fall back by default to any other MS. Of course, this is the case only if you haven’t done any failover config with PowerShell. Otherwise you must use PowerShell again to assign them to their new MS:

 

Set Primary and Failover Management Servers for Agent Managed Computers with Powershell

 

So, just move you agents to the other Management Server (if they are “push” installed, via the console). If not (if they are installed manually, GPO or SCCM), then you can either make them remotely managed and then change their MS or you can just let them do the failover after they find out their Primary MS is down (this is associated with a small agent downtime).

  1. Now uninstall the software from control panel and delete it from SCOM console under “management servers”
  2. Now upgrade the OS  (Please refer Windows Upgrade)
  3. Now wait for 48 hrs. so that data for GW will be groomed from database.
  4. Now install the role again by following the below article.

 

https://blogs.technet.microsoft.com/pfesweplat/2012/10/15/step-by-step-walkthrough-installing-an-operations-manager-2012-gateway/

 

https://gallery.technet.microsoft.com/SCOM-2012-R2-Gateway-8e7df1b0

 

 

Action Plan 4:

 

For MS:

 

  1. Please change the primary management server to another server for the agents which are reporting to the management server.

 

You can of course configure all agents to report to the other MS, so that you can avoid the small downtime they have until the fall back by default to any other MS. Of course, this is the case only if you haven’t done any failover config with PowerShell. Otherwise you must use PowerShell again to assign them to their new MS:

 

Set Primary and Failover Management Servers for Agent Managed Computers with Powershell

 

So, just move you agents to the other Management Server (if they are “push” installed, via the console). If not (if they are installed manually, GPO or SCCM), then you can either make them remotely managed and then change their MS or you can just let them do the failover after they find out their Primary MS is down (this is associated with a small agent downtime).

  1. If MS plays as watcher node, redirect watcher node role into another machine
  2. If you have configure email notification, remove present MS from notification resource pool

 

  1. Uninstall the SCOM application from control panel.
  2. Now upgrade the OS as mentioned in the below Windows section (Windows Upgrade).
  3. Now install the pre-requisites as mentioned in the below.

 

  1. Add the .NET 3.5.1 feature to windows. Use the Server Manager UI, or use PowerShell:

 

 

Open PowerShell (as an administrator) and run the following:

 

Import-Module ServerManager

Add-WindowsFeature NET-Framework-Core

 

  1. Install .NET 4.0 to all servers

  2. Install the Report Viewer controls to all Management Servers. Install them from http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=6442

  3. Install all available Windows Updates.
  4. Add the “SCOMAdmin group” domain global group to the Local Administrators group on each server.
  5. Install IIS on any management server that will also host a web console:

Open PowerShell (as an administrator) and run the following:

Import-Module ServerManager

Add-WindowsFeature NET-Framework-Core,Web-Static-Content,Web-Default-Doc,Web-Dir-Browsing,Web-Http-Errors,Web-Http-Logging,Web-Request-Monitor,Web-Filtering,Web-Stat-Compression,Web-Mgmt-Console,Web-Metabase,Web-Asp-Net,Web-Windows-Auth -Restart

 

  1. Reboot the server.
  2. Now copy the installation media and run the below command.

 

Setup.exe /silent /AcceptEndUserLicenseAgreement

 /recover

  /EnableErrorReporting:[Never|Queued|Always]

  /SendCEIPReports:[0|1]

  /UseMicrosoftUpdate:[0|1]

  /DatabaseName:<OperationalDatabaseName>

  /SqlServerInstance:<server\instance>

  /DWDatabaseName:<DWDatabaseName>

  /DWSqlServerInstance:<server\instance>

  /UseLocalSystemDASAccount

  /DatareaderUser:<domain\username>

  /DatareaderPassword:<password>

  /DataWriterUser:<domain\username>

  /DataWriterPassword:<password>

  /ActionAccountUser:<domain\username>

  /ActionAccountPassword:<password>

 

 

You must use the same parameter values for account credentials, management group, and database names as the old server you are trying to recover.

 

https://docs.microsoft.com/en-us/previous-versions/system-center/system-center-2012-R2/hh531578(v=sc.12)

 

https://kevinholman.com/2018/10/29/recovering-a-scom-management-server/

 

for gateway:

 

  1. Please change the primary gateway server to another server for the agents which are reporting to the gateway server.

 

You can of course configure all agents to report to the other GW, so that you can avoid the small downtime they have until the fall back by default to any other MS. Of course, this is the case only if you haven’t done any failover config with PowerShell. Otherwise you must use PowerShell again to assign them to their new MS:

 

Set Primary and Failover Management Servers for Agent Managed Computers with Powershell

 

So, just move you agents to the other Management Server (if they are “push” installed, via the console). If not (if they are installed manually, GPO or SCCM), then you can either make them remotely managed and then change their MS or you can just let them do the failover after they find out their Primary MS is down (this is associated with a small agent downtime).

  1. Now upgrade the OS (Please refer Windows Upgrade)

 

 

Windows Upgrade:

 

From Windows OS point of view, the following are some of the important points to be considered:

 

  1. To perform in-place upgrade of Windows 2008 R2 SP1 please follow the article – How to Perform an In-Place Upgrade on Windows Vista, Windows 7, Windows Server 2008 & Windows Server 2008 R2 – https://support.microsoft.com/en-us/help/2255099/how-to-perform-an-in-place-upgrade-on-windows-vista-windows-7-windows.

 

  1. Some important points to keep in mind before in-place upgrade:
  • Ensure we have good amount of disk space on C:\ drive like minimum 20-30GB free space for each time of the in-place.
  • Ensure we make a note of the IP address (just to be on safer side) of the server. It’s good to take a copy of IPCONFIG /ALL output in text note.
  • Get the ISO data copied to local hard drive and then run in-place upgrade.
  • Ensure if any application backup is not present or there’s no application setup file, then it’s good to have a backup of the application.
  • To be on safer side it’s good to have System State Backup & C:\ Drive backup, so your windows & application copy will be present to restore back in any worst case scenario. If the application binaries are residing on D:\ drive then there is no need for application backup.

 

  1. Normally when we do in-place upgrade, the process will touch only C:\Windows\Servicing & C:\Windows\Winsxs folder apart from that we will not touch other folders like C:\Program Files where your application reside. So you don’t have to worry about the application side, until unless we are using age old ASP.net version that might cause some issues i.e. if your application is built on old ASP.net then it’s binaries old binaries in that case the application might not perform correctly.

 

  1. And Application backup should be MUST and also your application should be compatible with new OS version as well. Since your OS is Windows 2008 R2, it’s important that your application to be compatible with those new OS versions.

 

  1. Another important point after in-place upgrade, you have to run Windows Update once to check if there are any binaries to be updated i.e. important updates only & then get them installed. If any binaries are replaced to RTM, then it might require the Windows Update to check & install updates.