Inside Windows Server 1002003 [Electronic resources] نسخه متنی

Troubleshooting Replication Problems

Active Directory replication involves a series of complex transactions. If one of these transactions fails, the problems it causes tend to be . . . well . . . complex. In general, replication problems are caused by unstable server hardware, poor network connections, and DNS errors.

The symptoms usually are a series of Error Log messages about failed replication. The logged message tells you the cause of the failure, but it doesn't necessarily tell you the cause of the problem. Several tools are available to help you get more information. They include the following:

• Special diagnostic traces that put more information in the Event log




• Command-line replication administration utility, REPADMIN




• Graphical Replication Monitor utility, Replmon

Also watch for problems with other services that depend on Active Directory. This includes services that have service accounts that must authenticate in the domain.

Directory Diagnostics Traces

One tool for tracing replication problems is hidden in the Registry. A variety of Diagnostics settings under HKLM | System | CurrentControlSet | Services | NTDS | Diagnostics dump information into the Event log. Three possible settings exist for each diagnostic trace (in addition to 0 for disabled):

1.

3.

5.

Full reporting gives the most information but can fill up the Directory Services log quickly in a production environment. This doesn't hurt anything, but you may miss an important piece of data.

The contents of an Event log can be exported to a CSV or TXT tab-delimited file and then imported into a database or spreadsheet. From the Event Log menu, select F

ILE | S

AVE A

S and then select the file type and location for the export.

An E

XPORT menu option also exists, but it has the same functions as S

AVE A

S . In addition, you can use a tool in the Resource Kit, called Dumpel, for the three standard logsapplication, security, and system. As of this writing, Dumpel does not work with the Directory Services log or the File Replication log.

Using the Command-Line Replication Administrator, REPADMIN

Microsoft supplies a command-line Resource Kit utility, called the Replication Administrator, or REPADMIN, for managing the inner workings of replication. A graphical tool, Replmon, shows much of the same information. For details, see the section "Using the Graphical Replication Monitor, Replmon" later in this chapter.

The online help (repadmin /?) shows the syntax for options and switches. What follows is a brief rundown of the nomenclature, in case the terms are unfamiliar:

• DSA is X.500 terminology for Directory Services Agent. An Active Directory domain controller is a DSA.




• When entering the name of a DSA, use the fully qualified DNS name. For example, enter dc-01.branch.company.com.




• GUID stands for Globally Unique Identifier. This is an octet string that is assigned to a domain controller. A domain controller actually has two GUIDs: an object GUID and an invocation GUID. The object GUID designates the DSA itself. The invocation GUID designates Active Directory replica hosted by that DSA.




• The naming context designates one of the Directory partitions hosted by the DSA. Only GC servers host copies of all domain naming contexts, and those are read-only.




• Object DN designates the LDAP distinguished name of the object you want to list.

In some respects, the functions in the Replication Administrator duplicate those in the AD Sites and Services console. For example, if you want to know whether a domain controller is configured as a Global Catalog server, you can open the console, navigate to the NTDS Settings object for that server, and check the properties. Or, you can open a command prompt and type repadmin /options.

Standard REPADMIN Functions

The AD Sites and Services console lacks many of the details available in REPADMIN. The following are a few of the questions that REPADMIN can answer:

• What is the status of knowledge consistency for this replication ring?

  
  repadmin /kcc
  Consistency check on local host successful.
  




• What was the result of the last replication event from each replication partner? (This listing shows the results for the Schema naming context at the DC-01 DSA.)

  
  repadmin /showreps
  Phoenix\DC-01
  DSA Options : IS_GC
  objectGuid  : 61d9fcd2-1172-11d3-b902-00c04f536a4d
  invocationID: 61d9fcd2-1172-11d3-b902-00c04f536a4d
  ==== INBOUND NEIGHBORS ======================================
  CN=Schema,CN=Configuration,DC=company,DC=com
  Phoenix\DC-02
  DEL:604ba650-124d-11d3-b903-00c04f536a4d via RPC
  objectGuid: 85e37932-124d-11d3-b903-00c04f536a4d
  Last attempt @ 2002-02-26 19:53.35 failed, result 1722:
  The RPC server is unavailable.
  Last success @ 2002-02-25 17:45.37.
  27 consecutive failure(s).
  Phoenix\DC-03 via RPC
  objectGuid: ce87aef1-1232-11d3-b903-00c04f536a4d
  Last attempt @ 2002-02-26 19:53.35 was successful.
  Atlanta\DC-04 via IP
  objectGuid: fba7a044-1176-11d3-b903-00c04f536a4d
  Last attempt @ 2002-02-26 19:53.35 was successful.
  ==== OUTBOUND NEIGHBORS FOR CHANGE NOTIFICATIONS ============
  CN=Schema,CN=Configuration,DC=company,DC=com
  Phoenix\DC-05 via RPC
  objectGuid: fba7a044-1176-11d3-b903-00c04f536a4d
  

  Without a connection, the system can report only the GUID of the failed replication partner, not its name. You can interrogate the domain controllers to find their GUIDs and then figure out which one failed. A glance at the Event log is helpful because it lists the GUIDs in the context of the server that caused the error. The showreps listing for a site with many domain controllers can be difficult to interpret. If you want to look at just the failures, use the /unreplicated switch.

Expert REPADMIN Functions

Windows Server 2003 includes many additional functions that can be performed by REPADMIN. To see the instructions for these functions, run repadmin /experthelp. Here is an example listing:

Expert Help
/add <Naming Context> <Dest DC> <Source DC> [/asyncrep] [/syncdisable]
[/dsadn:<Source DC DN>] [/transportdn:<Transport DN>] [/mail]
[/async] [/readonly]
/mod <Naming Context> <Dest DC> <Source GUID>
[/readonly] [/srcdsaaddr:<dns address>]
[/transportdn:<Transport DN>]
[+nbrflagoption] [-nbrflagoption]
/delete <Naming Context> <Dest DC> [<Source DC Address>] [/localonly]
[/nosource] [/async]
/removelingeringobjects <Dest DC> <Source DC GUID> <NC> [/ADVISORY_MODE]
/addrepsto <Naming Context> <DC> <Reps-To DC> <Reps-To DC GUID>
/updrepsto <Naming Context> <DC> <Reps-To DC> <Reps-To DC GUID>
/delrepsto <Naming Context> <DC> <Reps-To DC> <Reps-To DC GUID>
/options [DC] [{+|-}IS_GC] [{+|-}DISABLE_INBOUND_REPL]
[{+|-}DISABLE_OUTBOUND_REPL] [{+|-}DISABLE_NTDSCONN_XLATE]
/siteoptions [DC] [/site:<Site>] [{+|-}IS_AUTO_TOPOLOGY_DISABLED]
[{+|-}IS_TOPL_CLEANUP_DISABLED] [{+|-}IS_TOPL_MIN_HOPS_DISABLED]
[{+|-}IS_TOPL_DETECT_STALE_DISABLED]
[{+|-}IS_INTER_SITE_AUTO_TOPOLOGY_DISABLED]
[{+|-}IS_GROUP_CACHING_ENABLED] [{+|-}FORCE_KCC_WHISTLER_BEHAVIOR]
/testhook [DC] [{+|-}lockqueue] [{+|-}link_cleaner]
[{+rpctime:<call_name>,<ip or hostname>,<seconds_to_run>|-rpctime}]
[{+rpcsync:<call_name>,<ip or hostname>|-rpcsync}]
nbrflagoptions:
SYNC_ON_STARTUP DO_SCHEDULED_SYNCS TWO_WAY_SYNC
NEVER_SYNCED IGNORE_CHANGE_NOTIFICATIONS DISABLE_SCHEDULED_SYNC
COMPRESS_CHANGES NO_CHANGE_NOTIFICATIONS

There are two important options in these expert features. The first is the ability to remove so-called

lingering objects from a replica of Active Directory. A lingering object can appear following the restoration of the Active Directory database, especially if the tape is older than the garbage collection tombstone interval of 60 days.

The second option that warrants your attention is the ability to disable compression on inter-site replication. This is done with the -compress changes option. If a bridgehead server has multiple replication partners, as you might have in a hub-and-spoke arrangement, the processor(s) on the bridgehead may become swamped with compression requests for the replication packets. If you have sufficiently fast WAN connections, you can significantly reduce the CPU load on the bridgehead by disabling compression.

Using the Graphical Replication Monitor, Replmon

In addition to the command-line tool, REPADMIN, the Support Tools includes a graphical replication management tool called

Replication Monitor , or Replmon.

Replmon puts a lot of information on the screen in a highly useful format. To open the Replication Monitor and select a domain controller to monitor, follow Procedure 7.13.

Procedure 7.13 Configuring Replmon

1. Open the Replication Monitor from the Support Tools menu. Although it is not an MMC console, it has the same look-and-feel.

2. When the main window opens, right-click Monitored Servers and select A

   DD M

   ONITORED S

   ERVERS from the flyout menu. The Add Monitored Server Wizard starts.

3. Select the Search The Directory For The Server To Add option. After a brief pause while the server does a Directory lookup, the name of the server's domain is inserted into the drop-down box field.

4. Click Next. The Add Server To Monitor window opens. The top pane shows the list of available sites in the forest. Select a server by expanding the tree and double-clicking a server icon. You can also select the Enter The Name Of The Server option at the bottom of the window and enter the name of the server you want to monitor. You can enter the flat name.

   If you are going to monitor a server in another domain and you are not logged on with administrative rights in that domain, select the Use Alternate Credentials option, click Change, and enter suitable credentials in the target domain.

5. Click Finish. The server is added to the main Replication Monitor window. Expand the tree to show the naming contexts (see Figure 7.21 for an example). Highlight one of the servers in the tree to view the replication log for that connection.

   Figure 7.21. Replication Monitor (Replmon) main window showing naming contexts on server DC-01.

   

The following is a quick rundown of the information shown on the main Replication Monitor window, as shown in Figure 7.21:

• Naming Contexts.
  Each naming context hosted by the server is listed. If the server is a Global Catalog server, the list includes every domain in the forest. If the server is a standard domain controller, the list includes the domain naming context and the Schema and Configuration naming context from the root domain.




• Replication partners.
  The tree under each naming context lists the inbound replication partners for that naming context. The names are listed by site and then by flat name. In the example, DC-01 has four replication partners for the Schema and Configuration naming contexts.




• Server icons.
  The double-server icon with a link indicates an intra-site replication partner. A server icon that looks as though it is talking on a futuristic phone represents an intra-site connection. A miniature PC indicates the local server.




• Log entries.
  The right pane lists the replication history for the connection. New entries are added to the end.

[View full width]
Key:    HKCU | Software | VB and VBA Program Settings | Active 
Directory Replication Monitor | Settings
Values: View Menu Options

Replmon View Options

After you configure Replmon to monitor a domain controller, set viewing options by selecting V

IEW | O

PTIONS from the menu. The Active Directory Replication Monitor Options window opens with the focus set to the General tab. See Figure 7.22 for a sample of this window.

Most of the options in this window are self-explanatory. Some that might be a little obscure include the following:

• Show Retired Replication Partners.
  These are server objects that were tombstoned but not yet deleted by the ESENT database engine. They are usually deleted over time. The NTDSUTIL utility has an option for cleaning up metadata that can delete these old entries.




• Show Transitive Replication Partners and Extended Data.
  This option enables Replmon to show USN and metadata information from servers outside the local site that are multiplexed on the same Site Link.




• Notify When Replication Fails After This Number Of Attempts.
  This option, coupled with the Notification Options entry in the next field, can be used to send email if a connection fails to replicate. Set the attempt number at 3-5 to account for a couple of missed attempts that might happen in the ordinary course of operations.




• Log Files.
  This changes the default path for the log files. The default location is the Resource Kit directory.




• Enable Debug Logging.
  This option is for debugging Replmon, not for debugging replication. Debug Logging writes a great deal of information about the Replmon application to the Application log. This fills the Event log very quickly, so only use this option during troubleshooting.

Replmon Connection Properties

You can view a great deal of information about a particular replication connection by opening the Properties window for the connection. Figure 7.23 shows an example. The General tab shows the connection type and information about the connection itself. The important statistics are the last three lines, which show whether replication attempts failed and the associated error message.

The Update Sequence Numbers tab shows the current USN received from each replication partner. This option requires you to select the Show Transitive Replication Partners and Extended Data option in the V

IEW options menu.

The Flags tab lists the configuration settings for the replication connection. The flags shown in the example are standard for an inter-site connection.

Replmon Replica Synchronization Options

Right-click a Naming Context icon and select S

YNCHRONIZE T

HIS D

IRECTORY P

ARTITION W

ITH A

LL S

ERVERS . This opens a window of the same name. Figure 7.24 shows an example. This list of options enables you to override the default replication behavior in a variety of ways:

• Disable Transitive Replication.
  By default, all inter-site replication uses the same default Site Link, and the KCC is free to build connections between domain controllers regardless of their site affiliation. If you are troubleshooting problems with replication loops or failed replication to a particular server, you can disable transitive replication when initiating a replication event to see if it succeeds.




• Push Mode.
  By default, the DRA "pulls" updates from a replication partner. This selection enables Push mode for a single replication transaction.




• Cross Site Boundaries.
  This option enables you to directly initiate an inter-site replication, but it is effective only for RPC connections. The default inter-site connection transport is IP. You can use the Properties window for a connection to change the transport to RPC and then select this option. If you change the transport for a connection, the connection status changes to a static connection that requires manual control.




• Skip Initial Topology Check.
  This speeds up replication across a slow network with many domain controllers. It takes the chance that a server or link is down.




• Generate Fatal Error On Unreachable Server.
  Not enabled.




• Disable All Synchronization.
  Not enabled.




• Return Server DN.
  Not enabled.

Replmon Server Property Menu Selections

When you right-click the server icon in Replication Monitor, a flyout (P

ROPERTIES ) menu appears. Several of the options in this menu can give you highly useful information about replication in particular and the domain controller status in general:

• Generate Status Report.
  Select this option to get a comprehensive report on the domain controller's Active Directory configuration. The list of items in the report is selected from a Report Options window that opens prior to running the report. Figure 7.25 shows an example.

  Figure 7.25. Report Options window for Replmon server status report.

  

  
  



• Show Group Policy Status.
  Lists all the Group Policy objects for the domain and whether the object was synced. Use this information if users on some domain controllers are getting policies and other users aren't.




• Show Trust Relationships.
  This option shows the same information as the AD Domains and Trusts window, but much more conveniently.




• Display Metadata Properties.
  When you select this option, you are prompted to enter a set of alternate credentials, if necessary, and then the distinguished name of an object whose replication data you want to view. This is equivalent to repadmin/showmeta. Metadata information is invaluable when trying to isolate a problem with a corrupt property or corrupt user object. By comparing the metadata on various replicas, you can discover whether you have a corruption problem and how extensive the problem has become.

Server Properties

Right-click the server icon in the Replmon window and open the Properties window. The tabs in this window give you an update of the server's replication status:

• Server Flags.
  Lists special domain controller options including GC status, KDC status, and W32Time status.




• FSMO Roles.
  The window lists all the FSMO role masters by name and site with a Query button for each to verify that the server is still online. Figure 7.26 shows an example.

  Figure 7.26. Replmon Server Properties windowFSMO Roles window.

  

  
  



• Inbound Replication Connections.
  This window answers the who, why, and how for each inbound replication connection. Figure 7.27 shows an example.

  Figure 7.27. Replmon Server Properties windowInbound Replication Connections.