Troubleshooting Active Directory and FRS ReplicationA GPO will not be downloaded to the client if the client cannot authenticate to the domain. When this occurs, the cached policy is applied. Any changes made since the policy was cached will not be applied. A number of Active Directory and related GPO issues, such as problems with trust relationships or Active Directory Replication, Domain Controller/Active Directory Configuration, and even the File Replication System (FRS) can either prevent GPOs from downloading or hinder proper processing for appropriate computer and user accounts. Trust Relationship ProblemsAlthough it is not advised, GPOs can be linked across domains if a trust relationship exists between the domains. By default, all domains in the forest have a trust relationship, and additional external and forest trusts can be configured. If a GPO is linked across a trust, and the trust fails, the GPO cannot be processed. A list of denied GPOs can be found in the GPMC Resultant Set of Policy report, and if a GPO cannot be processed due to a failed trust, it will appear there. You can also use NTtest.exe or view Active Directory Domains and Trusts to verify the trust relationship and to repair it if this is necessary. However, best practices recommend that a copy of the GPO be placed in the domain and used for the link, rather than linking GPOs across domains. TIP: Best Practices for GPO Links A GPO should never be linked across domains. Instead, place a copy of the GPO in the domain and link to the copy. Active Directory Replication ProblemsIn addition to name resolution, DNS stores Active Directory-related records that must be accessible and that must resolve to domain controller computers for authentication and replication to occur. AD replication is necessary for Group Policy information to reach all domain controllers. When there are problems with replication, many users and computers will not have the most current versions of the GPOs they must process. Problems with DNS can impact replication. Before replication can occur, a domain controller must find its replication partners. Replication partners are those domain controllers assigned to replicate with each other. Assignment can be automatic or can be assigned manually. Each domain controller knows the names of its replication partners and uses DNS to find the DC IP address. However, this is not simple DNS name resolution. Instead the following steps take place:
If anything is missing, such as the GUID CNAME, its alias, or the correct IP address for the server, replication cannot occur. Use DNSLint to Test ReplicationDNSLint simulates this process and reports back on any issues and successes. The DNSLint /ad switch uses LDAP to connect to the root forest of AD and gets the GUIDs for all DCs in the forest. Then it queries DNS to resolve the CNAME and obtain the alias and then the IP addresses of the DCs. This information reveals if the proper SRV records are registered on the correct DNS server, and therefore if all replication partners are listed and accessible in DNS. Because DNSLint must authenticate to AD to find the GUID information, it may also find netlogon problems and report them. DNSLint also looks for zone delegation and queries additional DNS servers. Use the command dnslint /ad IP_address_of_the_forest_root /s IP address of the DNS server. Local host can be used with the /s switch if the command is issued at the DNS server. The following command was used at the DNS server authoritative for the _msdcs.chicago.com zone (the forest root _msdcs zone) and produced the report in Figure 9-21. Figure 9-22 shows the CNAME and glue records for DCs with GUIDs in the _msdc.root_of_forest_zone. dnslint /ad /s localhost Figure 9-21. Finding GUIDs with DNSLint.Figure 9-22. DNSLint report shows CNAME and glue record for those DCs with GUIDs in the _msdc.root_of_forest zone.Troubleshoot DNS" section. Because DNSLint contacts the DC at the IP address after /ad and uses an LDAP query to get all the GUIDs in the Active Directory forest, the user running DNSLint must have proper permissions to access the information. If he does not, the command will issue an error and terminate. If the command is successful, DNSLint contacts the DNS server identified by the /s switch to obtain the CNAMEs for each GUID, and then the glue or A record for each CNAME. If the IP address is not the address of a server authoritative for the _msdcs.root_domain, then the CNAME records may not be found. If other DNS servers that are authoritative for the root of the forest can be found, DNSLint will look for CNAME records there, too. Finally, DNSLint produces its report, which details what it finds and the records that should be available that are not found. This report can be used to identify necessary corrections that must be made to DNS. Use replmon.exe to Check ReplicationReplmon.exe, the Replication Monitor, is a GUI tool that can be used to find information about replication and replication partners. To use the program, follow these steps:
Use Repadmin.exe to Check Replication Links Between Replication PartnersRepadmin is a command-line tool that can be used to check replication links and replication latency, summarize information about replication, and report information on replication exchanges. The following commands are displayed in Figures 9-26 and 9-27, respectively. Repadmin /showreps regalinn.chicago.local Repadmin /replsummary regalinn.chicago.local Figure 9-26. Testing for replication success.Figure 9-27. Displaying replication statistics.Active Directory and Domain Controller Configuration and Health ProblemsIf the Active Directory is not working well, and if domain controllers are not correctly configured, many problems can occur. One of these problems can be with Group Policy. Periodic checkups are in order. Use DCDIAG to Check Health of DCA healthy domain controller is one that is properly registered in DNS, is replicating with its replication partners, and is authenticating clients. To verify domain controller health, the dcdiag /replication test can be used. This command tests DNS registration, checks replication permissions, and reports any replication errors that exist between domain controllers. Dcdiag /test:replications Other tests and commands can be used to find information specific to a single domain controller or to examine the replication topology of the domain. To test a specific domain controller, use the following: Dcdiag /s:regalinn This command is shown in Figure 9-28. It tests connectivity and then performs replication-related tests and other domain controller tests. Figure 9-28. Testing replication using DCDiag.Use GPOtool to Check Template and Policy Version SynchronizationIf the GPO policy gets updated but the templates and other files have not yet replicated, GPO processing will not give the correct results. Each Group Policy container and each Group Policy template is assigned a version number, which is incremented when the GPO is changed. Use GPOtool to check for version synchronization and to determine if policies are valid. It will also display the details of GPOs that have replicated. GPOtool is a Windows Server 2003 resource kit tool. GPOtool can do the following: Check GPO consistency. Check GPO object replication. Display information about a GPO, such as functionality, version, and extension GUIDs. Search GPOs based on friendly name or GUID. Partial match is supported. Use specific domain controllers or all domain controllers. Check policies in remote domains. Provide more information if run in verbose mode.
Run gpotool.exe at the command line to see a summary report on policy consistency and replication. Each GPO is tested, and either errors will be listed, or a "Policy OK" statement will be generated, as shown in Figure 9-29. More details will be shown if running gpotool.exe /verbose. Figure 9-29. Checking policy consistency.Use LDP.exe to Check DC LDAP PortsThe LDP.exe support tool can be used to find out if a domain controller is responding to LDAP queries ports 389 and 3268.
The default port used will be the 389 port and will produce results as shown in Figure 9-30. Port 3268, used with SSL, can be tested if a certificate is available for use by the DC. The results show that the DC is listening on LDAP ports, and they provide some additional useful information. Figure 9-30. Displaying replication statistics.Use Portqry to Troubleshoot AD ConnectivityPortqry can be used to check the status of TCP/IP and UDP ports on a remote computer. Use it to test the status of Active Directory-specific ports. If these ports are not listening, replication and other functions will not be possible. In addition to simply reporting if a port is listening, portqry obtains additional information. Testing the RPC endmapper port, 135, for example, lists the ports used by RPC for specific services. When the LDAP port, 389, is queried, a range of useful information is reported, as shown in Figure 9-31. Figure 9-31. Use portqry to test LDAP connectivity and learn about the domain.More information on using portqry can be found in Knowledge Base article 816103, http://support.microsoft.com/default.aspx?scid=kb;en-us;816103. Portqry reports the status of a port in one of three ways: Listening Portqry received a response from the port. Not Listening Portqry received an ICMP "Destination UnreachablePort Unreachable" from the UDP port or a TCP acknowledgement packet with the Rest flag set. Filtered Portqry did not receive a response from the port and doesn't know if the port is listening or not.
To verify connectivity with AD, use portqry to verify ports for LDAP, DNS, RPC, and AD. To verify LDAP connectivity for the DC, use the following command: Portqry n nameofDC p udp e 389 This command will cause portqry to first send an unformatted datagram to UDP port 389 on the domain controller. Because LDAP only responds to a formatted request, no response is returned, so portqry reports that the port is listening or filtered. Next, portqry sends a properly formatted LDAP query. If a response is received, the port is reported as listening, and the response is printed to the screen. If no response is received, the port is listed as filtered. The port should be listening on a domain controller, so a "filtered" report indicates a problem. Use GPOTool.exe, Event Viewer, and GPMC to Check for Missing or Corrupted FilesGroup Policy information is contained in numerous files. If these files are missing or corrupted, Group Policy processing is affected. A number of tools, including GPOTool.exe, GPMC, Event Viewer, and manual inspection, may be used to spot problems with files. You can use GPOTool.exe to check for the presence and integrity of these files in the SYSVOL share and its subfolders on DCs. These include files in the template and registry.pol. GPMC will report file access problem events in the Policy Events tab in the Group Policy Results report. You can also view the events listed here in the Application log events in the DCs Event Viewer. Setting up verbose logging for UserEnv and for specific client-side extensions (CSEs) will also provide information. Verbose LoggingVerbose logging enables the collection of more detailed event messages. Log files are created for the core client engine (userenv) and all CSEs except the scripts processing CSE. You can also enable verbose logging for other Group Policy processing issues and tools you use. You'll find CSE-related logs in the following locations: Group Policy Core and registry CSE (userenv.log) in the %windir%\debug\usermode Security CSE (Winlogon.log) at %windir%\security\logs\ winlogon.log Folder Redirection CSE (fdeploy.log) at %windir%\debut\usermode Software Installation CSE (appmgmt.log) at %windir%\debut\usermode Windows issues (MSI*.log) at %windir%\temp\
Turning on verbose logging for CSEs requires registry changes as noted in Table 9-5.
To set up verbose logging for GPMC and GPO Editor errors, use Table 9-6.
The entries generated are placed in %temp%gpmgmt.log for all GPMC errors, %windir%\debug\usermode\gpedit.log for all GPO Editor core entries, and %windir%\debug\usermode\gptext.log for all GPO Editor CSE-specific entries. Check for client files, and if necessary, replace corrupted or missing files from the installation CD. (Use System File Checker (SFC.exe) to scan all files to verify versions.) Client files are found in the %windir%\system32 folder and are as follows: Userenv.dll Dskquota.dll Fdeploy.dll Gptext.dll Appmgmts.dll Scecli.dll Examine the File Replication and Directory Service Event LogFRS errors will have the source NTFRS. Before replication occurs, problems that prevent it may be reported in the directory services log. The error messages 1925 as shown in Figures 9-32 and 9-33 appeared when DC2 was first dcpromoed. The error message confirms the DNS lookup failure. Figure 9-32. Displaying a replication failure.Figure 9-33. Revealing the error due to DNS lookup.Once replication has occurred, the directory service log will report replication errors. Replication errors have the source KCC. Typical errors, which may impact Group Policy, are as follows: 1311: The replication configuration information in the Active Directory Sites and Services does not accurately reflect the physical topology of the networkThis can be as simple as one of the domain controllers being offline, as complex as configuration problems with bridgehead servers and site link configuration, or as annoying as problems with the WAN links. 1265: DNS Lookup Failure or RPC server is unavailableThis error is also produced, along with Target account name is incorrect, when the problem exists and repadmin is run. The errors all mean DNS problems, typically that the GUID-based domain name of the source replication partner cannot be resolved. Use dcdiag /test:connectivity to verify CNAME and A records. If records are missing, it may be possible to fix the situation. Four steps toward resolution are as follow:
1265: Access Denied or the repadmin command returns the sameThe domain controller failed to authenticate against its replication partner. This can happen when the DC is offline for a long time. When this occurs, its password is not synchronized with its computer account password in the Active Directory of its replication partner. To correct, do this:
Generic AD Troubleshooting StepsIn addition to the specific tools and commands, there are items that should be checked manually to eliminate them as possible causes of replication failure. Check each of these: CPU utilization at each DC to make sure it's not overloaded The time on the DC and on the client Event log records on each DC For the existence of the SYSVOL share on every domain controller Check File ReplicationThe .adm templates, scripts, and other files that are used for Group Policy processing are not part of Active Directory. Instead, they are stored in the SYSVOL share and its subfolders and are replicated via the File Replication Service (FRS). (The Group Policy container is an Active Directory container that stores GPO properties.) If FRS is not functioning properly, then the information needed to apply the policy may be missing or out of date. The files are automatically replicated to all domain controllers by default according to a schedule and cannot be forced. GPO File Components and LocationsEach GPO's file-based components are located in their own subfolder of the System Volume folder (Sysvol) \Policies subfolder. The GUID of the GPO becomes the folder name. Each GPO fills a Group Policy template that includes security settings, Administrative Templates-based policy settings, applications for software installation, and script files. Each Group Policy folder contains the gpt.ini file at its root. This file stores information on which client-side extension contains user or computer data, whether user or computer portions are disabled, and the version number of the GPO editor extension that created the GPO. The collection of files and subfolders and their contents is the Group Policy template. NOTE: Local GPO Don't confuse the GPO folders with the local GPO folder and files. Each computer also stores a local GPO, which contains security policy only. It is stored in %systemroot%\system32\grouppolicy. Each GPO folder contains multiple subfolders, including these: User A registry.pol file, it contains the registry setting for users. It may also include Applications advertisement files (.aas files) if applications are indicated for installation via Group Policy. Documents and Settings fdeploy.ini, which is status information about folder redirection options. Microsoft /remoteinstall The OSCfilter.ini file, which contains user options for operating system installation through the Remote Installation Service (RIS). Microsoft \IEAK Setting for Internet Explorer maintenance snap-in. Scripts All Group Policy-based scripts. Machine registry.pol for computer (registry settings for the computer). Microsoft\windows NT\secedit The Gpttmpl.ini file, the default security configuration settings for a Windows 2000 domain controller. Adm All the .adm files for this GPO. The .adm files are refreshed when service packs are installed. If the correct .adm files are not installed, or if they vary between domain controllers, the results of Group Policy can change. While .adm files are replicated, problems with replication will prevent this. In addition, .adm files are overwritten by .adm files of a later date, which also can cause problems. Suppose, for example, that .adm files are updated to provide up-to-date configuration for Windows XP computers after the installation of Windows XP Service Pack 2. Then suppose that a service pack for Windows Server 2003 is added. The new Windows Server 2003 service pack .adm files will have a later date than the Windows XP .adm files and will therefore replace them. This can cause problems. Check the File Replication Service Event LogThe File Replication Service Event Log will record errors and information about the function of FRS. The DNS issues at the promotion of DC2 also produce an FRS error, 13508, as shown in Figure 9-34. Figure 9-34. Displaying a FRS error message.The sonar.exe help file provides a table of all FRS messages and their meanings in addition to a list of some helpful troubleshooting tips. Table 9-7 is adapted from this information and my own experiences and provides several common event warnings and errors and how to respond to them.
Use Sonar.exe to Check FRS FunctioningSonar.exe is a Microsoft Resource Kit tool that can be used to check FRS functioning. Sonar is a file replication service status viewer that can be used to monitor FRS statistics and the health of the SYSVOL share by providing information on traffic levels, backlogs, and free space. Download sonar from http://www.microsoft.com/downloads/details.aspx?FamilyID=158cb0fb-fe09-477c-8148-25ae02cf15d8&DisplayLang=en. To use sonar, run the program by double-clicking on the executable in the file system. Sonar begins monitoring FRS in the domain, as shown in Figure 9-35. The tool queries AD and displays the last information it received. You can limit the columns shown by using the Columns drop-down box, change the collection timeframe, and produce a log of all statistics. Figure 9-35. Using Sonar to check FRS.Use the Filters box drop-down to produce a number of reports, such as: Backlogged member Current involved in join DC not sharing sysvol FRS in error condition Low disk space Sharing violation Staging area problems Worst backlogged members
Use the Columns box to determine the display: All columns Backlog Error conditions Replication status Resource usage Settings Sources of change Topology
|