Troubleshooting the On-Demand Assessments (AMA)

Most common troubleshooting issues

1. DCR Creation Failed error message when adding an assessment to a machine from ServicesHub portal

Ensure that your Azure subscription has the Microsoft.Insights provider registered, then retry adding the assessment solution from ServicesHub portal.

For details on how to register a provider, see Azure resource providers and types.

If this did not correct the issue for you, you can attempt to create a basic DCR for your machine/workspace from the Azure portal. To test if you have the right permission to do so, see Create and edit data collection rules (DCRs) in Azure Monitor.

2. Error message: "You don't have access to Azure Log Analytics" in Services Hub -> IT Health -> On-Demand Assessments

Follow the guidance in Azure Roles for Log Analytics and How they Relate to the Services Hub to ensure you have the right permission level for your role.

3. Unable to find ARC/Azure VM when adding ODA Assessment in ServicesHub Connector

Currently SH Connector has a limitation that only allows it to discover ARC/VM machines inside the same resource group as the Log Analytics workspace selected, if you are unable to see a specific machine in the list when adding an assessment, check if both machine and Log Analytics workspace are found in the same resource group/subscription.

4. Recommendations are not uploading from data collector machine to Azure Log Analytics workspace

If you can't see the results of the collected data in the Azure Workbook, follow these steps:

1. Ensure that you have a set of recommendation files in the data directory for your assessment. The usual format is new.recommendation.*.solutionrec (ex: new.recommendations.*.adrecs for AD).
2. Verify that the content of the files has been uploaded to the Azure Log Analytics workspace by checking the Logs section for the workspace you are using, and run a query based of the time stamp.

Example: Run `AzureAssessmentRecommendations`` and the time set to match your last run time (such as 24 hours ago).

If the upload was successful, you can see the same content you have in the recommendation files inside the Azure table table as well

If the data is present, verify that you've selected the correct workbook associated with your workspace data.

If you're using an ARC-enabled machine for data collection, check the setting for your DCR to ensure you've mapped the correct working directory path when adding the assessment.

1. Navigate to the Azure portal.

2. Look for Data Collection Rules (DCR).

3. Locate the DCR associated with your assessment (it should contain the ODA reference and the assessment type).

4. Open the DCR and select Export Template.

5. Look for "filePatterns" in the json, and ensure the path is pointing to the folder where you created your recommendation files.

   

If the path is correct, try rerunning data collection on the machine to ensure the data is not old/expired. If rerunning data collection doesn't help, try rebooting the machine to avoid any glitches or pending updates.

If you haven't previously done so, review the network requirements for AMA to rule out any connectivity issues.

5. Workbook results are showing content unavailable due to permission issues

1. Try Refreshing the Content query.

2. Edit the Workbook. Select to the Edit button on the top right and in order to expand the parameters.

   1. Select SubsId, then edit the parameters: check the box for "Hide parameter in reading mode", then save your changes and close the editor.

   

If the issue isn't resolved, try re-registering the Microsoft.ServicesHub resource provider for your subscription. For guidance on registering or re-registering a resource provider,see Azure resource providers and types.

Linking and Permissions

1. Verify that you have the Azure Subscription permissions necessary for your role on the same email ID that you use to sign in to Services Hub.

2. You should be able to see the following page in Services Hub under the Health tab, then Assessments after successful linking.

3. Confirm that the Log Analytics workspace you have access to is the one linked in Services Hub. If it's not, ask them to relink: select your profile at the top right, then select Edit Log Analytics Workspace and link the desired workspace.

4. Confirm that you've added the desired assessment from the catalog.

   

   

   

Add-*AssessmentTask Commandlet related issues

This section covers some of the most frequent problems you might encounter when you invoke a command such as AddExchangeAssessmentTask.

Windows Server 2008 R2 does not recognize Add-*AssessmentTask commandlets

We recommend that you not run assessments on a Windows Server 2008 machine. But if you absolutely must run assessments on one, you need to follow these steps:

1. Install latest version of Powershell and .net 4.6.2 on the Windows 2008 server.
   1. Most Windows 2008 Servers have Powershell which uses .net 2.0 and that does not meet the requirements.
2. Run Powershell in Administrator mode.
3. Check that the custom ODA module is available on your machine: Get-Module Microsoft.PowerShell.Oms.Assessments -listavailable. Ensure the path displayed is C:\ODA\Binaries\bin\Microsoft.PowerShell.Oms.Assessments.

If you can't find the module using the get command, check whether the module is found in your environment variables list. Run $env:psmodulepath in PowerShell to show the list.

The correct path is typically added as part of your extension installation. If it's missing, you do have the option to add the correct path manually by running:

$env:PSModulePath = $env:PSModulePath + ";C:\ODA\Binaries\bin\Microsoft.PowerShell.Oms.Assessments"

As a precaution, verify that the path exists on your machine to ensure the module is installed on your machine.

After you execute this command, you should be able to invoke Add-*AssessmentTask commandlets.

The previous version of the module that is used for MMA does not prompt you to enter the workspace ID.

On any platform, if the Add-*AssessmentTask commandlets are not recognized

1. Verify that the OMS Assessments Powershell module has been downloaded.
   1. To do this, verify that C:\ODA\Binaries\bin has a subdirectory named Microsoft.PowerShell.Oms.Assessments, and in that subdirectory there's a file named Microsoft.PowerShell.Oms.Assessments.dll.
2. If Microsoft.PowerShell.Oms.Assessments.dll isn't there, make sure that your Log Analytics workspace is linked from Services Hub.
3. Verify that the installed version of PowerShell is at least 4.0 (to check this, type $PsVersionTable in the Powershell window) and that Powershell uses a CLRVersion equal to or greater than 4.0.
4. Ensure you have the right module path.

Troubleshooting Assessment Installation Errors when executing an Add-*AssessmentTask cmdlet

1. Review the log file.

2. The log file location is displayed on the Powershell console window. It's an informational message, in white, as visible in the following screenshot:

   

3. The -ScheduledTaskUserName and -ScheduledTaskPassword may be invalid--no such user exists, or the password is invalid, or has expired.

   

Requirements for successfully running the scheduled task

Verify the user account Group Policies: Logon as Batch Job Permission

1. Right click or long-press on "Log on as batch job", then select Properties.
2. Select "Add User or Group" and include the relevant user.

Do not forcefully unload the user registry at user logoff

On the data collection machine, change the following setting in the group policy editor (gpedit.msc) from "not configured" to "enabled".

Go to Computer Configuration -> Administrative Templates -> System -> User Profiles.

Enable "Do not forcefully unload the user registry at user logoff".

Disable the FIPS Policy

1. In Control Panel, select Administrative Tools, then select Local Security Policy.

2. Within Security Settings, expand Local Policies, then select Security Options.

   

3. Under Policy in the right-side pane, select System cryptography.

4. Use FIPS compliant algorithms for encryption, hashing, and signing, and then select Disabled.

Network Access: Do not allow storage of passwords and credentials

This error occurs with the message "A specified logon session does not exist. It may already have been terminated." To resolve this:

1. Navigate to SECPOL.MSC -> Security Settings -> Local Policies -> Security Options.

2. Don't allow storage of passwords and credentials for network authentication.

   

3. Set the policy to disabled.

Assessment has not been added to your workspace

If invoking the start-XXXAssessment results in this error: "There is no file XXXAssessment.execPkg associated to the current Log Analytics workspace...", the reason is that the specific assessment-type must be added to your machine via ServicesHub portal.

Add the assessment on the ServicesHub page:

1. Select the Health tab, then select Assessments.

2. At the end of the page, select the desired assessment listed under "Available On-Demand Assessments".

3. On the modal dialog that appears for adding the selected assessment, select "Add Assessment".

4. Wait at least five to ten minutes for the solution extension to be added.

   

Data from OnDemand assessment is no longer seen in Log Analytics, but it was seen in the past

1. Find the scheduled task in the Task Scheduler.
2. Run the task manually from the Task Scheduler.
3. Verify that an OmsAssessment.exe process is running.

If no such process is running, one possible reason is that the password specified at the time the Add-XXXAssessmentTask was run for this assessment is no longer valid (e.g. has expired or has been changed).

If this is the case, you can see the error both on the History tab of the Task Scheduler, and also in the Task Scheduler Event Log--with EventId 101 (and 104) and ErrorValue 2147943726, as shown in the following screenshots:

OmsAssessment.exe is not running or appears stuck for long periods of time

This can be the result of your antivirus blocking certain scripts or dependencies related to the process.

While the assessment can be run with the antivirus on, depending on your product or environment settings, there can be times when AV affects the performance of the process or blocks certain files from being created.

To avoid related issues, we recommend adding an exception in your antivirus for the folder you're running the assessment from.

Check for any conflicting omsassessment.exe processes running

Open Task Manager and look for a process named omsassessment.exe. If visible, it indicates that the assessment is still running.

If it's been running for quite a long time (for example, if you find this process has been running for over a day), it's possible that the assessment agent couldn't process data. In this case, proceed to the following troubleshooting steps.

Go through any errors in the prerequisite file

Go to the assessment working directory and look at the pre-requisites (processed.prerequisites) files to find any errors mentioned for the assessment targets.

If any errors are found (for example, WMI connectivity issues), the target names and the error is mentioned in this file.

Resolve any mentioned errors, then trigger the assessment:

1. Navigate to Task Scheduler -> Microsoft -> Operations Management Suite -> AOI***** -> Assessments.
2. Right click or long press on the desired assessment scheduled task, then select Run.

Go through error in the discovery log file

1. Navigate to the assessment working directory, then into the 6-8 digit numbered folder inside the directory.
2. Look for a folder named Logs. Within it, you can find a file named DiscoveryTrace***.
3. Look for any errors or exceptions in this file and resolve them, since they relate to credential/permissions issue, WMI failure, network issues, etc.

Large file ingestion

Try to reduce the number of targets per assessment schedule

If you're running the Windows Server, Windows Client, or SQL Assessment, and you've added more than 5 targets in a single scheduled task, sometimes the assessment agent might not be able to process so many targets in one go.

If you encounter this situation, use the following cmdlet to remove any existing configuration:

Remove-WindowsClientAssessmentTask -LogAnalyticsWorkspaceId "YourWorkspaceId"
Remove-WindowsServerAssessmentTask -LogAnalyticsWorkspaceId "YourWorkspaceId"
Remove-SQLAssessmentTask -LogAnalyticsWorkspaceId "YourWorkspaceId"

Now run Add-AssessmentTasks again with fewer targets. You can add multiple such tasks, and create batches of tasks with 3-5 targets per task, resulting in a faster evaluation of your entire environment.

Go through Scheduled Task dispatch and uploader log files

In the Assessments working directory, there is a folder named \Logs\ that contains *Commandlet*.log and *Module*.log.

The Commandlet.log file contains data about the scheduled task starting the Powershell commandlet. You can use this data to find why the scheduled task didn't start.

If this file isn't produced when the ODA scheduled task is started, it usually indicates a password error.

The Applications and Services Logs\Operations Manager Event Log also contains information you can use to troubleshoot various issues.

No prompt for Log Analytics Workspace ID when adding Assessment Task in PowerShell

You should be prompted for a Log Analytics Workspace ID (LAW-ID) during the creation of the Assessment Task in PowerShell. This process is what creates the Scheduled Task to run the assessment.

If you aren't prompted to provide the LAW-ID, you're not using the correct PowerShell modules. This can happen if the Microsoft Monitoring Agent is still installed, or if your data collection machine hasn’t picked up the path of the Azure Monitoring Agent’s PowerShell modules.

To configure the environment variables:

1. Select the Start button.

2. Use the search box to search for "Environment".

3. In the System Properties window, select "Environment Variables."

   

4. In the Environment Variables window, find the variable called "PSModulePath" and select it, then select the Edit button.

   

5. Move the line C:\ODA\Binaries\bin\Microsoft.PowerShell.OMS.Assessments above the line C:\ProgramFiles\MicrosoftMonitoringAgent\Agent\PowerShell\.

   

6. Select "OK" three times to exit System Properties.

7. Open up a new Administrative PowerShell window.

8. Try to add the assessment again.

   

9. If you still see no change, reboot your machine.