Troubleshooting Desktop Analytics - Part 1

For some time now, I've been hearing Desktop Analytics will provide some excellent data for your environment to assist with planning and deploying for your next Windows 10 feature update.  And from the demos that I have seen, especially at MMS Jazz Edition, I believe it.  The other thing I've been told multiple times over is that it's easy to set up.  Just a few clicks and then wait a little bit and you have data.  That's awesome!  And I'm sure it is easy to set up, except.......when it isn't easy and doesn't work like it should.

About a month ago, I deployed it to a pilot collection of just 25 systems.  From that 25, 100% of them FAILED to configure properly.  I waited 2, then 3, then 4 days...  And still 100% FAIL.  Ok.... So, now what?

Well, first things first, lets take a look at M365AHandler.log.  Ok, so digging through that produces some less than useful details seen below, "Census returned a non zero ReturnCode:1".

Ok, Returncode:1, not very useful at all.  Now what?

Looking around, I found some details referring to the desktopanalyticslogscollector.ps1 script found in the .\Microsoft Configuration Manager\tools directory on you CM server.  So, copied this to one of my test machines, yes, my primary workstation (what else) and ran it. So, running the script and it returns the following:

So, it says that "RunCensus failed with Unexpected exception... ...Cannot convert null to type "System.DateTime".". Basically, digging into the powershell, its looking for the end date/time of the Census but, it never completes so, it throws generic null failure because there is no end date.

So, what is C:\Windows\System32\devicecensus.exe and Why is it failing?  I don't know its full history but, its a component of the former Windows analytics, or upgrade analytics.  Why is it failing in my environment is the bigger story here, and I'll get into the history of that in a bit.  First, lets look at some more of the logs that desktopanalyticslogscollector.ps1 is collecting.

Running desktopanalyticslogscollector.ps1 with no arguments will be enough to collect the logs.  These logs will be placed in the root of the C drive under C:\M365AnalyticsLogs.  Each time it is run, it creates a time stamped folder under this directory.  In each folder, these are some of the files that you'll find:

• installedKBs.txt -- Lists what windows updates/hotfixes are installed on the computer.  Not useful because my systems always have everything installed.
• M365AHandler.log -- This is a copy of the CM client log.  So, I've already looked at this above.
• M365AnalyticsLogs_20_02_03_08_38_36.txt -- Timestamped details of what we just saw in the output from desktopanalyticslogscollector.ps1.
• RegAppCompatFlags.txt -- Lots of stuff in there.  Didn't look at it much.
• RegCensus.txt -- This is collecting the registry values and data under the HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Census key that tells you when devicecensus.exe has last run.  (this may be useful)
• RegDataCollection.txt -- Group policy settings for desktop analytics, HKLM\SOFTWARE\Policies\Microsoft\Windows\DataCollection.  (this may be useful)
• RegDiagTrack.txt -- Lots of stuff in here as well.  Didn't look at it much either.
• RegPoliciesDataCollection.txt -- This is pulling local policy data from the computer from HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\DataCollection.  And beyond that, it looks like it is logging policy data from individual user accounts as well. (this may be useful as well)
• RegSQM.txt -- I didn't see much interesting info in here either.

Ok, so I found 3 logs of interest from here. Let's take a look.

RegCensus.txt

This file contained the following data:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Census]
"ReturnCode"=dword:00000001
"StartTime"="2020-02-03T14:38:42.290"
"FullSync"=dword:00000001
"RunCounter"=dword:00000000

Look at that, Census has no end time. That's what desktopanalyticslogscollector.ps1 was barking about with the null value.  Great!  But, why is it not finishing.  So, not very useful to troubleshooting the issue only to understanding the error from the log collector.  Let's look at the next log.

RegDataCollection.txt

This file contained the following data:

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\DataCollection]
"DisableEnterpriseAuthProxy"=dword:00000001
"LimitEnhancedDiagnosticDataWindowsAnalytics"=dword:00000001
"AllowDeviceNameInTelemetry"=dword:00000001
"AllowTelemetry"=dword:00000000
"DoNotShowFeedbackNotifications"=dword:00000001
"RequestAllAppraiserVersions"=dword:00000001

So, the one that popped out at me right away was "AllowTelemetry"=dword:0000000.  Seems like most of the time 0 is off.  The Microsoft article on configuring Desktop Analytics confirms this. 2 or greater.  And so does This Article.  Well, there seems to be the problem there.  So, where is the zero coming from because that's not what I set when I did the easy part of setting up DA.  Well, with a little digging, it's coming from Group Policy.  An old group policy.... Something that was set up about 3 years ago following some security guidelines (because you know, you can't trust your data with Microsoft... Or something silly like that).

So, domain group policy always trumps local group policy (which is what CM will set for this).  So, regardless of the 2 I was assigning via CM, it was overridden by the 0 from group policy.  Also note, this can be set on both Computer and User policies, which it was.  No problem, we'll just unset this in the policy and then give it a little time and CM will run its baseline again and all will be good.

Side Note: Did you know that the Desktop Analytics settings from the client perspective is just a baseline that configures some group policies via the CM client?  This baseline is only set to evaluate daily, by the way.  Remember, group policy is on login and then every 90 minutes.

So, I cleared this policy in our domain's group policy and figured that we were home free.  I'd just have to wait a day or so for everything to catch up, right?  Wrong there's still something else going on as nothing is uploading telemetry.  Onward!

RegPoliciesDataCollection.txt

Alright, one final collected log file that looked interesting.  What's it got:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\DataCollection]
"AllowTelemetry"=dword:00000003
"CommercialId"="1111aaaa-33dd-5555-3344-123456abcdef"
"IEDataOptIn"=dword:00000003
"RequestAllAppraiserVersions"=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\DataCollection\Users]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\DataCollection\Users\S-1-5-21-7777777777-666666666-1433333333-3333]
"AllowTelemetry"=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\DataCollection\Users\S-1-5-21-1433333333-7777777777-2222222222-44444]
"AllowTelemetry"=dword:00000000

Ok, so, looking at this, I translated that it is collecting the AllowTelemetry policy setting from the user policies on the users with those SIDs from AD.  Looking up the SIDs, those made sense for my computer.  But, I haven't logged on with those accounts in a while and don't log in with them regularly.  So, group policy won't apply to that user account and therefore, it will continue to collect these policy settings and prevent census from running.

Options:

1. Delete the user profiles for those accounts (not ideal, but I can deal on my system).
2. Log in with each of those accounts to update policy (again, not ideal across the enterprise).
3. Crawl each user's registry hive and remove the policy setting. (probably can script that).

Ok, so option 3 seems to be the best option and least impactful... Let's see how it goes...

In testing, if I load their hives, delete the AllowTelemetry value and then unload the hive, it's no longer there. But, running desktopanalyticslogscollector.ps1 still produces the same result.  So, next, the keys in regpolicydatacollection.txt for the user SIDs are still there.  How about I delete the SID keys for all the users? Running desktopanalyticslogscollector.ps1 takes a few minutes to run now.... And no angry red text!  It worked!  Census has and Enddate value now as well!

A few days later, I have data in the Desktop Analytics portal and this test machine shows green in the \Software Library\Overview\Desktop Analytics Servicing\Connection Health in the CM console!  Alright, looks like I have a solution, now I'll just need to script it.

Also, looking at the M365AHandler.log, it's no longer throwing the useful error code 1.

I hope to check back in later with my scripted solution (or something else)...  Stay tuned...

Update!

• 4/13/20 - Script solution now available in Part 2!