Azure-cli: `az ad sp list` should by default show the first few pages, rather just hang to get all of them

[SteveLasker]

Kinda says it all

[yugangw-msft]

are you using your microsoft.com account? If yes, then this hangs might be just long-delay. Not saying this is ideal, but there are 100k+ users account to retrieve. There are server side filters you can use to narrow down the data.

[yugangw-msft]

The cli need to support streaming (start to display before all data are pulled down), rather wait for thousands of get operations returned.

[djyou]

I was wondering if az ad sp list can have an option (or by default) to show MY service principals rather than ALL service principals. For applications, the portal has an option to show All apps or My apps. It doesn't make sense to me if by default it returns all service principals.

[mayurid]

@yugangw-msft : List vs list-all?

[yugangw-msft]

It winds down to whether there is ownership such information inside the graph application object. AFAIK, there are not.
The following is the object dump

{
      "odata.type": "Microsoft.DirectoryServices.Application",
      "objectType": "Application",
      "objectId": "005b1e87-7383-4f8c-9a84-a913f9137a49",
      "deletionTimestamp": null,
      "appId": "0675995d-e1dc-4943-beb6-4df25e02c6d5",
      "appRoles": [],
      "availableToOtherTenants": false,
      "displayName": "updated Saml display name",
      "errorUrl": null,
      "groupMembershipClaims": null,
      "homepage": null,
      "identifierUris": [
        "http://samlCustom",
        "http://customappsso/d18d09a13dd340fa8b6a1733bad042e1/"
      ],
      "keyCredentials": [],
      "knownClientApplications": [],
      "logoutUrl": null,
      "oauth2AllowImplicitFlow": false,
      "oauth2AllowUrlPathMatching": false,
      "oauth2Permissions": [
        {
          "adminConsentDescription": "Allow the application to access SAML Custom App #2 on behalf of the signed-in user.",
          "adminConsentDisplayName": "Access SAML Custom App #2",
          "id": "5279cafd-2efd-4067-9a05-2e4d529876c0",
          "isEnabled": true,
          "type": "User",
          "userConsentDescription": "Allow the application to access SAML Custom App #2 on your behalf.",
          "userConsentDisplayName": "Access SAML Custom App #2",
          "value": "user_impersonation"
        }
      ],
      "oauth2RequirePostResponse": false,
      "passwordCredentials": [],
      "publicClient": null,
      "replyUrls": [
        "https://samlCustom"
      ],
      "requiredResourceAccess": [
        {
          "resourceAppId": "00000002-0000-0000-c000-000000000000",
          "resourceAccess": [
            {
              "id": "5778995a-e1bf-45b8-affa-663a9f3f4d04",
              "type": "Role"
            }
          ]
        }
      ],
      "samlMetadataUrl": null
    }

[djyou]

The portal does have a way to show My apps or All apps with two different calls in Azure Active Directory extension.

[yugangw-msft]

@djyou
Again, could you please share the object dump, which you can get by downloading the manifest from portal, or just use graph explorer. Only if there is a property in the graph, then we can filter; otherwise it is a portal specific UI profile setting which can't be used by cli command.

[djyou]

I agreed that the manifest doesn't have user information. It was just interesting to see the portal can actually do My apps vs All apps. I am not too familiar with Graph API just curious how they did it.

[yugangw-msft]

@djyou, you will have to ask portal team. To me it is a very UI related. First I have to make sure the combo has 'my apps' selected, otherwise the new application will be created under 'all apps', even though it was me who created. Also all applications I created in classic portal belong to "all apps" which doesn't make too much sense

[yugangw-msft]

I logged #3323 for the original perf issue. Closing this one as it was re-topic'd

[johngossman]

I don't find this resolution acceptable. Streaming isn't going to help. We need to do what the portal does and by default show just my sps.

[yugangw-msft]

Actually, it appears portal shows the first page, which is still better than CLI just hanging there.
Suggest CLI should align to that behavior, and also outputs a warning to tell people

1. There are more
2. Suggest supplying arguments to accelerate the command through --spn, --display-name, etc...

[squillace]

@yugangw-msft the problem with the suggestion of 2, is that you'd NEVER use list if you really knew what you were looking for. You'd just use show. By definition, list must return. I'm with john here; we need to return my SPs by default.

[johngossman]

+1 ... in fact, I'd also suggest that the portal default to "My apps"

[yugangw-msft]

CLI probably should also expose a new flag --my-apps or --all, depending on what the default should be.

[squillace]

That would be a fine resolution, but the default should be my apps, where you specifically _request_ the firehose with --all if you want it, but risk it never returning (until there's paging)....

[mmacy]

What are the chances we can get this popped off the backlog? As working with RBAC service principals becomes more and more common in various services (AKS and ACR, for example), it's becoming critical that we and our customers can effectively manage our SPs.

Such management includes deletion of stale principals, but without a built-in facility for listing only the service principals associated with my subscription, I've no clear path to delete them quickly and easily (such as via script).

[tjprescott]

cc/ @yugangw-msft @mayurid

[yugangw-msft]

2 things

1. Paging support from CLI
2. GraphRBAC should be updated to support https://graph.windows.net/{tenantId}/me/ownedObjects so CLI can create and list users' own SPs

The 2nd item is SDK work. I have asked last month, but AAD GRAPH is not in the priority. Guess I will have to find cycles, and do it myself

[squillace]

OMG. @yugangw-msft ping me and let me know who to discuss this with. :-) This is a very high priority for some. :-)

[squillace]

I love me some @yugangw-msft on this. What are you thinking now?

[squillace]

I wonder what the odds are here. :-) and :-(

[palma21]

Any news here?
@yugangw-msft
@mayurid

[squillace]

It's on the way. meanwhile, use https://twitter.com/ralph_squillace/status/1055245250380259328 @noelbundick's extension to get what you want there. @palma21

[yugangw-msft]

So, in a month, CLI will tweak the behavior for az ad app list and az ad sp list:

1. Both will have the flag of --mine to only list your own service principals or applications.
2. If no filter argument are supplied, you will get the first 100 with a warning which tells you the incomplete result.
3. There is a flag of --all for you to opt in the old behavior, but you are advised not to do so as it is just not useful to help you find the one

So the recommended pattern are either az ad sp list --mine or az ad sp list --display-name johndoe

Any suggestions Let me know.

Since it is a breaking change, we will first emit out a warning for 2 releases and then make the change.

[squillace]

Now, @yugangw-msft, in #3481 you mentioned that "we have a different way to fix this". This DOES fix the "need to see only your SPs", and I'm happy about that. However, it doesn't help the situation in which you need to examine SPs _generally_ and find or do something with some of them.

Is there another solution for paging that is coming, or have we just made that issue disappear? It's OK, I just want to be precise here.

[yugangw-msft]

Take MS tenant for example, we have 200,000+ service principals, even with paging (50 per page), w/o
filters, it doesn't really help you find the one you want. It is like a phone book, you can't find anything by starting from the first page till turning to the last one. Filters have to be involved.
Also once the performance issue gets resolved with filter support, jmespath based client end filters can be used to pin down the result for you.
We have no perfect solution here, but it is a trade-off we are making to help you find the one you need to work on, which is the goal anyway when you use any list commands.

[squillace]

yeah, I thought about it some more, and what I really was thinking about was the fact that you need to get an SP before you can examine the json and decide what to filter _for_. But, with the --mine option, you can get those first, dump the json and have a look and THEN write your filter.

It's something, but I'd really like the online help to update and say something like, "Tenants with large numbers of service principals and applications will require a query filter in order to receive only the results you want, as paging could take an extremely long time."

[yugangw-msft]

   > Tenants with large numbers of service principals and applications will require a query filter in order to receive only the results you want, as paging could take an extremely long time.

@squillace, I will call this out in command's help.
I plan to get the change merged in the upcoming release.