Azure-cli: Example reading large quantities of data from an Azure Storage Table?

[nomisbeme]

I'm trying to read large quantities of data from an Azure Storage Table. Large is greater than 1000 rows.

I'm able to read the first 1000 rows successfully by configuring my environment and using:

$ az storage entity query -t SomeTable --num-results 1000

The help and documentation mention the --marker switch to retrieve the next set of results but I can't find an example that works. Passing in the nextMarker.nextrowkey value from the JSON result (and various other combinations) give a python stacktrace:

$ az storage entity query -t MarketplaceLeads --num-results 1000 --marker '<redacted>'
'str' object has no attribute 'get'
Traceback (most recent call last):
  File "/usr/local/Cellar/azure-cli/2.0.27/libexec/lib/python3.6/site-packages/knack/cli.py", line 194, in invoke
    cmd_result = self.invocation.execute(args)
  File "/usr/local/Cellar/azure-cli/2.0.27/libexec/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 331, in execute
    six.reraise(*sys.exc_info())
  File "/usr/local/Cellar/azure-cli/2.0.27/libexec/lib/python3.6/site-packages/six.py", line 693, in reraise
    raise value
  File "/usr/local/Cellar/azure-cli/2.0.27/libexec/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 304, in execute
    result = cmd(params)
  File "/usr/local/Cellar/azure-cli/2.0.27/libexec/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 168, in __call__
    return super(AzCliCommand, self).__call__(*args, **kwargs)
  File "/usr/local/Cellar/azure-cli/2.0.27/libexec/lib/python3.6/site-packages/knack/commands.py", line 109, in __call__
    return self.handler(*args, **kwargs)
  File "/usr/local/Cellar/azure-cli/2.0.27/libexec/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 348, in default_command_handler
    result = op(**command_args)
  File "/usr/local/Cellar/azure-cli/2.0.27/libexec/lib/python3.6/site-packages/azure/multiapi/cosmosdb/v2017_04_17/table/tableservice.py", line 730, in query_entities
    resp = self._query_entities(*args, **kwargs)
  File "/usr/local/Cellar/azure-cli/2.0.27/libexec/lib/python3.6/site-packages/azure/multiapi/cosmosdb/v2017_04_17/table/tableservice.py", line 776, in _query_entities
    next_partition_key = None if marker is None else marker.get('nextpartitionkey')
AttributeError: 'str' object has no attribute 'get'

Update 2 here: https://stackoverflow.com/questions/49680850/how-retrieve-all-of-records-in-azure-storage-tables-using-powershell-or-azure-cl/49683088#49683088 appears to be a similar, related issue.

---

Environment summary

Brew on Mac. Version 2.0.27. System bash, High Sierra 10.13.4.

[williexu]

This is a bug in our CLI, the sdk requires a dictionary and we never parse the string entered in the command line.
https://github.com/Azure/azure-cli/pull/6210 addresses this by accepting a list of key=values pairs for the --marker parameter, i.e:
--marker nextPartitionKey=sdfsfsfs nextRowKey=1sflksdfsf

Should be reminiscent of the --entity parameter when inserting into the table.

[nomisbeme]

Woah, thanks @williexu for tagging, reproducing and fixing this in record time.
I hope to be able to repay your assistance by testing the fix soon.

NewMicrosoft

[troydai]

You're welcome, @noamben. Thank you for filing this issue and providing detailed trace (and in code format!)

[mayurid]

Adding the appropriate milestone to have the fix available.