Moto: Timeline for boto/deprecated_decorator removal

[spulec]

It would be nice to remove all of the deprecated code and the bundled httpretty.

There are some tests we have where code is only covered by deprecated decorators so we would need to migrate those tests first to ensure we don't lose coverage.

The boto library hasn't had a commit in over a year. It doesn't look like the library will be getting much for updates.

I would be in favor of starting this process soon (like now), but I want to hear if others have a good reason for delaying. Given that we would have to migrate tests, this process will take some time and we would obviously cut a larger release after that. Anyone that still needs to use boto could still use Moto in server-mode.

For anyone advocating for delaying, please propose what you think would be a reasonable timeframe.

[bblommers]

I'm not aware of any outstanding boto bugs/requests, so I have no problems with the proposed timeline.

Happy to help create a plan of attack to tackle this.

[bpandola]

I know that for RDS and Redshift there are differences between boto and boto3 in the way that (list) parameters are passed through. Additionally, Redshift must respond with JSON payloads for boto calls but XML is expected for boto3 calls. Just pointing this out to say that migrating the RDS/Redshift tests that currently use boto (and the deprecated decorator) to use boto3 would eliminate some important test coverage. It would require someone with very specific domain knowledge to migrate those tests in a way that maintained their integrity. I could probably do this for RDS/Redshift, but I wonder how many landmines like this exist throughout the other mocked services.

[spulec]

@bpandola great call. We could migrate some of the tests to be server-mode only tests and check the JSON.

Bigger picture, if we aren't supporting boto, how long should we continue to support the JSON endpoints?

For anyone following along, I have started a branch where I have removed some of the deprecated decorators: https://github.com/spulec/moto/tree/deprecate-httpretty

RDS and Redshifts are fairly far down my list though.

[bpandola]

Something else to consider: the moto.ec2 module directly imports some models from boto.ec2 (https://github.com/spulec/moto/blob/master/moto/ec2/models.py#L18).

[bblommers]

Does it make sense to also deprecate the old DynamoDB API version, while we're doing a Spring cleaning?
Would be good to reduce the 4 decorators (mock_dynamodb, mock_dynamodb_deprecated, mock_dynamodb2, mock_dynamodb2_deprecated) and only keep a single mock_dynamodb.

It does mean we wouldn't be able to implement this feature request.

[bpandola]

@spulec I'm not sure how far along you are with this, but I'm wondering if there might be an intermediate step here (given issues like #3013 and #3418) where moto still works for people using boto but removes it as an explicit dependency. People who long ago left boto behind need not bother with it any longer, but those who still use it can still use moto.

To be clear: those developing moto would still need boto in order to run the tests, but that could be handled with a tests_require directive in setup.py.

I'm happy to work on a PR for this, if it falls in line with your roadmap for the project.

[spulec]

I pushed a branch with my work a little while back: https://github.com/spulec/moto/tree/deprecate-httpretty (it is easier to read commit by commit).

Unfortunately, I haven't had time to pick it up and worry it has started to drift now. In hindsight, I should have attempted to tackle this with 1 pull request per service instead of trying to do it all at once.

I think your approach makes sense and would welcome any work along those lines.