Hypothesis: PyCon Mentored Sprints meta-issue

[Zac-HD]

This is the meta-issue for Hypothesis at the PyCon US Mentored Sprints.
We'll try to keep conversation in the Discord channel and treat this as a reference document, but you can still comment with questions or suggestions about what issues we might work on.

---

Writing property-based tests for Python is a great way to start contributing. The build system is simple, and you'll get a nice balance between using Hypothesis and exploring the standard library (and trying to break it :wink:). Specifically, you might:

• [x] Take the 'good first issue' to add tests for the zlib module. This was a very popular option!
• [ ] Adopt some tests from PyPy, as in https://github.com/Zac-HD/stdlib-property-tests/issues/7. This requires navigating a larger codebase with less help, and may or may not work out, but it's an option.

Working on Hypothesis itself can be rewarding, but it's also challenging - the codebase is larger, the build system more complicated, and things are generally more complex. If you're already a confident coder or have used Hypothesis though, they would be a valuable open source contribution.

• [x] https://github.com/HypothesisWorks/hypothesis/issues/2425, making the functions() strategy smarter about the return type. This is a whole new feature, if a small one, so you'll also need to write tests and release notes.

My hypothesis-jsonschema library has some nice feature improvement issues. These involve calculating the 'intersection' of two JSON schemas, which describe the sets of allowed values, as a performance optimisation. If all goes well, I'll release a new version with your code on Monday!

• [x] https://github.com/Zac-HD/hypothesis-jsonschema/issues/15 needs a small calculation of the least common multiple. Then add an example which didn't work before to the tests, and we're done!
• [x] https://github.com/Zac-HD/hypothesis-jsonschema/issues/38 is a small enough feature - 'just' calculating how many integers are allowed - that it doesn't need any new tests :sparkles:
• [x] https://github.com/Zac-HD/hypothesis-jsonschema/issues/39 is to actually list out all the allowed integers if there are only a few of them, for performance reasons. Still no new tests needed, but #38 has to be finished first.
• [x] https://github.com/Zac-HD/hypothesis-jsonschema/issues/30 is a more complicated case, because several keywords interact, but the workflow and tests are very similar.

Scientific projects are also starting to use Hypothesis, so if you're keen on scientific or data-related code you could help write some tests from https://github.com/pydata/xarray/issues/1846 - but note that I can only help you prepare a pull request here, not help get it merged.

Visual art is not my strong suit, but I would love logo proposals for hypothesis-jsonschema and Hypothesmith! You can draw on the hypothesis logo, imitate https://github.com/rjsf-team/react-jsonschema-form/issues/1349 or the CSmith logo (adding the Python snakes somehow?), or design something from scratch.

[marielledado]

Hi @Zac-HD ! Thanks for this meta-issue. I'm participating in Mentored Sprints this weekend. I'm a data scientist and relatively new to the more "technical" parts of the job, but I've been meaning to learn more about testing. I heard Hypothesis is really helpful for data science, so I'd like to try writing tests for data-related code (as you mentioned).

Would you be able to offer support, even if I've never written a test before? (I must admit I read through pydata/xarray#1846 and I wouldn't know where to start...)

[Zac-HD]

Hey @marielledado - welcome to the mentored sprints, I'd be very happy to help!

(the catch is that I'm in Australia and the UTC+10 timezone, so I won't be awake for the Europe/USA sprints session. But maybe we can work asynchronously and/or on Sunday morning Berlin-time?)

1. To learn about testing, my favorite quick introduction is https://realpython.com/python-testing/ - it gives you a nice overview of "why test" and the common tools in Python. Most people use pytest, so you can skip over the other test runners :wink:

2. To learn about property-based testing, In Praise of Property-Based Testing is the best introduction I know of, and you can practice on the problems in this tutorial

3. Now you're ready to work on the Xarray issue! If this sounds like a lot to learn... well, it is - but learning is also part of the sprint and I'm here to help with that too.

Because the Xarray tests are large and complicated, I suggest doing (1), the first part of (2), and then writing colorsys tests as above. Once those are merged, you'll know that you can do this and be ready to apply the tools to a larger and harder problem. Sound good?

[marielledado]

Thanks @Zac-HD these are fantastic resources! I'll do my best to go through them by Sunday and challenge myself to write at least one test as my contribution :) I might try to start on Sunday morning to catch you online!

[Zac-HD]

Sounds great! Feel free to ping me on Discord if that would be helpful 😁

[akash-suresh]

Hey @Zac-HD. I use hypothesis, on one of my projects. And, I love it. I am interested in contributing to hypothesis. I am a first time contributor, is there a guide or any easy issues, I can begin with?

[Zac-HD]

Hi @akash-suresh, :grin: glad you like it!

To get started, I suggest writing some tests for the binascii module in the top comment. While stdlib-property-tests doesn't have a contributing guide yet, the walkthrough from hypothesis-jsonschema also applies - I use the same project setup for both.

[marielledado]

Hi @Zac-HD thanks for your advice! Spent the last half day learning how to write unit tests, then gave your workshop materials a go, and I'm finding it surprisingly doable! 😄 I just finished block 1 of the workshop, but do you think I'll be ready to write at least one simple test for the sprint? Maybe for zlib or lzma as you mentioned? Open to others as well!

I'll try to be up by 9 AM CEST tomorrow, which should be 5pm where you are, hope to catch you then!

P.S. I should mention I've only done OS contribution once, so there might be growing pains at the sprint 😬

[akash-suresh]

Hey @Zac-HD!

Thanks for the quick response. That sounds awesome. I will start setting it up, and get started on tests for binascii.

[Zac-HD]

Congratulations @marielledado, that's awesome! Starting to use Hypothesis the same day you learn testing is way faster than most people manage :grin:

Don't worry about the 'growing pains', that's what the mentored sprints are for!

I think that compared to the "pbt 101" module, writing tests for lzma is probably pretty similar - use the bz2 tests as a template, and let me know how it goes.

[marielledado]

@Zac-HD thank you! 😊 great, so I'll work on writing tests for lzma module! Will post further questions on Discord!

[marielledado]

I've decided to work on writing tests for colorsys instead.

Writing tests for the lzma module is up for grabs 😄

[akash-suresh]

Hey @Zac-HD. I went through your workshop, and got a better understanding of hypothesis. :)

I would like to pick up writing tests for base64 first. I hope that's alright?

[Zac-HD]

Sure - go for it!

[Cheukting]

I may have a look at the zlib or lzma module. I got this warning message for Python 3.7.3 on my Mac OS about lzma so I am interested to have a look. Is that ok @Zac-HD

[Zac-HD]

Sounds great! The lzma API is more similar to Zlib bz2 (which already has tests you can imitate), if that helps?

[Cheukting]

Where is the Zlib tests? It's not in https://github.com/Zac-HD/stdlib-property-tests/blob/master/tests/test_compression.py

I may start working on it later today (my tutorial in streaming in 1.5 hours) hope you will see some work done tomorrow.

[marielledado]

@Zac-HD I'll give Zac-HD/hypothesis-jsonschema#38 a go :) hope I can pull it off!

[Zac-HD]

@Cheukting oops! Should have been bz2 :blush:

[akash-suresh]

@Zac-HD I have raised a PR for the test cases on base64. I would like to give binascii a try now. Is that alright? 😄

[Zac-HD]

Yes, but give me five minutes and I'll review the PR you already have open first :rocket:

[Zac-HD]

Thanks so much to everyone who participated in the mentored sprints! Togther, we:

• Released a new version of hypothesis-jsonschema with some sweet numeric logic
• Found bugs in the standard library, including a PyPy-specific error
• Found a release-blocker bug for Python 3.9 (!!)
• Learned a lot about testing and about open source
• Inspired a better error message for Hypothesis (#2439)

And while the sprints are over now, open source projects will keep going... albeit at a less frantic pace. Whether you stick around, apply your skills elsewhere, or share what you've learned with others - go well, and good luck :heart_eyes: :wave: