Hypothesis: Reference of projects which provide custom strategies based on hypothesis

[fkromer]

I recently began a project called rospbt which aims at providing customized hypothesis strategies for the Robot Operating System (including its successor ROS2.0). I appreciate the list of open source projects which use hypothesis / libraries.io - hypothesis (dependent repositories) as reference where to look for inspiration (design and implementation). Most dependent projects use hypothesis capabilities for testing but not for providing advanced custom strategies as a library. This makes it very time consuming to find examples about how to design custom strategies which enable unlimited support for strategy configuration and adaption (via map(), filter() and flatmap) of these custom strategies.

Would it be reasonable to split the docs section "Open Source Projects using Hypothesis" into a "I just use hypothesis" and a "I built something on top of hypothesis" kind of sub sections?

[DRMacIver]

We do! Though I guess the fact that you missed it is a sign that it's not that obvious (we really do need to work on better docs organisation). But there's a separate section of projects extending Hypothesis

[fkromer]

Great. I somehow missed that section because I used the rtd search instead of looking into the TOC.

[DRMacIver]

Great. I somehow missed that section because I used the rtd search instead of looking into the TOC.

No worries. BTW feel free to open a pull request adding your project to the list whenever you think it's ready! It sounds neat

[fkromer]

Right now I am still struggling with getting the API done right without limiting configuration and adaption capabilities. I hope I get some inspiration from Projects extending Hypothesis and by discussing possible designs with other people also interested in an adaption of PBT to ROS. As soon as rospbt will be usable (without guaranteeing a stable API) I plan to release it as v0.X.0 for "API usability" testing. Probably it would be suitable to add it to the list as well then...

[DRMacIver]

Right now I am still struggling with getting the API done right without limiting configuration and adaption capabilities.

Sure thing! That's what I meant by "whenever you think it's ready". No pressure on our end.

[Zac-HD]

I look forward to seeing what you do with rospbt! If you'd like some design or implementation feedback just @-mention me somewhere or send me an email; I'd be delighted to help out :smile:

As a starting point, I actually think that the Numpy (and possibly Django) extras would be a really nice example of how to infer a strategy for a specific message from some kind of schema (a dtype or a model, respectively) - a ROSpy msg is pretty similar to a Numpy compound dtype, after all!

More generally, a well-defined schema is a wonderful thing - if you can say "any message that matches this schema should be OK", that's a good solution to the "oracle problem". That is, the problem of describing what an implementation should do separately from re-implementing it: in this case, the implementation should always accept a compliant message with at most a well-known error type and not e.g. crash. It's also super convenient if or when you can just say @given(from_ros_message(msg_type)) and have that work!

[fkromer]

I look forward to seeing what you do with rospbt!

Me too :smile:

If you'd like some design or implementation feedback just @-mention me somewhere or send me an email; I'd be delighted to help out :smile:

Would be great to get some hints. Thanks a lot in advance!

As a starting point, I actually think that the Numpy (and possibly Django) extras would be a really nice example of how to infer a strategy for a specific message from some kind of schema (a dtype or a model, respectively) - a ROSpy msg is pretty similar to a Numpy compound dtype, after all!

I will look into there next.

It's also super convenient if or when you can just say @given(from_ros_message(msg_type)) and have that work!

Right now I don't think that it makes sense to implement this specific example (a receiver of messages subscribes to specific topics with one specific message type, messages with another message type should be "filtered" on a low level). In general I am aiming at an as general and flexible solution as possible.

[fkromer]

No worries. BTW feel free to open a pull request adding your project to the list whenever you think it's ready! It sounds neat

I deployed the initial version of hypothesis-ros on pypi. Seemed reasonable to add it to the docs #1312 :slightly_smiling_face:

[fkromer]

:confetti_ball: :tada: Package announcement on discourse.ros.org :tada: :confetti_ball: