Joss-reviews: [REVIEW]: pirecorder: Controlled and automated image and video recording with the raspberry pi

Created on 20 Aug 2020 · 87Comments · Source: openjournals/joss-reviews

Submitting author: @JolleJolles (Jolle Wolter Jolles)
Repository: http://github.com/jollejolles/pirecorder
Version: v3.2.0
Editor: @marcosvital
Reviewer: @lucask07, @DerJH
Archive: 10.5281/zenodo.4058628

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/cd4781aefbc0f72a36470fd81e48c2fb"><img src="https://joss.theoj.org/papers/cd4781aefbc0f72a36470fd81e48c2fb/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/cd4781aefbc0f72a36470fd81e48c2fb/status.svg)](https://joss.theoj.org/papers/cd4781aefbc0f72a36470fd81e48c2fb)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@lucask07 & @DerJH, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

Make sure you're logged in to your GitHub account
Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @marcosvital know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @lucask07

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@JolleJolles) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Review checklist for @DerJH

Conflict of interest

[x] I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

[x] I confirm that I read and will adhere to the JOSS code of conduct.

General checks

[x] Repository: Is the source code for this software available at the repository url?
[x] License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
[x] Contribution and authorship: Has the submitting author (@JolleJolles) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
[x] Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines

Functionality

[x] Installation: Does installation proceed as outlined in the documentation?
[x] Functionality: Have the functional claims of the software been confirmed?
[x] Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
[x] Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
[x] Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
[x] Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
[x] Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

[x] Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
[x] A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
[x] State of the field: Do the authors describe how this software compares to other commonly-used packages?
[x] Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
[x] References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

Jupyter Notebook Python TeX accepted published recommend-accept review

Source

whedon

All 87 comments

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @lucask07, @DerJH it looks like you're currently assigned to review this paper :tada:.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

:star: Important :star:

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/openjournals/joss-reviews) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

Set yourself as 'Not watching' https://github.com/openjournals/joss-reviews:

watching

You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

whedon on 20 Aug 2020

Reference check summary:

OK DOIs

- 10.1038/s41598-019-43845-9 is OK
- 10.1111/2041-210X.13005 is OK
- 10.1371/journal.pcbi.1006959 is OK
- 10.1371/journal.pbio.2003026 is OK
- 10.1016/j.cub.2017.08.004 is OK
- 10.1098/rspb.2017.2629 is OK
- 10.1016/j.anbehav.2019.06.022 is OK
- 10.1371/journal.pone.0169758 is OK
- 10.1016/j.dsr.2019.103136 is OK
- 10.1111/jofo.12182 is OK
- 10.1371/journal.pone.0220751 is OK
- 10.1089/zeb.2016.1412 is OK
- 10.3389/fevo.2018.00026 is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon on 20 Aug 2020

:point_right: Check article proof :page_facing_up: :point_left:

whedon on 20 Aug 2020

Dear @JolleJolles: your manuscript will be reviewed in this issue, and you can reply any comments and suggestions that the reviewers might address right here.

marcosvital on 20 Aug 2020

@lucask07 and @DerJH: thank you all for for accepting review this submission for JOSS.

Even if you are not starting the review right now, please accept the invite, as it has an expiration date (there is a link under Reviewer instructions & questions and you should also get an email notification). Furthermore, please check the instructions and checklists above, and let me know if you need any assistance.

You can also tag @JolleJolles if you need to ask specific questions about the submission.

marcosvital on 20 Aug 2020

Thanks everyone for your help. Looking forward.

On 20 Aug 2020, at 12:46, Marcos Vital (LEQ-UFAL) notifications@github.com wrote:

Dear @JolleJolles https://github.com/JolleJolles: your manuscript will be reviewed in this issue, and you can reply any comments and suggestions that the reviewers might address right here.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/openjournals/joss-reviews/issues/2584#issuecomment-677521845, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACC6PYPS6YTK3MUYMAYXTDTSBT5HNANCNFSM4QF5CM5A.

JolleJolles on 20 Aug 2020

Hi,
I am trying to install pirecorder at the moment, but both using _sudo pip install pirecorder_ and the GitHub version i am running into the following error code (I attached the whole commandline dialog as a text file). It seems to have something to do with "h5py". I then tried to install with "sudo pip install h5py", which also failed.

Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-eCsuDf/h5py/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-Vphw1e/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-eCsuDf/h5py/

Is there an obvious workaround or fix for that?
Best, Jan
full_error.txt

DerJH on 26 Aug 2020

Hi Jan,

That seems to be an error with the h5py package, not with pirecorder. I have never encountered it. I have installed pirecorder on fresh raspbian systems on rpi2,3, and 4 and then do not get this error. I am not sure how to solve the h5 install error, perhaps try updating your raspberry pi?

h5py is not actually used by pirecorer but only by 1 function in my pythutils package that is also not used by pirecorder. I updated pythutils to not require the h5py package anymore, so hopefully now when installing pythutils and pirecorder you can get past this stage. You have to use the github version (1.3.6). I will update it on pypi.org after the package and paper are passed the review stage.

I hope this helps, do let me know if something else related to this point has come up.

Thanks,
Jolle

JolleJolles on 26 Aug 2020

Hi Jolle,
ok after upgrading raspbian and using apt-get python-h5py and apt-get python3-h5py the installation worked!
I'll try to test all the functions in your program in the next 2 days.
Greetings,
Jan

DerJH on 26 Aug 2020

Hi @marcosvital,
Sorry, I just came to this and the invite has expired. Is there a way you could resend?

lucask07 on 27 Aug 2020

👍1

Hi @JolleJolles,
A quick question before I dig too deep. Is your package targeting a specific Raspberry Pi compatible camera? Have you documented the cameras your software has been tested with? I suspect the compatible cameras are set by the Picamera package. I see these "official" Raspberry Pi cameras that connect via the MIPI port here

lucask07 on 27 Aug 2020

Hi @lucask07. No the package does not target a specific camera and should work with all cameras, official and non-official, that connect with the MIPI port, as pirecorder builds on the picamera package.
Thanks

JolleJolles on 28 Aug 2020

@whedon re-invite @lucask07 as reviewer

marcosvital on 31 Aug 2020

OK, the reviewer has been re-invited.

@lucask07 please accept the invite by clicking this link: https://github.com/openjournals/joss-reviews/invitations

whedon on 31 Aug 2020

Let me know if it's working, @lucask07.

marcosvital on 31 Aug 2020

Hi @marcosvital
It's working, I can click on the check-boxes now.

lucask07 on 1 Sep 2020

👍1

@JolleJolles

A few comments as I'm working through the Documentation section of the reviewer checklist.

README page is missing an explanation of how others may "1) Contribute to the software"
The _Statement of Need_ should be elaborated on in both the README and the github.io page.
The software does not have automated tests. "Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?"

lucask07 on 1 Sep 2020

For the scholarly contribution a check with CLOC ("count lines of code") shows:

Python: 1093 lines of code, and 402 lines of comment
Markdown 1150 lines of code
Jupyter Notebook: 52 lines of code, 973 lines of comment

So this work is above the LOC threshold. Submissions with LOC less than 1000 are "flagged" per the JOSS scholarly effort guidelines.

lucask07 on 1 Sep 2020

@JolleJolles

A few comments as I'm working through the Documentation section of the reviewer checklist.

README page is missing an explanation of how others may "1) Contribute to the software"

The _Statement of Need_ should be elaborated on in both the README and the github.io page.

The software does not have automated tests. "Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?"

Thanks @lucask07. I will address your comments 1 and 2 immideately.

For Comment 3, I read in the author guidelines that automated tests are not a requirement and that "Documented manual steps that can be followed to objectively check the expected functionality of the software" would also be okay. I have detailed tutorials to run through the whole software (jollejolles.github.io/pirecorder), which I thought, perhaps wrongly so, that that would conform to that requirement. Would that indeed be enough if I would add a clear statement that the software can be tested by following the online manual? Or alternatively if I add python files that can be run to show the software is working? I would appreciate your feedback on this so I can address the comment appropriately.

JolleJolles on 1 Sep 2020

For the scholarly contribution a check with CLOC ("count lines of code") shows:

Python: 1093 lines of code, and 402 lines of comment

Markdown 1150 lines of code

Jupyter Notebook: 52 lines of code, 973 lines of comment

So this work is above the LOC threshold. Submissions with LOC less than 1000 are "flagged" per the JOSS scholarly effort guidelines.

Thanks. I have actually very limited comments throughout my code. The only commented code would be the function documentation. Could you guide me on what to here? Thank you.

JolleJolles on 1 Sep 2020

@JolleJolles
A few comments as I'm working through the Documentation section of the reviewer checklist.

README page is missing an explanation of how others may "1) Contribute to the software"

The _Statement of Need_ should be elaborated on in both the README and the github.io page.

The software does not have automated tests. "Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?"

Thanks @lucask07. I will address your comments 1 and 2 immideately.

For Comment 3, I read in the author guidelines that automated tests are not a requirement and that "Documented manual steps that can be followed to objectively check the expected functionality of the software" would also be okay. I have detailed tutorials to run through the whole software (jollejolles.github.io/pirecorder), which I thought, perhaps wrongly so, that that would conform to that requirement. Would that indeed be enough if I would add a clear statement that the software can be tested by following the online manual? Or alternatively if I add python files that can be run to show the software is working? I would appreciate your feedback on this so I can address the comment appropriately.

Hi, I think a python file which executes several functions would be a good idea, but as I am also new to the software testing business (and github reviewing) not sure if that is the way these things go about.

DerJH on 1 Sep 2020

I have some more questions/comments

Is there a way to save the ROI directly into the config file? So 1. opening stream, 2. making rectangle, 3. press "s", 4. save to the current .conf file?
In the convert function I might have discovered a tiny bug.

from pirecorder import Convert
Convert(indir = "/home/pi/Videos", outdir = "/home/pi/Videos/converted", resizeval = 0.5)
Convert(indir = "/home/pi/Videos", outdir = "/home/pi/Videos/converted", resizeval = 0.5, withframe = True)

when i do the "withframe = True" then the result becomes a scrambled video mostly grey and random pix, but it works without.

The following example on the website has a small error. Twice "imgwait":
rec.settings(imgwait = 1, imgnr = 10, imgtime = 15, imgwait = 60, imgquality = 90 \
imgdims = (3280, 2464))

4.This is more a wish for a future update: On a small screen the pirecorder.Camconfig() slider window only allows access until the first Red channel slider. Maybe this could be made more responsive, or all the UI elements a bit smaller.

all the above is tested using python 2.7. I somehow didn't manage to get it to work under python 3, but given my general linux skills this is probably more a problem of me failing to manage to install it properly. But maybe you can help me to figure out what could be wrong here.
pi@raspberrypi:~ $ python3
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> import pirecorder
>>> rec = pirecorder.PiRecorder(configfile = "irsettings.conf")
Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'pirecorder' has no attribute 'PiRecorder'
>>> exit()

thanks

DerJH on 1 Sep 2020

For the scholarly contribution a check with CLOC ("count lines of code") shows:

Python: 1093 lines of code, and 402 lines of comment

Markdown 1150 lines of code

Jupyter Notebook: 52 lines of code, 973 lines of comment

So this work is above the LOC threshold. Submissions with LOC less than 1000 are "flagged" per the JOSS scholarly effort guidelines.

Thanks. I have actually very limited comments throughout my code. The only commented code would be the function documentation. Could you guide me on what to here? Thank you.

@JolleJolles : I think this is fine and there's no needed action on your part. I was just documenting my findings.

lucask07 on 2 Sep 2020

@lucask07 it would be great to get your response to my reply earlier this week so I can best address your points. Thanks

@JolleJolles
A few comments as I'm working through the Documentation section of the reviewer checklist.

README page is missing an explanation of how others may "1) Contribute to the software"

The _Statement of Need_ should be elaborated on in both the README and the github.io page.

The software does not have automated tests. "Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?"

Thanks @lucask07. I will address your comments 1 and 2 immideately.

For Comment 3, I read in the author guidelines that automated tests are not a requirement and that "Documented manual steps that can be followed to objectively check the expected functionality of the software" would also be okay. I have detailed tutorials to run through the whole software (jollejolles.github.io/pirecorder), which I thought, perhaps wrongly so, that that would conform to that requirement. Would that indeed be enough if I would add a clear statement that the software can be tested by following the online manual? Or alternatively if I add python files that can be run to show the software is working? I would appreciate your feedback on this so I can address the comment appropriately.

JolleJolles on 5 Sep 2020

With regards to revising the paper/package, would it be best to wait for all comments to incorporate them all at once in a new release or address them as they come in? I have no experience in this submission and revision process via Github yet so appreciate your understanding. Thanks!

JolleJolles on 5 Sep 2020

With regards to revising the paper/package, would it be best to wait for all comments to incorporate them all at once in a new release or address them as they come in? I have no experience in this submission and revision process via Github yet so appreciate your understanding. Thanks!

It's pretty much up to you, @JolleJolles. Most authors will work on improvements and corrections as they come, but it's not a rule at all. So feel free to follow the path you think is better.

Now, if you choose to go working as the suggestions come in, I would suggest you to address the changes you make here in this issue, so reviewers can easily track down what is already covered. Also, both you and the reviewers can open issues on the software's repository to track down important issues/changes and link them here.

marcosvital on 7 Sep 2020

Hi @JolleJolles
Just a small comment about the "statement of need". I think it would be sufficient to basically just place a header after the first summary paragraph, as you clearly outline why you made the software there.

You could also place a similar statement of need paragraph on the front page of the software documentation and then I guess the two JOSS points about the "statement of need" would be addressed.

DerJH on 8 Sep 2020

Hi @JolleJolles,
I'm going through the paper and have a few concerns/nitpicks that I'd suggest you clarify.

The paper references "get the optimal camera shutter speed". This is a rather complex calculation that attempts to limit saturation while simultaneously having the darkest pixels collect the most possible light. I can't find any auto exposure calculations in your code. Can you clarify how the optimal shutter speed is calculated/retrieved?
The paper reference “Calibrating the camera.” This is not appropriate usage of the term _calibrate_ for a camera. Camera calibration may involve determining the focal length and image center or estimating the lens distortion. Other calibration efforts may center around aspects of sensor performance such as measuring the color correction matrix or locating hot pixels. This does not appear to be what _pirecorder_ is doing. I think your package enables real-time streaming so that a user is given feedback while positioning a camera? Can you change the paper wording to clarify what _pirecorder_ helps the user accomplish?
_Paper grammar_: “Multiple configuration files can be used that can be edited directly, such as to set the video duration, quality, or number of images, or dynamically via an interactive video stream, such as to draw the region of interest, set the desired white-balance, or get the optimal camera shutter speed.”
The “or”s in this list are confusing. Should these actually be “and”s? Consider splitting this sentence into 2 or 3 sentences for clarity.
The Jupyter notebook references the three possible cameras to use:
v1.5 camera model (2592x1944)
v2 camera (3280x2464)
HQ camera (4056x3040)
I think these 3 cameras should be more clearly called out in the README.
The code _camconfig.py_ allows for a limited range of ISO.

isos = [100,200,320,400,500,640,800]

Is this consistent with the capabilities of the 3 camera modules? Is it possible to probe the camera module for the available ISO settings?

The code _camconfig.py_ appears to limit resolution to 1640x1232:

if (res[0]*res[1])>=(1640*1232):
        res = maxrect(dims, maxdims = (1640,1232), decimals = 0)
cam.resolution = picamconv(res)

Is my interpretation correct? Is this limitation required?

Thanks for all your work.

lucask07 on 9 Sep 2020

Thanks for all your comments. I will address them all together from next Monday (14th), making sure to add each point as an issue to the repository.
Thanks

JolleJolles on 10 Sep 2020

4.This is more a wish for a future update: On a small screen the pirecorder.Camconfig() slider window only allows access until the first Red channel slider. Maybe this could be made more responsive, or all the UI elements a bit smaller.

I opened this issue as an enhancement request (https://github.com/JolleJolles/pirecorder/issues/28) and addressed it there with this commit.

JolleJolles on 14 Sep 2020

The following example on the website has a small error. Twice "imgwait":
rec.settings(imgwait = 1, imgnr = 10, imgtime = 15, imgwait = 60, imgquality = 90
imgdims = (3280, 2464))

Thanks, fixed the typo with commit c2f23e4

JolleJolles on 14 Sep 2020

A few comments as I'm working through the Documentation section of the reviewer checklist.

README page is missing an explanation of how others may "1) Contribute to the software"

The _Statement of Need_ should be elaborated on in both the README and the github.io page.

The software does not have automated tests. "Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?"

Following the above point, I improved the statement of need in the paper and added the statement to the README and the web documentation (issue #23, commit edae82f).

JolleJolles on 14 Sep 2020

The code _camconfig.py_ allows for a limited range of ISO.
isos = [100,200,320,400,500,640,800]
Is this consistent with the capabilities of the 3 camera modules? Is it possible to probe the camera module for the available ISO settings?

In reply to this point, pirecorder builts on the picamera package to control the native camera settings, which are indeed limited to the above range, see the pirecorder documentation. pirecorder uses a default iso setting of 200.

JolleJolles on 14 Sep 2020

All the above is tested using python 2.7. I somehow didn't manage to get it to work under python 3, but given my general linux skills this is probably more a problem of me failing to manage to install it properly. But maybe you can help me to figure out what could be wrong here.
pi@raspberrypi:~ $ python3
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import cv2
import pirecorder
rec = pirecorder.PiRecorder(configfile = "irsettings.conf")
Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'pirecorder' has no attribute 'PiRecorder'
exit()

@DerJH in reply to your above question, I cannot reproduce your issue. Doing a fresh install of pirecorder on python3 I do not get this error. Could you show the output of pirecorder.__version__ and cv2.__version so I can help further? I also recommend to use a virtual environment (as I detail in the online documentation)

JolleJolles on 15 Sep 2020

Is there a way to save the ROI directly into the config file? So 1. opening stream, 2. making rectangle, 3. press "s", 4. save to the current .conf file?

Integrated saving directly from stream by the possibility of adding a configuration file when running stream from the command line (e.g. stream -c pirecorder.conf). When the 's' key is pressed after drawing a rectangle, the roi coordinates are stored in the config file. Working after commit d335de9.

JolleJolles on 15 Sep 2020

2. In the convert function I might have discovered a tiny bug.

from pirecorder import Convert
Convert(indir = "/home/pi/Videos", outdir = "/home/pi/Videos/converted", resizeval = 0.5)
Convert(indir = "/home/pi/Videos", outdir = "/home/pi/Videos/converted", resizeval = 0.5, withframe = True)

when i do the "withframe = True" then the result becomes a scrambled video mostly grey and random pix, but it works without.

@DerJH I also cannot reproduce this issue. When the withframe parameter is set to True then instead of fast ffmpeg conversion, opencv will be used for conversion (and drawing the frame number on each frame of the video). So I think there is an issue with the version of opencv you are using. Python2.7 is no longer supported and I think it could well be that your version of opencv is not up to par anymore. Again I recommend to try again on python3 and to use a virtual environment to install the right version of opencv. I wrote a special documentation page for that, but it should hopefully work after running this simple command: pip3 install opencv-contrib-python==4.1.0.25.

JolleJolles on 15 Sep 2020

I have some more questions/comments

Is there a way to save the ROI directly into the config file? So 1. opening stream, 2. making rectangle, 3. press "s", 4. save to the current .conf file?

In the convert function I might have discovered a tiny bug.

from pirecorder import Convert
Convert(indir = "/home/pi/Videos", outdir = "/home/pi/Videos/converted", resizeval = 0.5)
Convert(indir = "/home/pi/Videos", outdir = "/home/pi/Videos/converted", resizeval = 0.5, withframe = True)

when i do the "withframe = True" then the result becomes a scrambled video mostly grey and random pix, but it works without.

The following example on the website has a small error. Twice "imgwait":
rec.settings(imgwait = 1, imgnr = 10, imgtime = 15, imgwait = 60, imgquality = 90
imgdims = (3280, 2464))

4.This is more a wish for a future update: On a small screen the pirecorder.Camconfig() slider window only allows access until the first Red channel slider. Maybe this could be made more responsive, or all the UI elements a bit smaller.

all the above is tested using python 2.7. I somehow didn't manage to get it to work under python 3, but given my general linux skills this is probably more a problem of me failing to manage to install it properly. But maybe you can help me to figure out what could be wrong here.
pi@raspberrypi:~ $ python3
Python 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import cv2
import pirecorder
rec = pirecorder.PiRecorder(configfile = "irsettings.conf")
Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'pirecorder' has no attribute 'PiRecorder'
exit()

thanks

I hope to have hereby addressed all your comments from this thread. Please follow-up on the specific comments above if needed.

JolleJolles on 15 Sep 2020

A few comments as I'm working through the Documentation section of the reviewer checklist.

README page is missing an explanation of how others may "1) Contribute to the software"

The _Statement of Need_ should be elaborated on in both the README and the github.io page.

The software does not have automated tests. "Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?"

I have now included a python test file in the package (tests/test.py) to run all functionalities, including running pirecorder, setting configuration files, running different types of recordings, scheduling recordings, converting recordings, and testing the stream and camconfig interactive functions. The possibility to test the software is also described in the README and in the online documentation.

JolleJolles on 16 Sep 2020

In the convert function I might have discovered a tiny bug.
from pirecorder import Convert
Convert(indir = "/home/pi/Videos", outdir = "/home/pi/Videos/converted", resizeval = 0.5)
Convert(indir = "/home/pi/Videos", outdir = "/home/pi/Videos/converted", resizeval = 0.5, withframe = True)
when i do the "withframe = True" then the result becomes a scrambled video mostly grey and random pix, but it works without.

@DerJH I also cannot reproduce this issue. When the withframe parameter is set to True then instead of fast ffmpeg conversion, opencv will be used for conversion (and drawing the frame number on each frame of the video). So I think there is an issue with the version of opencv you are using. Python2.7 is no longer supported and I think it could well be that your version of opencv is not up to par anymore. Again I recommend to try again on python3 and to use a virtual environment to install the right version of opencv. I wrote a special documentation page for that, but it should hopefully work after running this simple command: pip3 install opencv-contrib-python==4.1.0.25.

I am also convinced that this is some installation hiccup with python and opencv on my side. I will try to check this on a fresh install in the coming days. But given that it worked for @lucask07 it should not hinder the publication.

DerJH on 16 Sep 2020

In the convert function I might have discovered a tiny bug.
from pirecorder import Convert
Convert(indir = "/home/pi/Videos", outdir = "/home/pi/Videos/converted", resizeval = 0.5)
Convert(indir = "/home/pi/Videos", outdir = "/home/pi/Videos/converted", resizeval = 0.5, withframe = True)
when i do the "withframe = True" then the result becomes a scrambled video mostly grey and random pix, but it works without.

@DerJH I also cannot reproduce this issue. When the withframe parameter is set to True then instead of fast ffmpeg conversion, opencv will be used for conversion (and drawing the frame number on each frame of the video). So I think there is an issue with the version of opencv you are using. Python2.7 is no longer supported and I think it could well be that your version of opencv is not up to par anymore. Again I recommend to try again on python3 and to use a virtual environment to install the right version of opencv. I wrote a special documentation page for that, but it should hopefully work after running this simple command: pip3 install opencv-contrib-python==4.1.0.25.

I am also convinced that this is some installation hiccup with python and opencv on my side. I will try to check this on a fresh install in the coming days. But given that it worked for @lucask07 it should not hinder the publication.

Hi @DerJH, unfortunately, I do not have a Raspberry Pi and so am not able to test for functionality.

lucask07 on 16 Sep 2020

@JolleJolles
Looks great that most issues are closed. I'm OK with signing off on publication once the wording on camera calibration is cleared up in the paper. Note that I will need to leave the functionality verification box unchecked since I lack the hardware.

lucask07 on 16 Sep 2020

Hi @JolleJolles,
I'm going through the paper and have a few concerns/nitpicks that I'd suggest you clarify.

The paper references "get the optimal camera shutter speed". This is a rather complex calculation that attempts to limit saturation while simultaneously having the darkest pixels collect the most possible light. I can't find any auto exposure calculations in your code. Can you clarify how the optimal shutter speed is calculated/retrieved?

In this context, it was meant optimal in terms of the user's perspective. I rewrote the sentence to say "...set the ideal camera shutter speed" to make this clearer.

The paper reference “Calibrating the camera.” This is not appropriate usage of the term _calibrate_ for a camera. Camera calibration may involve determining the focal length and image center or estimating the lens distortion. Other calibration efforts may center around aspects of sensor performance such as measuring the color correction matrix or locating hot pixels. This does not appear to be what _pirecorder_ is doing. I think your package enables real-time streaming so that a user is given feedback while positioning a camera? Can you change the paper wording to clarify what _pirecorder_ helps the user accomplish?

One of the definitions of "calibrate" is to "carefully assess, set, or adjust". This is what is meant here. By means of an interactive user stream, both with the stream and camconfig functions, the user is for example able to carefully position the camera and center it, adjust the camera focus using a zoomed-in view of the camera, and adjust external lighting and camera sensitivity settings to get the ideal image brightness and shutter speed. I, therefore, do feel that the term "calibrate" is correctly used here. But I agree that this was not clear enough in the paper. I have therefore carefully revised the paper to better clarify what pirecorder helps accomplish in terms of the steps until recording.

_Paper grammar_: “Multiple configuration files can be used that can be edited directly, such as to set the video duration, quality, or number of images, or dynamically via an interactive video stream, such as to draw the region of interest, set the desired white-balance, or get the optimal camera shutter speed.”
The “or”s in this list are confusing. Should these actually be “and”s? Consider splitting this sentence into 2 or 3 sentences for clarity.

I have revised this paragraph to improve grammar and readability, including by splitting the sentence into multiple sentences.

The Jupyter notebook references the three possible cameras to use:

v1.5 camera model (2592x1944)

v2 camera (3280x2464)

HQ camera (4056x3040)
I think these 3 cameras should be more clearly called out in the README.

These are the three official cameras, all with their own maximum resolution. But besides these three official cameras there are many unofficial cameras that also connect to the MIPI port and should generally be supported. I have revised the sentences related to this point in the Jupiter notebook to clarify to the user that the maximum resolution depends on the camera used and that "for example" the three official cameras have max resolutions of (2592,1944), (3280x2464) and (4056x3040). I have also renamed the cameratype parameter to maxres with the ability to set the resolution via a tuple as well as a string detailing either "v1.5", "v2", or "hq", to help users get the maximum out of their camera.

The code _camconfig.py_ allows for a limited range of ISO.
isos = [100,200,320,400,500,640,800]
Is this consistent with the capabilities of the 3 camera modules? Is it possible to probe the camera module for the available ISO settings?

See my reply in a previous comment.

The code _camconfig.py_ appears to limit resolution to 1640x1232:
if (res[0]*res[1])>=(1640*1232):
        res = maxrect(dims, maxdims = (1640,1232), decimals = 0)
cam.resolution = picamconv(res)
Is my interpretation correct? Is this limitation required?

Yes, this limitation was included to prevent users from using a too high resolution for streaming and getting a shuttering video. The limitation could be removed but most users want to use a smaller video with good frame rate anyway to calibrate the camera settings as that is what the camconfig function is about.

JolleJolles on 17 Sep 2020

@JolleJolles
Looks great that most issues are closed. I'm OK with signing off on publication once the wording on camera calibration is cleared up in the paper. Note that I will need to leave the functionality verification box unchecked since I lack the hardware.

I have revised the paper in a number of ways, including removing the word "calibration" and adding a couple of sentences that clarify the streaming functionalities among some other changes.

JolleJolles on 17 Sep 2020

I hope to have hereby addressed all your comments! Many thanks for the very helpful feedback, I really appreciate you taking the time. And please let me know if there are any final things you'd like me to fix or clarify.

JolleJolles on 17 Sep 2020

I am also happy with the changes and the current state of the submission. And as I could confirm the functionality on python 2.7 I can also tick the functionality part. I really enjoyed playing around with the software, and I hope to apply it in my coming experiments myself.

DerJH on 17 Sep 2020

Great! I acknowledged both your help in the paper. I have created an explanatory video for pirecorder that I want to release when the paper will be available online. I will integrate a final slide with the details of the paper, but you can find a temporary version here: https://youtu.be/aEWIGlDY3f8

JolleJolles on 18 Sep 2020

Thanks @JolleJolles. @marcosvital I approve this submission.

lucask07 on 19 Sep 2020

Dear @marcosvital, please let me know if there is anything else you'd like me to do. Many thanks.

JolleJolles on 22 Sep 2020

Hi, @JolleJolles, I think we are ready to go through our final steps before publication. I'll start with a simple reference's DOI check.

@lucask07 and @DerJH, thank you so much for all the time and effort you put into reviewing this submission for JOSS.

marcosvital on 28 Sep 2020

@whedon check references

marcosvital on 28 Sep 2020

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1038/s41598-019-43845-9 is OK
- 10.1111/2041-210X.13005 is OK
- 10.1371/journal.pcbi.1006959 is OK
- 10.1371/journal.pbio.2003026 is OK
- 10.1016/j.cub.2017.08.004 is OK
- 10.1098/rspb.2017.2629 is OK
- 10.1016/j.anbehav.2019.06.022 is OK
- 10.1371/journal.pone.0169758 is OK
- 10.1016/j.dsr.2019.103136 is OK
- 10.1111/jofo.12182 is OK
- 10.1371/journal.pone.0220751 is OK
- 10.1089/zeb.2016.1412 is OK
- 10.3389/fevo.2018.00026 is OK

MISSING DOIs

- None

INVALID DOIs

- None

whedon on 28 Sep 2020

Good, all references are fine.

@JolleJolles, I'll generate another manuscript proof, so you can take a carefull (maybe final?) look. I'll read it myself too, and will let you know if I find any issues, ok?

marcosvital on 28 Sep 2020

@whedon generate pdf

marcosvital on 28 Sep 2020

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

whedon on 28 Sep 2020

Good, all references are fine.

@JolleJolles, I'll generate another manuscript proof, so you can take a carefull (maybe final?) look. I'll read it myself too, and will let you know if I find any issues, ok?

Hi @marcosvital, I have read the proofs and they look good. I just have three small changes, all for the last paragraph:

one small spelling mistake to the very last sentence ("their" should be "the")
for the references in the last paragraph, fix order (2018 should be before 2019) and change reference jolles et al 2017 by 2020 (rather than add both of them as I don't want to over cite my own studies)
add "(in prep") for the description of the study "more recently, the autonomous long-term recording of fish each day, every day, for the first four-month of their life"

I will wait to hear if you have any final corrections to make and then commit these changes. I will then also generate a final release of the package and publish that to pypi.

Jolles et al 2020 BibTeX:
@article{Jolles2020,
author = {Jolles, Jolle W and Mazu{\'{e}}, Geoffrey P F and Davidson, Jacob and Behrmann-Godel, Jasminca and Couzin, Iain D},
doi = {10.1038/s41598-020-69057-0},
journal = {Scientific Reports},
pages = {12282},
title = {{{\textless}i{\textgreater}Schistocephalus{\textless}/i{\textgreater} parasite infection alters sticklebacks' movement ability and thereby shapes social interactions}},
volume = {10},
number = {1},
year = {2020}
}

JolleJolles on 28 Sep 2020

👍1

@JolleJolles, everything seems fine, and I would like to ask for one more change: please include the "statement of need" as a titled paragraph. I believe that could be done with your second paragraph, as it seems to cover this content. It should look like in this paper.

marcosvital on 29 Sep 2020

I have incorporated your requested change and committed my suggested changes to the paper.