Circuitpython: Improve our reference docs

[tannewt]

ReadTheDocs is awesome for hosting reference docs based on GitHub code. However, we've been bad about ensuring its set up and formatted consistently. Ideally, they would all be like this.

We should do a pass over all of the drivers to ensure that they do these things:

• Have a docs folder with the conf.py moved there. (change sys.path at top of conf.py from '.' to '..') (Adabot verified.)
• Use an docs/index.rst file which includes the README.rst contents and has a table of contents. (Also remove API reference from old README.rst and update conf.py to point to it master_doc = 'index'.) (Adabot verified.)
• Have an examples.rst file which includes all of the example code with titles and brief explanations of what they do. (Adabot verified.)

• Have an updated .travis.yml which will build the docs using sphinx for every PR. Any warnings will cause the Travis run to fail. (Adabot verified.)

  • Add a new folder docs/_static to remove a static warning.
  • Automock any missing modules, such as micropython and busio, in conf.py.
  • Correct any rST syntax errors.
  • Exclude .env and CODE_OF_CONDUCT.md from docs in conf.py.
    python exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', '.env', 'CODE_OF_CONDUCT.md']

  • Add sphinx.ext.todo to extensions in conf.py and these settings.

    # If true, `todo` and `todoList` produce output, else they produce nothing.
    todo_include_todos = False
    
    # If this is True, todo emits a warning for each TODO entries. The default is False.
    todo_emit_warnings = True
    

• Add a blinka favicon at docs/_static/favicon.ico and update conf.py to point to it.
  python html_favicon = '_static/favicon.ico'
• Rename readthedocs.yml to .readthedocs.yml so its hidden on some systems and listed near .travis.yml
• Have Discord and Travis CI badges (Adabot verified.)
• All images in the README have alt text. This is useful for Adabot finding badges and for screen readers. (Adabot verified.)

I'll follow up with more info below.

If you change example files, please check the Learn guide to see if it's linked

• See the comment below for clarification. Thanks!

[willingc]

Have an updated .travis.yml which will build the docs using sphinx for every PR.

Once a month, maybe by scheduled cron job, run make linkcheck on all the docs.

[willingc]

Just an FYI for future docs, you may want to check out sphinx-gallery for examples: https://sphinx-gallery.github.io/index.html

[sommersoft]

Assigned myself, but anyone feel free to jump in. @ me if you start one so I can mark it, or if you get one completed and need it marked as complete.

So, I took step one and made a list of the _current_ drivers in the bundle. Then, went through each one to see if any were already completed (knew it would be a short list).

At any rate, here's the task list:

DRIVERS

• [x] ads1x15
• [x] amg88x
• [x] apds9960
• [x] bme280
• [x] bme680
• [x] bmp280
• [x] bno055
• [x] ccs811
• [x] charlcd
• [x] circuitplayground
• [x] dht
• [x] dotstar
• [x] drv2605
• [x] ds1307
• [x] ds18x20
• [x] ds2413
• [x] ds3231
• [x] fingerprint
• [x] fxas21002c
• [x] fxos8700
• [x] gps
• [x] ht16k33
• [x] ina219
• [x] irremote
• [x] is31fl3731_2
• [x] lis3dh
• [x] lsm9ds0
• [x] lsm9ds1
• [x] max31855
• [x] max31865
• [x] max7219
• [x] max9744
• [x] mcp4725
• [x] mcp9808
• [x] mma8451
• [x] mpr121
• [x] neopixel
• [x] pca9685
• [x] pcf8523
• [x] rfm69
• [x] rgb_display
• [x] sdcard
• [x] seesaw
• [x] sgp30
• [x] sht31
• [x] si5351
• [x] si7021
• [x] ssd1306
• [x] tcs34725
• [x] thermal_printer
• [x] thermistor
• [x] tlc5947
• [x] tlc59711
• [x] trellis
• [x] tsl2561
• [x] tsl2591
• [x] vc0706
• [x] vcnl4010
• [x] veml6070
• [x] vl6180x
• [x] vl53010x

HELPERS

• [x] avrprog
• [x] bus_device
• [x] fancyled
• [x] featherwing
• [x] hid
• [x] motor
• [x] onewire
• [x] register
• [x] rtttl
• [x] simpleio
• [x] waveform

[sommersoft]

README Badges: some have Gitter some don't. Do we want all, some, or none to have it?

[tannewt]

@sommersoft I remove them in favor of the Discord badge when I come across them.

[sommersoft]

Realized a small(?) problem. By getting image files in the repos to pass Sphinx builds (.. image:: ../blah/blah.jpg), they no longer show up in GitHub.

Here is what I have so far

• CircuitPlayground: images were in top/_static/. I moved them to top/docs/_static and updated README with the appropriate path. Images show in RTD build, but do not show in GitHub. My PR was merged, so RTD should reflect my changes.

• DS1307: image was in top/. I originally left it there, and got Sphinx to eventually build using .. image:: ../image.jpg. Moved on to next lib to work, then it popped in my head to check. GitHub is no longer showing the image (PR isn't merged yet, so I can't check RTD). Not fully understanding the problem, I moved the image to top/docs/_static and updated README. That's when I went back to check CircuitPlayground to verify...

• DS3231: (updating original comment) Same as DS1307 above. Just to keep track of where the images are.

I don't remember any other libraries having images that I've done so far, but I'll check. (update: no others effected before CircuitPlayground)

Since we're using the README.rst for both GitHub and RTD, the only way I see having images in both places is to use a .rst and a .md version. Granted, my Sphinx knowledge is barely basic...

[sommersoft]

For clarity, is this what we need to check for on learn pages?

And again, I'm sorry for flooding email boxes! I had a fleeting thought that they might be connected, and even browsed a guide or two but didn't scroll all the way to the end (or the ones I looked at didn't have examples linked).

[willingc]

Folks, I don't think that there is a need for two versions (rst and markdown). I'm trying to understand what is happening now and what you wish the desired behavior to be. Links to specific files would be helpful too. Thanks :-)

[kattni]

If you change example files, please check the Learn guide to see if the code is linked

1. Find the guide for the product associated with the library you're updating.
2. Find the page that explains the code, and scroll down to where the example code is linked. NOTE: This may involved scrolling past a significant number of code snippets before finding the final code, which is what may be linked to GitHub. The snippets are simply built into the guide.
3. Check to see if the example file link would still be accurate. Note: There may be more than one example linked from the GitHub repo in each guide. Remember: Until I approve the PR, the link will continue to be valid, so simply clicking on it to see if it is still valid will not give you the information you need. Visually verify that the link that it currently goes to is something that you altered in your Pull Request.
4. If it would not, please include this information in your pull request (either in the PR itself, or as a comment on your PR).

For example:
The guide has a link to .../libname/main.py (where "..." is https://www.github.com etc) and you changed it to .../libname/examples/sensorname_simpletest.py. If you go to the guide, and the URL to the example is .../libname/main.py, and you know you changed it, then you would include that information in your pull request.

See @sommersoft's comment above for a screenshot of what the file will look like when it's linked to GitHub.

It's ok if you miss it! Even if it's been fixed, we'll likely be getting the email notifying us of the change, so I'll have another opportunity to check it.

It is possible that you'll find examples not linked to GitHub, in which case they will not have a "View on GitHub" link. These will not be altered when you make changes to GitHub.

[sommersoft]

@willingc , the issue is has to do with the file paths. I'll use the DS3231 repo as a reference.

• Pre-update:

  • README showed image in GitHub here
  • The image path is .. image:: 3013-01.jpg

• Post-update (not merged upstream yet):

  • README no longer shows the image in GitHub here
  • The image path is .. image:: ../docs/_static/3013-01.jpg. (note: I moved it into the nested folder)

---

The major difference is the inclusion of .. in the file path. If it is not included, Sphinx will not build, but it does show on GitHub. And vice versa if .. is included in the file path.

Here is the result using an image path of /docs/_static/3013-01.jpg:

$ cd docs && sphinx-build -E -W -b html . _build/html
Running Sphinx v1.7.1
............
Warning, treated as error:
../README.rst:37:image file not readable: docs/_static/3013-01.jpg

Here is the link to the Travis build that fails (I just did this to verify): Travis

And here is the link to the GitHub README linked to that Travis failure: GitHub

I hope that helps explain what I'm seeing. Word salad is a talent of mine. 😄

[tannewt]

The different file path is needed because we moved conf.py from the top directory to docs/conf.py. The top-level README.rst is now included by index.rst which also has the table of contents because the conf.py can't reference it directly anymore. Its also nice because the table of contents doesn't render nicely on GitHub anyway.

[willingc]

Thanks for the detailed explanation @sommersoft, @kattni, and @tannewt. I'll take a closer look. My initial reaction is that the conf.py needs a bit of tweaking re: paths. Overall, folks, the docs are looking fantastic 👍

[sommersoft]

RE: Checking Learn Guides for GitHub linked example files
Make sure you view the webpage with a wide enough scale, otherwise you may miss a "View On GitHub".

The dynamic width of the code blocks will clip it off. ctrl+f "View On GitHub" works, but you still can't see/read it...

This is on Microsoft Edge; can't speak to other browsers.

Also, there may be more than one learn guide affected for each library...

[sommersoft]

Should we close this, now?

[kattni]

We still have some updates to make. Here's the Adabot report:

Missing license. -

• [x] Adafruit_CircuitPython_LSM303
• [x] Adafruit_CircuitPython_TSL2561
• [x] Adafruit_CircuitPython_APDS9960
• [x] Adafruit_CircuitPython_DHT

Not in bundle. -

• [ ] Adafruit_CircuitPython_FocalTouch

Missing CODE_OF_CONDUCT.md -

• [x] Adafruit_CircuitPython_MCP9808
• [x] Adafruit_CircuitPython_CCS811
• [x] Adafruit_CircuitPython_DS1307

ReadTheDocs latest build has warnings and/or errors -

• [x] Adafruit_CircuitPython_AMG88xx
• [x] Adafruit_CircuitPython_LSM303
• [x] Adafruit_CircuitPython_AVRprog
• [x] Adafruit_CircuitPython_VL6180X
• [x] Adafruit_CircuitPython_VL53L0X
• [x] Adafruit_CircuitPython_Thermal_Printer
• [x] Adafruit_CircuitPython_OneWire
• [x] Adafruit_CircuitPython_TCS34725
• [x] Adafruit_CircuitPython_VCNL4010
• [x] Adafruit_CircuitPython_CharLCD
• [x] Adafruit_CircuitPython_LSM9DS0
• [x] Adafruit_CircuitPython_INA219
• [x] Adafruit_CircuitPython_MPR121
• [ ] Adafruit_CircuitPython_APDS9960
• [x] Adafruit_CircuitPython_Fingerprint
• [x] Adafruit_CircuitPython_LSM9DS1
• [x] Adafruit_CircuitPython_MAX31865
• [x] Adafruit_CircuitPython_seesaw

README missing Travis badge -

• [x] Adafruit_CircuitPython_FocalTouch
• [x] Adafruit_CircuitPython_VEML6070
• [x] Adafruit_CircuitPython_LSM303

Autodoc failed on ReadTheDocs. (Likely need to automock an import.) -

• [x] Adafruit_CircuitPython_AMG88xx
• [x] Adafruit_CircuitPython_VL53L0X
• [x] Adafruit_CircuitPython_TCS34725
• [x] Adafruit_CircuitPython_VCNL4010
• [x] Adafruit_CircuitPython_CharLCD
• [x] Adafruit_CircuitPython_LSM9DS0
• [x] Adafruit_CircuitPython_INA219
• [ ] Adafruit_CircuitPython_APDS9960
• [x] Adafruit_CircuitPython_Fingerprint
• [x] Adafruit_CircuitPython_LSM9DS1
• [x] Adafruit_CircuitPython_MAX31865
• [x] Adafruit_CircuitPython_seesaw

Missing .py files in examples folder -

• [x] Adafruit_CircuitPython_LSM303
• [ ] Adafruit_CircuitPython_FeatherWing

Sphinx missing files -

• [x] Adafruit_CircuitPython_CharLCD
• [x] Adafruit_CircuitPython_MPR121

README image missing alt text -

• [x] Adafruit_CircuitPython_DS3231

[willingc]

Great work folks. Moving things forward well.

[sommersoft]

As you can see by its inclusion in almost all categories, LSM303 & FocalTouch need the Full Monty. They were added after I compiled the task list, and I didn't go back and see what was added during the 2 week docstravaganzo.

Preliminary "check the checker" results for the autodoc failures: all of those RTD builds are over a month old, and aren't for the new doc structure. Also, CharLCD is still giving a 404... I'm working on a full scale report; may take a couple days.

[kattni]

Brilliant, thank you!

[tannewt]

Are there doc builds that I need to poke? Its possible the webhooks are setup correct and so they are out of date and therefore look like they are failing.

[sommersoft]

@tannewt CharLCD has been out of commission for some time... It appears that most, if not all, of the ones in the 'autodoc failed' section are older builds (1+ month). I will get a full list together shortly.

[tannewt]

I checked and fixed all the build issues except APDS9960 which is failing due to a data descriptor getting passed a mock object I think.

[sommersoft]

Missing .py files in examples folder -

• Adafruit_CircuitPython_FeatherWing

FeatherWing example.py files are nested in subdirectories. Should we figure out a way to have adabot check nested folders, or maybe just put a dummy/general .py in the examples/ folder?

Autodoc failed on ReadTheDocs. (Likely need to automock an import.) -

• Adafruit_CircuitPython_APDS9960

For the APDS9960, do you want me to remove the const on the APDS9960_ENABLE assignment? I just hope it doesn't waterfall for all of the others and we just keep chasing it down the line.

Sphinx missing files -

• Adafruit_CircuitPython_CharLCD
• Adafruit_CircuitPython_MPR121

I think these two should be good now after the new builds. I checked their RTDs, and all seems well. Just don't want to mark them complete since I didn't do anything with them..

[kattni]

I noticed the FeatherWing issue. If we can update Adabot to check subfolders, that's great, but I think in the case of FeatherWing, it might still be a good idea to have a readme.py in that folder anyway since it's an unusual library. Something explaining that each folder contains examples to go along with the associated FeatherWing.

We'll need to test it if we remove const but I think we should. We need to make sure it gets tested and doesn't sit for ages, though. And, if it's something we have to chase, then we chase it. Once we've chased it to the end, it'll be over and we can do as we like moving forward. I'll defer to Scott on this one if he has a differing opinion.

Thanks for the update!

[tannewt]

We can updated Adabot to be ok with subfolders. I think a README.md would be fine but I don't think a .py makes sense.

I'll take a quick look now at APDS9960 and see if I can get a better mock function in there. I think another library may have done it already.

[tannewt]

Here is a fix for APDS9960: https://github.com/adafruit/Adafruit_CircuitPython_APDS9960/pull/8

We can use the same technique to remove any MockObjects we see references in the auto-gened docs.

Great work everyone!

[sommersoft]

I agree on both accounts. A README would be useful in the FeatherWing examples folder, but a .py might get confusing. Would RTD render a .md (lazy/at work; haven't looked it up myself)? If not, we could setup a toctree for the examples with an examples.rst, or sub-tree it in the index.rst.

[tannewt]

I believe RTD would render the .md unless we explicitly ignore it.

On Mon, Mar 19, 2018 at 8:46 AM sommersoft [e-mail] wrote:

I agree on both accounts. A README would be useful in the FeatherWing
examples folder, but a .py might get confusing. Would RTD render a .md
(lazy/at work; haven't looked it up myself)? If not, we could setup a
toctree for the examples with an examples.rst, or sub-tree it in the
index.rst.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/adafruit/circuitpython/issues/566#issuecomment-374259697,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADNqcrCIRhxgtJTaus9xyRjR5LY0D0Eks5tf9LFgaJpZM4RzRbj
.

[willingc]

I believe @tannewt is correct. If you do not for some reason see it, double check that it is included in any toctree (local or master) and that recommonmark is installed.

[tannewt]

Isn't this done?

[sommersoft]

@tannewt I think the only thing left from the lists above, is taking care of the FeatherWing example "readme". Other than that, guess it depends on what adabot is still complaining about...

[tannewt]

I don't think we'll get to zero, we'll just hover around it. Thanks for getting us close! I'll close this and we can do more Adabot driven stuff in new issues as needed.